Blueprint(Overlink) Candidates Example

This PR:

Adds an example executable that takes as input two blueprint unstructured hex meshes (converted from Overlink silo meshes in Visit), and finds their candidates for intersection using either a BVH or Implicit Grid spatial index.

(note: does not perform the actual intersection operation)
Initial outline for the example was taken from the AWS tutorial's final lesson: link

Adds the example blueprint meshes to the data submodule:
LLNL/axom_data#21

Here are some average numbers (in seconds) for the spatial index performances:

Policy - Spatial Index
Initialize Spatial Index
Query candidates
Write candidates (host)
Total average processing runtime

Sequential - BVH
2.57
11.6
0.9908
15.2

OpenMP - BVH
0.5602
0.5681
1.051
2.267

CUDA - BVH
1.128
0.05817
2.778
3.981

HIP - BVH
4.219
0.06138
1.975
6.294

Sequential - Implicit Grid
1.273
285.8
0.684
287.9

OpenMP - Implicit Grid
0.7592
10.46
0.7577
12.07

CUDA - Implicit Grid
1.187
2.189
2.79
6.216

HIP - Implicit Grid
4.352
8.467
2.283
15.17

Testing setup:

Test command: time ./examples/quest_candidates_example_ex -i ../data/quest/ucart23z.cycle_000000.root  -q ../data/quest/ucart23z_shifted.cycle_000000.root -p <raja policy number> -m <method, either "bvh" or "implicit">
HIP command: flux run -N 1 -g 1
CUDA command: lrun -n 1 -g 1
OpenMP allocation: salloc -N 1 -n 36

Did try getting an allocation with 112 threads with one node, but that does not seem to be allowed anymore:       $ salloc -N 1 -n 112
       salloc: error: Job submit/allocate failed: Requested node configuration is not available

@bmhan12 in your table, is the last cell in each row (total time) supposed to be the sum of the other cells in that row? If so, then the totals don't show that.

@bmhan12 in your table, is the last cell in each row (total time) supposed to be the sum of the other cells in that row? If so, then the totals don't show that.

It's supposed to be the average totals of the runtimes. Updated the table for clarity.
On that note, I did notice the CUDA and HIP total runtimes should be similar looking at the breakdowns, but there is an additional 5-6 seconds for HIP. I  have not narrowed down the cause yet...

Would it also be possible to output and post some basic stats about the query results? For this example (w/ a Cartesian mesh shifted by half a cell width), we'd expect about 8 candidates per cell.

Would it also be possible to output and post some basic stats about the query results? For this example (w/ a Cartesian mesh shifted by half a cell width), we'd expect about 8 candidates per cell.

Yes, here are is the OpenMP-BVH output:
$ ./examples/quest_candidates_example_ex -i ../data/quest/ucart23z.cycle_000000.root  -q ../data/quest/ucart23z_shifted.cycle_000000.root -p 2
[INFO]
     Parsed parameters:
      * First Blueprint mesh to insert: '../data/quest/ucart23z.cycle_000000.root'
      * Second Blueprint mesh to query: '../data/quest/ucart23z_shifted.cycle_000000.root'
      * Verbose logging: false
      * Spatial method: 'Bounding Volume Hierarchy (BVH)'
      * Resolution: 'Not Applicable'
      * Runtime execution policy: 'raja_omp'

[INFO] Reading Blueprint file to insert: '../data/quest/ucart23z.cycle_000000.root'...

[INFO] Mesh bounding box is { min:(-1,-1,-1); max:(1,1,1); range:<2,2,2> }.

[INFO] Reading Blueprint file to query: '../data/quest/ucart23z_shifted.cycle_000000.root'...

[INFO] Mesh bounding box is { min:(-0.995,-0.995,-0.995); max:(1.005,1.005,1.005); range:<2,2,2> }.

[INFO] Reading in Blueprint files took 5.87 seconds.
[INFO] Running BVH candidates algorithm in execution Space: [OMP_EXEC]
[INFO] 0: Initializing BVH took 0.562 seconds.
[INFO] 1: Querying candidate bounding boxes took 0.573 seconds.
[INFO] 2: Initializing candidate pairs on host took 1.04 seconds.
[INFO] Stats for query
    -- Number of insert-BVH mesh hexes 8,000,000
    -- Number of query mesh hexes 8,000,000
    -- Total possible candidates 64,000,000,000,000
    -- Candidates from BVH query 63,521,199

[INFO] Computing candidates took 2.25 seconds.
[INFO] Mesh had 63,521,199 candidates pairs

...and the OpenMP - Implicit Grid output:
$ ./examples/quest_candidates_example_ex -i ../data/quest/ucart23z.cycle_000000.root  -q ../data/quest/ucart23z_shifted.cycle_000000.root -p 2 -m implicit
[INFO]
     Parsed parameters:
      * First Blueprint mesh to insert: '../data/quest/ucart23z.cycle_000000.root'
      * Second Blueprint mesh to query: '../data/quest/ucart23z_shifted.cycle_000000.root'
      * Verbose logging: false
      * Spatial method: 'Implicit Grid'
      * Resolution: '0'
      * Runtime execution policy: 'raja_omp'

[INFO] Reading Blueprint file to insert: '../data/quest/ucart23z.cycle_000000.root'...

[INFO] Mesh bounding box is { min:(-1,-1,-1); max:(1,1,1); range:<2,2,2> }.

[INFO] Reading Blueprint file to query: '../data/quest/ucart23z_shifted.cycle_000000.root'...

[INFO] Mesh bounding box is { min:(-0.995,-0.995,-0.995); max:(1.005,1.005,1.005); range:<2,2,2> }.

[INFO] Reading in Blueprint files took  5.9 seconds.
[INFO] Running Implicit Grid candidates algorithm in execution Space: [OMP_EXEC]
[INFO] 0: Initializing Implicit Grid took 0.73 seconds.
[INFO] 1: Querying candidate bounding boxes took 11.2 seconds.
[INFO] 2: Initializing candidate pairs on host took 7.06 seconds.
[INFO] Stats for query
    -- Number of insert mesh hexes 8,000,000
    -- Number of query mesh hexes 8,000,000
    -- Total possible candidates 64,000,000,000,000
    -- Candidates from Implicit Grid query 504,358,336

[INFO] Computing candidates took 19.2 seconds.
[INFO] Mesh had 504,358,336 candidates pairs

Yes, here are is the OpenMP-BVH output:
-- Candidates from BVH query 63,521,199
-- Candidates from Implicit Grid query 504,358,336

Thanks @bmhan12 -- that's what I suspected, and it accounts for the big difference in runtimes. I think the suggested BBox-BBox filter -- and perhaps also a larger grid resolution -- will significantly improve the runtime.

Thanks @bmhan12 -- that's what I suspected, and it accounts for the big difference in runtimes. I think the suggested BBox-BBox filter -- and perhaps also a larger grid resolution -- will significantly improve the runtime for the ImplicitGrid

That's good to know! I'll look into the filters!

Adding a filter definitely made a difference for the Implicit Grid!
Was able to add a filter by mirroring the BVH two-pass traversal
The OpenMP - Implicit Grid output after adding a filter:
$ ./examples/quest_candidates_example_ex -i ../data/quest/ucart23z.cycle_000000.root  -q ../data/quest/ucart23z_shifted.cycle_000000.root -p omp -m implicit
[INFO]
     Parsed parameters:
      * First Blueprint mesh to insert: '../data/quest/ucart23z.cycle_000000.root'
      * Second Blueprint mesh to query: '../data/quest/ucart23z_shifted.cycle_000000.root'
      * Verbose logging: false
      * Spatial method: 'Implicit Grid'
      * Resolution: '0'
      * Runtime execution policy: 'omp'

[INFO] Reading Blueprint file to insert: '../data/quest/ucart23z.cycle_000000.root'...

[INFO] Mesh bounding box is { min:(-1,-1,-1); max:(1,1,1); range:<2,2,2> }.

[INFO] Reading Blueprint file to query: '../data/quest/ucart23z_shifted.cycle_000000.root'...

[INFO] Mesh bounding box is { min:(-0.995,-0.995,-0.995); max:(1.005,1.005,1.005); range:<2,2,2> }.

[INFO] Reading in Blueprint files took 6.07 seconds.
[INFO] Running Implicit Grid candidates algorithm in execution Space: [OMP_EXEC]
[INFO] 0: Initializing Implicit Grid took 0.792 seconds.
[INFO] 1: Querying candidate bounding boxes took 10.7 seconds.
[INFO] 2: Initializing candidate pairs on host took 0.714 seconds.
[INFO] Stats for query
    -- Number of insert mesh hexes 8,000,000
    -- Number of query mesh hexes 8,000,000
    -- Total possible candidates 64,000,000,000,000
    -- Candidates from Implicit Grid query 63,521,199

[INFO] Computing candidates took 12.3 seconds.
[INFO] Mesh had 63,521,199 candidates pairs

Please also remove the I/O time from the "overall time" column in the results (and perhaps change the name to something like "processing time")

Table has been updated with "Total average processing runtime", and new timings from the implicit grid changes.

Table was updated to include only the time to get the candidate pairs on device, removing the time spent copying these pairs to host.
Currently looking at adding a smaller blueprint mesh for unit testing this example.

@publixsubfan
To cut down on the number of candidates initially given/space allocated for by the Implicit Grid, I tried to mirror the Device Traverser API in the BVH by using the Implicit Grid's queryObject to filter candidates with a test for the bounding box intersection. They seem to work in the same way, but wanted to make sure there were not any caveats or differences between the two? I also noticed the UniformGrid has a queryObject, and wonder if that also can be used in the same way?

Yeah, the query object API should work similarly across the same across the spatial indexes. There are some differences in the APIs, and maybe we can unify them better:

BVH::Traverser::traverseTree() is similar to the ImplicitGrid::QueryObject::visitCandidates() and UniformGrid::QueryObject::visitCandidates() methods.
UniformGrid::QueryObject has a countCandidates method for a bounding box, and ImplicitGrid::QueryObject has a countCandidates method for bounding boxes and points.
For point-based queries, UniformGrid::QueryObject also has a getCandidates method, which just directly returns the cell containing all the candidates.

Hmmm. HIP looks ~4x slower that CUDA. This is something we should dig into in the near futures.

@tomstitt suggested that we look into our usage of atomic operations on hip -- specifically using unsafeAtomicAdd on hip in some cases instead of RAJA::atomicAdd

I may be mistaken, but I believe you can enable HIP unsafe FP atomics in RAJA by compiling with the -munsafe-fp-atomics option in the CMAKE_CXX_FLAGS. We've looked at this in some detail with the RAJA Perf Suite.


fix: A fix for Controller disposed error

Hi,
In the logs, we sometimes see something like this:
E/flutter ( 9644): [ERROR:flutter/lib/ui/ui_dart_state.cc(198)] Unhandled Exception: Bad state: Cannot add new events after calling close
E/flutter ( 9644): #0      _BroadcastStreamController.add (dart:async/broadcast_stream_controller.dart:243)
E/flutter ( 9644): #1      CameraScannerLogMixin.addLog (package:scanner_shared/src/scanner_log.dart:11)
E/flutter ( 9644): #2      MLKitCameraScanner.onDispose (package:scanner_mlkit/src/mlkit_camera_scanner.dart:80)
E/flutter ( 9644): #3      MLKitScannerPageState._stopImageStream (package:smooth_app/pages/scan/ml_kit_scan_page.dart:457)
E/flutter ( 9644): <asynchronous suspension>

This PR ensures the controller is opened before sending anything into it.

Codecov Report

Merging #3351 (f7f54b8) into develop (5304614) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff            @@
##           develop    #3351   +/-   ##
========================================
  Coverage    10.49%   10.49%
========================================
  Files          254      254
  Lines        12362    12362
========================================
  Hits          1297     1297
  Misses       11065    11065

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

Thanks @g123k and thanks @monsieurtanuki for the review


Start of a terminology page

Start of a terminology page. These should be very clearly defined, so I expect my definitions might not cover things completely. The point of this page is to be a succinct collection of terms, so we shouldn't go into too much detail then can link out to other pages.
Some more terms that can be added (that I didn't feel comfortable defining as I have less experience with them)

Blocks
GIL
Rechunk

I am going to be away from open source work for a few weeks so I might not have time to make updates to this branch. I would welcome any commits from others.

[x] Closes #7677
[NA ] Tests added / passed
[x] Passes black dask / flake8 dask / isort dask

Closing this for now. The branch will be around if anyone else wants to pick it up!


Added a mechanism for caching and error handling

caches the url and the vscode.Hover so that Axios is only called for new urls.
Works in debug mode, you need to check that it works in production for the vscode workspace and clears the cache when vscode closes.

Oh I didn't finished the review

@Shaked-g tell me if you plan to continue this or I will take it

@baruchiro Thank you for the reminder, ill continue this.

@baruchiro, i committed the changes


S9992 error for one web application but not for 2nd web application and 2 unit test projects

I have posted this question to stackoverflow... https://stackoverflow.com/questions/44520289/eway-rapid-error-s9992-keeps-occuring-on-asp-net-mvc-application
var ewayClient = RapidClientFactory.NewRapidClient(Platform.eWayApiKey, Platform.eWayPassword, EndPoint);
ewayClient.SetVersion(31);

var transaction = CreateTransaction();
var preAuthTransaction = ewayClient.Create(PaymentMethod.ResponsiveShared, transaction);

Assert.IsNull(preAuthTransaction.Errors);

It sets up a basic transaction but it is failing in 1 out of 4 contexts.
I have two solutions each with a web application and a unit test project.  They all return a response with no Errors except for one of the web applications which returns that S9992 error code.
The web applications are running under IIS Express 10.  The app that fails locally on IIS Express also fails when deployed to external web site.
I am pulling my hair out over this.
NOTE: I'm using  NuGet package version 1.6.0-beta1

Hi @BlackMael
Thanks for raising this, have you spoken to the eWAY customer support team? They are the best people to ask with any issues you have with the SDKs.
As I'm sure you are aware, S9992 is a communications error. Most of these are caused by either your system not having up-to-date root certificates or .NET not being able to connect over TLS 1.2.
If you haven't already, are you able to debug and step through from a break point in your local IIS instance to get the exact error?

Hi @incarnate
I finally found the solution.
I had to set the SecurityProtocol
ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12

I felt my issue was more .NET related and apparently my Google-fu was lacking.
I know eWay were changing to require TLS 1.2 (see https://myeway.force.com/success/s/article/ka828000000L2mlAAC/I-am-testing-in-Sandbox-and-I-am-receiving-an-error-that-the-connection-has-been-closed)
But reason that one .NET web app running on my development machine needed me to add that line, when another .NET web application and two unit test projects across two separate solutions didn't still alludes me.
Everything is running .NET 4.6+
It seems 4.0 and 4.5 don't support TLS 1.2 by default and that 4.6+ does. My one app that is failing is starting up set to Ssl3.  But it is .NET 4.6.1 so I'm confused.
Anyhoo... I just thought I'd mention my findings here as it may help someone else in the future.

Thanks for following up with your fix @BlackMael
It is certainly strange that you're seeing the issue while running .NET 4.6 since it should default to TLS 1.2 as you say. The stable release of the eWAY .NET SDK has got management of the TLS settings built in since v1.5.1, but that fix isn't in the beta yet.
Hope the rest of your project goes smoothly!


TYP: fix typing error in hookspecs, refactor NebariStage

Reference Issues or PRs
xref #1800 ##1894

What does this implement/fix?
Put a x in the boxes that apply

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds a feature)
[ ] Breaking change (fix or feature that would cause existing features not to work as expected)
[ ] Documentation Update
[ ] Code style update (formatting, renaming)
[x] Refactoring (no functional changes, no API changes)
[ ] Build related changes
[ ] Other (please describe):

Testing

[x] Did you test the pull request locally?
[ ] Did you add new tests?

Any other comments?

@costrouc could you have a look at this PR?

This has some big refactorings in the Nebari Stage which needs to be retested now that we changed a bit the behavior for the default schema dumps -- I will follow up on this, next week

@viniciusdc should we close this one or is there still value here?

Hi @dcmcand, There is a lot of value here. This is good work that we will need to encompass sooner or later (related to that schema refactoring I was talking about recently). I was waiting on this due to the exposing default config values Adam was working on.
My opinion on this would be to close, and I will incorporate and cite those changes in a follow up PR


tvOS compatibility

Due to time limits I didn't handled tvOS as a second target but I simply adapted the project removing or changing the conflicting parts of the code.
The best thing would be to create a new target and manage code blocks using #if TARGET_OS_IOS or #if TARGET_OS_TV as you made in your tvOS branch of SRGMediaPlayer

Letterbox is now compatible with tvOS. We are waiting on TagCommander binaries for tvOS (in fact, we'll probably do the work ourselves), so no official release can be made yet. But everything else is ready on our side.
As already discussed elsewhere, we will be in touch when you feel ready to integrate our tvOS-compatible Letterbox in RSI for Apple TV. I would recommend waiting for an official release, though. I am quite optimistic one will be made before the end of the year, but we'll keep you informed.
I'll close this PR, as it is now of no real use anymore. Thanks for initiating this Apple TV-stuff in the first place!

Thank you for your PR. It was an help to push this feature at the top of our backlog.
We just delivered the 2.0.0 release, with an official tvOS support.
We recommend to use this new release, and give us any feedback.


[VTK] build of pcl:x64-windows

Host Environment

Windows 10 Pro x64
Compiler: VS2017

To Reproduce
Steps to reproduce the behavior:
./vcpkg install pcl:x64-windows
Failure logs
Building package vtk[core]:x64-windows...
-- Using cached C:/vcpkg/downloads/Kitware-VTK-v8.2.0.tar.gz
-- Using source at C:/vcpkg/buildtrees/vtk/src/v8.2.0-b37306c27c
-- Configuring x64-windows
-- Building x64-windows-dbg
-- Building x64-windows-rel
CMake Error at scripts/cmake/vcpkg_fixup_cmake_targets.cmake:98 (message):
'C:/vcpkg/packages/vtk_x64-windows/debug/share/vtk' does not exist.
Call Stack (most recent call first):
ports/vtk/portfile.cmake:139 (vcpkg_fixup_cmake_targets)
scripts/ports.cmake:85 (include)
Error: Building package vtk:x64-windows failed with: BUILD_FAILED
Please ensure you're using the latest portfiles with .\vcpkg update, then
submit an issue at https://github.com/Microsoft/vcpkg/issues including:
Package: vtk:x64-windows
Vcpkg version: 2019.08.27-nohash

See #7967

Just to clarify that the issue is related to the VTK:
.\vcpkg install vtk:x64-windows
leads to the same error message

If you could give the PR a shot to see if it fixes your issue that would be great!

I am so sorry, dear cbezault, but it is my first time that I am using vcpkg to add the dependencies into the VS. Tried to take a look into #7967 but did not understood. Can you give light push what should I do to try to fix this issue? Appreciate your help in advance

@davidbun you need to remove glew once and reinstall it. Otherwise the required wrapper fr glew is not going to be installed

@Neumann-A - Thx, figured out that few minutes ago. Both updates from #7967 worked well for VTK. Now I am running the PCL and will let you know

Finally, works including PCL
Thx a lot for your help

@davidbun, PR#7967 merged, VTK works well now, and looks no issue for PCL.


Let Instantiables conform to Instantiable by default

Objectives
Enhance the ease of declaring instantiators.
Discussion
With the recent update to SafeDI 0.5.0, to create an insatiable builder, users need to perform two steps:

Apply the @Instantiable macro to the desired concrete type.
Ensure that the concrete type conforms to Instantiable.

Proposal
Could SafeDI automatically generate the conformance using macro code generation, eliminating the need for users to manually add it?
@Instantiable
public class ConcreteType {
    ...
}

Generated Code
extension ConcreteType: Instantiable { }

No problem, I understand the complexity. The Fixit seems good for now.


[WIP] snake_case naming convention

Regarding your comment on the issue:

instead of just injecting the name translator, lets just inject the whole Database

I'm finding this a bit confusing. Building of all entities and columns happening in dbConfigurator.BuildConfiguration(configurationOptionsBuilder);  is the process of constructing the database. Passing down the database in order to aid it's own construction creates a chicken and egg problem.
I think it makes sense to just pass the DatabaseConfigurationOptionsBuilder and grab the instance from there or if it's overridden on the entity builder.

I totally agree, that's what I was trying to say in my last comment, if you didn't see it.

yes completely missed it :D

I have read the CLA Document and I hereby sign the CLA

Seems promising, lets see what the unit tests say :)

The build is actually failing, can you verify that locally on your end as well?

I got it running locally, had something f* up, but all sorted now.
I am debugging and it seems that the native postgesql naming doesn't align with that of venflows.
so SELECT "Something" AS "Count" FROM ..., the Npgsql command thinks it's Count where venflow thinks it's \"Count\"
I am just learning this but seems that there is a difference between how a case sensitive table/column exists in postgresql and how it's interpreted via SQL.
This makes things a little bit more complicated, if you have any advice for me otherwise it might be a bit slow going for me to get these fixed.

might have been a lot easier than I thought kek ))

Looks quite good, but I am curious to why you added a second ColumnName e.g. QueryColumnName?

Looks quite good, but I am curious to why you added a second ColumnName e.g. QueryColumnName?

The reason I've kept the ColumnName is for it to be used in https://github.com/TwentyFourMinutes/Venflow/blob/82c9f3aad9a3a10d3fe274d60f687d01d470af0f/src/Venflow/Dynamic/Materializer/MaterializerFactory.cs#L116
Custom entities use that to map to properties (I think I can use the property name there but wasn't sure)
The tests in https://github.com/TwentyFourMinutes/Venflow/blob/dev/test/Venflow.Tests/QueryTests/CustomQuery.cs will fail without it. For query like SELECT Count(""Id"") As ""Count"" FROM ""People""

reader.GetColumnSchema() will see "Count" as Count
Venflow will translate Count property to "Count"

so this evaluates to false: https://github.com/TwentyFourMinutes/Venflow/blob/82c9f3aad9a3a10d3fe274d60f687d01d470af0f/src/Venflow/Dynamic/Materializer/MaterializerFactory.cs#L122
The reason I didn't change that to property name is if you want to have something like this in the future:
SELECT silly_column_name FROM ""People""

public class Custom
{
    [Column("silly_column_name")
    public long GoodName { get; set; }
}

Please do tell me which way you want this, I think having the flexibility to do the above example may not be the best to give as a tool to new solutions but may be good to adapt existing solutions. Or you say let's force people to adapt queries to models by using as because then we have 1 way of doing this which might be best with ORMS althought I can see how some may find this annoying.

Well Data Annotations will never arrive in Venflow, so that wouldn't be a worrying point. I see why you created a new property for that, would do something similar. Maybe we should find a different name for it though, as it is being used in more places other than for queries.
Maybe something along the line of NormalizedColumnName?

Did you forget to rename some other things? Unit Tests are failing again.

squashed last commit, I made the mistake of doing the rename in Rider which is not ready to work with experimental features

Classic Rider moment :D

Looks good I'll merge, if you are good with that.

@TwentyFourMinutes that's fine with me thank you


"docker-compose up -d" step failing with error "sync-images-to-local-registry container "0d851d894440" is unhealthy"

I am trying out steps to bring-up Tinkerbell on baremetal server with Windows 2019 installed. I am following steps mentioned in location "https://github.com/tinkerbell/sandbox/blob/main/docs/quickstarts/COMPOSE.md".
In the above location at Step-4, after executing "docker-compose up -d" command, getting following error and not able to proceed.
ERROR : for sync-images-to-local-registry container "0d851d894440" is unhealthy.
ERROR: Encountered errors while bringing up the project.
Expected Behaviour
Step-4 should complete with out errors and should be able to start the provisioner successfully and proceed further bring up of new machine.
Current Behaviour
Getting error in step-4 as explained above.
Possible Solution

Steps to Reproduce (for bugs)
Refer : https://github.com/tinkerbell/sandbox/blob/main/docs/quickstarts/COMPOSE.md

Make sure pre-requisite are met and all components working fine.
Start executing from Step-1

Context
I am trying to bring-up test machine.
Your Environment
Trying to setup provisioner on Windows 2019 DC.
Running Tinkerbell on baremetal

I resolved that on sandbox\deploy\compose\sync-images-to-local-registry\upload.sh, adding  dest=$(echo $dest|sed 's/\r$//') after read -r src dest <<<"$l". But, unfortunately, I still couldn't get the tinker bell to work.

@gabrielbarata , Thanks a lot for the response.
I tried adding the line you mentioned  ( dest=$(echo $dest|sed 's/\r$//') ) in upload.sh and re-ran the command "docker-compose up -d". Unfortunately issue is not resolved, still seeing the same error as before i.e "ERROR: for sync-images-to-local-registry  Container "c1d2fcdaeda3" is unhealthy."
Copy/pasting the snippet from upload.sh below:
for l in "${lines[@]}"; do
read -r src dest <<<"$l"
dest=$(echo $dest|sed 's/\r$//')    ------> The line you mentioned to add.
echo "::::: syncying $src → $url/$dest :::::" >&2
skopeo copy --all --dest-tls-verify=false --dest-creds="$user:$pass" "docker://$src" "docker://$url/$dest"
done
( I am seeing same error in Ubuntu also. Code snippet given above is from Ubuntu )

Can you run docker-compose logs sync-images-to-local-registry please?

@gabrielbarata , I ran the command you gave but I am not seeing any logs getting displayed. Please find the console ouput.
xy@ab:~/sandbox/deploy/compose$ sudo docker-compose logs sync-images-to-local-registry
Attaching to
xy@ab:~/sandbox/deploy/compose$


HSEARCH-3846 Filtering multi values for sort and aggregation rebase

https://hibernate.atlassian.net/browse/HSEARCH-3846
Fixing the problem of aggregating multiple HSEARCH-3846 values. Added filtering of nested filter minister values for sorting and aggregation. A solution for sorting multiple values in user provided mode.
Rebase has been fixed to the master branch.

Thanks for the rebase, but this PR breaks almost every test related to sorts or aggregations: https://ci.hibernate.org/blue/organizations/jenkins/hibernate-search/detail/PR-2255/5/tests

@wklaczynski This looks a bit complex, and is apparently a bit hard to get in working shape.
May I suggest that you just remove all the changes related to aggregations, and focus on sorts as a first step? If you reduce the scope of this PR to just sorts, the changes to low-level classes related to docvalues become unnecessary (it's just a matter of filtering children, after all) and the PR will be much simpler to debug.
Then as a second step we can consider changes to aggregations, in a second PR addressing a separate ticket.
The aggregation changes are unlikely to get merged anyway. First because they are out of scope for HSEARCH-3846, but also because the solution you chose cannot be implemented easily and reliably on Elasticsearch. As I already mentioned before, Elasticsearch does not pick one value per document for aggregations, but instead considers all values of each document. That's a sensible default, and it'd be better to have the same default on the Lucene backend.

Aggregation is together with sorting. Filtering is a tiny problem. I've solved all problems before. I just have to get the tests done. These are only minor errors, I think that it should start working and meet all test conditions in a moment.

I have implemented the default condition none for aggregation if none is set to mode. I saw that you did there max. If no value for multivalue is set for aggregation, it will return the count of all possible values without calculating any function. Unfortunately, elasticsearch does not support the aggregation filter. But since Lucene can do it after modifications. I suspect that they will make some extra money, we will be the first in this matter.

I saw that you transferred the type name to SortMode. Aggregation is also a type for many values, now virtually duplicated. I didn't change it, I left it for SortMode, and for aggregation is MultiValue. Maybe it would be good to consider to establish a common type name?

It defaults to multiple values for aggregation. Of course, after previous filtering, if the filter has been defined. Just selecting the function causes that the values will be converted to one value according to the functions MIN, MAX, AVG, etc. ...

Myślę, że problem gotowy do zamknięcia.

Although it now automatically builds nested questions, sometimes it is necessary to define such a query. Especially when using native lucene questions. In order not to build separate queries for sorting, aggregation and search, this query when using a nested predicate should be ignored when building the same nested context.
            PermissionQuery allowQuery = new PermissionQuery("prices.permissions", operation, user, true);
            PermissionQuery dannyQuery = new PermissionQuery("prices.permissions", operation, user, false);

            BooleanQuery query = new BooleanQuery.Builder()
                    .add(allowQuery, BooleanClause.Occur.MUST)
                    .add(dannyQuery, BooleanClause.Occur.MUST_NOT)
                    .build();

            filter = predicate.nested().objectField("prices")
                    .nest(f -> f.extension(LuceneExtension.get()).fromLuceneQuery(query))
                    .toPredicate();

            sort.add(sorter.field("prices.bruttoPrice_sort")
                    .asc()
                    .mode(SortMode.MIN)
                    .filter(filter));

            SearchQuery<Assortment> query = session.search(scope)
                    .where((f) -> f.bool().must(select)
                    .filter(filter))
                    .sort(sort.toSort())
                    .toQuery();

Using the same filter for sort will be possible because nesting will be ignored. But nesting is recognized when using the same filter for search.


Composing objects in groups of 32 leads to rateLimitExceeded error

Answering these questions before submitting your bug report will help us give
you a quicker answer.  Thank you!
If one or more of these questions are not applicable, feel free to remove them.
Does this issue affect the google-cloud-cpp project?
Yes, specifically the ComposeObject api
If the problem is with the service exposed by the google-cloud-cpp APIs
instead of the client libraries you may consider opening a support request
instead.  The google-cloud-cpp developers cannot help you troubleshoot
problems with the service itself.
What component of google-cloud-cpp is this related to?

Cloud Storage

What version of google-cloud-cpp are you using?
https://github.com/googleapis/google-cloud-cpp/commit/6430b38c4f600c5fb04f1ed62b8e6debe9463699
0.11
What compiler and version are you using?
Apple LLVM version 10.0.0 (clang-1<IP_ADDRESS>)
What operating system and version are you using?
Mac 10.13.6
What were you trying to do?
I work within a business space that's concerned with transferring large files quickly and securely to destinations such as cloud storage. We maximize upload throughput by multi-part concurrent uploads which are then assembled into the original object. This paradigm is quite common in cloud storage providers and indeed most offer api to handle the details of file assembly automatically. However, certain limitations of the google cloud storage "part" paradigm are problematic for us. The specific scenario that we are investigating here was a 10Gb file broken into 1000 part files.
What did you expect to see?
The ComposeObject api return success as we reassemble the final 10Gb object.
###What was the behaviour you expected from the library?
Calling the ComposeObject api with groups of 32 objects (as is the documented limit of cpp api) until all 1000 parts are successfully composed into the single file.
What did you see instead?
Once the part files are uploaded, if we upload too fast and the calls to ComposeObject api are too close together the ComposeObject api fails with a 429 error code
ComposeObject api returns
Error in non-idempotent operation ComposeObject: {
"error": {
"code": 429,
"message": "The rate of change requests to the object bucket/folder/10G.bin exceeds the rate limit. Please reduce the rate of create, update, and delete requests.",
"errors": [
{
"message": "The rate of change requests to the object bucket/folder/10G.bin exceeds the rate limit. Please reduce the rate of create, update, and delete requests.",
"domain": "usageLimits",
"reason": "rateLimitExceeded"
}
]
}
}
Users of the ComposeObject api are forced to limit the calls to 32 objects at a time. We should not then be penalized because we need to do more than 32 faster than the backend services can handle. This is one of the reasons other cloud storage providers hide the details of composing parts into larger files.
Anything else you would like us to know?
The implementation of the gcs ComposeObject (specifically its limitations) should not be forcing callers of the api to have to 'slow down' the composes. Allow a larger compose limit or remove the ratelimiting of composeobject calls.
Yes, Increasing the part size would reduce the number of parts and thus # calls to ComposeObject api but this only masks the issue. Besides, we need to be able to adjust part sizes to maximize throughput. And for any given part size there is a overall file size that can yield a large number of parts.
Include here information about your environment that is not captured above, or
any other information you think might be relevant.

Thank you for filing a detailed bug report.  I will pass this on to my colleagues working on GCS, maybe they can raise the 32 object limit and/or increase the object change rate limit in the service.  Unfortunately, there is little we can do in the library to fix either.
We can, however, build better functionality in the library to help you use the existing APIs as efficiently as possible.  We recently released a new version of the library with two functions that might help you:

google::cloud::storage::ComposeMany (docs) can create an object from an unlimited number of components. It does need a prefix to store temporary objects, as it creates an object for the first 32 components, another object for the remaining 32 components, and so forth (and then builds up a hierarchy of objects until the final object is built).

You may want to take a look at google::cloud::storage::ParallelUploadFile() (docs) which uploads a file using multiple shards (and threads) and then uses ComposeMany to build the larger object.

Please let us know if these help.

@coryan I have a question. Back before the ::ComposeMany api was introduced we composed in groups of 32, then manually deleted files. One side effect we noticed was that uploading a multipart file this way required DELETE permissions enabled on the bucket. Is this true with the new ComposeMany() api ?
Similarly, a retention policy actively interferes with deleting parts. Is this true with the new ComposeMany() api ?
Thanks

@coryan I have a question. Back before the ::ComposeMany api was introduced we composed in groups of 32, then manually deleted files. One side effect we noticed was that uploading a multipart file this way required DELETE permissions enabled on the bucket. Is this true with the new ComposeMany() api ?

Yes.

Similarly, a retention policy actively interferes with deleting parts. Is this true with the new ComposeMany() api ?

My guess is "yes", because ComposeMany() is just a helper to call Compose() repeatedly (no new server-side stuff).  Though I am not sure what interference you have in mind, if you could explain I would be grateful, I think I will be learning something.

Ugh. IMO this is more reason to do as AWS and Azure and treat part files as second class citizens - they are hidden from the end user and are completely managed by the storage service, hence there is no i) rate limiting on compose, ii) no delete permissions to clean up parts, iii) no retention policy that binds the parts
My company works in an industry where companies are very security conscious-they do not understand (rightly so) why they need to enable DELETE permissions in order to upload large files. Ditto for retention policy

Got it. I think the GCS understands the value of multi-part uploading too.
One idea we have been bouncing around is using a separate bucket to hold the parts during the upload, and then "moving" the final object (using Client::RewriteObject()) to the destination bucket. RewriteObject is (mostly) server-side: you do not need to download and upload the data again, but you might need multiple calls before the operation completes. Would that be useful to you?

Let me first say that I really appreciate your comments and input on this, and I am happy to work towards the best solution.
As for an additional bucket, I can think of a couple problem with that:
i) it pushes implementation details to the customer - they will wonder why they need two buckets to upload to one bucket? Our customers often use multiple cloud storage vendors, so they are perplexed when we tell them they need DELETE permissions and/or two buckets for GCS but not for S3/Azure
ii) we service many industries by providing fast transfers of very large files to storage, any storage, including cloud storage. Performance is critical and is in fact a differentiator for our company. So an additional move/copy at the end undoes much of what we are striving to accomplish.

I hear the problems, nothing we can do to hide the inherent differences between multi-part uploads and upload+compose in the client library. All we can do is make it easier to work with upload+compose.

Performance is critical and is in fact a differentiator for our company. So an additional move/copy at the end undoes much of what we are striving to accomplish.

Ack. I would encourage you to test though, because RewriteObject does not copy the metadata in many cases.  As I recall if (a) if there are no changes to the encryption key, (b) there are no changes to the location and storage class, then  RewriteObject is just a metadata operation.

nothing we can do to hide the inherent differences between multi-part uploads and upload+compose in the client library

Can you elaborate? i.e. What do you call a multipart upload vs upload+compose ? Are they two different things? In my mind they're the same.  How does google::cloud::storage::ParallelUploadFile() fit in ?

u elaborate? i.e. What do you call a multipart upload vs upload+compose ?

I was drawing a difference between the behavior of S3 vs. GCS.  I am not sure there is an widely recognized name for the current GCS behavior.

Are they two different things? In my mind they're the same.

Well, in an alternative universe (or some desirable future) GCS could offer an API that behaved exactly like S3 with respect to uploading large files in parts... until then:

How does google::storage::ParallelUploadFile() fit in ?

An easier way to use multiple uploads and then compose and delete.

Gotcha, thanks

I think the questions in this bug report are answered, there is work for the GCS backend team maybe, but this is not the place to track it. Closing.


Side bar to close when user click out of side bar.

The side bar should close whenever user click anywhere but the side bar.
The close button to side bar is only available on top of the side bar. If user navigate or scroll to the bottom, the side bar can't be close except going back to the top and close it.

[x] I searched or browsed the repo’s other issues to ensure this is not a duplicate.
[x] This issue is for enhancement/bug fixing.
[ ] This issue is an addition to JSON database.
[x] I have gone through Contribution Guidelines.
[ ] Assign this issue to me.

Hey @YernYee thanks for this, in fact, I was gonna open an issue for the same. Are you up for this?

Can u put hacktoberfest label for this enhancement?
I've added the functionality plus escape button to close the sidebar


Non-PI Grant associations

The grants_complete table includes non-PI roles as well.  This is not captured in the current grant query.  The roles are found in the CS_EFFORT_COMMITMENT table, which is joined on the awdext.UC_LOC_CD field.
, ec.EMPLOYEE_ID
    , per2.PERSON_NM AS "EffortCommittmentName"
    , ec.EMPLOYEE_ROLE_ID
    , ec.STATUS_CODE
``

We need a mapping of the ROLE_ID to VIVO Grant roles

UCD warehouse definitions
PI = Principal Investigator
OTHR = Other
COPI = Co-PI
KP  = Key Personnel
PLDR = Project Leader
PDIR = Program Director
CLDR = Core Leader

UCDWh: Co-PI
VIVO
<owl:Class rdf:about="http://vivoweb.org/ontology/core#CoPrincipalInvestigatorRole">
<rdfs:label xml:lang="en">Co-Principal Investigator Role</rdfs:label>
<rdfs:subClassOf rdf:resource="http://vivoweb.org/ontology/core#InvestigatorRole"/>
<obo:IAO_0000115 rdf:datatype="http://www.w3.org/2001/XMLSchema#string">
Role of co-principal investigator of an Agreement (for example, a grant), who devotes a specified percentage of time and is considered key personnel.
</obo:IAO_0000115>
</owl:Class>
MIV: Co-Principal Investigator

UCDWh:  Project Leader
<owl:Class rdf:about="http://vivoweb.org/ontology/core#LeaderRole">
<rdfs:label xml:lang="en">Leader Role</rdfs:label>
<rdfs:subClassOf rdf:resource="http://purl.obolibrary.org/obo/BFO_0000023"/>
<obo:IAO_0000112 rdf:datatype="http://www.w3.org/2001/XMLSchema#string">
A broad-ranging leader concept, from leading a small temporary committee to head of a large international organization.
</obo:IAO_0000112>
<obo:IAO_0000115 rdf:datatype="http://www.w3.org/2001/XMLSchema#string">A leadership role</obo:IAO_0000115>
</owl:Class>
MIV: unclear

UCDWh: pending
VIVO:
<owl:Class rdf:about="http://vivoweb.org/ontology/core#InvestigatorRole">
<rdfs:label xml:lang="en">Investigator Role</rdfs:label>
<rdfs:subClassOf rdf:resource="http://vivoweb.org/ontology/core#ResearcherRole"/>
<obo:IAO_0000115 rdf:datatype="http://www.w3.org/2001/XMLSchema#string">
A role in an Agreement (for example, a grant) as a named investigator or key personnel.
</obo:IAO_0000115>
</owl:Class>
MIV: Co-Investigator

Catch-all
<owl:Class rdf:about="http://vivoweb.org/ontology/core#ResearcherRole">
<rdfs:label xml:lang="en">Researcher Role</rdfs:label>
<rdfs:subClassOf rdf:resource="http://purl.obolibrary.org/obo/BFO_0000023"/>
<obo:IAO_0000115 rdf:datatype="http://www.w3.org/2001/XMLSchema#string">
A role of conducting funded or unfunded research, sometimes linked to an Agreement.
</obo:IAO_0000115>
<obo:IAO_0000112 rdf:datatype="http://www.w3.org/2001/XMLSchema#string">
Examples of research can be seen at: http://www.ufl.edu/research/products/index.html. Note these may have been funded, but the research doesn't have to be funded. Also, the research may be linked to an Agreement (for example, a Grant), but does not need to be.
</obo:IAO_0000112>
</owl:Class>
or
<owl:ObjectProperty rdf:about="http://vivoweb.org/ontology/core#contributingRole">
<rdfs:label xml:lang="en">contributor</rdfs:label>
<rdfs:range rdf:resource="http://purl.obolibrary.org/obo/BFO_0000023"/>
<owl:inverseOf rdf:resource="http://vivoweb.org/ontology/core#roleContributesTo"/>
</owl:ObjectProperty>
MIV: Other (can't resolve down to Assistant Researcher, Collaborator, Instructor, Trainer)

From Whitney Cheung, OR;
•	Program Director – Often used interchangeably with “Principal Investigator” (PD/PI); the person responsible for the project or program if awarded.
•	Key Personnel – Umbrella term for an investigator who will contribute / do work on / carry out the proposed project, part of the research team.  May also be referred to as a primary contributor or senior personnel.  Per NIH: “…individuals who contribute to the scientific development or execution of a project in a substantive, measurable way, whether or not they receive salaries or compensation under the grant…Consultants and those with a postdoctoral role also may be considered senior/key personnel if they meet this definition. Senior/key personnel must devote measurable effort to the project whether or not salaries or compensation are requested. "Zero percent" effort or "as needed" are not acceptable levels of involvement for those designated as Senior/Key Personnel.”
•	Core Leader – Investigator who is designated as the lead for a “core”, as a component/project in large multi-project proposals like center grants (e.g. Center ABC, with an Administrative Core, an Education Core, and two projects).  NIH refers to it as a “distinct, reviewable part…for which there is a business need to gather detailed information identified in the funding opportunity announcement”.
•	Project Leader – Investigator who is designated as the lead for a particular component within a multi-project proposal.  “Project” may be used interchangeably with “Core” as defined above.
Other resources for definitions
UC Contract & Grant Manual Chapter 1-500: Principal Investigator Guidelines
1-520 Leadership of a Sponsored Project
An essential qualification of the individual who proposes to head an extramurally supported research, training, or public service project is that he or she will personally participate in it to a significant degree. It is contrary to University policy, and is viewed with extreme disfavor by sponsoring agencies, to list as head of a project the name of an individual, however prestigious that person may be, who will contribute only a minimum or nominal portion of their own time and effort to the furtherance of the work. Within this basic qualification:
a.	The head of a research project is called a Principal Investigator. On occasion, he or she may have one or more Co-Principal Investigators who share with him or her responsibility for participatory conduct of the project.
b.	The head of a training or public service project may be also be called a Project Administrator or Project Director.
c.	The head of a program involving more than one project may be also be called a Program Administrator.
Cayuse SP – UC Davis Handbook for Department Contract & Grant Staff, Principal Investigators and Approvers

I am using the following roles, and definitions:
  VALUES (?grant_role ?ucd_role ?vivo_role) {
    ("PDIR" ucdrp:GrantProgramDirectorRole vivo:LeaderRole)
    ("KP" ucdrp:GrantKeyPersonnelRole vivo:ResearcherRole)
    ("OTHR" ucdrp:GrantOtherRole vivo:ResearcherRole)
    ("PLDR" ucdrp:GrantProjectLeaderRole vivo:LeaderRole)
    ("COPI" ucdrp:GrantCoPrincipalInvestigatorRole vivo:CoPrincipalInvestigatorRole)
    ("PI" ucdrp:GrantPrincipalInvestigatorRole vivo:PrincipalInvestigatorRole )
    ("CLDR" ucdrp:GrantCoreLeaderRole,vivo:ResearcherRole)
    }

ucdrp:GrantProgramDirectorRole
    a vivo:ResearcherRole ;
    rdfs:label "Program Director" ;
    ucdrp:abbrev "PDIR" ;
    .

ucdrp:GrantKeyPersonnelRole
    a vivo:ResearcherRole ;
    rdfs:label "Key Personnel" ;
    ucdrp:abbrev "KP" ;
    .

ucdrp:GrantOtherRole
    a vivo:ResearcherRole ;
    rdfs:label "Researcher" ;
    ucdrp:abbrev "OTHR" ;
    .

ucdrp:GrantProjectLeaderRole
    a vivo:ResearcherRole ;
    rdfs:label "Project Leader" ;
    ucdrp:abbrev "PLDR" ;
    .
ucdrp:GrantCoPrincipalInvestigatorRole
    a vivo:ResearcherRole ;
    rdfs:label "Co-Principal Investigator" ;
    ucdrp:abbrev "COPI" ;
    .
ucdrp:GrantPrincipalInvestigatorRole
    a vivo:ResearcherRole ;
    rdfs:label "PrincipalInvestigator" ;
    ucdrp:abbrev "PI" ;
    .
ucdrp:GrantCoreLeaderRole
    a vivo:ResearcherRole ;
    rdfs:label "Core Leader" ;
    ucdrp:abbrev "CLDR" ;
    .


ClickHouse Unknown Codec Family Error on Version Update

I have been working on a three node ClickHouse Cluster with a Table created using ReplicatedMergeTree. I recently updated one of its nodes from version 18.14.2 to 19.3.7. However, after the upgrade, I could not establish a connection with the upgraded node as the data did not sync up.
The ClickHouse error logs are as follows.
(StorageReplicatedMergeTree): DB::StorageReplicatedMergeTree::queueTask()::<lambda(DB::StorageReplicatedMergeTree::LogEntryPtr&)>: Code: 432, e.displayText() = DB::Exception: Unknown codec family code : 130, Stack trace:

0. clickhouse-server(StackTrace::StackTrace()+0x16) [0x5d9fdf6]
1. clickhouse-server(DB::Exception::Exception(std::string const&, int)+0x1f) [0x2ceb79f]
2. clickhouse-server(DB::CompressionCodecFactory::get(unsigned char) const+0x185) [0x5af35b5]
3. clickhouse-server(DB::CompressedReadBufferBase::readCompressedData(unsigned long&, unsigned long&)+0x911) [0x5af0e31]
4. clickhouse-server(DB::CompressedReadBuffer::nextImpl()+0x26) [0x5aef7b6]
5. clickhouse-server(DB::MergeTreeDataPartChecksums::read_v3(DB::ReadBuffer&)+0x834) [0x5738604]
6. clickhouse-server(DB::MergeTreeDataPartChecksums::read_v4(DB::ReadBuffer&)+0xb2) [0x5738fd2]
7. clickhouse-server(DB::MergeTreeDataPartChecksums::read(DB::ReadBuffer&, unsigned long)+0x28) [0x5739f68]
8. clickhouse-server(DB::MergeTreeDataPartChecksums::read(DB::ReadBuffer&)+0x94) [0x573a114]
9. clickhouse-server(DB::MergeTreeDataPart::loadChecksums(bool)+0x119) [0x572f6b9]
10. clickhouse-server(DB::MergeTreeDataPart::loadColumnsChecksumsIndexes(bool, bool)+0x39) [0x57338a9]
11. clickhouse-server(DB::DataPartsExchange::Fetcher::fetchPart(std::string const&, std::string const&, std::string const&, int, DB::ConnectionTimeouts const&, std::string const&, std::string const&, std::string const&, bool, std::string const&)+0xdde) [0x56d955e]
12. clickhouse-server() [0x567a556]
13. clickhouse-server(DB::StorageReplicatedMergeTree::fetchPart(std::string const&, std::string const&, bool, unsigned long)+0x7a9) [0x569c379]
14. clickhouse-server(DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&)+0x999) [0x569ebf9]
15. clickhouse-server(DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&)+0x9f) [0x56a3c9f]
16. clickhouse-server() [0x56a4610]
17. clickhouse-server(DB::ReplicatedMergeTreeQueue::processEntry(std::function<std::shared_ptr<zkutil::ZooKeeper> ()>, std::shared_ptr<DB::ReplicatedMergeTreeLogEntry>&, std::function<bool (std::shared_ptr<DB::ReplicatedMergeTreeLogEntry>&)>)+0x3e) [0x57aadfe]
18. clickhouse-server(DB::StorageReplicatedMergeTree::queueTask()+0x192) [0x5678b62]
19. clickhouse-server(DB::BackgroundProcessingPool::threadFunction()+0x396) [0x56d3b86]
20. clickhouse-server() [0x56d4484]
21. clickhouse-server(ThreadPoolImpl<std::thread>::worker(std::_List_iterator<std::thread>)+0x199) [0x5da8aa9]
22. clickhouse-server() [0x65ce86f]
23. /lib64/libpthread.so.0(+0x7e25) [0x7fb7b43ffe25]
24. /lib64/libc.so.6(clone+0x6d) [0x7fb7b3c2334d]

There was a serious bug https://github.com/yandex/ClickHouse/pull/4623
I'm not sure if fix  was merged into 19.3 branch.
We recommend to pick latest 19.4 for update.

Upgraded to <IP_ADDRESS>. Now continuously getting the following error.
(StorageReplicatedMergeTree): DB::StorageReplicatedMergeTree::queueTask()::<lambda(DB::StorageReplicatedMergeTree::LogEntryPtr&)>: Poco::Exception. Code: 1000, e.code() = 0, e.displayText() = Null pointer: _pInstance in file "/home/centos/clickhouse-rpm/rpmbuild/BUILD/ClickHouse-<IP_ADDRESS>-stable/contrib/poco/Util/include/Poco/Util/Application.h", line 457

Found that the above is another bug in ClickHouse v<IP_ADDRESS> - #5005

AFAIK in #5005 was somehow related with database with dictionary=Engine, and some dictionaries initialization problems.
Do you use database with Engine=Dictionary? odbc dictionaries? Can you try if ClickHouse starts without dictionaries?
It would be great it we could find the root of that problem.

Please look for the following lines in the logs BEFORE the errors you mentioned.
Application: Caught exception ...
Application: Destroyed global context ...

Upgraded to the latest version which fixed the issue. Closing the issue.


Cannot focus input while image chooser modal is open

Issue Summary
When an image chooser or page chooser modal is open, it's impossible to focus an <input> in another overlay that's appended to the <body>.

Steps to Reproduce

Go to a page with an image chooser or page chooser modal
Paste the following code into the browser console to generate a custom overlay with some inputs:

const mymod = document.createElement('div')
document.body.append(mymod)
mymod.style.position='absolute'
mymod.style.width="300px"
mymod.style.height="300px"
mymod.style.left="300px"
mymod.style.top="300px"
mymod.style.zIndex="9999999"
mymod.style.backgroundColor="#99000088"

const mybutt = document.createElement('button')
mybutt.setAttribute('type', 'button')
mymod.append(mybutt)
mybutt.innerHTML='Click me'
mybutt.addEventListener('click', function(){console.log('clicked')})

const myinp = document.createElement('input')
myinp.setAttribute('type', 'text')
mymod.append(myinp)
myinp.value='foo'

Click the input on the custom overlay. You can focus it and write into it
Open the image chooser or page chooser modal
Although the custom overlay is in front of the Wagtail modal, you can't focus the input on the overlay any more. Clicking the button works.

What did I expect to happen instead?
I should be able to focus the input on my custom overlay. It's a common practice to append overlays to the document body, most libraries do that as it makes absolute positioning way easier (among other reasons). For our project, we need to implement a custom modal that opens on top of the Wagtail modal. We cannot append it to the body as this bug prevents our inputs from getting the focus.

I have confirmed that this issue can be reproduced as described on a fresh Wagtail project: no

Technical details

Wagtail version: 4.2.2
Browser version: Chromium 111

Possibly related to https://github.com/wagtail/wagtail/issues/9326 ?


Could you provide the synthetic datasets?

Hi, I'm very curious about how I could train a new model with your synthetic dataset. Could you tell me the way to download and use the synthetic dataset.

Hi, If you run the command in Step 1 of the Readme, this will automatically create the synthetic dataset for you. It should take a few minutes, so it is as easy as downloading it.
To train a new model, I suggest that you look at the code and replicate what has been done for our own line detector. This is in particular the dataloader to process the synthetic images: https://github.com/cvg/SOLD2/blob/main/dataset/synthetic_dataset.py

Thank you very much for your quick reply!
---Original---
From: "Rémi @.>
Date: Tue, Dec 7, 2021 19:01 PM
To: @.>;
Cc<EMAIL_ADDRESS>Subject: Re: [cvg/SOLD2] Could you provide the synthetic datasets? (Issue #32)
Hi, If you run the command in Step 1 of the Readme, this will automatically create the synthetic dataset for you. It should take a few minutes, so it is as easy as downloading it.
To train a new model, I suggest that you look at the code and replicate what has been done for our own line detector. This is in particular the dataloader to process the synthetic images: https://github.com/cvg/SOLD2/blob/main/dataset/synthetic_dataset.py
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.


Proposal: Add '.' as a valid column number for :subst %c patterns

I've been editing markdown formatted tables, and have made great use of the \%<23c zero-width pattern atoms.
My particular table had "structured" fields on the left, and "unstructured" text on the right. My changes were to the structured fields, so I spent some time using \%<23c as a constraint on my subst commands: "make these changes, but only in the structured fields!"
However, that number 23 can be hard to come by. The best "direct" method I came up with was to use the expression register, something like:
s/... \%< [Ctrl-r] =col(".") [Enter] c ... / ... /

The alternative was to try reading the number off the status line.
I think that adding just a dot character (.) as a valid column number, signifying "position of the cursor at the start of the command," would address this nicely. And it would make the subst more usable in '@'macros.
I thought about suggesting that \%c (and \%v) without numbers take the current position, but that lacks specificity. Typing the dot says "I mean to do this".
This change should probably apply to %v (virtual column) and %l (line number), as well.

I have implemented that in PR https://github.com/vim/vim/pull/8497.
Note, if we omit the . and no number is given, this might actually look slightly better.
So instead of using \%<.c to match all positions before the cursor position, use \%<c.
Not sure what would be better.


Upload community call recording video to YouTube

Problem: The community call recording is not uploaded to YouTube.

The recording video of community call should be uploaded to YouTube by default for openness, transparency, and tracking purpose
Upload the existing recordings to YouTube
Figure out a way of automatically uploading to save effort from manual work.

We got an access invitation to CNCF-related platforms a couple of days ago. It requires another CNCF email account access to log in to the Notary Zoom account. I submitted a ticket to CNCF to ask for their help and request automation for this flow. https://cncfservicedesk.atlassian.net/servicedesk/customer/portal/1/CNCFSD-1684?created=true
I will update the progress here if I receive any updates from the CNCF Service Desk.

Now the livestreaming to Youtube option has been set in Notary's Zoom meeting room. We can enable the livestreaming option when starting the meeting. The recordings will be automatically uploaded to Youtube.

For previous meeting records, we may need to download from Zoom and upload them to YouTube manually. I am still confirming this question with CNCF.

@FeynmanZhou can this issue be closed now?

Yes. We can close it as the Zoom-to-Youtube livestream configuration was set. We had a guideline for streaming https://lists.cncf.io/g/cncf-notary-maintainers/topic/98852881


spelling and clarifications

Fixing some simple spelling mistakes.
Also, Keras is not only for Python anymore. For example, TensorFlow.js supports Keras-like layers. Telling the readers which language it is written in might not be necessary here though anyway.

Sounds good. thanks for the contribution


#42 What is in the main, what is in the CORE, what is somewhere else

For the moment there is no explicit logic on this but it could be interesting discussing about it !

This as an example : should y, Y_s should be explicitely in main or inside the core ?

Solved by new structure


Convert API payloads to snake_case

The expectation is that Ruby applications will be consuming the API predominantly, so it makes sense to accept snake_case attributes instead of the default camelCase to make the implementation simpler on the service consuming the API.

We had a discussion about the merits of this and decided to leave it as camelCase and convert to snake_case in the client instead.


Idea

Is your feature request related to a problem or do you just have an idea that fit the theme of the mod? Please describe.
No not a problem but a cool idea for Graves/graveyards
Describe the solution you'd like
I was hoping to see like maybe a rare chance a Zombie with the player head is spawned from your grave or when opened
Describe alternatives you've considered
just by rare chance or on opening the grave
Additional context
gotta keep my server on their toes so figured a zombie rising out of their graves would do and also Fit the theme of you are in grave danger :)

Sorry for not replying to your issue. I usually try to, but at the time you created the feature suggestion, I was busy and didn't have the time to tend to this mod. I've slowly tried increasing productivity on this project as well as other projects, and I've implemented a config allowing you to do this very thing.
The config is called randomSpawnSettings and can vastly controll entities spawned when retrieving items. You can configure the chance of an entity spawning, what type of mob it should be, and what NBT data it should contain. In the NBT data you can use values as grave owner UUID, grave owner username, grave looter name and UUID, as well as every item in the inventory of the grave owner. You can alos decide if using the item in the NBT should be removed from the grave drops or not (in case you would like to spawn a zombie with all the stuff, making you have to fight for it back for example).
I realize that the current default config, even though it's supposed to work like you asked (zombie with player head), it will not do that. You have to replace the NBT data with this for it to work: {ArmorItems:[{},{},{},{id:"minecraft:player_head",tag:{SkullOwner:{Name:"${owner.name}",Id:"${owner.uuid}"}},Count:1b}]}
This was probably more than you expected. And someone else asked about a similar feature (with another entity type), so I decided to just make it as configurable as I possibly could. Let me know if you have any questions, or if you would like me to create a config setting for your need.
This feature is in the latest version for minecraft 1.18.2 and 1.19.2


[gitlab-use-module] Use gitlab python module instead of raw http requests

What does this PR do?
Replaces the gitlab rest api calls by python-gitlab calls

Motivation

Additional Notes

The only business logic difference after refactoring is that tasks.pipeline.wait_for_pipeline_from_ref takes the last updated pipeline instead of the last created one
The gitlab module is now named gitlab_api to avoid conflicts with the package
The package uses gitlab api v4
Now the gitlab_api module provides only tokens and function to get the gitlab api / repository object
To have all items from the api per_page=100 is the most efficient (max 100 items per page)
To access functions from objects obtained by list(), we need to get the full object using get(..., lazy=True), lazy avoid doing another api call

Possible Drawbacks / Trade-offs

Describe how to test/QA your changes

Each updated task / function has been tested by hand

/trigger-ci

/trigger-ci --variable RUN_ALL_BUILDS=true --variable RUN_KITCHEN_TESTS=true --variable RUN_E2E_TESTS=on --variable RUN_UNIT_TESTS=on --variable RUN_KMT_TESTS=on


How to generate the  "openshape-objaverse-embeddings" ?

HI, @Colin97 @eliphatfs  Thanks for sharing your excellent work.
I wonder how to extract the embeddings of the objects in the dataset objaverse? which model was used ?

The OpenShape/openshape-pointbert-vitg14-rgb model was used.
You can check here for how to use the model for inference. Preprocessing logic is here.
Note that you need to handle the gravity axis mismatch between Objaverse and the model by swapping y and z coordinates in the point cloud.

You may also try the following snippet with this repository:
def test_objaverse(self):
    import tqdm
    import torch.nn.functional as F
    us = []
    feats = []
    self.model.eval()
    if self.config.training.use_text_proj:
        self.text_proj.eval()
    clip_text_feat = torch.from_numpy(self.objaverse_lvis_loader.dataset.clip_cat_feat).to(self.config.device)
    if self.config.training.use_text_proj:
        clip_text_feat = self.text_proj(clip_text_feat)
    with torch.no_grad():
        for data in tqdm.tqdm(self.objaverse_lvis_loader):
            if not self.config.model.get("use_dense", False):
                pred_feat = self.model(data['xyz'], data['features'], \
                                        device = self.config.device, \
                                        quantization_size = self.config.model.voxel_size)
            else:
                pred_feat = self.model(data['xyz_dense'], data['features_dense'])
            us.extend(data['name'])
            feats.append(torch.tensor(quantize(pred_feat.cpu().numpy())))
    torch.save({"us": us, "feats": torch.concat(feats)}, "/root/objaverse.pt")

test_objaverse

@eliphatfs thanks for your reply
I have tried to extract the embeddings of the objects from the objaverse dataset with the model "openshape-pointbert-vitg14-rgb". But the embedding is different from that in the "openshape-objaverse-embeddings" with the same uid.
Is it due to the operation sampling the 10000 points?

You may also try the following snippet with this repository:
def test_objaverse(self):
    import tqdm
    import torch.nn.functional as F
    us = []
    feats = []
    self.model.eval()
    if self.config.training.use_text_proj:
        self.text_proj.eval()
    clip_text_feat = torch.from_numpy(self.objaverse_lvis_loader.dataset.clip_cat_feat).to(self.config.device)
    if self.config.training.use_text_proj:
        clip_text_feat = self.text_proj(clip_text_feat)
    with torch.no_grad():
        for data in tqdm.tqdm(self.objaverse_lvis_loader):
            if not self.config.model.get("use_dense", False):
                pred_feat = self.model(data['xyz'], data['features'], \
                                        device = self.config.device, \
                                        quantization_size = self.config.model.voxel_size)
            else:
                pred_feat = self.model(data['xyz_dense'], data['features_dense'])
            us.extend(data['name'])
            feats.append(torch.tensor(quantize(pred_feat.cpu().numpy())))
    torch.save({"us": us, "feats": torch.concat(feats)}, "/root/objaverse.pt")

@eliphatfs thanks for your reply
I have tried to extract the embeddings of the objects from the objaverse dataset with the model "openshape-pointbert-vitg14-rgb". But the embedding is different from that in the "openshape-objaverse-embeddings" with the same uid.
Is it due to the operation sampling the 10000 points?

How different are they? We have farthest point sampling in the tokenization layer that involves randomness, and minor differences are expected.

HI, @eliphatfs
For the the file 0155b81b90fd4585b388f5c0e34bbdfb.glb, the cosine similarity score is about 0.95 which is calculated between the embedding extracted using the model "openshape-pointbert-vitg14-rgb" and that from the "objaverse_embeddings" with the same uid.
Does it mean that they are different?  or Did I do something wrong?
The following codes  are extracted from  the repository "openshape-demo-support"
1. Functions to read the data
def trimesh_to_pc(scene_or_mesh):
    if isinstance(scene_or_mesh, trimesh.Scene):
        meshes = []
        for node_name in scene_or_mesh.graph.nodes_geometry:
            # which geometry does this node refer to
            transform, geometry_name = scene_or_mesh.graph[node_name]

            # get the actual potential mesh instance
            geometry = scene_or_mesh.geometry[geometry_name].copy()
            if not hasattr(geometry, 'triangles'):
                continue
            geometry: trimesh.Trimesh
            geometry = geometry.apply_transform(transform)
            meshes.append(geometry)
        total_area = sum(geometry.area for geometry in meshes)
        if total_area < 1e-6:
            raise ValueError("Bad geometry: total area too small (< 1e-6)")
        pcs = []
        for geometry in meshes:
            pcs.append(model_to_pc(geometry, max(1, round(geometry.area / total_area * 10000))))
        if not len(pcs):
            raise ValueError("Unsupported mesh object: no triangles found")
        return numpy.concatenate(pcs)
    else:
        assert isinstance(scene_or_mesh, trimesh.Trimesh)
        return model_to_pc(scene_or_mesh, 10000)

def model_to_pc(mesh: trimesh.Trimesh, n_sample_points=10000):
    f32 = numpy.float32
    rad = numpy.sqrt(mesh.area / (3 * n_sample_points))
    for _ in range(24):
        pcd, face_idx = trimesh.sample.sample_surface_even(mesh, n_sample_points, rad)
        rad *= 0.85
        if len(pcd) == n_sample_points:
            break
    else:
        raise ValueError("Bad geometry, cannot finish sampling.", mesh.area)
    if isinstance(mesh.visual, trimesh.visual.ColorVisuals):
        rgba = mesh.visual.face_colors[face_idx]
    elif isinstance(mesh.visual, trimesh.visual.TextureVisuals):
        bc = trimesh.proximity.points_to_barycentric(mesh.triangles[face_idx], pcd)
        if mesh.visual.uv is None or len(mesh.visual.uv) < mesh.faces[face_idx].max():
            uv = numpy.zeros([len(bc), 2])
            print("Invalid UV, filling with zeroes")
        else:
            uv = numpy.einsum('ntc,nt->nc', mesh.visual.uv[mesh.faces[face_idx]], bc)
        material = mesh.visual.material
        if hasattr(material, 'materials'):
            if len(material.materials) == 0:
                rgba = numpy.ones_like(pcd) * 0.8
                texture = None
                print("Empty MultiMaterial found, falling back to light grey")
            else:
                material = material.materials[0]
        if hasattr(material, 'image'):
            texture = material.image
            if texture is None:
                rgba = numpy.zeros([len(uv), len(material.main_color)]) + material.main_color
        elif hasattr(material, 'baseColorTexture'):
            texture = material.baseColorTexture
            if texture is None:
                rgba = numpy.zeros([len(uv), len(material.main_color)]) + material.main_color
        else:
            texture = None
            rgba = numpy.ones_like(pcd) * 0.8
            print("Unknown material, falling back to light grey")
        if texture is not None:
            rgba = trimesh.visual.uv_to_interpolated_color(uv, texture)
    if rgba.max() > 1:
        if rgba.max() > 255:
            rgba = rgba.astype(f32) / rgba.max()
        else:
            rgba = rgba.astype(f32) / 255.0
    return numpy.concatenate([numpy.array(pcd, f32), numpy.array(rgba, f32)[:, :3]], axis=-1)

def load_pc_data(glb_file: str, swap_yz_axes: bool=False):
    glb_obj = trimesh.load(glb_file)

    pc = trimesh_to_pc(glb_obj)

    assert pc.ndim == 2, "invalid pc shape: ndim = %d != 2" % pc.ndim
    assert pc.shape[1] in [3, 6], "invalid pc shape: should have 3/6 channels, got %d" % pc.shape[1]
    pc = pc.astype(f32)
    if swap_yz_axes:
        pc[:, [1, 2]] = pc[:, [2, 1]]

    pc[:, :3] = pc[:, :3] - numpy.mean(pc[:, :3], axis=0)
    pc[:, :3] = pc[:, :3] / numpy.linalg.norm(pc[:, :3], axis=-1).max()
    if pc.shape[1] == 3:
        pc = numpy.concatenate([pc, numpy.ones_like(pc) * 0.4], axis=-1)

    if pc.shape[0] >= 10000:
        pc = pc[numpy.random.permutation(len(pc))[:10000]]
    elif pc.shape[0] == 0:
        raise ValueError("Got empty point cloud!")
    elif pc.shape[0] < 10000:
        pc = numpy.concatenate([pc, pc[numpy.random.randint(len(pc), size=[10000 - len(pc)])]])

    return pc.astype(f32)

2. extract the embedding of point cloud
name = 'openshape-pointbert-vitg14-rgb'
pc_encoder = openshape.load_pc_encoder(name=name)

glb_file = "/data/0155b81b90fd4585b388f5c0e34bbdfb.glb"
pc_data = load_pc_data(glb_file=glb_file, swap_yz_axes=False)

pc_encoder.eval()
with torch.no_grad():
    pc_enc = pc_encoder(torch.tensor(pc_data[:, [0, 2, 1, 3, 4, 5]].T[None], device="cuda:0")).cpu()

3. get the embedding from "objaverse_embeddings"
meta_path = "/data/objaverse_meta.json"
feat_path = "/data/objaverse.pt"
meta, us, feats = load_objaverse_embeddings(meta_path=meta_path, feat_path=feat_path)
dict_uid_to_idx = {k: v for v, k in enumerate(us)}

obj_id = "0155b81b90fd4585b388f5c0e34bbdfb"
idx = dict_us_to_idx[obj_id]

obja_emb = feats[idx]
obja_emb = obja_emb.type('torch.FloatTensor')

4. The cosine similarity score
obja_emb /= obja_emb.norm(dim=-1, keepdim=True)
pc_enc /= pc_enc.norm(dim=-1, keepdim=True)

score = torch.dot(obja_emb, pc_enc[0])

I think the similarity is fairly high, and shall not interfere much with the downstream applications.
I currently cannot tell where the difference comes from and I might check it later.

I think the similarity is fairly high, and shall not interfere much with the downstream applications. I currently cannot tell where the difference comes from and I might check it later.

thanks a lot for your reply

I did some toy experiments and basically the FPS randomness can cause the embeddings to fluctuate with similarity mostly in 97-99; resampling can cause the embeddings to fluctuate with similarity between 94-99. Thus it is natural to have an object embedding that has 95 similarity with ours.

FPS is inside the model (check the PointBERT paper), where 10000 points are split into k (k = 384 in the g14 model) patches randomly distributed across the point cloud.
Sampling is the code you just pasted. It randomly samples 10000 points from the surface of the objects. By 're'-sampling I simply mean running it twice, as the distribution of these points has its randomness.

FPS is inside the model (check the PointBERT paper), where 10000 points are split into k (k = 384 in the g14 model) patches randomly distributed across the point cloud.
Sampling is the code you just pasted. It randomly samples 10000 points from the surface of the objects. By 're'-sampling I simply mean running it twice, as the distribution of these points has its randomness.

I get it now. And thanks again for your reply.


metadata_model creating wrong load_path for KNNRegressor

From Metadata.toml:
[NearestNeighbors.KNNRegressor]
":input_scitype" = "`ScientificTypes.Table{_s13} where _s13<:(AbstractArray{_s12,1} where _s12<:ScientificTypes.Continuous)`"
":target_scitype" = "`AbstractArray{ScientificTypes.Continuous,1}`"
":is_pure_julia" = "`true`"
":package_name" = "NearestNeighbors"
":package_license" = "MIT"
":load_path" = "MLJModels.NearestNeighbors_.MLJModels.NearestNeighbors_.KNNRegressor"

Isolated problem to metadata_model tool and will fix shortly.

Oh no that's my fault I believe, I'll try to fix this and will report if I manage to.


diffoscope: 29 -> 44

Relevant changes:

Python version switched to Python 3
ssdeep library got replaced with tlsh
the 'magic' Python package got replaced with a different one
Minor build system improvements == less work for us

In total the changes are fairly large and I'm not terribly familiar with Python packaging, so maybe @domenkozar could have a quick look?

Heh, I wanted to try this, but the closure... it wants > 2.5 GB of new packages installed! The collection of compilers is really impressive:
47160   /nix/store/pcmmfcn0f8wpbn2lsj27z1zdaizs186p-perl-5.20.3
51748   /nix/store/y4f8fr5nqizn5ii9vp5wwj0wwlkmds33-python-2.7.11
61576   /nix/store/5a4ixyqczpnpqfcwjl4mgkds28hn9cjm-python3-3.4.3
79480   /nix/store/a5f56ipqjmkjnf4xrv19368k2hcv3dp8-openjdk-7u91b01
89756   /nix/store/9v4rwdywv1a5rnk5nz7akkl7lsxq8yrm-openjdk-7u91b01-jre
100564  /nix/store/rpdv6pi9z5dq7mcfh3rsavx0aixvcgl1-gcc-4.9.3
192128  /nix/store/x17j0g3kj4bg32cmrffyiaqx8r7n88vl-gcj-4.9.3
237300  /nix/store/7bkbyd2vi2h618k3gp86q8qbg1zb5bj9-fpc-2.6.4
324884  /nix/store/g8xpgyf6gszj1v032swvvb9kbb7sgqsc-mono-<IP_ADDRESS>
1026248 /nix/store/r77siffpvl6a7y9z6d1ql6swimk0s6pn-ghc-7.10.3

Heh, I wanted to try this, but the closure... it wants > 2.5 GB of new packages installed! The collection of compilers is really impressive:
47160   /nix/store/pcmmfcn0f8wpbn2lsj27z1zdaizs186p-perl-5.20.3
51748   /nix/store/y4f8fr5nqizn5ii9vp5wwj0wwlkmds33-python-2.7.11
61576   /nix/store/5a4ixyqczpnpqfcwjl4mgkds28hn9cjm-python3-3.4.3
79480   /nix/store/a5f56ipqjmkjnf4xrv19368k2hcv3dp8-openjdk-7u91b01
89756   /nix/store/9v4rwdywv1a5rnk5nz7akkl7lsxq8yrm-openjdk-7u91b01-jre
100564  /nix/store/rpdv6pi9z5dq7mcfh3rsavx0aixvcgl1-gcc-4.9.3
192128  /nix/store/x17j0g3kj4bg32cmrffyiaqx8r7n88vl-gcj-4.9.3
237300  /nix/store/7bkbyd2vi2h618k3gp86q8qbg1zb5bj9-fpc-2.6.4
324884  /nix/store/g8xpgyf6gszj1v032swvvb9kbb7sgqsc-mono-<IP_ADDRESS>
1026248 /nix/store/r77siffpvl6a7y9z6d1ql6swimk0s6pn-ghc-7.10.3

And it's not currently even depending on everything that it could be (since a bunch aren't packaged in NixOS)! Most of the dependencies are optional though, so you could just comment out/override to null anything you don't need in propagatedBuildInputs. In fact, maybe it'd be useful to add a diffoscope-full (i.e. what it currently is) and then make the plain diffoscope depend only on say, those tools that are only needed to diff the NixOS LiveCD images or the nixos.closures.* derivations in `release.nix'

FYI closure size has been reduced in https://github.com/NixOS/nixpkgs/commit/bbfbadf327af8454fc449fb567db00ab254cfc3c


Ignore outside click events and escape when recent overlays opened exist

Fixes #43

This change is

Reviewed 2 of 2 files at r1.
Review status: all files reviewed at latest revision, all discussions resolved.

Comments from Reviewable

vaadin-overlay.html, line 379 at r1 (raw file):

       * returns true if this is the last in the opened overlay stack
       */
      get last() {

No need for this to be public last -> _last

Comments from Reviewable

Did you try the approach suggested here ?

Comments from Reviewable

Reviewed 2 of 2 files at r1.
Review status: all files reviewed at latest revision, 1 unresolved discussion.

Comments from Reviewable

I tried that approach, but means that all elements using the overlay needs to be updated to listen for the custom events fired by the overlay in order to prevent them.  This approach is simpler and fixes the problem at once.

Review status: all files reviewed at latest revision, 1 unresolved discussion.

Comments from Reviewable

Review status: 0 of 2 files reviewed at latest revision, 1 unresolved discussion.

vaadin-overlay.html, line 379 at r1 (raw file):
Previously, tomivirkki (Tomi Virkki) wrote…
No need for this to be public last -> _last

Done.

Comments from Reviewable

Reviewed 2 of 2 files at r2.
Review status: all files reviewed at latest revision, all discussions resolved.

Comments from Reviewable


Laravel 5.3

the below issue occuring in laravel 5.3
Could not add route model binding for the route 'phone'.
Please add the following line manually in C:\xampp\htdocs\blog\app\Providers/RouteServiceProvider.php:
$router->model('phone', 'App\Phone');

This package is not yet valid for Laravel 5.3 completely . It needs lots of tweak.
As lots of file structure has changed in laravel 5.3


Wagtail 6.2rc1 - Migration issue

Issue Summary
Wagtail 6.2rc1, python manage.py migrate.  Looks like while doing something with users in the migrations, something is breaking.
Here is the stack trace:
(.venv) % python manage.py migrate
Operations to perform:
  Apply all migrations: accounts, admin, ai, articles, auth, authors, awards, banking, brokerages, calculators, contenttypes, conversions, creditcards, cryptocurrency, db, django_rq, ecap, insurance, loans, marketing, mortgages, pages, personalfinance, press, quizzes, research, sessions, shopping, sites, smallbusiness, snippetfinder, student_loans, taggit, taxes, wagtail_ai, wagtail_content_import, wagtail_transfer, wagtailadmin, wagtailcore, wagtaildocs, wagtailembeds, wagtailforms, wagtailimages, wagtailredirects, wagtailsearch, wagtailusers
Traceback (most recent call last):
  File "/Users/twhite/PycharmProjects/the-ascent/manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/Users/twhite/PycharmProjects/the-ascent/.venv/lib/python3.11/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
    utility.execute()
  File "/Users/twhite/PycharmProjects/the-ascent/.venv/lib/python3.11/site-packages/django/core/management/__init__.py", line 436, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/Users/twhite/PycharmProjects/the-ascent/.venv/lib/python3.11/site-packages/django/core/management/base.py", line 413, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/Users/twhite/PycharmProjects/the-ascent/.venv/lib/python3.11/site-packages/django/core/management/base.py", line 459, in execute
    output = self.handle(*args, **options)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/twhite/PycharmProjects/the-ascent/.venv/lib/python3.11/site-packages/django/core/management/base.py", line 107, in wrapper
    res = handle_func(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/twhite/PycharmProjects/the-ascent/.venv/lib/python3.11/site-packages/django/core/management/commands/migrate.py", line 302, in handle
    pre_migrate_apps = pre_migrate_state.apps
                       ^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/twhite/PycharmProjects/the-ascent/.venv/lib/python3.11/site-packages/django/utils/functional.py", line 47, in __get__
    res = instance.__dict__[self.name] = self.func(instance)
                                         ^^^^^^^^^^^^^^^^^^^
  File "/Users/twhite/PycharmProjects/the-ascent/.venv/lib/python3.11/site-packages/django/db/migrations/state.py", line 566, in apps
    return StateApps(self.real_apps, self.models)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/twhite/PycharmProjects/the-ascent/.venv/lib/python3.11/site-packages/django/db/migrations/state.py", line 627, in __init__
    self.render_multiple([*models.values(), *self.real_models])
  File "/Users/twhite/PycharmProjects/the-ascent/.venv/lib/python3.11/site-packages/django/db/migrations/state.py", line 665, in render_multiple
    model.render(self)
  File "/Users/twhite/PycharmProjects/the-ascent/.venv/lib/python3.11/site-packages/django/db/migrations/state.py", line 949, in render
    body = {name: field.clone() for name, field in self.fields.items()}
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/twhite/PycharmProjects/the-ascent/.venv/lib/python3.11/site-packages/django/db/migrations/state.py", line 949, in <dictcomp>
    body = {name: field.clone() for name, field in self.fields.items()}
                  ^^^^^^^^^^^^^
  File "/Users/twhite/PycharmProjects/the-ascent/.venv/lib/python3.11/site-packages/django/db/models/fields/__init__.py", line 665, in clone
    name, path, args, kwargs = self.deconstruct()
                               ^^^^^^^^^^^^^^^^^^
  File "/Users/twhite/PycharmProjects/the-ascent/.venv/lib/python3.11/site-packages/wagtail/fields.py", line 169, in deconstruct
    block_types = [
                  ^
  File "/Users/twhite/PycharmProjects/the-ascent/.venv/lib/python3.11/site-packages/wagtail/fields.py", line 170, in <listcomp>
    (name, lookup.add_block(block))
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/twhite/PycharmProjects/the-ascent/.venv/lib/python3.11/site-packages/wagtail/blocks/definition_lookup.py", line 70, in add_block
    deconstructed = block.deconstruct_with_lookup(self)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/twhite/PycharmProjects/the-ascent/.venv/lib/python3.11/site-packages/wagtail/blocks/struct_block.py", line 326, in deconstruct_with_lookup
    [
  File "/Users/twhite/PycharmProjects/the-ascent/.venv/lib/python3.11/site-packages/wagtail/blocks/struct_block.py", line 327, in <listcomp>
    (name, lookup.add_block(block))
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/twhite/PycharmProjects/the-ascent/.venv/lib/python3.11/site-packages/wagtail/blocks/definition_lookup.py", line 70, in add_block
    deconstructed = block.deconstruct_with_lookup(self)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/twhite/PycharmProjects/the-ascent/.venv/lib/python3.11/site-packages/wagtail/blocks/list_block.py", line 407, in deconstruct_with_lookup
    if isinstance(args[0], Block):
                  ~~~~^^^
IndexError: tuple index out of range

@gasman Not that I can find.  Here's the ListBlock extensions we are doing:
class ActionTileListBlock(ListBlock):
    """
    NO LONGER USED
    """

    def __init__(self, **kwargs):
        # ListBlock wants to be passed in a block type to make a list of
        super().__init__(ActionTileBlock, **kwargs)

    class Meta:
        icon = "list-ul"

class BioListBlock(ListBlock):
    """
    List of Team Bios
    """

    def __init__(self, **kwargs):
        # ListBlock wants to be passed in a block type to make a list of
        super().__init__(BioBlock, **kwargs)

class QAListBlock(ListBlock):
    """
    Block for a list of question/answer blocks
    """

    def __init__(self, **kwargs):
        # ListBlock wants to be passed in a block type to make a list of
        super().__init__(QABlock, **kwargs)

    class Meta:
        icon = "list-ul"
        admin_text = mark_safe("QA List")
        help_text = "List of Questions and Answers (Also generates Google FAQ schema)"
        label = "QA List"
        template = "pages/blocks/qa_list.html"

class AccordionBlock(ListBlock):
    """
    Block for a list of question/answer blocks
    """

    def __init__(self, **kwargs):
        # ListBlock wants to be passed in a block type to make a list of
        super().__init__(AccordionItemBlock, **kwargs)

    class Meta:
        icon = "list-ul"
        admin_text = mark_safe("Accordion")
        help_text = "Choose what to show on page, vs hide"
        label = "Accordion"
        template = "pages/blocks/accordion_list.html"

Ah yep, I see the problem - your __init__ methods change the signature of the block away from what the base ListBlock is expecting. This wasn't a problem with the old migration logic, because (unlike StructBlock and StreamBlock) ListBlock didn't have any special handling of subclasses and just kept hold of the args/kwargs to return them unchanged when writing the migration - but the new logic expects to see the child block definition in there.
ListBlock isn't designed with subclassing in mind, but there's no fundamental reason why it shouldn't be valid, so I'll try and come up with a fix for this. For now, you should be able to work around this by adding the following methods to each class:
@classmethod
def construct_from_lookup(cls, lookup, *args, **kwargs):
    return cls(*args, **kwargs)

def deconstruct_with_lookup(self, lookup):
    return self.deconstruct()

Thanks! Migrations ran fine with those new methods added. No worries on my part if adding those is just part of the release notes upgrade process.

FYI @gasman, I tripped into this problem too while trying out 6.2rc1, as I have two subclassed ListBlocks. I substituted the new list_block.py from #12168 and the migration worked fine.


Please Ship a Flatpak

I would like to see the flatpak distribution become officially maintained by Microsoft. There are currently a few ways to get vscode on linux (package managers, snap, tarball) but flatpak is notably missing. Adding this distribution method would round out the install options.
Flatpak is rapidly gaining support as a safe (sandboxed) and cross-distro solution. It is developed by Gnome and Fedora and is the primary way to install GUI applications on Fedora Silverblue. There is currently an unofficial flatpak being distributed on flathub with a large built-in user base at https://github.com/flathub/com.visualstudio.code
There are two major issues with this distribution:

It is unofficial so there is no hard guarantee on what exactly is being shipped
It lacks an official maintainer and often lags behind the official releases by around a month
The insiders package is a completely different project and is often even more out of date and lacking in convenience additions that the stable package provides (https://github.com/flathub/com.visualstudio.code.insiders)

Microsoft taking ownership of the repo should fix all of these problems, serve a large and growing userbase, and be minimal extra packaging effort as github actions handles most of the heavy lifting. I would guess the largest effort would be bringing the insiders package up to par with the stable package.
The previous maintainer has said explicitly that they do not want to be involved in the flathub distribution anymore so there shouldn't be any friction with vscode devs taking it over.

We closed this issue because we don't plan to address it in the foreseeable future. If you disagree and feel that this issue is crucial: we are happy to listen and to reconsider.
If you wonder what we are up to, please see our roadmap and issue reporting guidelines.
Thanks for your understanding, and happy coding!

We've considered this in the past but we're still in that same position: this is out of scope for the time being. Sorry about that!

I recommend leaving this open to gather feedback from the community, flatpak is becoming an extremely important part of the Linux ecosystem and is generally better received than snap which is an officially supported distribution
I would also recommend addressing my some of the specific technical points instead of leaving a generic "it's out of scope" comment

@Queuecumber The decision isn't based on any specific technical points. This is both a product decision as well as an engineering cost decision. You can imagine that running a project like VS Code takes considerable effort and every new target we pick adds a new dimension to that effort table.

Then it sounds like the technical point is that it would be "minimal extra packaging effort as github actions handles most of the heavy lifting."?

No. This is both a product decision as well as an engineering cost decision. When the Flatpak distribution causes issues, we have to fix them. When the Flatpak CI fails, we have to fix it. When the Flatpak distribution channels are down, we have to fix them.
Every new supported target introduces non-constant engineering effort. It's not as simple as writing a CI job and forgetting about it.

Right, so from that I take it that you disagree that it would be minimal effort to maintain a flatpak distribution.
I'm not sure what a product decision is or if it's related to your technical point about the CI.
At any rate I still recommend keeping this open to see if your Linux users, or even the flatpak users, care about this distribution channel. It's certainly possible that I'm the only one who cares about it and then you can freely ignore it

I care too! With it becoming more common to use “immutable base” linux distro a (fedora silverblue, steam os) a flatpak install is a fantastic choice for me!

open this pls

Unfortunately this isn't profitable for Microsoft, so it's up to the community to fend for themselves as they always have!

Unfortunately this isn't profitable for Microsoft, so it's up to the community to fend for themselves as they always have!

The flaw in your argument is that they maintain the Snap version of it. Arguably, Flatpak is much more widely used compared to Snap yet Snap is officially supported.

No. This is both a product decision as well as an engineering cost decision. When the Flatpak distribution causes issues, we have to fix them. When the Flatpak CI fails, we have to fix it. When the Flatpak distribution channels are down, we have to fix them.
Every new supported target introduces non-constant engineering effort. It's not as simple as writing a CI job and forgetting about it.

If you do a good job on fully functional flatpak Microsoft can afford to stop working on all other builds and that in theory would save a lot of costs, part of the money that VsCode generates and for what VsCode became successful is thanks to its open source nature, Microsoft can also give facilities to the community such as not closing these tickets, or facilitate open source in a better way

I'm rather baffled that this was discarded so easily. Microsoft, please reopen this. The current method for installing code is untenable and overly complicated. A flatpak simplifies the process and makes reachability much better.

Well. A lot has happened since then. And I want to leave my comment here. I've using VS Code since IDK when. From Windows to Linux and who knows on macOS in the future. But for now I'm struggling with my OS choice because of that.
I'm using Fedora Silverblue, because I'm a developer and Toolbox seems to me a real change breaker in my environment/workflow. But (as always there's), there's a catch: to isolate all from system, Silverblue mostly use Toolbox and Flatpak to handle apps installation in favor of rpm-ostree, since this should be my last option - so I was told.
As I'm writing this now, the VS Code flatpak version has - for now - two months out of date from the original RPM file distributed on VS Code's website. Besides the lack of a guaranteed support, I'm now using outdated software that has his risk as itself (and it's blame of anyone - I would fix it if I know how).
But, since the community does not exist without community's feedback, I'm now leaving mine: it's hard to use VS Code on Flatpak and my OS of choice - that has a lot of apps based on flatpak. They don't fit together.

I'm in a similar situation to that described by the comment above. I'm using OpenSuse Aeon which is also immutable and therefore I can not install the classic vscode. Unfortunately the flatpak version of vscode is hard to use as it has some issues that the community is struggling to solve. Given the rising popularity of flatpaks and immutable OSes I think you should consider reopening this issue.

a few of the issues I experienced with the community vscode flatpak:

folder/file access
vscode extension runner doesn't work
issues with terminal integration

while some of these issues may be addressed by how the flatpak is bundled, i think some issues will require actual changes to vscode

I'm in a similar situation too, fedora silverblue + flatpak + wayland, and some important applications like vs code do not want to adapt to one of the best linux experience

I see flatpaks as an eventuality, and I imagine vscode devs will revisit this topic years down the line.

As a fedora silverblue user, vscode as a flatpak would be extremely valuable

As a fedora silverblue user, vscode as a flatpak would be extremely valuable

Not for Microsoft...

Leave this open and tag it as won't fix.

Re-open this


[zh-cn] Need to update the code-contribution.md with full link

This is a Bug Report
I have fixed the same issue in English docs, same issue is followed by the other localization team. This page has one broken issue which needs to fix.

Refer my merged PR of english docs.
kubernetes/website
/assign

/triage accepted


DNS SRV support

From thread.
I think the problem is Proxy func which uses grpc client that initialized with host:ip from static configuration file (remote.Host[0]), while host and port of services may dynamically change with service discovery(proxy.Request.URL.Host):
https://github.com/devopsfaith/krakend-grpc/blob/3c20bd6faa3bef5d1325d9d7f0f37d476d5116b4/grpc.go#L104-L127
https://github.com/devopsfaith/krakend-grpc/blob/3c20bd6faa3bef5d1325d9d7f0f37d476d5116b4/grpc.go#L43

The connection is created long before the first request and it's reused by the internal grpc client. I'd rather load the load balancer and the SD subscriber before establishing the connection
lb := sd.NewRandomLB(sd.GetSubscriber(remote))
host, err := lb.Host()
if err != nil {
	host = remote.Host[0]
}
cc, err := grpc.Dial(host, do)

I've tried with load balancer. In that case if host which returned with lb.Host() (for example, host1.example.com:11111) fails, grpc client don't automatically switch to alive host (host2.example.com:11112), because host1.example.com:11111 statically configured when creating connection.

Indeed, that's when a retry strategy should kick in and, after requesting another host from the LB, it should reconnect to the backend service

But where is/should be the reconnection logic in grpc plugin?
When first host is down krakend don't even try to connect to another host from LB hosts

just before returning the new connection here https://github.com/devopsfaith/krakend-grpc/blob/master/grpc.go#L127 I think a goroutine should be triggered, so the status of the connection can be monitored and restored after an error by calling the newConnection function


rados : resolve_stuck_peering.yaml - restructure downstream

Signed-off-by: Rachanaben Patel<EMAIL_ADDRESS>

http://pulpito.ceph.redhat.com/racpatel-2020-05-19_14:06:17-rados:downstream:singleton-nautilus-distro-basic-argo/