Code Refactoring : Allow CMIP integration

This PR does the following refactoring prior intake-cmip integration within intake-esm:

[x] Move cesm_definitions to config.py
[x] Add test_config.py
[x] Rename CesmMetadataStoreCatalog to ESMMetadataStoreCatalog and add functionality for supporting CMIP integration.

@matt-long, when you get time, can you take a look this? I made quite a lot of changes and would like this to be merged once you give it a green light.

@matt-long,
Because of cmip integration, I had to change the interface.
To build collections, you would need to run the following:
col = intake.open_esm_metadatastore(collection_input_file=collection_type_def_file)
col.build_collections()

This returns an instance of ESMMetadataStore class from intake_esm. Once the collections are built, you would need to call open_colllection() method of this class by specifying the name of the collection as defined in collection_input_file and what type of collection this is: cesm or cmip.
col = col.open_collection(collection_name="cesm_dple", collection_type="cesm")

I am open to ideas on how to make this interface better.

@andersy005, this is an impressive amount of work and a great step forward. I like the solution of putting the catalog definitions in config.
I am concerned, however, about the complexity of this interface. Here a few observations.

It seems most intuitive to me if there is a single call that enables opening, and optionally building the collection.
The collection_name argument is problematic: at present, we have this in the input file and specified by the user—they might not match. Also, what if the user builds different collections in different projects with the same name?
The CESM collection input file contains all the information defining the collection; we don't have a comparable input file for CMIP.

I think it would be helpful to discuss in person.

It seems most intuitive to me if there is a single call that enables opening, and optionally building the collection.

This sounds like an ideal approach. However, the issue arises when you have collections of different types in the collection_input_file. For instance, if you had a mixture of cmip5 and cesm collections in the collection_input_file, what would you return after all the collections are built? When this input file contains homogeneous collections, interface you are proposing would definitely be the right solution.

The collection_name argument is problematic: at present, we have this in the input file and specified by the user—they might not match. Also, what if the user builds different collections in different projects with the same name?

I added a partial solution to this that makes it easier for the user. Once the collections are built, self.collections attributes of ESMMetadataStore is updated. This a dictionary of built collections and their corresponding paths. I use the same dictionary when loading built collections:
https://github.com/NCAR/intake-esm/blob/d36e84f2410fa4f696c346a5b9d4d2e6b1a5dc2d/intake_esm/core.py#L55-L60
https://github.com/NCAR/intake-esm/blob/d36e84f2410fa4f696c346a5b9d4d2e6b1a5dc2d/intake_esm/core.py#L87-L98

The CESM collection input file contains all the information defining the collection; we don't have a comparable input file for CMIP.

We will need to put together a definition file for CMIP.

I think it would be helpful to discuss in person.

I am available this afternoon. Let me know, and I will stop by your office. If you have a lot on your place, I can stop by another time tomorrow

Regarding the issue with type:

constrain the input file specification to allow one and only one collection.

Another idea: permit the "contents" of the input file to be passed in as arguments:

name
type
data_sources

name
locations
component_attrs
case_members

The problem is we are "discovering" these things from the CMIP-AP holdings, while "specifying" them with the CESM input file.
I am book solid today.

constrain the input file specification to allow one and only one collection.

Sounds like a great constraint :). I will work on this

The problem is we are "discovering" these things from the CMIP-AP holdings, while "specifying" them with the CESM input file.

This won't be a problem since I separated the implementation for both collection types. For the time being, we can keep using the same approach we used in intake-cmip to discover files' attributes.

@andersy005, what's the status here?

@matt-long, this is still work in progress. I didn't get to work on it last week. I will address issues you pointed out some time tomorrow.

@matt-long, this is ready for another review whenever you have time

Thank you, @matt-long! I've addressed the issues you pointed out.


minimal_ns rust example

add minimal_ns example for rust

guys could you please guide me whats wrong with the failed builds?

@anakryiko  could you please guide me on this?

will reopen after figuring out the problem


Fix broken tests!

This should work, we'll see what Travis says...

@SUPERCILEX d'oh!  That's genius.  I think this will work.

😢 Now the tests work on my machine... Not sure why Travis is mad at them.

Well this is a good thing anyway, I'll merge this and keep hunting.


[Sectors] Non-abelian fusion

This is a work in progress for a full implementation of non-abelian fusion rules in Sectors. It also contains several patches on other components of Sectors. A core aspect is that the dependency order between GradedAxes and Sectors has been reversed: Sectors now depends on GradedAxes.
To do:

[x] implement fusion rules for CategoryProduct
[x] implement SymmetryStyle trait with values AbelianGroup, NonAbelianGroup and NonGroupSymmetry

To be decided:

[x] type return for c1 ⊗ c2: either always a GradedUnitRange or a GradedUnitRange or a Category depending on  SymmetryStyle
[x] whether to keep SU2 and SU{2}.
[ ] whether to implement SU(N) fusion rules or add dependency / fork SUNRepresentations.jl

Decisions:

[x]  c1 ⊗ c2: return type depends on  SymmetryStyle
[x] work on replacing SU2 with SU{2} and try to have a half-integer interface specific to SU{2}

Long term: how to handle braiding for fermions and anyons.

Regarding the question of the SU(2) types, I believe we had settled on the following plan

remove the specialized SU(2) type and just have a single type for all SU(N)
add a type parameter which sets the convention of using integer versus half-integer labels (though does this concept generalize to SU(N) well?)
specialize certain methods to the SU{2} case whenever performance is a concern. Though at first maybe don't do this at all and just check later by profiling if this is an issue.

The current version implements fusion rules for simple categories, ordered category product and dictionary category product. The dictionary category product is still a a work in progress as it is not type stable yet.
I am puzzled by @inferred behavior, it indicates a type instability in quantum_dimension that only depends on compiling order. I wonder if it is an issue with @inferred, as @code_warning finds no issue. I am interested on feedback on this (see test_category_product.jl line 144 and below, @testset "Quantum dimension and GradedUnitRange" begin).

The current version implements fusion rules for simple categories, ordered category product and dictionary category product. The dictionary category product is still a a work in progress as it is not type stable yet.
I am puzzled by @inferred behavior, it indicates a type instability in quantum_dimension that only depends on compiling order. I wonder if it is an issue with @inferred, as @code_warning finds no issue. I am interested on feedback on this (see test_category_product.jl line 144 and below, @testset "Quantum dimension and GradedUnitRange" begin).

I think I have seen some funny things with @inferred but I don't think I have seen that. Maybe you could try @constinferred from TestExtras.jl, though I'm not sure if that is relevant for what you are seeing.

All stability issues are now solved. I still need to check the behavior of dual.

I made the requested changes: all explicit type comparison are replaced by dispatch. I propose conventions for tensor_product, fusion_product and dual. I leave the unification of SU2 and SU{2} for future work.

I had a look at the failures in CI. They all come from julia 1.6, all tests pass with julia 1.10. Failures boil down to two cases: first a return type is not correctly inferred in one case, this is not a big deal. Second getindex(::NamedTuple, ::Tuple{Symbol})is not supported. I think writing a dictionary-like implementation of CategoryProduct without using this function would be a lot of work. As order product implementation is already fine for julia 1.6, maybe the best would be to require julia >= 1.7 to use the dictionary-like implementation and keep the current code?

I had a look at the failures in CI. They all come from julia 1.6, all tests pass with julia 1.10. Failures boil down to two cases: first a return type is not correctly inferred in one case, this is not a big deal. Second getindex(::NamedTuple, ::Tuple{Symbol})is not supported. I think writing a dictionary-like implementation of CategoryProduct without using this function would be a lot of work. As order product implementation is already fine for julia 1.6, maybe the best would be to require julia >= 1.7 to use the dictionary-like implementation and keep the current code?

It's more verbose, but we could use getindices from Indexing.jl:
julia> using Indexing

julia> getindices((; a=2, b=3, c=4), (; a=:a, b=:b))
(a = 2, b = 3)

which works across Julia versions. We will likely drop support for Julia 1.6 at some point, but I would rather not do that here.

We could wrap that functionality up into a nicer interface, maybe:
subkeys(collection::NamedTuple, indices::Tuple) = getindices(collection, (; (indices .=> indices)...)

to be used like:
julia> subkeys((; a=2, b=3, c=4), (:a, :b))
(a = 2, b = 3)

which could generalize to other collections like dictionaries.

Thank you for the clues, I ended up using another way. I also removed a few @inferred from the tests and now they all pass with julia 1.6. I understand jenkins failure to be unrelated to this PR.

I'll try to take another look through it to see if I have any more comments. @emstoudenmire could you do another pass over it as well?

One minor thing I just noticed when pulling this branch is that there's a Project.toml file in NDTensors/src/lib/Sectors/ but which is empty (i.e. the file has no lines). I don't believe the Project.toml is needed, so let's please delete it.

Thank you @mtfishman for the review. As you noticed, there are many parts involving UnitRangeDual that are not satisfying. I There is no good solution with the current design UnitRangeDual <: AbstractUnitRange, the solution is to have a GradedUnitRangeDual <: AbstractGradedUnitRange. I plan to do this change in a separate PR (see #1529), once this PR is merged. Here my focus is on Sectors and the fusion rules.
Concerning the style, I find it much easier to read and write code when namespaces are explicit. At first I really struggled to enter ITensors code basis as I had no idea which functions were called and whose API they belonged. It also avoids to systematically rewrite imports when a function call is added or deleted. Finally, I find it more convenient in my workflow when I swap between the source code and the REPL. Therefore I would rather keep imports at the beginning of the module and not touching them.
using BlockArrays

using NDTensors.LabelledNumbers
using NDTensors.GradedAxes

Thank you @mtfishman for the review. As you noticed, there are many parts involving UnitRangeDual that are not satisfying. There is no good solution with the current design UnitRangeDual <: AbstractUnitRange, the solution is to have a GradedUnitRangeDual <: AbstractGradedUnitRange. I plan to do this change in a separate PR (see #1529), once this PR is merged. Here my focus is on Sectors and the fusion rules.

Sounds reasonable to address some of these issues in #1529, however many of the suggestions I've made seem to be addressable without introducing GradedUnitRangeDual <: AbstractGradedUnitRange so I'm a bit confused by that comment.

Concerning the style, I find it much easier to read and write code when namespaces are explicit. At first I really struggled to enter ITensors code basis as I had no idea which functions were called and whose API they belonged. It also avoids to systematically rewrite imports when a function call is added or deleted. Finally, I find it more convenient in my workflow when I swap between the source code and the REPL. Therefore I would rather keep imports at the beginning of the module and not touching them.
using BlockArrays

using NDTensors.LabelledNumbers
using NDTensors.GradedAxes

I understand that is your preference, and that was the style we used for a while in ITensors.jl since we started developing Julia code, however that style leads to maintenance issues since it is harder to tell which module/package a function or type is defined in originally when looking at a file in the source code, and additionally if two packages decide to export the same name, using implicit imports (the style you are advocating for) can lead to naming conflicts. See https://github.com/ericphanson/ExplicitImports.jl and the README of that package for a discussion of this issue. Please change to using explicit imports listed at the top of the files like we do everywhere else in the library.
Sometimes when you are developing a package built by someone else, you need to make compromises in your coding style. I'm happy to discuss choices like this with you (and I'm happy to admit I am wrong when I hear a convincing argument) but demanding that you do things your own way that conflict with ways we have decided to do them is not a friendly and productive way to develop code as part of a group. Perhaps in the future, if you don't like the way we do things, instead of demanding that you do it your own way and saying you don't like our way, you can be more humble and start by asking why we do things the way we do them.

One minor thing I just noticed when pulling this branch is that there's a Project.toml file in NDTensors/src/lib/Sectors/ but which is empty (i.e. the file has no lines). I don't believe the Project.toml is needed, so let's please delete it.

@emstoudenmire @ogauthe I have been defining Project.toml files for submodule libraries of NDTensors to prepare them to be moved out of the library into separate packages, so ideally we would list whatever packages are loaded in the Sectors module in NDTensors/src/lib/Sectors/Project.toml.

I was not doing implicit import but what your documentation calls qualified access. I realize that indeed these lines were not the correct ones as I was intending to do
using BlockArrays: BlockArrays

using NDTensors.LabelledNumbers: LabelledNumbers
using NDTensors.GradedAxes: GradedAxes

I agree with you that implicit import is problematic, I just got confused when I wrote these 3 lines and use qualified access everywhere. Since you seem not to like it I changed to explicit imports.

@ogauthe thanks for the update to categories_fusion_rule, that's definitely already easier to understand. I'll take a look over it again and leave comments.

In order to help keep track of things (since I am having trouble keeping track and remembering what needs to be addressed), this is a list of the bigger issues to address in this PR:

[ ] Simplify tensor_product(a1::AbstractBlockedUnitRange, a2::AbstractBlockedUnitRange), ideally by removing or redesigning flatten_maybe_nested, which may require redesigning fuse_blocklengths to have more consistent outputs for abelian and non-abelian spaces.
[ ] Redesign label_fusion_rule to always output the same data format, i.e. a list of degeneracies and sector labels, for both abelian and non-abelian symmetries. We can define a simple_label_fusion_rule or abelian_label_fusion_rule for abelian symmetries that have simpler rules (where fusion outputs only one sector of length 1), and label_fusion_rule can call out to simple_label_fusion_rule in those cases.
[ ] Delete TrivialSector and definitions of fusion_rule that are hardcoded to handle fusion of empty category products at compile time. Ideally, those corner cases can be handled by the generic fusion code.
[ ] Continue to refactor and simplify CategoryProduct fusion, which is implemented in categories_fusion_rule.

These issues, along with some smaller issues, are being discussed in comments on the code in this PR as well.
@ogauthe @emstoudenmire

Thank you @mtfishman for the review and the summary.
Concerning TrivialSector, I understand you want to delete it. From my perspective, I have a use case for it in FusionTensors. There, I need a placeholder for the symmetry sector obtained by fusing zero axis, and it should compare equal to any singlet. As a CategoryProduct with zero category inside automatically exists as a byproduct of the current implementation, it seems natural to use it for that purpose. I also think a dedicated way to construct it makes the code easier to read. Now, this placeholder could be something else, for instance a TrivialSector that would not be a CategoryProduct, would you like it better?
The reason for the hardcoded version of fusion_product with TrivialSector is that Trivial ⊗ U1(1) should return U1(1). If I first cast the inputs of fusion_rule to CategoryProduct, then the output would be CategoryProduct{Tuple{U1(1)}}, which is not the same. Then there has to be several version to lift ambiguities.

Concerning TrivialSector, I understand you want to delete it. From my perspective, I have a use case for a TrivialSector in FusionTensors. There, I need a placeholder for the symmetry sector obtained by fusing zero axis, and it should compare equal to any singlet. I also think a dedicated way to construct it makes the code easier to read. As a CategoryProduct with zero category inside automatically exists as a byproduct of the current implementation, it seems natural to use it for that purpose. Now, this placeholder could be something else, for instance a TrivialSector that would not be a CategoryProduct, would you like it better?

That's an interesting example, it's helpful to have an explicit use case where you want to use TrivialSector outside of this PR. As you can tell, I may have a bit of a myopic view in this PR review, where my primary focus is on making the implementation of this PR as simple as possible while keeping all of the functionality you set out to have.
I don't think it is obvious which direction to go in. Part of my motivation for asking about deleting the TrivialSector type alias is to try to decrease the number of types in the code right now, especially ones that aren't really needed since they can be handled by existing types. Some may be necessary in the end if there is a compelling reason for them motivated by real world examples, but sometimes it is good to try to tear things down to the bare minimum to then see what is really necessary, and we can always add things back as we find use cases for them (but it is harder to take things out than put them back, once they are in the code and being used, so keeping things simpler at first is generally better).
For those kinds of use cases, I was picturing that you could just use CategoryProduct() directly. I understand that may look a bit more mysterious, i.e. not be as explicit as using a devoted TrivialSector(), so again I don't think the choice is obvious.

The reason for the hardcoded version of fusion_product with TrivialSector is that Trivial ⊗ U1(1) should return U1(1). If I first cast the inputs of fusion_rule to CategoryProduct, then the output would be CategoryProduct{Tuple{U1(1)}}, which is not the same. Then there has to be several version to lift ambiguities. A dedicated TrivialSector would allow for more generic handling of empty CategoryProduct but would need its own dedicated fusion rules.

I see, that's a helpful perspective that I didn't consider. I think the part that I don't like about CategoryProduct(()) for that purpose is that choice is kind of arbitrary, i.e. why not CategoryProduct((;)) or some other type? You originally circumvented that issue by not allowing users to construct CategoryProduct((;)) but that feels like a band-aid solution to me, and also a bit arbitrary. Another thing that doesn't sit right with me about the way CategoryProduct(()) is being treated as special in the code is that tuples happen to have static/compile time lengths, but as I've mentioned we will want to support storage types inside CategoryProduct that don't have compile time lengths, so why should CategoryProduct(()) act differently from CategoryProduct([])?
So I think your two examples you've mentioned are convincing me that we should have a devoted type struct TrivialSector <: AbstractCategory end type, and not treat CategoryProduct(()) as special in any way, so that fusion of CategoryProduct(()) with AbstractCategory returns a CategoryProduct (i.e. it promotes non-products to products). TrivialSector can be the type that has special rules and preserves whatever it is being fused with.
It also makes me wonder about the name AbstractCategory, since all of the subtypes are in fact sectors. I think that is misleading and will start to get confusing. I guess "sector" is a terminology from group theory, is there a generalization of the term sector in category theory? Would it make sense to rename AbstractCategory to AbstractSector? @emstoudenmire curious about your thoughts on that point as well.

In a follow-up to my comment about maybe renaming AbstractCategory to AbstractSector, we should also consider renaming CategoryProduct to SectorProduct or something similar, since it is really a product of sectors in practice. Again, curious to hear both of your thoughts on that, @ogauthe and @emstoudenmire.

In TensorKit.jl, they use the name Sector for the abstract supertype that is analogous to what we now call AbstractCategory and what I'm proposing to change to AbstractSector. Additionally, they use ProductSector for the analog of what we now call CategoryProduct and what I'm proposing to change to ProductSector.
Finally, they define AbstractIrrep, which in the case of abelian groups becomes AbelianIrrep, for sectors of a group. I don't know if that is needed in our case, I think our traits like AbelianStyle and NotAbelianStyle should serve a similar purpose to distinguish what kind of sector it is, which also handles composite sectors/irreps like CategoryProduct/SectorProduct, which is naturally handled by our current design based on traits that are outside of the abstract type hierarchy of AbstractCategory/AbstractSector.
Anyway, interesting to see we might be converging towards similar designs.

@ogauthe I think you forgot to upload trival.jl.

Never mind, I see it now.

About the trivial type, I think here it's all about the code and what makes the code cleanest. With the QN system we didn't have this problem since there was only one kind of internal storage, analogous to the NamedTuple storage here. Since here we can have different internal, yet empty, states of a SectorProduct, that does create an unfortunate problem. I'm not offering a solution, but just to provide that background if it's helpful.

Yes, that's a good point about how it wasn't as compelling to have a special trivial sector type before since we had only one storage type (QN with a small set of possible sectors), so now that we have multiple storage types and also the group irreps are themselves types it became clear that it is helpful to have a devoted trivial sector type. Also it has made the code simpler now that we've introduced it.

So I refactored product_sectors_fusion_rule. I kept recover_sector_product_type as this is a function I need in FusionTensors. I could replace factorize_gradedaxis(fused) by a systematic call to reduce(×, fused, init=SectorProduct()), not sure it makes things more clear.

So I refactored product_sectors_fusion_rule. I kept recover_sector_product_type as this is a function I need in FusionTensors. I could replace factorize_gradedaxis(fused) by a systematic call to reduce(×, fused, init=SectorProduct()), not sure it makes things more clear.

Great, I'll take a look at the new version. I didn't look carefully at the new version yet, but if factorize_gradedaxis(fused) is just defined as reduce(×, fused), let's get rid of factorize_gradedaxis and just use reduce(×, fused) directly where it is being used now.

I kept recover_sector_product_type as this is a function I need in FusionTensors.

If that is the case, my recommendation would be to remove recover_sector_product_type from this PR and make the code in this PR as simple as possible, just with the goal of making SectorProduct fusion work with the simplest code possible. If that doesn't cover what you need for FusionTensors then define what you need for that in that branch. I'm still finding the logic of fix_fused_product_type, recover_sector_product_type, recover_gradedaxis_product_type, etc. hard to follow (there are many similar functions calling each other, many conversions happening, etc.), so please try to simplify that down as much as you can.

@ogauthe here is a proposal for a simpler implementation of SectorProduct fusion:
function fusion_rule(::NotAbelianStyle, s1::SectorProduct, s2::SectorProduct)
  return to_gradedrange(
    product_sectors_fusion_rule(product_sectors(s1), product_sectors(s2))
  )
end

function fusion_rule(::AbelianStyle, s1::SectorProduct, s2::SectorProduct)
  return label(only(fusion_rule(NotAbelianStyle(), s1, s2)))
end

function product_sectors_fusion_rule(sects1, sects2)
  shared_sect = shared_product_sectors_fusion_rule(
    product_sectors_common(sects1, sects2)...
  )
  diff_sect = SectorProduct(product_sectors_diff(sects1, sects2))
  return shared_sect × diff_sect
end

function shared_product_sectors_fusion_rule(shared1::T, shared2::T) where {T<:Tuple}
  fused = map(to_gradedrange ∘ fusion_rule, shared1, shared2)
  return reduce(×, fused)
end

function map_blocklabels(f, r::AbstractUnitRange)
  return gradedrange(labelled.(unlabel.(blocklengths(r)), f.(blocklabels(r))))
end

function shared_product_sectors_fusion_rule(shared1::NT, shared2::NT) where {NT<:NamedTuple}
  tuple_fused = shared_product_sectors_fusion_rule(values(shared1), values(shared2))
  return map_blocklabels(SectorProduct ∘ NT ∘ product_sectors, tuple_fused)
end

I didn't test it thoroughly, just on a few examples of abelian and non-abelian fusion. Hopefully even if it isn't totally correct it gives you an idea of what I have in mind.

I'll do a final look over the PR, but after that I think it is ready to merge. Thanks for sticking with all the comments @ogauthe.
@emstoudenmire can you look over it as well to see if you notice anything that needs to be changed?

I had a last look, I removed adjoint(::SU{N}) as it was conflicting with Base.adjoint and added more tests. From my side this is ready to merge.

I had a last look, I removed adjoint(::SU{N}) as it was conflicting with Base.adjoint and added more tests. From my side this is ready to merge.

It would be fine to define adjoint in SymmetrySectors without overloading Base.adjoint, and then call it like SymmetrySectors.adjoint(SU{2}) so it doesn't conflict with Base.adjoint. Up to you though if you want to keep it, I don't know what it is used for.

Besides the last two very small comments I left above, this is good to go from my end. I'll merge and register as soon as we get the go-ahead from @emstoudenmire.

Alright, this looks good to go from my end and addresses the latest round of comments. @emstoudenmire, any final comments before I merge?

Good to go from me. Looks very clean and I think it will be a powerful system!

Thanks for sticking with the long review process @ogauthe!


Issue #212 fixes - examples fail to compile

MCO example enabled both HSI and HSE, fixed by disabling HSE in funconf.h and enabling it in the example code.
SPI library still requires APB_CLOCK to be defined, which was removed in #208

Thanks!


Minimum doctrine/orm version

Currently we're requiring >=2.1. Is there any compelling reason to change this? I ask because I'm finding issues with the existing tests on at least 2.1 and 2.2. Guess I never went back and tested with previous versions. So far it's only required changes in the test classes, but I haven't resolved all the problems yet.

I think you can safely upgrade requirements to 2.3 or 2.4 as you need. 2.1 and 2.2 have not been updated in the last 3 years ....

Date
Version

2012-07-29
2.1.x

2012-11-21
2.2.x

2012-09-20
2.3.0

2014-06-03
2.3.x

2013-09-07
2.4.0

2015-08-31
2.4.x

Reading through UPGRADE.md I'd vote for 2.4 based on this alone:
https://github.com/doctrine/doctrine2/blob/master/UPGRADE.md#parenthesis-are-now-considered-in-arithmetic-expression

Sounds good !

Doctrine version updated in #86


Add in missing docker login to long running container

Description of changes:
fixed bug where pulling kind/node during bootstrap-cluster-init phase failed with "unauthorized to access repository: eks-anywhere/l0g8r8j6/kubernetes-sigs/kind/node"
Testing (if applicable):
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

/cherrypick release-0.13


Add check for ngram_len for calculating distinct ngrams

@madaan OK I guess we can define it as 0 for the edge case.


Is there a function that can visualize the BEV Box in XYWHR format?

Hello.
Thank you so much for sharing a great project.
I want to visualize the bounding boxes obtained in LiDAR format with BEV.
I succeeded in visualizing the bounding boxes with no rotation as below using nearest_bev,
but I'm not sure how to visualize a bev with yaw.

Is there any function in this project that can visualize the rotated box using the parameter R?
If it doesn't exist, can you give me the idea of using this yaw to rotate the coordinates of the box?
Thank you very much.

We support 3D-view visualization. Please refer to our documentation for visualization tools. If you would like to get a relative "standard" bird-view visualized result, I recommend you can refer to this function in nuScenes devkit for some inspiration.

Thank you very much for the quick reply.
In your answer, the 'documentation' link and the 'nuScenes devkit' link appear to be the same. Could you give me a link again if this is wrong?

Sorry for my carelessness. Links updated.

I really appreciate your kind help. Have a nice day!


Class 'int' has no instance getter

Is there an existing issue for this?

[X] I have searched the existing issues
[X] I have read the guide to filing a bug

Steps to reproduce
when try to debug any code in VS code , I got error :
Unhandled exception:
NoSuchMethodError: Class 'int' has no instance getter 'controller'.
Receiver:<PHONE_NUMBER>
Tried calling: controller
#0      Object.noSuchMethod (dart:core-patch/object_patch.dart:38:5)
#1      Eval ()

but I'm sure this code is working without debug
Expected results
show debug results
Actual results
error
Unhandled exception:
NoSuchMethodError: Class 'int' has no instance getter 'controller'.
Receiver:<PHONE_NUMBER>
Tried calling: controller
#0      Object.noSuchMethod (dart:core-patch/object_patch.dart:38:5)
#1      Eval ()

Code sample
Code sample
CachedNetworkImage(
            imageUrl:
                controller.worksList[controller.worksIndex.value].photo.path ??
                    "",
            imageBuilder: (context, imageProvider) => Container(
              decoration: BoxDecoration(
                borderRadius: BorderRadius.circular(10),
                image: DecorationImage(
                  image: imageProvider,
                  fit: BoxFit.cover,
                ),
              ),
            ),
            placeholder: (context, url) => Image.asset(unknownPerson),
            errorWidget: (context, url, error) => Image.asset(defaultUser),
          ),

Screenshots or Video

Screenshots / Video demonstration

Logs
Logs
[Paste your logs here]

Flutter Doctor output
Doctor output
Doctor summary (to see all details, run flutter doctor -v):
[✓] Flutter (Channel stable, 3.13.9, on macOS 14.1.1 23B81 darwin-x64, locale en-EG)
[✓] Android toolchain - develop for Android devices (Android SDK version 34.0.0)
[✓] Xcode - develop for iOS and macOS (Xcode 15.0.1)
[✓] Chrome - develop for the web
[✓] Android Studio (version 2022.3)
[✓] VS Code (version 1.84.2)
[✓] Connected device (3 available)
[✓] Network resources

• No issues found!

Hi @heshesh2010, please provide a complete minimal reproducible so that we can investigate this. From your screenshot it seems like you are using get. It has some extensions which may not be working as intended. So in order for this to be properly investigated, please provide a complete minimal reproducible code sample that does not use getx so that we can investigate this issue.
Thank you

@heshesh2010 since you didn't give proper stacktrace and code, are you trying to execute the getter controller as a function () ?
Instead of This  int len = recievedData.controller(); use  int len = recievedData.controller;
(where controlller might be a getter)
If you are trying to access a getter.

no I have passed the controller instance to my widget and in parent widget I init the controller by extends
GetWidget

Hi @heshesh2010, please provide a complete minimal reproducible so that we can investigate this. From your screenshot it seems like you are using get. It has some extensions which may not be working as intended. So in order for this to be properly investigated, please provide a complete minimal reproducible code sample that does not use getx so that we can investigate this issue.
Thank you

Yes I'm using Getx , here is my code sample
`class ShowWorksItemsScreen extends GetWidget {
const ShowWorksItemsScreen({super.key});
@override
Widget build(BuildContext context) {
return NetworkAwareScaffold(
child: Scaffold(
resizeToAvoidBottomInset: true,
appBar: CustomAppBarWidget(
'our_works'.tr,
isBackEnabled: true,
backButtonCallback: () {
Get.delete();
Get.offNamed(Routes.worksScreen);
},
),
body: SafeArea(
child: GestureDetector(
onHorizontalDragEnd: (dragDetail) {
// Swiping in right direction.
if (dragDetail.velocity.pixelsPerSecond.dx < 1) {
if (controller.worksIndex.value == 0) {
// controller.worksIndex.value =
//     controller.worksList.length - 1;
return;
} else {
controller.worksCarouselController.previousPage();
                controller.worksIndex.value =
                    controller.worksIndex.value - 1;
              }
            }

            // Swiping in left direction.
            else {
              if (controller.worksIndex.value ==
                  controller.worksList.length - 1) {
                //controller.worksIndex.value = 0;
                return;
              } else {
                controller.worksCarouselController.nextPage();
                controller.worksIndex.value =
                    controller.worksIndex.value + 1;
              }
            }
          },
          child: Padding(
            padding: EdgeInsets.symmetric(horizontal: 16.0.w),
            child: Center(
              child: Obx(
                () => controller.worksCommentControllerList.isEmpty
                    ? Center(
                        child: Text(
                          'no_works_found'.tr,
                          style: Get.textTheme.bodyLarge!,
                        ),
                      )
                    : SingleChildScrollView(
                        child: Column(
                          crossAxisAlignment: CrossAxisAlignment.center,
                          mainAxisAlignment: MainAxisAlignment.start,
                          children: [
                            // 25.verticalSpace,
                            // grid view
                            CarouselSliderWidget(controller: controller),
                            24.verticalSpace,
                            // show selected works photo widget
                            SelectedWorksPhotoWidget(
                                controller: controller),

                          ],
                        ),
                      ),
              ),
            ),
          ),
        ),
      )),
);

}
}`
`
class SelectedWorksPhotoWidget extends StatelessWidget {
const SelectedWorksPhotoWidget({
super.key,
required this.controller,
});
final WorksScreenController controller;
@override
Widget build(BuildContext context) {
return Obx(() => controller.isWorksPicUploadingList.isEmpty
? Center(
child: Text(
'no_works_found'.tr,
style: Get.textTheme.bodyLarge!,
),
)
: controller.isWorksPicUploadingList[controller.worksIndex.value].value
? Helper().shimmerSmallImageWidget()
: Stack(
children: [
SizedBox(
height: 326.h,
//    width: 326.w,
child: GetUtils.isURL(controller
.worksList[controller.worksIndex.value].photo.path!)
? CachedNetworkImage(
imageUrl: controller
.worksList[controller.worksIndex.value]
.photo
.path ??
"",
imageBuilder: (context, imageProvider) => Container(
decoration: BoxDecoration(
borderRadius: BorderRadius.circular(10),
image: DecorationImage(
image: imageProvider,
fit: BoxFit.cover,
),
),
),
placeholder: (context, url) =>
Image.asset(unknownPerson),
errorWidget: (context, url, error) =>
Image.asset(defaultUser),
)
: ClipRRect(
borderRadius: BorderRadius.circular(10.0),
child: Image.file(
File(controller
.worksList[controller.worksIndex.value]
.photo
.path!),
fit: BoxFit.cover,
)),
),
// edit works photo button
Positioned(
left: 10,
top: 10,
child: InkWell(
onTap: () => controller.updateWorksPic(),
child: Container(
height: 40.h,
width: 40.w,
decoration: BoxDecoration(
color: Colors.white,
borderRadius: BorderRadius.circular(8),
),
child:
Icon(Icons.edit, color: AppColors().greenColor(1)),
),
),
),
],
));
}
}
`

Hi @heshesh2010, please provide a minimal reproducible sample that does not use getx or any third-party packages so that we can investigate the issue. If it is an issue with the framework, it should be reproducible without the third-party packages.

I think the use of "super" keyword in your case is incorrect. You should not use a superclass constructer with the "key" parameter.  A classic syntax error in my opinion
Use this instead:
const SelectedWorksPhotoWidget({   Key? key,   required this.controller, }) : super(key: key);
This should solve your issue.


Error deleting a dicom study or retrieving a study using REST API

Describe the bug
The application does not respect the HTTP 'Accept' headers in the request. When we provide -H 'accept:multipart/related; type=application/dicom'  The application only receives. application/dicom and not the multipart/related part of the header which causes some checks later in the request to fail (the checks enforce both application/dicom as the only supported media type and also expect the multipart/related section.

The headers are conformat to the spec as specified here
After debugging, it seems there is a bug with ASP.NET Typed headers as used in \dicom-server\src\Microsoft.Health.Dicom.Api\Extensions\HttpRequestExtensions.cs  The call to the method httpRequest.GetTypedHeaders().Accept returns a list of MediaTypeHeaderValue which seems strange, As per the class doc on Microsoft, the class is used for content type and not accept headers. The call to httpRequest.GetTypedHeaders().Accept only returns application/dicom and not the multipart/related section. However, a call to  httpRequest.Headers.GetCommaSeparatedValues(HeaderNames.Accept) returns the correct headers.
Ideally there should be a bug request for ASP.NET to fix how accept headers are handled. or in the meantime, the dicom code can change from the MediaTypeHeaderValue for accept header to a custom implemnted header handler.
I tried forcing the MediaTypeHeaderValue to accept the correct headler by calling new MediaTypeHeaderValue(new StringSegment(x, 0, x.Length)) where x = "multipart/related; type=application/dicom", but that threw a formatting exception confirming it is not the correct abstraction.

To Reproduce
Steps to reproduce the behavior:

Enable the openAPI Spec (optional).
Follow the instructions in the README to upload a dicom file using fiddler or postman
Try to delete or get that study using the swagger button which correpsonds to
curl -X 'GET' 'https://localhost:63838/v1/studies/1.2.8<IP_ADDRESS>80043.8.498.13230779778012324449356534479549187420' -H 'accept:multipart/related; type=application/dicom'  -H 'content-type: application/dicom' substitue the study instace id with the apporpiate study instance if of the file you uploaded in step 2.
You will get an exception:
"Headers are not supported"

Expected behavior
Successful Response with the study file or an empoty response when deleting the study
Actual behavior
Headers not supported exxeption

Hi @mhassaninmsft, can you clarify what tool you're using to send the GET request? If you enabled the Swagger UI, that's not currently supported due to accessibility issues.
I did just check your scenario against the Postman collection, and I was able to retrieve a study as a multipart response successfully. Looking at your request, there are two things that stand out:

When specifying the content type in either the Accept or Content-Type header, you need to include surrounding quotes, like Accept:multipart/related; type="application/dicom". According to the spec, the media type is surrounded by quotes when a multipart response is requested.
You don't include a Content-Type header in a WADO-RS request since there's no body.

Please let me know if this works for you!

And the spec does say that the media type param value can be quoted or unquoted - that's a bug on us.

Correction of above - our current handling of quotes is correct according to the Dicom standard:

according to 8.7, in general, media type parameter values can be quoted or unquoted (i.e. text/html; q=0.3 and text/html; q="0.3" are both valid)
according to 8.7.1, in the special case where the media type is multipart/related there must be a type parameter present, and it must be quoted (i.e. multipart/related; type=application/dicom is not valid)

Closing this issue - thanks for raising it!


New profile pages for institutions / institutional sellers

This is an issue for describing updates to profile pages for institutions, institutional sellers with subscriptions, and institutional sellers without subscriptions.
After checking in with Chung-yi he suggested to handle this transition with the same "2.0" flag on the partner that we've used for subscribing galleries such that each of the features in this issue would be applied for the relevant partner when the institution or institutional seller has the "2.0" flag checked.
Note: Across the board these borrow most heavily from the current styling of the Preferred 2.0 profile pages
Institutions

[ ] all partners with partnerType = Institution should be upgraded to have the new profile page overview style]

[ ] This overview style should mirror the style of preferred 2.0 galleries like PACE in which they have:

a carousel with up to two current or upcoming shows with the option to instead have a carousel of recently updated artworks (this preference will be set in Volt)
About text and news section (that appears if upcoming events have been entered)
a shows carousel (if published shows >0)
a fair booths carousel (if published fair booths >0)
an articles carousel
Note: there will be no Artists section since Institutional profile pages have no artist tab

[ ] The remaining tabs should be the same as the existing Institutional profile page with a "Works", "Articles", and "About" tabs

Institutional Sellers

[ ] all partners with partnerType = Institutional Seller
[ ] This overview style should mirror the style of preferred 2.0 galleries like PACE in which they have:

a carousel with up to two current or upcoming shows with the option to instead have a carousel of recently updated artworks (this preference will be set in Volt)
About text and news section (that appears if upcoming events have been entered)
a shows carousel (if published shows >0)
a fair booths carousel (if published fair booths >0)
an artists list (if they've chosen the option to have an artist tab in Volt)
an articles carousel

[ ] Institutional Sellers currently do not have an Artists tab on the profile, but should have the option for having one controlled through preferences in Volt. This artists tab should display/function just as gallery artist tabs. If selected to display this tab should appear after the "shows" and "works" tabs, but before the "Articles", "Shop", and "About" tabs
[ ] The remaining tabs should be the same as the existing Institutional profile page with a "Works", "Articles", "Shop", and "About" tabs

Amendments to existing functionality of existing tabs:

[ ] The shop tab should only display if the count of published for sale artworks by the partner is >0
[ ] The works tab currently includes both for sale works and not for sale works. The works tab should only include not for sale works since the shop tab includes all for sale works.

 New profile pages for institutions / institutional sellers


fix(deps): bump

Applied mkpr on undefined
/tmp/mkpr.sh

:tada: This PR is included in version 1.29.20 :tada:
The release is available on:

v1.29.20
npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


WindowCovering: YAML Add GoTos tests 4.1 4.2

Problem
Activate the previously incomplete YAML test WNCV 4.1 4.2
Change overview
We activate on Chip-tool linux and darwin those test with their default waiting duration time
A real device DUT will use a parameter adapted for it opening/closing speed
Testing
Tests are functional and updated

Tested on EFR32 demo app
Passed the CI test suite

/rebase


[SPARK-21224][R] Specify a schema by using a DDL-formatted string when reading in R

What changes were proposed in this pull request?
This PR proposes to support a DDL-formetted string as schema as below:
mockLines <- c("{\"name\":\"Michael\"}",
               "{\"name\":\"Andy\", \"age\":30}",
               "{\"name\":\"Justin\", \"age\":19}")
jsonPath <- tempfile(pattern = "sparkr-test", fileext = ".tmp")
writeLines(mockLines, jsonPath)
df <- read.df(jsonPath, "json", "name STRING, age DOUBLE")
collect(df)

How was this patch tested?
Tests added in test_streaming.R and test_sparkSQL.R and manual tests.

cc @felixcheung and @cloud-fan, could you maybe have some time to take a look for this?

Test build #78685 has started for PR 18431 at commit 3b21231.

LGTM

Test build #78685 has finished for PR 18431 at commit 3b21231.

This patch fails due to an unknown error code, -10.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78685/
Test FAILed.

retest this please

Test build #78694 has started for PR 18431 at commit 3b21231.

Test build #78694 has finished for PR 18431 at commit 3b21231.

This patch fails PySpark pip packaging tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78694/
Test FAILed.

retest this please

Test build #78700 has started for PR 18431 at commit 3b21231.

Test build #78700 has finished for PR 18431 at commit 3b21231.

This patch fails PySpark pip packaging tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78700/
Test FAILed.

retest this please

Test build #78711 has started for PR 18431 at commit 3b21231.

@srowen I think python is simply broken and it won't work even if retest.

@felixcheung, hm.. I saw few tests were passed occasionally so I thought some might work. Let me probably wait until it is fixed.

will you be adding schema ddl string in other cases? It'll be good to be consistent.
for example, from_json, gapply, dapply, createTable, createDataFrame/as.DataFrame - I think that's it.

Yea, but for now we don't support this for few cases, for example,  createDataFrame for now in Scala side. I think I can do this for gapply and dapply (which I believe R-speicific) but I am a bit worried of as.json.array which we added R specifically but let me try to fix it here at my best.
For createTable, createDataFrame and as.DataFrame, would you mind if I do this latter when Scala-side has the equivalent APIs?

Test build #78711 has finished for PR 18431 at commit 3b21231.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78711/
Test PASSed.

fair enough - IMO, good idea to add to scala api but I'd defer to @cloud-fan

Test build #78757 has started for PR 18431 at commit 7c47934.

Test build #78757 has finished for PR 18431 at commit 7c47934.

This patch fails due to an unknown error code, -9.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78757/
Test FAILed.

retest this please

Test build #78772 has started for PR 18431 at commit 7c47934.

Test build #78772 has finished for PR 18431 at commit 7c47934.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78772/
Test PASSed.

Merged build finished. Test PASSed.

merged to master.

Thank you @felixcheung.


How to do concurrent Jobs in different machines? [question]

Is there a way to to orchestrate concurrent jobs on multiple boxes ?

Not at the moment. See Strider-CD/core for ongoing development of a future build system.

Not at the moment meaning it is the plane ?  :)
How do you treat an instance of tests running. Do you have some kind of concept of an agent?

Yes we call them Drones: https://github.com/Strider-CD/drone


How to pass the confirm/cancel button result to the component?

Right now, if the users click confirm/cancel button on the modal, it just  close the modal. How can it pass any result back to the component? so the component can perform any actions based upon confirm/cancel.
Please help.!

The demo app has an example of how to do this. From line 255 in demo-app.component.ts:
this.modal.show().then( (res:string) => this.result = res);
The modal returns a promise. The promise resolves to a string that gives the name of the button clicked.
To see it in action, look at the demo and check the "show result" option.


All metrics reporting methods should allow for uniform sampling settings

These settings should be uniformly configurable for all metrics reporting channels, so that when users compare data between them, they know that the data is being captured and reduced in the same way.

HDR digits
sampling interval
reservoir type

This is to enable empirical cross-checking between the various metrics output formats. This should not be considered complete unless/until unit tests are added which provides numerical verification for all metrics channels.

Superseded by OpenMetrics integration


DOD redacted logic too trusting

Some DOD reports that say they are redacted actually have links to the reports.
See DODIG-2014-123 on http://www.dodig.mil/pubs/index.cfm?fy=2014. I believe these are reports that were originally redacted, but later released.

Does the scraper skip over these entirely? I can't see the logic that would, though it's a tricky scraper.
I suspect this report was originally unreleased, and then when it was released, the link was added and the word "(Redacted)" added.
If that's the case, and the report's title/ID was released in 2014 but its redacted text released in 2015, we would not catch it in the usual course of nightly events, since it only looks at the latest year. Maybe that's an assumption worth revisiting.
Running ./inspectors/dod.py --year=2014 picks up this report and doesn't mark it as unreleased, so I think this is instead an issue of whether we should be fetching information from farther back in time than we do, on a regular basis. I'm opening a new issue for that.
cc @LindsayYoung

Oops, you are correct. I just saw that it wasn't on oversight.io and forgot that we don't scrape historical regularly.


⚠ Fortnite has degraded performance

In 4a2b101, Fortnite (https://fortnite.com) experienced degraded performance:

HTTP code: 200
Response time: 1587 ms

Resolved: Fortnite performance has improved in db3b8e9.


update to 1.19.3 or 1.20

i love the far lands but i play on the newest minecraft  versions can u updata the mod
thank u

maybe not only to 1.19.3 and 1.20 but also to fabric 1.19?


ARROW-1088: [Python] Only test unicode filenames if system supports them

In what environment does this occur?

@wesm here is a Dockerfile to create the environment:
FROM ubuntu:17.10

MAINTAINER Jeff Knupp<EMAIL_ADDRESS>
RUN apt-get update && apt-get install -yq python3 \
        python3-dev \
        g++-4.9 \
        libboost-dev \
        libboost-filesystem-dev \
        libboost-program-options-dev libboost-regex-dev \
        libboost-system-dev libboost-test-dev \
        libssl-dev libtool bison flex pkg-config git vim cmake \
        zlibc python3-pip

WORKDIR /home
COPY .ssh /home/.ssh/

WORKDIR /home/arrow

RUN git clone https://github.com/apache/parquet-cpp
RUN git clone https://github.com/apache/arrow

ENV ARROW_HOME=/usr/local
RUN pip3 install pandas cython
WORKDIR /home/arrow/arrow/cpp/build
RUN cmake -DARROW_PYTHON=on .. && make -j4 && make install

WORKDIR /home/arrow/parquet-cpp
RUN cmake . && make -j4 && make install

WORKDIR /home/arrow/arrow/python
RUN python3 setup.py build_ext --with-parquet --with-jemalloc

I digged a bit through this and the main difference between my ubuntu 16.04 machine and the 17.04 docker image is the filesystemencoding:
Ubuntu 16.04:
Python 2.7.12 (default, Nov 19 2016, 06:48:10)
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.getfilesystemencoding()
'UTF-8'

Ubuntu 17.04:
Python 3.5.3 (default, Jan 19 2017, 14:11:04) [GCC 6.3.0 20170118] on linuxType "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.getfilesystemencoding()
'ascii'

Setting the appropriate environment variables like LC_ALL and LANG to en_US.UTF-8 didn't help.

I was able to fix this by appending this to the Dockerfile
RUN apt-get -yq install locales
RUN locale-gen en_US.UTF-8
ENV LANG en_US.UTF-8
ENV LANGUAGE en_US:en
ENV LC_ALL en_US.UTF-8

@wesm I think you've just proven that if you change a system that doesn't accept unicode filenames into one that does, the tests work as expected. I wasn't really making that point. Rather, if the underlying system doesn't (at the time of test execution) support unicode filenames, we should skip the unit test that relies on their support.

@wesm thanks. Can you (or someone else) merge when you get a chance?

I'm going to merge; since @xhochy had said -1 I didn't want to override that right away. If there are more concerns about how this is handled I will be help with another patch


How to update the database in batches?

I have a small database where changes are "saved" to the database each time a field is modified, like this:
using SQLite;

public class Entry
{
	[PrimaryKey, AutoIncrement]
	public int ID { get; set; }
	public string Field { get; set; }
}

public class Database
{
	SQLiteConnection connection;

	//...

	public int UpdateEntry(Entry entry)
	{
		return connection.Update(entry);
	}
}

Somewhere in my app:
entry.Field = "some data";
database.UpdateEntry(entry);

It does the trick, but it's inefficient, so I was wondering what's the recommended way to update the database in batches, so that I can do the batch update when the app enters in background mode, or somewhere else in the app's live cycle.

You have the method InsertAll here or you can use transactions if you have to apply some logic to your data. Or you can always do a raw Execute<T> too but you actually bypass the ORM so be careful.
I would encourage you to have a look at the API to discover its features.
Hope this helps!


Precision not working

Precision variable while querying data is not working and I guess the reason for that is:
params + ("precision" -> precision.toString)

buildQueryParameters is making map for (precision -> precision.toString) but influxdb expects parameter to be "epoch" and not "precision". So, shouldn't this be ("epoch" -> precision.toString)?
https://docs.influxdata.com/influxdb/v0.10/guides/querying_data/

0.4.4 is out with a fix for this.
Thanks for the report!


Set.animate(array) wont work as desired

Sorry about my english :O)
I try to animate a Set (Snap.js 0.5.1) with Set.animate(). In the Manual I can read ([http://snapsvg.io/docs/#Set.animate]) this:
// animate first element to radius 10, but second to radius 20 and in different time
set.animate([{r: 10}, 500, mina.easein], [{r: 20}, 1500, mina.easein]);

In the manual I also can read, that I can pass an array of animations as the first argument like
Set.animate(arr)
but this doesn't work.
If I simply try
set.animate([{r: 10}, 500, mina.easein], [{r: 20}, 1500, mina.easein]);
this woks well. But if I try
var arr = [[{r: 10}, 500, mina.easein], [{r: 20}, 1500, mina.easein]];
set.animate(arr);

will result in this Error
InvalidCharacterError: String contains an invalid character snap.svg.js:1000
I ask this question in stackoverflow but got no answer to solve this problem.
Is this a bug?

I added a comment to the SO question previously. Thinking more, I don't think it's a bug actually, it's the fact you are passing an array of arrays, instead of an array into the parameters, which is where I was leading before.
So I think this may work (as it will unfold the arrays into the parameter list...
set.animate.apply( set, arr );
eg http://jsfiddle.net/3nmL6tz1/


Downloading more tweets + tweet caching

This contains the changes from #4 plus a change to the core ebook method to cache tweets already obtained and only request new tweets after the initial scrape. This should make it much less likely to exceed the rate limit. The bot would have to post 162 new tweets in the 15min window after starting to exceed it, which the default time limits should prevent.

Ok yeah I think that was the issue, thanks!
Also thanks for the markov bug catch!

Also sorry about the revert earlier, I'm still figuring out git.

It's cool.


HttpParse .IndexOf(...) >= 0 to .Contains(...)

Is faster and added to coreclr for this purpose https://github.com/dotnet/corefx/issues/27526

Thanks Ben 🤗


@global binding not updated on ractive.update

Description:
While writing an example for @global two-way binding, I notice that changing the global outside of Ractive and calling ractive.update(), the instance isn't updated to the new value.
I might be blind staring at the screen all day or just using it wrong. Wouldn't hurt to ask for another pair of eyes to take a look. 🤓
Versions affected:
Platforms affected:
Reproduction:
https://ractive.js.org/playground/#N4IgFiBcoE5QdgVwDbIL4BoQBcogDwDOAxjAJYAO2ABITMQLwA6422FhkA9F4vBQGsA5gDpiAewC2XGAENi2MgDcApiwB8+LiXJV1ILITwB3MvAAm44yMkrChWUJXUG1AOQAJFanHUA6uIwyOYAhG5M8BEASvKKqgAUwBFM2N6Q7gBG4uYAnm4YydiyFBQqFunYMIgqBZFsKpIUyLKp6QAGhSn4ZhSINNg5pcw4KgAe2CzUSrLI1cPAwNQAAkLI4hkzNnYOTtRoaBqd2PgZfdji8NQDQyynbBeTFwC0xMhkxALDS9hgZIQiayEURUADMVDAysQVPEAJQaAAy4iE1AhYIh8ChUxm1S0d3O8HURxOZwuV0GKmGeIeIGoz1e70+LG+v3+gIA4msNshYQikdRVusZljZipcSSCUSqZdrhTbuLHvAXm8Pl8fn8RIQVNgAGLicQAIVkMENAC8eSB1ABhMCyeC7c7UFggvXUDYwV2yE2TcxkCEKZA5MX3CV1LpSsk3EBShVKhmqlkiRAUcwtaFwi2W5UCK6+JMp1JB-GE0MTEv4cTIYspavHN5WrOOkCI5Go8GQ5zTEWTW3mahrcTZlpXMDOCTwQgV0VcOtHLp1zMMxvN-mcoWd6rdix9vWD-oj6hjifIKczktzshWm125w-Zw9PrC6rUHsolSlIeEVIcagARmfW4AJi0U8a3PetFxYa1bXtXw3GdXw3Q9E03GoH0-WwAMWGAi9Z1rC9gXfGhPzfQhf3-XsAPI6h4HERRMR+D8vzI4xZFImiaFkFiIVpEFh2cNCVAUA8rycbCqwmCT8HnBtIIbB081TLDpxws88PUAiVEYkiyJfSiX3Y94bxtGgsh+Wgv1Io1nBo4xny4k8VJrLQK3EtpahSQFgTRdtYSSEtD0nAEkXiNV-icbB4jcFZV2QLZ7EcFQ3BhOE6kwQp2Ri3yjgC48gqEeJTAsKw4p2FQYQAbkKNK6k1HU9UNY1PSyktCssaxbHi3ZXCdF1EI2L0QCqiI0AqkA0CAA
Code:
window.message = 'Hello World!'

Ractive({
  el: 'body',
  append: true,
  template: `
    <input type="text" value="{{ @global.message }}">
    <button type="button" on-click="@this.logReference()">Log reference value</button>
    <button type="button" on-click="@this.logGlobal()">Log global value</button>
    <button type="button" on-click="@this.setFooBarBaz()">Change to "foo bar baz" directly</button>
    <button type="button" on-click="@this.update()">Click to update</button>

Click "Log reference value" and look at the console

Click "Log global value" and look at the console

Change the input value and repeat steps 1 and 2

Click "Change to 'foo bar baz' directly"

Repeat steps 1 and 2 and notice that step 1 was not aware of the direct change

Click "Click to update"

Repeat steps 1 and 2 and notice that both steps are now aware

  `,
  logReference(){
    console.log(this.get('@global.message'))
  },
  logGlobal(){
    console.log(window.message);
  },
  setFooBarBaz(){
    window.message = "foo bar baz"
  }
});

So it looks like only a blank ractive.update() doesn't work. But specifying the keypath does.

That actually seems like mostly reasonable behavior to me. I wouldn't really expect any old update to touch @global and @shared refs in other instances. If it needs to be changed, we'll just need to update update to include those special keypaths. I lean towards requiring explicit updates (ractive.update('@global')), though, because the next logical step is @this, and that's another can of worms.

If it's expected behavior, it's ok by me. I'll just update ractive.update to make note of that. 👍


Ensure consistent Java types for fallback values when converting

https://issues.jboss.org/browse/DBZ-1149

any chance of this being solved soon ?
My entire db is filled with time zone dates and the connector breaks completely on deletes

Thanks @gunnarmorling. I can try writing a test later in the week, I'm a bit slammed at work right now.

@kbarber2 I've sent #795 which contains this one here and a few more things, so I'm going to close this one.


Print load in float value not Integar

Currently, I am working on multiple vehicles doing multiple trips problem and I am passing the demand of particular location in float value like 3.82 or 5.67 and I want to print the cumulative load of the vehicle at a particular location in float value. Currently when the load is .87 or .43 at a particular location the optimizer printing it as 0 and when the load is 3.57 or 2.34 the optimizer printing it as 3 or 2. how I can do this?
Also, I want to know that the optimizer is considering the decimal values as well to give the best route or not? or it's just considering the integer value?
from __future__ import print_function
import math
from ortools.constraint_solver import pywrapcp
from ortools.constraint_solver import routing_enums_pb2
from numpy import genfromtxt
import pandas as pd

def create_distance_callback(dist_matrix):
  # Create a callback to calculate distances between cities.

  def distance_callback(from_node, to_node):
    return int(dist_matrix[from_node][to_node])

    for to_node in range(num_locations):
      if from_node == depot or to_node == depot:
            # Define the distance from the depot to any node to be 0.
            self.matrix[from_node][to_node] = 0
    else:
          dist_matrix[from_node][to_node] = (
            manhattan_distance(locations[from_node], locations[to_node]))
  return distance_callback

dist_matrix = genfromtxt('full_validation_points_dist_matrix_xy.csv', delimiter=',')

# Demand callback
class CreateDemandCallback(object):
  """Create callback to get demands at each node."""

  def __init__(self, demands):
    self.matrix = demands

  def Demand(self, from_node, to_node):
   # del to_node
    return self.matrix[from_node]

# Service time (proportional to demand) + transition time callback.
class CreateServiceTimeCallback(object):
  """Create callback to get time windows at each node."""

  def __init__(self, demands, time_per_demand_unit):
    self.matrix = demands
    self.time_per_demand_unit = time_per_demand_unit

  def ServiceTime(self, from_node, to_node):
    return self.matrix[from_node] * self.time_per_demand_unit

# Create total_time callback (equals service time plus travel time).
class CreateTotalTimeCallback(object):
  def __init__(self, service_time_callback, dist_callback, speed):
    self.service_time_callback = service_time_callback
    self.dist_callback = dist_callback
    self.speed = speed

  def TotalTime(self, from_node, to_node):
    return self.service_time_callback(from_node, to_node) + self.dist_callback(from_node, to_node) / self.speed

def DisplayPlan(routing, assignment):

  # Display dropped orders.
  dropped = ''

  for order in range(1, routing.nodes()):
    if assignment.Value(routing.NextVar(order)) == order:
      if (dropped.empty()):
        dropped += " %d", order
      else: dropped += ", %d", order

  if not dropped.empty():
    plan_output += "Dropped orders:" + dropped + "\n"

  return plan_output

def main():
  # Create the data.
  data = create_data_array()
  dist_matrix = data[0]
  demands = data[1]
  start_times = data[2]
  end_times=data[3]
  #print(data[2])
  num_locations = len(dist_matrix)
  #print(num_locations)
  #depot = 68
  num_vehicles = 2

  search_time_limit = 180

  # Create routing model.
  if num_locations > 0:

    start_location = [0,0]
    end_location=[141,141]
    # The number of nodes of the VRP is num_locations.
    # Nodes are indexed from 0 to tsp_size - 1. By default the start of
    # a route is node 0.
    routing = pywrapcp.RoutingModel(num_locations, num_vehicles, start_location,end_location)

    search_parameters = pywrapcp.RoutingModel.DefaultSearchParameters()
    search_parameters.first_solution_strategy = (
                                    routing_enums_pb2.FirstSolutionStrategy.PATH_CHEAPEST_ARC)

    # Callback to the distance function.
    #print(dist_matrix)
    dist_callback = create_distance_callback(dist_matrix)
    routing.SetArcCostEvaluatorOfAllVehicles(dist_callback)

    routing.SetArcCostEvaluatorOfAllVehicles(dist_callback)
    demands_at_locations = CreateDemandCallback(demands)
    demands_callback = demands_at_locations.Demand

    # Add capacity dimension constraints.
    vehicle_capacity = 120;
    null_capacity_slack = 0;
    fix_start_cumul_to_zero = True
    capacity = "Capacity"
    routing.AddDimension(demands_callback, null_capacity_slack, vehicle_capacity,
                         fix_start_cumul_to_zero, capacity)

    # Adding time dimension constraints.
    time_per_demand_unit = 0.005
    horizon = 720
    time = "Time"
#     tw_duration = 680
    speed = 40*60/3.6
    #print(speed)

    service_times = CreateServiceTimeCallback(demands, time_per_demand_unit)
    service_time_callback = service_times.ServiceTime
    total_times = CreateTotalTimeCallback(service_time_callback, dist_callback, speed)
    total_time_callback = total_times.TotalTime

    # Note: In this case fix_start_cumul_to_zero is set to False,
    # because some vehicles start their routes after time 0, due to resource constraints.

    fix_start_cumul_to_zero = False
    # Add a dimension for time and a limit on the total time_horizon
    routing.AddDimension(total_time_callback,  # total time function callback
                         720,
                         720,
                         fix_start_cumul_to_zero,
                         time)

    time_dimension = routing.GetDimensionOrDie("Time")

    for order in range(1, num_locations-1):
      start = start_times[order]
      end=end_times[order]
      #print("---")

      #print(end_times[num_locations-1])
      #print(start_times[num_locations-1])
      ##print("---")
 #     print(start_times[num_locations])
  #    print(end_times[num_locations])
      #print("---")
      x=start.item()
      y=end.item()

      #print(time_dimension.CumulVar(int(cust.index)).Max())
      time_dimension.CumulVar(routing.NodeToIndex(order)).SetRange(int(x), int(y))

    vehicle_start_time = [0,0,0]
    for vehicle_id in range(num_vehicles):
            index = routing.Start(vehicle_id)
            time_dimension.CumulVar(index).SetRange(vehicle_start_time[vehicle_id], vehicle_start_time[vehicle_id])

#     # Add resource constraints at the depot (start and end location of routes).

#     vehicle_start_time = [180,240,360]
# #     vehicle_unload_time = 0
#     solver = routing.solver()
#     intervals = []
#     for i in range(num_vehicles):
#       # Add time windows at start of routes
#       intervals.append(solver.FixedDurationIntervalVar(routing.CumulVar(routing.Start(i), time),
#                                                      vehicle_start_time[i],
#                                                      "depot_interval"))
#       # Add time windows at end of routes.
# #       intervals.append(solver.FixedDurationIntervalVar(routing.CumulVar(routing.End(i), time),
# #                                                      vehicle_unload_time,
# #                                                      "depot_interval"))
# #     print(intervals)
# #     print("*"*50)
#     # Constrain the number of maximum simultaneous intervals at depot.
#     depot_capacity = 1;
#     depot_usage = [1 for i in range(num_vehicles * 1)]
#     solver.AddConstraint(
#       solver.Cumulative(intervals, depot_usage, depot_capacity, "depot"))
#     # Instantiate route start and end times to produce feasible times.
#     for i in range(num_vehicles):
#       routing.AddVariableMinimizedByFinalizer(routing.CumulVar(routing.End(i), time))
#       routing.AddVariableMinimizedByFinalizer(routing.CumulVar(routing.Start(i), time))

    # Solve, displays a solution if any.
    assignment = routing.SolveWithParameters(search_parameters)
    if assignment:

      # Solution distance.
#      print("Total distance of all routes: " + str(assignment.ObjectiveValue()) + "\n")
      # Display solution.

      capacity_dimension = routing.GetDimensionOrDie(capacity)
      for node_index in [1, 2, 3]:
            index = routing.NodeToIndex(node_index)
            capacity_dimension.SlackVar(index).SetRange(0, vehicle_capacity)
      time_dimension = routing.GetDimensionOrDie(time)

      total_distance = 0
      total_load = 0
      total_time = 0
      visited_locations = []
      dropped_locations = list(range(num_locations))

      for vehicle_nbr in range(num_vehicles):
        index = routing.Start(int(vehicle_nbr))
        plan_output = 'Route {0}:'.format(vehicle_nbr)
        route_dist = 0
        route_load = 0
        while not routing.IsEnd(index):
          node_index = routing.IndexToNode(index)
          next_node_index = routing.IndexToNode(assignment.Value(routing.NextVar(index)))
          visited_locations.append(next_node_index)
          if next_node_index in dropped_locations:
                dropped_locations.remove(next_node_index)
          load_var = capacity_dimension.CumulVar(index)
          time_var = time_dimension.CumulVar(index)
#           print(time_var)
          route_dist += routing.GetArcCostForVehicle(node_index, next_node_index, vehicle_nbr)
#           route_load += [demands][node_index]
          plan_output += \
                    " {node_index} Load({load}) Time({tmin}, {tmax}) -> ".format(
                        node_index=node_index,
                        load=assignment.Value(load_var),
                        tmin=str(assignment.Min(time_var)),
                        tmax=str(assignment.Max(time_var)))
#           plan_output += ' {0} Load({1}) -> '.format(node_index, route_load)
          index = assignment.Value(routing.NextVar(index))

        node_index = routing.IndexToNode(index)  # Convert index to node
        total_distance += route_dist
        load_var = capacity_dimension.CumulVar(index)
        time_var = time_dimension.CumulVar(index)

        plan_output += \
                  " {node_index} Load({load}) Time({tmin}, {tmax})".format(
                      node_index=node_index,
                      load=assignment.Value(load_var),
                      tmin=str(assignment.Min(time_var)),
                      tmax=str(assignment.Max(time_var)))

        print("\n")

        plan_output += 'Load of the route: {}\n'.format(assignment.Value(load_var))
        plan_output += 'Time of the route: {}\n'.format(assignment.Value(time_var))
# #         plan_output += ' {0} Load({1})\n'.format(node_index, route_load)
        plan_output += 'Distance of the route: {0}m\n'.format(route_dist)
#         plan_output += 'Load of the route: {0}\n'.format(route_load)

        total_load += assignment.Value(load_var)
        total_time += assignment.Value(time_var)
        print(plan_output)
#         print(total_time)
        print("\n")

        print('Total Load of all routes: {}'.format(total_load))
        print('Total Time of all routes: {}min'.format(total_time))
        print('Total Distance of all routes: {0}m'.format(total_distance))
        print('Dropped visits: ', list(set(range(num_locations)) - set(visited_locations)))

    else:
      print('No solution found.')
  else:
    print('Specify an instance greater than 0.')
def create_data_array():

  tv=pd.read_csv('full_collection_location_demand_df_or.csv')
  demands = tv.Demand_hun

  start_times=tv.lower_limit_time_Summary
  end_times=tv.upper_limit_time_Summary

  data = [dist_matrix, demands, start_times,end_times]
  return data

if __name__ == '__main__':
  main()

@Mizux

Just multiply all floats by 100, and cast to int.

by multiplying floats to hundred script is taking to much time to run and not giving the output

@lperron

yes, but it has a chance to be correct.
Laurent Perron | Operations Research |<EMAIL_ADDRESS>| (33) 1 42 68 53
00
*De : *Abhishek Agrawal<EMAIL_ADDRESS>*Date : *sam. 11 mai 2019 à 23:24
*À : *google/or-tools
*Cc : *Laurent Perron, Mention
@lperron https://github.com/lperron

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/google/or-tools/issues/1266#issuecomment-491544974,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACUPL3N5AN3KNUV3REU65D3PU42Q5ANCNFSM4HMIXWKA
.

first the demand at the particular location was high like 231, 234,546 (int values)  but script was not giving any output and running for hours( because total demand was high 24000 )then i divided it by 100 now it's giving output really quick. so is there any method from which i can print decimal values?

@lperron

also to optimize the route the optimizer taking the decimal values (2.37 as 2.37) or just taking the integer values like 2.37 as 2 ?? @Mizux

The optimizer is integral only. Therefore 2.
Laurent Perron | Operations Research |<EMAIL_ADDRESS>| (33) 1 42 68 53
00
*De : *Abhishek Agrawal<EMAIL_ADDRESS>*Date : *sam. 11 mai 2019 à 23:37
*À : *google/or-tools
*Cc : *Laurent Perron, Mention
also to optimize the route the optimizer taking the decimal values (2.37 as

2.37) or just taking the integer values like 2.37 as 2 ?? @Mizux
https://github.com/Mizux
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/google/or-tools/issues/1266#issuecomment-491545706,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACUPL3PSSPLR3GCKBTAZWOTPU44BNANCNFSM4HMIXWKA
.

how to run the script fast with high demand?  can i add anything so that it can give the output quick when i am increasing the demand?

How big is your problem ?
*De : *Abhishek Agrawal<EMAIL_ADDRESS>*Date : *dim. 12 mai 2019 à 00:01
*À : *google/or-tools
*Cc : *Laurent Perron, Mention
how to run the script fast with high demand? can i add anything so that it

can give the output quick when i am increasing the demand?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/google/or-tools/issues/1266#issuecomment-491547068,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACUPL3OV7HET3AMMKWC4QBTPU463JANCNFSM4HMIXWKA
.

Actually total demand is 24000 and total location need to cover is 142. So i divided each demand point by 100 now total demand is 240. Also previously capacity of vehicle was 12000 now it's 120

I meant number of visits, number of vehicles...
*De : *Abhishek Agrawal<EMAIL_ADDRESS>*Date : *dim. 12 mai 2019 à 00:08
*À : *google/or-tools
*Cc : *Laurent Perron, Mention
Actually total demand is 24000 and total location need to cover is 142. So

i divided each demand point by 100 now total demand is 240. Also previously
capacity of vehicle was 12000 now it's 120
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/google/or-tools/issues/1266#issuecomment-491547481,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACUPL3JTRRDZTBU46SZVSYLPU47VVANCNFSM4HMIXWKA
.

number of vehicle is 3 and I am making one dumping point.
and vehicle capacity is 12000 so maximum a specific vehicle will start from depot go to dummy dumping then end at dumping.

@Mizux @lperron


Sensor to Light Switch remapping

Describe the idea or improvement
There is a clever solution available to use window/door sensor as smart light switch where some alias workaround needed to convert sensor to switch. Home Assistant example:
- alias: Light switch
  trigger:
    platform: state
    entity_id: binary_sensor.door_window_sensor_158d000xxxxxc2
  action:
    service: light.toggle
    entity_id: light.somelight

Describe the solution you'd like
Is there any options already available to do that with this plugin or additional development needed?
Additional context
No context.

Hi,
interesting to use of the sensors😅 I understand using the sensor as a trigger to turn on off a light switch is not possible hence this hack?
I do not think that it is possible currently, but I mean you could try to manually specify the device model and use properly monitor or control to expose the properties which you want. The thing is that probably the properties will not match between the real or fake device, but you could try that out anyway...

@merdok Thank you! Can you help me with that? How to manually specify the device?

Please read the README, all the information which you need is there!

I already did, but it's way out of my knowledge in this area. Any examples available for remapping one device type to another? Appreciate your help here.
Sensor details:
NAME:     Light switch Virtual
ID:       lumi.158d00016cd97a
MODEL:    lumi.sensor_magnet.v2

PROPS:

magnet-sensor:contact-state

EVENTS:

magnet-sensor:close
magnet-sensor:open
magnet-sensor:door-not-closed
battery:low-battery

You have to specify the device type manually in the config. That will be used to create the accessory. Afterwards, as the properties will it match between what you specified and the actual device, you should propertyMonitor or propertyControl to configure the device. All the info on how to do that is in the README.

Oh, wait. It's a zigbee device, so, there is no token/ip for this device available. It will not work anyway, correct?


Updated to latest devcontainer specification

114
PR Checklist

[ ] I have updated the documentation accordingly.
[ ] I have added tests to cover my changes.
[ ] All new and existing tests passed.
[X] My code follows the code style of this project.
[ ] I ran lint checks locally prior to submission.
[ ] Have you checked to ensure there aren't other open Pull Requests for the same update/change?

Description
On the current starter branch the devcontainer.json file needs an update to latest devcontainer specification
Currently the file shows warnings as below:

Does this introduce a breaking change

[ ] YES
[X] NO

@microsoft-github-policy-service agree
Regards,
Sujay Pillai
M: +60-1129623182
On Wed, Mar 22, 2023 at 11:51 PM microsoft-github-policy-service[bot] <
@.***> wrote:

@sujaypillai https://github.com/sujaypillai please read the following
Contributor License Agreement(CLA). If you agree with the CLA, please reply
with the following information.
@microsoft-github-policy-service agree [company="{your company}"]
Options:

(default - no company specified) I have sole ownership of
intellectual property rights to my Submissions and I am not making
Submissions in the course of work for my employer.

@microsoft-github-policy-service agree

(when company given) I am making Submissions in the course of work
for my employer (or my employer has intellectual property rights in my
Submissions by contract or applicable law). I have permission from my
employer to make Submissions and enter into this Agreement on behalf of my
employer. By signing below, the defined term “You” includes me and my
employer.

@microsoft-github-policy-service agree company="Microsoft"
Contributor License Agreement Contribution License Agreement
This Contribution License Agreement (“Agreement”) is agreed to by the
party signing below (“You”),
and conveys certain license rights to Microsoft Corporation and its
affiliates (“Microsoft”) for Your
contributions to Microsoft open source projects. This Agreement is
effective as of the latest signature
date below.

Definitions.
“Code” means the computer software code, whether in human-readable
or machine-executable form,
that is delivered by You to Microsoft under this Agreement.
“Project” means any of the projects owned or managed by Microsoft
and offered under a license
approved by the Open Source Initiative (www.opensource.org).
“Submit” is the act of uploading, submitting, transmitting, or
distributing code or other content to any
Project, including but not limited to communication on electronic
mailing lists, source code control
systems, and issue tracking systems that are managed by, or on behalf
of, the Project for the purpose of
discussing and improving that Project, but excluding communication
that is conspicuously marked or
otherwise designated in writing by You as “Not a Submission.”
“Submission” means the Code and any other copyrightable material
Submitted by You, including any
associated comments and documentation.
Your Submission. You must agree to the terms of this Agreement
before making a Submission to any
Project. This Agreement covers any and all Submissions that You, now
or in the future (except as
described in Section 4 below), Submit to any Project.
Originality of Work. You represent that each of Your Submissions
is entirely Your original work.
Should You wish to Submit materials that are not Your original work,
You may Submit them separately
to the Project if You (a) retain all copyright and license information
that was in the materials as You
received them, (b) in the description accompanying Your Submission,
include the phrase “Submission
containing materials of a third party:” followed by the names of the
third party and any licenses or other
restrictions of which You are aware, and (c) follow any other
instructions in the Project’s written
guidelines concerning Submissions.
Your Employer. References to “employer” in this Agreement include
Your employer or anyone else
for whom You are acting in making Your Submission, e.g. as a
contractor, vendor, or agent. If Your
Submission is made in the course of Your work for an employer or Your
employer has intellectual
property rights in Your Submission by contract or applicable law, You
must secure permission from Your
employer to make the Submission before signing this Agreement. In that
case, the term “You” in this
Agreement will refer to You and the employer collectively. If You
change employers in the future and
desire to Submit additional Submissions for the new employer, then You
agree to sign a new Agreement
and secure permission from the new employer before Submitting those
Submissions.
Licenses.

Copyright License. You grant Microsoft, and those who receive the
Submission directly or
indirectly from Microsoft, a perpetual, worldwide, non-exclusive,
royalty-free, irrevocable license in the
Submission to reproduce, prepare derivative works of, publicly
display, publicly perform, and distribute
the Submission and such derivative works, and to sublicense any or all
of the foregoing rights to third
parties.
Patent License. You grant Microsoft, and those who receive the
Submission directly or
indirectly from Microsoft, a perpetual, worldwide, non-exclusive,
royalty-free, irrevocable license under
Your patent claims that are necessarily infringed by the Submission or
the combination of the
Submission with the Project to which it was Submitted to make, have
made, use, offer to sell, sell and
import or otherwise dispose of the Submission alone or with the
Project.
Other Rights Reserved. Each party reserves all rights not
expressly granted in this Agreement.
No additional licenses or rights whatsoever (including, without
limitation, any implied licenses) are
granted by implication, exhaustion, estoppel or otherwise.

Representations and Warranties. You represent that You are
legally entitled to grant the above
licenses. You represent that each of Your Submissions is entirely Your
original work (except as You may
have disclosed under Section 3). You represent that You have secured
permission from Your employer to
make the Submission in cases where Your Submission is made in the
course of Your work for Your
employer or Your employer has intellectual property rights in Your
Submission by contract or applicable
law. If You are signing this Agreement on behalf of Your employer, You
represent and warrant that You
have the necessary authority to bind the listed employer to the
obligations contained in this Agreement.
You are not expected to provide support for Your Submission, unless
You choose to do so. UNLESS
REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING, AND EXCEPT FOR THE
WARRANTIES
EXPRESSLY STATED IN SECTIONS 3, 4, AND 6, THE SUBMISSION PROVIDED
UNDER THIS AGREEMENT IS
PROVIDED WITHOUT WARRANTY OF ANY KIND, INCLUDING, BUT NOT LIMITED TO,
ANY WARRANTY OF
NONINFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE.
Notice to Microsoft. You agree to notify Microsoft in writing of
any facts or circumstances of which
You later become aware that would make Your representations in this
Agreement inaccurate in any
respect.
Information about Submissions. You agree that contributions to
Projects and information about
contributions may be maintained indefinitely and disclosed publicly,
including Your name and other
information that You submit with Your Submission.
Governing Law/Jurisdiction. This Agreement is governed by the
laws of the State of Washington, and
the parties consent to exclusive jurisdiction and venue in the federal
courts sitting in King County,
Washington, unless no federal subject matter jurisdiction exists, in
which case the parties consent to
exclusive jurisdiction and venue in the Superior Court of King County,
Washington. The parties waive all
defenses of lack of personal jurisdiction and forum non-conveniens.
Entire Agreement/Assignment. This Agreement is the entire
agreement between the parties, and
supersedes any and all prior agreements, understandings or
communications, written or oral, between
the parties relating to the subject matter hereof. This Agreement may
be assigned by Microsoft.

—
Reply to this email directly, view it on GitHub
https://github.com/Azure/caf-terraform-landingzones-accelerator/pull/115#issuecomment-1479825630,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AA6BDSWT6LXB24TS7HNXGX3W5MNZDANCNFSM6AAAAAAWEAM4FU
.
You are receiving this because you were mentioned.Message ID:
@.***
com>


x/time: create new subrepo for x/time/rate, other code

I would like to open-source a package rate that we developed at Google.  It provides a rate limiter that supports Context for cancelation and deadlines: a caller can block until either the rate limiter allows the call to proceed or the Context is canceled.  It includes an optimization to skip the wait if it would exceed the Context's Deadline.  This package is widely used inside Google Go code.
There is no obvious home for this package in any of the golang repositories, so I propose we add a new subrepo, golang.org/x/time, that will contain the rate package as golang.org/x/time/rate.  To justify the creation of a new subrepo, it would be useful to identify additional packages that belong here.

The "net" repo seems the obvious place for this, to me.
But my opinion aside, issue #12914 is a request for a monotonic clock
source. That seems like a candidate for inclusion for a potential "time"
repository.
On 23 October 2015 at 03:32, Sameer Ajmani<EMAIL_ADDRESS>wrote:

I would like to open-source a package rate that we developed at Google.
It provides a rate limiter that supports (Context)[
https://godoc.org/golang.org/x/net/context] for cancelation and
deadlines: a caller can block until either the rate limiter allows the call
to proceed or the Context is canceled. It includes an optimization to skip
the wait if it would exceed the Context's Deadline. This package is widely
used inside Google Go code.
There is no obvious home for this package in any of the golang
repositories, so I propose we add a new subrepo, golang.org/x/time, that
will contain the rate package as golang.org/x/time/rate. To justify the
creation of a new subrepo, it would be useful to identify additional
packages that belong here.
—
Reply to this email directly or view it on GitHub
https://github.com/golang/go/issues/13022.

There's nothing about networking in the package Sameer is talking about. It's important that net not become a dumping ground for "anything a server might want".
I think x/time is reasonable. One might envision calendar code going there too.
Who knows? Maybe even monotonic time support, although I am less convinced of that.
Anyway, an x/time repo seems reasonable.
/cc @robpike

Yes, my argument to Sameer was that "tools" became a dumping ground and now "net" is becoming a dumping ground. We also have a few things pending perfect for an "x/sync" subrepo if there were one.
Rather than thinking that a subrepo has a high cost, we should really think of a subrepo as a subdirectory of "x" where you can think of "x" as the same as, say, "google3".

I agree with @bradfitz we should be creating more repositories.
I'm not familiar with the rate package; it sounded intrinsically
network-related from its description.
On 24 October 2015 at 16:32, Brad Fitzpatrick<EMAIL_ADDRESS>wrote:

Yes, my argument to Sameer was that "tools" became a dumping ground and
now "net" is becoming a dumping ground. We also have a few things pending
perfect for an "x/sync" subrepo if there were one.
Rather than thinking that a subrepo has a high cost, we should really
think of a subrepo as a subdirectory of "x" where you can think of "x" as
the same as, say, "google3".
—
Reply to this email directly or view it on GitHub
https://github.com/golang/go/issues/13022#issuecomment-150757520.

LGTM. Leaving for adg@ to create the subrepo.

Thank you!
On Tue, Nov 3, 2015 at 1:58 AM, Andrew Gerrand<EMAIL_ADDRESS>wrote:

Closed #13022 https://github.com/golang/go/issues/13022 via
golang/build@4154d4f
https://github.com/golang/build/commit/4154d4f503977780c76a601139fd67035411f4d4
.
—
Reply to this email directly or view it on GitHub
https://github.com/golang/go/issues/13022#event-452846923.


[add-resource] New Theme: Tutorial

I would like to add my new theme to the Grav Repository.
Here are the project details: Tutorial

Was added already.


Advanced search does not find principal's first name

A user couldn't find a principal's record searching by first name.  I was able to replicate the bug.
Test 1
I used Advanced search (with the "People" parameter) and entered the last name.  The search results returned the principal's record.
Test 2
I used Advanced search (with the "People" parameter) and entered the first name.  The search results did not return the principal's record.  In fact, none of the 24 returned results contain the principal's name.  There are 74 principal records which have the first name as part of a principal's name.  The first name can also be used as a last name.
Test 3
I used the first name in a general search without any parameters and the same 24 principal records were returned at Test 2 (along with 3,929 engagement reports).
Test 4
I used Advanced search (with the "People" parameter and a "Role" = principal) and entered the first name.  The search results did not return the principal's record.  In fact, none of the 24 returned results contain the principal's name.  The same search results as Test 2.
I have sent the principal's full name to @VassilIordanov on the high side.

Added to #1969


granted: 0.34.1 -> 0.35.1

Diff: https://github.com/common-fate/granted/compare/v0.34.1...v0.35.1
Changelog:

https://github.com/common-fate/granted/releases/tag/v0.35.0
https://github.com/common-fate/granted/releases/tag/v0.35.1

Maintainers: @ivankovnatsky
Also adds updateScript (nix-update-script) and versionCheckHook.
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

@ivankovnatsky I currently use it at work; would you want me to replace you as a maintainer?

@ivankovnatsky I currently use it at work; would you want me to replace you as a maintainer?
edit: Took the liberty of doing just that; please feel free to say you rather just have me be an additional maintainer. Because I feel maintainer changes are important I'm re-requesting your code review (hope you don't mind), and updating the PR description to reflect that.
(also addressed your whitespace suggestion in the process)

Sure thing, totally makes sense! Thanks.

I've tested on aarch64-darwin and  x86_64-linux, and it works as expected.
Thanks 🙏

wow it's jarring to be be about to click the merge button, and see it turn gray due to a merge-conflict
conflict with https://github.com/NixOS/nixpkgs/pull/347816

@pbsds May I kindly ask you to re-review + merge after the force-push? I've resolved the conflicts (caused by the nixfmt commit).
(easier to review that commit separate from the rest).


Reduce created segments when there is low traffic on a shard

Lucene creates 1 segment per active concurrent thread per shard. Number of active concurrent threads per shard is determined by bulk thread pool in ES, which is a worker on the queue holding sub bulk requests. Each sub-bulk request ends up being picked by a different thread, hence resulting in multiple segments during refresh, resulting in better performance during reads. When there is a low traffic on a particular shard on a node, we can potentially reduce the number of created segments by co-alescing bulk requests.
When there are lot of shards on a single node for different indices, this problem may aggravate further.
The proposal is to optimize this entire process is via the following tasks:

[ ] Pipeline the CPU work into different tasks further (parsing, translog, lucene add) per shard
[ ] Coalesce the lucene documents across different bulk requests by adding a separate queue for lucene adds.

Can you more clearly define the problem that is solved by reducing the number of segments that are created on low traffic shards? What is the use case that optimization is targeting?

Closing it due to lack of details. @itiyamas feel free to reopen once you have more details about it.

@anasalkouz I do not have permissions to re-open the thread. Can you please open this?

Can you more clearly define the problem that is solved by reducing the number of segments that are created on low traffic shards? What is the use case that optimization is targeting?

In order to understand the use case, you need to understand how lucene parallelizes data writes across multiple segments within the same shard. Whenever data is to be written to Lucene buffer, it tries to get a buffer object from a list of pending buffer objects and ties it to the thread. If it does not find one, a new buffer object is created and documents are added there. These buffer objects are then indepently flushed to a segment during refresh. If there are multiple concurrent threads writing data to Lucene, the system ends up creating more segments. These segments are then merged later which means that we end up spending more compute. Searches also do not work well on lot of small segments.

@dreamer-89 Can you help re-open this?

@dreamer-89 Can you help re-open this?

Issue re-opened. @itiyama : Can you please share more details around the issue as requested previously.

Fixed by https://github.com/apache/lucene/pull/921

@msfroh The issue that you linked solves the problem of searching through more segments but utilizes more resources to do merging. The proposal here is to create fewer segments in the first place so that you do not spend resources on merging later.

Ahh... okay. I guess we can reopen it.
It feels like a pretty low priority, though, since writing and then merging small segments will (or at least should?) have negligible impact on overall performance.
If/when we move to predominantly pull-based indexing, we can allocate indexing threads per node based on the number of pending documents. (If each shard writes with one thread or less, then each shard will only ever write one segment per flush.)