Feature/plants model

Description
This integrates the plants model data into the animals model. The specific changes:

Adds a placeholder plants model canopy data array containing fictitious layer_leaf_mass by canopy layer. This is community wide and eventually we will likely need cohort specific cohort_leaf_mass to allow selective foraging. Either way, the plants model now populates something that can be eaten.
There is now a plant_resources submodule providing PlantResources. This is very like the old PlantCommunity in that it implements the Resource protocol from animal_model.protocols but it no longer has grow and die  methods, because they are never used.
PlantResources instances are created inside AnimalCommunity.forage_community method. They are no longer an instance attribute added at __init__, because they have to be updated from the Data instance at each time step.
As a result AnimalCommunity now accepts the data argument and stores that Data instance as an attribute so that forage_community can access the data and hence the plant resources can be populated.
At the moment, this only exposes that single layer_leaf_mass resource, but this will expand as the plants model adds more resources.  The leaf mass is currently summed to give a total mass to eat, but we can change that when foraging becomes vertically stratified.
There is no feedback of herbivory to the plant model. This needs fixing but it is a Milestone 2 extension to the plants model.

I've also:

Done a bit of docstring restructuring
Fixed up the tests
Moved the animals model specific test fixtures into models/animals/conftest.py to keep things tidy.

Type of change

[x] New feature (non-breaking change which adds functionality)
[ ] Optimization (back-end change that speeds up the code)
[ ] Bug fix (non-breaking change which fixes an issue)

Key checklist

[x] Make sure you've run the pre-commit checks: $ pre-commit run -a
[x] All tests pass: $ poetry run pytest

Further checks

[ ] Code is commented, particularly in hard-to-understand areas
[ ] Tests added that prove fix is effective or that feature works

How far off are you on the mass re-org?

I've parked the canopy dataclass suggestion for now - I think it is likely to all change fairly soon.

Codecov Report

Merging #322 (bc4e8f9) into develop (f02b0d0) will increase coverage by 0.25%.
The diff coverage is 100.00%.

:exclamation: Current head bc4e8f9 differs from pull request most recent head 84aec8c. Consider uploading reports for the commit 84aec8c to get more accurate results

@@             Coverage Diff             @@
##           develop     #322      +/-   ##
===========================================
+ Coverage    96.54%   96.79%   +0.25%
===========================================
  Files           48       48
  Lines         2400     2530     +130
===========================================
+ Hits          2317     2449     +132
+ Misses          83       81       -2

Files
Coverage Δ

virtual_rainforest/models/animals/__init__.py
100.00% <ø> (ø)

...al_rainforest/models/animals/animal_communities.py
97.61% <100.00%> (+0.08%)
:arrow_up:

virtual_rainforest/models/animals/animal_model.py
100.00% <ø> (ø)

...rtual_rainforest/models/animals/plant_resources.py
100.00% <100.00%> (ø)

virtual_rainforest/models/animals/protocols.py
93.33% <ø> (ø)

virtual_rainforest/models/plants/canopy.py
94.54% <100.00%> (+0.31%)
:arrow_up:

virtual_rainforest/models/plants/plants_model.py
92.77% <100.00%> (+0.08%)
:arrow_up:

... and 4 files with indirect coverage changes
:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


VW MQB: Add FW for 2022 Volkswagen Polo

Add missing firmware for the 2022 Volkswagen Polo, from the South African market.
Route: 14da75f0b8f732e2|2023-04-27--15-15-53
Thanks to community Polo owner zamv7r7vo!

That's awesome! Thank you very much!


Organoid data from PMLB

Description
As a PDCM FIder user, I would like to see organoid data from PMLB in PDCM FInder.
Acceptance criteria

[x] call with PMLB about the data
[x] plan for deposition

Templates sent.


error when table namespace (schema) contains a hyphen (-)

I build Laravel project, test environment had schema was named "web-test" in postgresql. I have set database search_path to "web-test", everything worked in Laravel except changing column definitions. Migration code contains this line:
$table->text('notes')->nullable()->change();

Migration gives error:
[Doctrine\DBAL\Schema\SchemaException]
There is no column with name 'notes' on table 'reports'.

There is a column named notes in the table. I have checked Doctrine\DBAL\Schema\Table why does it throw Doctrine\DBAL\Schema\SchemaException. Dumped $_columns property in method hasColumn but it was an empty array.
I have changed schema name to "webtest" and renaming column worked.

I don't see web-test in any of the examples above.
A few questions:

is this an actual ORM bug? Are you using the ORM or just DBAL?
can this be reproduced with just the DBAL test suite?

Schema name "web-test" does not appear in the code because it comes from environment variable. It is also not visible in the error messages because DBAL gets only the table name "reports" without schema.
Laravel uses only DBAL, now I found https://github.com/doctrine/dbal repository, I think this issue should be reported there. Agree?

Yes, but we need more details on how to reproduce it outside the context of laravel.
Closing here meanwhile.

I have made tests in DBAL, localized the problem, made a solution that satisfies tests and created a pull request.

@kolesar-andras thanks!


STORM-1725: Kafka Spout New Consumer API - KafkaSpoutRetryExponential Backoff method should use HashMap instead of TreeMap not to throw Exception

Replace TreeMap with HashMap

This issue was found by @jianbzhou and credit should be given to him (for discussion details see https://github.com/apache/storm/pull/1131#issuecomment-213211034)

+1


Error transforming debug

Haven't looked into this at all, just opening a corresponding issue to https://github.com/visionmedia/debug/issues/468

Sometimes instead of:
exports = module.exports = function Something () { … }

exports.is = function (smth) { … } /* «static» member */

I do:
var Something = module.exports = function Something () { … }

Something.is = function (smth) { … }

for better readability.

@Rich-Harris, hey, need help here? I can prepare a PR but it may take some time for me to dive into codebase.
For some reason my usual workaround for this obstacle didn't work well in debug's case and I don't know why. Maybe it's time to solve that centralized in this plugin.

I have the same error, any idea?

Hey folks (this is a canned reply, but we mean it!). Thanks to everyone who participated in this issue. We're getting ready to move this plugin to a new home at https://github.com/rollup/plugins, and we have to do some spring cleaning of the issues to make that happen. We're going to close this one, but it doesn't mean that it's not still valid. We've got some time yet before the move while we resolve pending Pull Requests, so if this issue is still relevant, please @ me and I'll make sure it gets transferred to the new repo. :beer:

@shellscape this is definitely worth to check after moving into new repo. Some (popular) cjs modules may accidentally «not work» because of not straghtforward cjs exports. But Rich once said the heuristics to detect it is totally doable on Rollup side.

@StreetStrider good to know. Should that be a concern for this plugin, or for Rollup core? (I can transfer this issue to the rollup core repo as well)

@shellscape I think it's in rollup-plugin-commonjs responsibility, but I might be wrong.
visionmedia/debug#468 got a very detailed symphtoms on this. They fixed their usage of cjs, but it's a pretty common pattern for cjs exports.


Create tagged release

Hi!
I put it on the Arch User Repository for you: bir-git.
Maybe you could create a tagged release, so I can also put up a  "Stable" version instead of only a *-git version.

Thanks for pointing that out. I created one.

Stable is also on the AUR now: bir!

Great thanks!
On Mon, May 11, 2015 at 5:38 PM, Tom van der Lee
<EMAIL_ADDRESS>wrote:

Stable is also on the AUR now: bir!
—
Reply to this email directly or view it on GitHub.


Markdown parser not convert formatting characters without whitespace characters around them

It seems that the markdown parser for the user guide does not apply formatting characters properly. My guess is that they need whitespace characters around them.
For example, on https://docs.cryptpad.org/ja/user_guide/user_account.html#user-account, you should see そのため**サービスの管理者に、あなたのパスワードを閲覧、取得、リセットすることはできません**。 in the first paragraph, as the formatting characters ** are escaped. Since this kind of issue does not happen somewhere else, I assume this is a bug of the markdown parser.

Since Japanese (like Chinese and Korean) language does not have the concept of whitespace character inside a sentence, it should appear just as "そのためサービスの管理者に、あなたのパスワードを閲覧、取得、リセットすることはできません。" without whitespace between ** and characters.

Hello, the bug with the Markdown parser is that it's actually a reStructuredText parser :) and RST is a lot more stringent when it comes to whitespace and syntax quirks. I think you can achieve what you want by escaping at the character level.
Eventually I'd like to move the documentation to full Markdown or Myst but I can't say when we will have time to dedicate to this.

I see. Adding escape in each case requires a lot of tasks, and Markdown is definitely preferred to RST when it comes to compatibility with Asian languages.


Fixed incomplete Arr import on Login.php

The import use Arr; used at line 50: $user = $model::where(Arr::only($data, 'email'))->first(); was causing a class not found error.
Changed to use Illuminate\Support\Arr

Thanks for this PR @brunovt074


Python API

Break free from pandas

Top level class
Design chart API similar to Altair
Synchronize the python object with widget

Fixed #103


Face normal direction handling/selection on import

Prompted by the discussion in #28 .
Vertex coordinates are always provided in an $\hat{x}\hat{y}$ coordinate system, but there is an arbitrary choice for whether face normals should point in the direction $\hat{z} = \hat{x}\times \hat{y}$ or in the direction $-\hat{z}$, and either could be valid. Currently, flat-folder only works for FOLD format crease patterns whose face normals point in the direction $\hat{z}$ direction, and does not work for when face normals point in the direction $-\hat{z}$. We should handle that case when importing a FOLD file containing faces_vertices, and allow a user to select the face normal direction when importing a crease pattern without face information (SVG, OPX, CP, FOLD without faces_vertices).

Here is some commentary on the above commit:
Importing FOLD files containing face_vertices

For internal representations and for output FOLD-format crease patterns, flat-folder adopts the convention that the direction of crease pattern face normals is in the $\hat{z} = \hat{x}\times\hat{y}$ direction, where $\hat{x}$ and $\hat{y}$ are the unit vectors in vertex coordinate space.
On import, if a FOLD file does not adhere to this convention, the arrays in faces_vertices are reversed, and mountains and valleys in edges_assignment are swapped.
Notably, FOLD files output by Oriedita direct face normals in the $-\hat{z}$ direction, which was problematic for flat-folder prior to this commit (in particular in crease pattern text rendering).
Also by convention, flat-folder renders the color side of the paper in the direction of $\hat{z}$.
Because flat-folder renders the white side of the paper when visualizing the crease pattern, the crease pattern for a FOLD file with face normals pointing in the $\hat{z}$ direction will appear reversed; i.e., flat-folder takes the perspective that a FOLD file with face normals pointing in the $\hat{z}$ direction was drawn while looking at the color side.
Conversely, flat-folder takes the perspective that a FOLD file with face normals pointing in the $-\hat{z}$ direction was drawn while looking at the white side. So on import, the crease pattern is not rendered in reverse (importantly, FOLD files exported by Oriedita will look the same when rendered by flat-folder by default).

Importing files that do not contain face information

By default, flat-folder now assumes that input crease patterns without face information have crease assignments that are defined when looking from the $-\hat{z}$ direction (i.e., from the white side). Note that this is the reverse of the previous default behavior. This may seem counter-intuitive, but this convention allows flat-folder to render crease patterns stored in CP and OPX files in the same way as in Oriedita and Oripa when rendering the white side of the paper.

Changing the default import behavior

flat-folder now has a new button in the interface, specifically a "+" button next to file import which will toggle to "-" when clicked.
"+" will default to the import behavior above.
Toggling to "-" before import the crease pattern will take the opposite perspective from above when reading in a file. In particular:

for a FOLD file containing face_vertices, the importer will assume face normals point in the $-\hat{z}$ direction (taking the perspective that the crease pattern was drawn while looking at the white side); and
for a file without face information, the import will assume crease assignments are defined when looking from the $\hat{z}$ direction (i.e., from the color side).


Provider improvements / fixes

Codecov Report

Merging #95 (7293990) into develop (e7c8dfd) will increase coverage by 0.87%.
The diff coverage is 63.63%.

@@             Coverage Diff             @@
##           develop      #95      +/-   ##
===========================================
+ Coverage    37.44%   38.31%   +0.87%
===========================================
  Files           39       40       +1
  Lines         2812     2808       -4
===========================================
+ Hits          1053     1076      +23
+ Misses        1655     1630      -25
+ Partials       104      102       -2

Flag
Coverage Δ

unittests
38.31% <63.63%> (+0.87%)
:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

pkg/provider/pulumi/azure/containerapp.go
0.00% <0.00%> (ø)

pkg/stack/types.go
8.92% <0.00%> (-0.63%)
:arrow_down:

pkg/provider/pulumi/aws/aws.go
67.14% <100.00%> (+3.72%)
:arrow_up:

pkg/provider/pulumi/aws/lambda.go
74.10% <100.00%> (+1.56%)
:arrow_up:

pkg/provider/pulumi/azure/azure.go
9.74% <100.00%> (+9.74%)
:arrow_up:

pkg/provider/pulumi/azure/resourcename.go
100.00% <100.00%> (ø)

pkg/runtime/golang.go
26.15% <100.00%> (+1.15%)
:arrow_up:

pkg/stack/options.go
64.44% <100.00%> (+4.89%)
:arrow_up:

pkg/utils/strings.go
100.00% <100.00%> (ø)

... and 2 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e7c8dfd...7293990. Read the comment docs.


Dynamic LINQ to XML

I am trying to use Dynamic LINQ Expressions with LINQ to XML, but having troubles with dynamic parameters. E.g. for the following structure:
<DataCenter>
   <Server IP="<IP_ADDRESS>">
       <OS>Windows</OS>
   </Server>
   <Server IP="<IP_ADDRESS>">
       <OS>Linux</OS>
   </Server>
</DataCenter>

I am trying to read and parse it like this:
XElement XmlSource = XElement.Load(filePath)
var query = XmlSource.Elements().AsQueryable().Select("???????");

What do I put as a string parameter? For example if I try to retrieve IP and OS of all servers. When I try to use Element( ) or Attribute( ) methods of XElement I get error that XElement doesn't have such property or method.
Though firstAtteibute and firstNode seems available. And something like this works:
    var query = XmlSource.Elements()
             .AsQueryable()
             .Select("new (FirstAttribute.Value as IP, FirstNode.toString() as OS)");

I found that similar issue was solved by updating predefinedTypes, Could it be done here?

There should be a configuration method to add additional types to the accessible types.

I am looking at System.Linq.Dynamic.Core.CustomTypeProviders, which I think might be the right place, but can't figure it out.

There's also System.Linq.Dynamic.Core.Parser.PredefinedTypesHelper but it's an internal class

I added an answer to your StackOverflow question. I also added an issue suggesting documenting ParsingConfig.CustomTypeProvider :)

@NetMage Once again, thank you so much. This is exactly what I was looking for. Linking the answer here in case anybody else needs it.


Option for chosing base on push of merge

https://github.com/josh-project/josh/blob/f67c3709e36d7ed192bc93018f62ecb9859cfa85/src/history.rs#L334
Maybe it'd be possible to select which of the bases of the merge should be used as a base when pushing? This would allow to push merges through filters

This is solved by #524


[BUG] I've installed Bloxstrap many times but I still don't see it.

Acknowledgement of preliminary instructions

[X] I have read the preliminary instructions, and I am certain that my problem has not already been addressed.

What problem did you encounter?
I have installed it many times, but I sitll don't see it in my apps.

but I sitll don't see it in my apps.

Are you looking in the Start menu? If you're searching for it, look for it in the list yourself, and it should be there.

I did that but it still didn't work. But I found the problem, I wasen't
installing the newest version, and I installed the newest version and it
worked and I have it now. Thank you though.
On Sun, Mar 17, 2024 at 5:32 AM pizzaboxer @.***> wrote:

but I sitll don't see it in my apps.
Are you looking in the Start menu? If you're searching for it, look for it
in the list yourself, and it should be there.
—
Reply to this email directly, view it on GitHub
https://github.com/pizzaboxer/bloxstrap/issues/1460#issuecomment-2002420317,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/BHA3ET6NNCOSWQC354LXYG3YYV5L3AVCNFSM6AAAAABEZUE7CKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMBSGQZDAMZRG4
.
You are receiving this because you authored the thread.Message ID:
@.***>


SQL Anywhere: CURRENT TIMESTAMP as an alias for CURRENT_TIMESTAMP

Search first

[X] I searched and no similar issues were found

Description
While most RDBMS know CURRENT_TIMESTAMP, SQL Anywhere does not. Instead, it solely knows CURRENT TIMESTAMP (note the missing underscore replaced by blank).
For improved portability of changelogs, Liquibase should handle CURRENT TIMESTAMP as an alias for CURRENT_TIMESTAMP.
Steps To Reproduce
CREATE TABLE T (c TIMESTAMP CURRENT TIMESTAMP)

liquibase generateChangelog

Expected/Desired Behavior
<createTable name="T">
  <column name="c" type="TIMESTAMP" defaultValueComputed="CURRENT_TIMESTAMP"/>
</createTable>

### Liquibase Version

master

### Database Vendor & Version

SAP SQL Anywhere 17.0.11

### Liquibase Integration

N/A

### Liquibase Extensions

N/A

### OS and/or Infrastructure Type/Provider

Windows 10 Pro (German)

### Additional Context

N/A

### Are you willing to submit a PR?

- [X] I'm willing to submit a PR (Thank you!)

Similar problems exist for current user, current date and current time.


A few small updates for logging and testing.

A few small updates for logging and testing. Each commit has the details.

Why do you need the first commit ?

Why do you need the first commit ?
I did have a commit which took one away from the height.
I may not use it but isn't it better to make sure any subtraction is not going to wrap around below zero?


[Suggestion] Updater

Is your feature request related to a problem? Please describe.
It can be annoying sometimes when you're still on an oudated version.
Describe the solution you'd like
Create an update button in the settings page. So you still have a choice if you want to download/install the update or not.
Describe alternatives you've considered
An autoupdater maybe, but not that many people like autoupdaters.

Hi @KnuffelBeestje,
The idea of an autoupdater (or in any case of an update notification) had already occurred to me some time ago. At first I used the already included functionality of electron-builder with the help of TravisCI. The only problem was that in those days Travis had started to limit the number of operations that could be performed monthly and since for each commit there are three builds (Mac/Windows/Linux) just for testing (in one day) I had already burned 3000 of the 10,000 guaranteed as a test.
Then I tried another free service (UpdateRocks) and created the updater.js file. However, this method requires you to change the type of installer from NSIS to Squirrel, the latter being considered "deprecated" by electron-builder. I'll try to do some testing and see if Squirrel can be an alternative, otherwise I'll make do and write a script for it.

[x] New branch: adding-autoupdater
[x] Move from NSIS to Squirrel (Windows)
[ ] Create loading animation for installer
[ ] Test with GitHub Releases


Managing Dependent Resource Create/Update and Matching with Server Side Apply

The problem:
Currently we are using post/put operations to create and update resources in dependent resources  if needed. And we also introduce various way to match if the desired state is the same as the actual state from the server. In theory in some cases would be enough just to compare if the spec (and label and/or annotations) are same on the desired and actual to match the resources.  However Kubernetes fills some default values and/or mutation hooks and other controllers can also fill some additional fields in a resources, thus simple compare in those cases won't work, also replacing the spec during the update would remove those (would be again added, but that is not that nice).  So we introduced additional enhanced mechanisms, like just check if there were some additions on server side compared to desired state. This works mostly, however it causes a problem when some wants for example delete a list item from a desired resource, in that case it will still match the server since the diff looks like that just values added. Further possibilities were presented to the user, like ignore lists, but those needs to be explicitly configured.
Solution:
For create and update we can use SSA, this will make sure that we update all the target fields we manage / care about. This is basically trivial to implement.
What we can do however is to implement the matching also based on SSA  managedFields:
We just compare the desired resource to the resource from the server but only the fields which are managed by us. So if the managed fields (by controller) of the actual resource from server  are identical with compared desired fields of a resource, we consider it as a match.
The problem with this matching approach is that it quite hard to prune the actual resource based on the managed field on client side, mainly because k8s lacks documentation of the managed fields structure.

Related issues:
https://github.com/java-operator-sdk/java-operator-sdk/issues/1914
https://github.com/java-operator-sdk/java-operator-sdk/issues/1907
https://github.com/java-operator-sdk/java-operator-sdk/issues/1888


Forward port some improvements from the backport of this to 0.5

@mesosphere-ci run all integration tests


Allow to influence "damage" of results

Use case: visualization of partial results  with kind='single' and possibly custom shape, but the UDF knows what parts of it are valid. Examples are SSB/WDD or possibly iCoM.

Right now, there's a single damage attribute on UDFResults, which is always nav-shaped and represents the indices of the navigation space that are already contained in the results. This can be used, for example by the plotting functionality, to limit kind='nav' plots to valid areas.
In addition, we need to make available a valid_data_mask for each result buffer.
By default, the valid_data_mask should:

Equal the damage for kind='nav' buffers
Equal 1 for kind='sig' and kind='single' buffers

Then, UDF.get_results needs to be able to influence this to build buffers that have custom validity. I think it makes sense to allow overriding the defaults for all kinds. For example, with a kind='nav' buffer, there may be border effects in the calculation that make intermediate results invalid at the border, which we want to cut off until the full result is available. We need to figure out how the API for this should look like.
We need to be careful to also consider the roi - is the valid_data_mask in the flattened shape if a roi is applied? Is it in respect to raw_data or in respect to data? I think we need to make up some example uses to figure this out.
This per-result-buffer valid data mask then needs to be consumed by the visualization (for proper vmin/vmax determination), and possibly by other downstream users, like LiberTEM-live-server.
TODO

[ ] make the damage available to UDFs (see #1593)
[ ] Figure out how to handle rois
[ ] API for customizing valid_data_mask from get_results

[ ] method/function name bikeshedding
[ ] which namespace does it live in? UDF.<something>? Is it common enough to warrant a place there? Or is this something people would be happy to import if they need it?

[ ]  API for customizing valid_data_mask from get_results

[ ]  method/function name bikeshedding
[ ]  which namespace does it live in? UDF.<something>? Is it common enough to warrant a place there? Or is this something people would be happy to import if they need it?

I've pushed an initial design to #1593 in form of a (failing) test case. As discussed with @uellue, this uses masked arrays for both influencing the result and for data consumers using the masked_data property. This is backwards-compatible, and data and raw_data should behave exactly as before.
If we do it like this, there's actually no API change needed on the UDF side, just the addition of masked_data on the BufferWrapper.
We might want to think if we want to depend 100% on masked arrays in the interface, or if we want to make a thin wrapper around it such that we can switch out the implementation. I'm tending towards just accepting our fate and exposing the masked arrays as our API.


raceshape.com no longer works

it reports "server error" when acessing either the link for export, or accessing directly to http://wwwhttp://raceshape.com/
I believe it was bought by strava

It has now a different error which doesn't seem any better :

Warning:  file_put_contents(/vol/cache/rides/1488/243/full.json): failed to open stream: No such file or directory in /www/raceshape/shape/includes/fileio.inc.php on line 78 Warning: chmod(): No such file or directory in /www/raceshape/shape/includes/fileio.inc.php on line 79 Warning: file_put_contents(/vol/cache/rides/1488/243/extra.json): failed to open stream: No such file or directory in /www/raceshape/shape/includes/fileio.inc.php on line 78 Warning: chmod(): No such file or directory in /www/raceshape/shape/includes/fileio.inc.php on line 79 {"error":"Strava not available"}

I re-developed the feature by myself.
You can try with a test build here: https://ci.appveyor.com/project/thomaschampagne/stravistix/branch/develop/artifacts
How to install: https://github.com/thomaschampagne/stravistix/wiki/How-to-install-stravistix-build-archive
(I'm able to see downloaded courses on my edge 1000)

I exported and uploaded a gpx to my fenix 5. The map and elevation is shown correctly, as well as distance and time of the effort.
I still have not tried it on a ride but I'm confident that it will work well.
Thank you Thomas !

I tried to export this effort and it has a javascript error :
https://www.strava.com/activities/158017059/segments/3657729681
VirtualPartnerModifier.js:112 Uncaught TypeError: Cannot read property '0' of undefined
at VirtualPartnerModifier.genGpxData (VirtualPartnerModifier.js:112)
at VirtualPartnerModifier.createGpxAndSave (VirtualPartnerModifier.js:85)
at VirtualPartnerModifier.js:80
at VacuumProcessor.getActivityStream (VacuumProcessor.js:158)
at VirtualPartnerModifier.js:74
at Object. (VirtualPartnerModifier.js:52)
at Object. (strava-head-15339e2….js:2)
at c (strava-head-15339e2….js:2)
at Object.fireWith [as resolveWith] (strava-head-15339e2….js:2)
at n (strava-head-15339e2….js:3)

I saw that you fixed this issue in your last commit actually.
it was quick.


How do I easily calculate the determinant of the Jacobian when it's a pytree of pytrees?

I would like to test that the determinant of the Fisher information of an exponential family is positive, but it's not easy to do that since the Jacobian returns a pytree of pytrees.  As it is, jacfwd and jacrev already calculate the Jacobian as an array before "unravelling" it into the pytree of pytrees.
I'm not sure what the best design is, but these seem reasonable?

I could try to flatten that pytree-of-pytrees into an array, which would mean stacking arrays horizontally and vertically using the treedefs?
With an appropriate flag, the determinant could be calculated by jacrev or jacfwd and returned?
With an appropriate flag, the Jacobian could be returned as an array, and I could call jnp.linalg.det on that?
Maybe there's a way to calculate the determinant of a pytree-of-pytrees?

Thanks for the question - I'm having trouble understanding the context in which you're seeing jacobians return a pytree of pytrees. Can you show a concise example?

@jakevdp I don't know how concise this is, but my application is simply to find the Fisher information here, which is just
jacfwd(grad(type(self).log_normalizer))

where log_normalizer is a scalar method of a distribution like this one, essentially something like:
@dataclass
class Normal:
    mean: RealArray
    variance: RealArray

    def log_normalizer(self) -> RealScalar:
        ...

The problem is that some distribution have both real and complex parameters.

Ah, I see. The issue is that you're computing the jacobian with respect to your dataclass type (self here). If you want to compute the jacobian with respect to an array, then you should pass jacfwd a function which takes that array as an argument, rather than a function that takes your dataclass as an argument. Does that make sense?

@jakevdp Thank you, yes.  However, I don't just want the Fisher information of the individual parameters (which are the arrays).  I want the Fisher information of all of the parameters.  I think that's a reasonable thing to calculate.

Although I think you just gave me an idea of how to work around this problem.  I already have functions to flatten and unflatten the structure.  I could define a log_normalizer_flat that works on the flattened version and calculate the Hessian of that.  Then, I would get the Fisher information as a matrix as desired.  So I guess that solves my problem !

I guess I'm still not understanding what's happening. For example, here is a simple version of compuing the forward jacobian of a pytree, and I get a pytree of results:
import jax
import jax.numpy as jnp
from typing import NamedTuple

class Normal(NamedTuple):
  x: jnp.ndarray
  y: jnp.ndarray

  def func(self):
    return jnp.dot(self.x, self.y)

  gradfunc = jax.jacfwd(func)

val = Normal(jnp.arange(4.0), jnp.ones(4))
val.gradfunc()
# Normal(x=DeviceArray([1., 1., 1., 1.], dtype=float64), y=DeviceArray([0., 1., 2., 3.], dtype=float64))

Can you modify this example so that it results in the pytree-of-pytrees that you're seeing?

@jakevdp I think what you've missed is that I'm calculating the Jacobian of the gradient?

Ah, I missed that, thanks.
What if you did something like this? Does the non-nested version give you the result you're interested in?
import jax
import jax.numpy as jnp
from typing import NamedTuple

class Normal(NamedTuple):
  x: jnp.ndarray
  y: jnp.ndarray

  def func(self):
    return jnp.dot(self.x, self.y)

  gradfunc_nested = jax.jacfwd(jax.grad(func))

  def gradfunc_not_nested(self):
    def gradfunc(x, y):
      return self.__class__(x, y).func()
    return jax.jacobian(jax.grad(gradfunc))(self.x, self.y)

val = Normal(jnp.arange(4.0), jnp.ones(4))

print(val.gradfunc_nested())
# Normal(x=Normal(x=DeviceArray([[0., 0., 0., 0.],
#              [0., 0., 0., 0.],
#              [0., 0., 0., 0.],
#              [0., 0., 0., 0.]], dtype=float64), y=DeviceArray([[1., 0., 0., 0.],
#              [0., 1., 0., 0.],
#              [0., 0., 1., 0.],
#              [0., 0., 0., 1.]], dtype=float64)), y=Normal(x=DeviceArray([[1., 0., 0., 0.],
#              [0., 1., 0., 0.],
#              [0., 0., 1., 0.],
#              [0., 0., 0., 1.]], dtype=float64), y=DeviceArray([[0., 0., 0., 0.],
#              [0., 0., 0., 0.],
#              [0., 0., 0., 0.],
#              [0., 0., 0., 0.]], dtype=float64)))

print(val.gradfunc_not_nested())
# [[0. 0. 0. 0.]
#  [0. 0. 0. 0.]
#  [0. 0. 0. 0.]
#  [0. 0. 0. 0.]]

I think your idea is along the lines of flattening and then taking the Hessian of the flattened structure.  However, in what you wrote grad is only taking the gradient of x since it defaults to taking the gradient of the first argument only.

You can pass argnums=(0, 1) if you want to take the gradient with respect to the first two arguments

Yes, but then but then you'll have some kind of non-array structure again, right?

Right - computing the jacobian with respect to two parameters will give you a tuple of the two jacobians.

It's just a shame because I know that the array I want is being calculated in jacfwd and jacrev.  But I should be able to get this working by flattening first and we can close this 😄   Thanks for your help.

Can you write down mathematically what you're hoping to compute? I'm not certain I've entirely grasped that yet.

@jakevdp It's the Fisher information of an exponential family, which is equal to the Hessian of the log-normalizer.  If you look here, it's equations 53–56.

Ah, I see. Am I right in interpreting that $\eta$ in that equation is the vector of relevant parameters, which in your case would contain both the mean and the variance?

@jakevdp Yes, you got it!

Ah, great!
In that case I'd recommend something along these lines:
import jax
import jax.numpy as jnp
from typing import NamedTuple

class Normal(NamedTuple):
  x: jnp.ndarray
  y: jnp.ndarray

  def func(self):
    return jnp.dot(self.x ** 2 + self.y, self.y)

  def gradfunc(self):
    def wrapped(eta):
      x, y = eta[:len(eta) // 2], eta[len(eta) // 2:]
      return self.__class__(x, y).func()
    eta = jnp.concatenate([self.x, self.y])
    return jax.hessian(wrapped)(eta)

val = Normal(jnp.arange(4.0), jnp.ones(4))

val.gradfunc()
# DeviceArray([[2., 0., 0., 0., 0., 0., 0., 0.],
#              [0., 2., 0., 0., 0., 2., 0., 0.],
#              [0., 0., 2., 0., 0., 0., 4., 0.],
#              [0., 0., 0., 2., 0., 0., 0., 6.],
#              [0., 0., 0., 0., 2., 0., 0., 0.],
#              [0., 2., 0., 0., 0., 2., 0., 0.],
#              [0., 0., 4., 0., 0., 0., 2., 0.],
#              [0., 0., 0., 6., 0., 0., 0., 2.]], dtype=float64)

Since you're interested in the hessian with respect to the full parameter vector, you can compute your gradients from a function that accepts that vector.

Yup, that makes sense!  Let's close this then.  Thank you for your help.


Postgres Deployment post-task fails when external database is used

The post task is missing a when clause, so it is run even when Postgres is not installed because repo_db_url or sync_db_url where provided.
https://github.com/Alfresco/alfresco-ansible-deployment/blob/ecc58887a6072b5613ccd16048a2de1e4c60297a/playbooks/acs.yml#L44

internally tracked in OPSEXP-2407

The error actually triggers as you probably still have a database host set in your inventory file while also providing external urls.

@jalvarezferr can you confirm that removing every host from the database group in your inventory solve the issue?

Delayed fix to OPSEXP-2408

I was planning to test the external DB today and I just found your issue so please allow me to participate here.
My understanding of the external DB is the following one: when defining yourself repo_db_url, the playbook is supposed to not use the "internal" PG but instead use the following properties to setup Alfresco:
repo_db_name: "alfresco"
repo_db_username: "alfresco"
repo_db_driver: "org.postgresql.Driver"
repo_db_url: "jdbc:postgresql://<host>:<port>/{{ repo_db_name }}"

When using this repo_db_url, you are indeed supposed to not have any host as part of the group database, otherwise there could be some side effects. The role for PG installation will still be executed if sync_db_url is empty (default), even if there is no syncservice group member, which isn't optimal. I quickly checked above #720 link and it seems that this was taken into account but isn't yet in the master branch. So that means for now, you should either not have anything in the database group OR define both repo_db_url AND sync_db_url to something not empty, otherwise the postgres role will be executed.
I tested on my side and for me, it works properly when the database role is empty with the following properties defined:
repo_db_host: "DB_HOST_VIP"
repo_db_port: "5432"
repo_db_name: "alfresco"
repo_db_username: "alfresco"
repo_db_driver: "org.postgresql.Driver"
repo_db_url: "jdbc:postgresql://{{ repo_db_host }}:{{ repo_db_port }}/{{ repo_db_name }}"

sync_db_host: "{{ repo_db_host }}"
sync_db_port: "{{ repo_db_port }}"
sync_db_name: "alfresco-sync"
sync_db_username: "alfresco-sync"
sync_db_driver: "org.postgresql.Driver"
sync_db_url: ""

There is only 1 issue I faced related to the driver but I will open a PR for that.

@jalvarezferr can you confirm that removing every host from the database group in your inventory solve the issue?

Hi. Sorry I was absent from the thread. The notifications got lost in my email.
I see the update done and applied to 2.4.2, but I believe the issue persists as in the scenario that I experienced with 2.3.0, I was using the included inventory-localhos.yml file for a single sever deployment, where no database hosts are defined. Yet, the post-task was run anyway and caused the error.
I'll test with 2.4.2 as soon as I can adopt the new version in my automations.

Hi @jalvarezferr
Actually the inventory file we ship as inventory_local.yml DO contain a database host. It is specificied as all.children.database.children.repository which means that any host that's memeber of repository is member of database, and repository group has one host: localhost. you need to remove all.children.database.children to clear out any database host

Thanks. That's very true. However seems that the code uses two different ways for the same purpose. On one hand the role has the when clause depending on the jdbc url vars to be set or not, and on the other hand the post_tasks rely on the inventory.  Even if the JDBC urls are there for a side case I have not identified, then the post-taks would need a when clause too. Or neither of them do. Isn't it?

As stated by @gionn this code needs refactoring. These repo_db_url & sync_db_url are legacy vars we want to get rid of and align external database handling to other external services (through inventory).
This is work we can only undertake for 3.0 as this would break the configuration interface used IN 0.x, 1.x & 2.x


Open Images & Conceptual Captions

This is a nice work. When will the code/data for Open Images & Conceptual Captions be released?
Thanks.

Hi @liujiaheng,
Thanks for reaching out and your interest in the project.
I will try to clean up the code by the end of this week.
Please let me know if you have any further questions.
Best,
Dat

In 1, the model is named as "model_pretrained.pth".
In 2, the model is "model_final.pth".
Should I just rename the "model_pretrained.pth" to "model_final.pth"?

Hi @liujiaheng ,
Thanks for spotting this.
I have fixed the name from the script to match with the downloaded file.
Moreover, I have included the pretrained models so you can quickly test the code out.
Please check out my latest commit and let me know if you have any comments.
Cheers!
Dat

Thanks for your update.
By default, which setting (Constrained or Generalized) of the testing results in your codebase?

In your paper, lr is set as 0.001.

However, the lr in config file, lr is set as 0.005.
Should I change the lr to 0.001?
Thanks.

When I train the student model, this error appeared.

Hi,
I also try to run the teacher training as follows:
python -m torch.distributed.launch --nproc_per_node=8 tools/train_net.py --config-file configs/coco_cap_det/zeroshot_mask.yaml OUTPUT_DIR ./checkpoint/mscoco_teacher/ MODEL.WEIGHT ./model_weights/model_final.pth
where ''model_final.pth'' is the downloaded version from the OVR ``model_pretrained.pth''.
After training, the results are very low as follows:

Hi @liujiaheng
Sorry for my late response.
Please find below my answers to your questions.

Thanks for your update. By default, which setting (Constrained or Generalized) of the testing results in your codebase?

We evaluate both constrained and generalized performance. This is specified in the config file as in the following line: https://github.com/hbdat/cvpr22_cross_modal_pseudo_labeling/blob/a8f77a60cf294409600fa877a9ce3f0db155ac3e/configs/coco_cap_det/zeroshot_mask.yaml#L40

  When I train the student model, this error appeared.

I haven't been able to reproduce this error. Could you but a break point there and inspect the error a bit (such as what idx makes it out of range? what is the size of features?)

Hi, I also try to run the teacher training as follows: python -m torch.distributed.launch --nproc_per_node=8 tools/train_net.py --config-file configs/coco_cap_det/zeroshot_mask.yaml OUTPUT_DIR ./checkpoint/mscoco_teacher/ MODEL.WEIGHT ./model_weights/model_final.pth  where ''model_final.pth'' is the downloaded version from the OVR ``model_pretrained.pth''. After training, the results are very low as follows:

I think you might use the incorrect pretrained model. The model_pretrained.pth from OVR is only trained for detection so it will definitely perform poorly on segmentation task.
Please download the corresponding pretrained model from #pretrained-models
Cheers!
Dat

Hi, @liujiaheng

  When I train the student model, this error appeared.
Do you solve this problem???


Run build fail

Describe the bug
A clear and concise description of what the bug is.
To Reproduce
Steps to reproduce the behavior:

Go to '...'
Click on '....'
Scroll down to '....'
See error

Expected behavior
A clear and concise description of what you expected to happen.
Screenshots
If applicable, add screenshots to help explain your problem.
System Info (if dev / build issue):

OS: [e.g. iOS]
Node version (please ensure you are using 12+)
Npm version

Browser Info (if display / formatting issue):

Device [e.g. Desktop, iPhone6]
Browser [e.g. chrome, safari]
Version [e.g. 22]

Additional context
Add any other context about the problem here.

Fixed


Review global admin requirement

Typically, office policy configuration does not require Global Admin, but also works with Office Apps Admin.
Details: https://docs.microsoft.com/en-us/deployoffice/overview-office-cloud-policy-service
This document however does not offer any alternative permissions and requires Global Admin.
I am quite sure Office Apps Admin will also work here, so kindly review the permission statement.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: c8d2ba49-5e22-3373-5034-bb2fde743762
Version Independent ID: 3a02dcf2-1310-561d-4106-d495eaec63b7
Content: Manage Microsoft feedback for your organization - Microsoft 365 admin
Content Source: microsoft-365/admin/manage/manage-feedback-ms-org.md
Service: o365-administration
GitHub Login: @KwekuA
Microsoft Alias: Kwekua

@MrAzureAD Thank you for your feedback. You are correct! The document has been updated and does not state the global admin requirement anymore.
Hope this helps!
Thanks
Sri

@msbemba Thank you very much for the contribution and sharing this explanation. @MrAzureAD Hope this comment is helpful for you. If you see a documentation update is required, please feel free to open an issue for the same. We proceed here to close it. Thanks for taking out some time to open the issue. Appreciate and encourage you to do the same in future also.


Use git best-practices: do NOT force-push into the main branch

IMO force-push into the main branch should not be allowed at all, for many reasons:

for one  the community/contributors here  need to all the time rebase their forks, PRs, patches
and it seems PRs got auto-closed because of this, including mine #2  :(
(Of course in feature branches it's OK to force-push , if it helps to keep the history cleaner etc)

Also in general would be nice if Cloudera does changes via  feature branches , as does the community

and we would almost need some CI/CD incl. basic integration tests for this (with the many changes and contributions)

We had to do it earlier to fix the commit messages. Going forward there won't be any force pushes.
Sorry for the trouble and thanks for bringing it up.

Every change goes through CI / CD & Testing is happening internally within cloudera.


Correct links

Fixes typos and links after moving files.

@fzipi nice, I like the level of detail and the things it is picking it up


Inconsistent results between the scr and app versions

The io500.sh script is running both versions and suprisingly with the same input files and striping options the end score is totally different. Is that a know issue ? If so which version should be trusted ?
Thank you

You should be running the "app" version from the IO500/io500 repo.
See the README.md file as updated by PR #28

ok sounds good. Thank you.


Sound: Play one-off sound, then stop audio thread again?

I would like to add some sound an application. However, I only need to play a short sound (a few seconds) every few minutes. When using rg3d-sound, the Context spawns a mixer thread in the background which consumes 100% cpu (of one core). Unfortunately, I cannot find a way to stop this thread again (not even if I execute std::mem::drop(context) after playing the sound).
Is there some way to create a Context in such a way that it automatically stops itself again after playing the sound, so that my application doesnt need that high CPU resources all the time?

Hello! It actually sounds like a bug in a backend. Which OS do you use?
I just checked how it works on Windows and single core load was around 2-5%.

Hi! Thanks for the quick response, given that its holiday season :)
I'm on Linux, so using the alsa backend.

Should be fixed now. Changes are in https://github.com/mrDIMAS/rg3d/commit/34591e9553ace577b8f2616ea5062a8ee947764a . Could you please confirm if CPU usage is ok now?

I checked on virtual machine and it was something around 0.6-1.0%, previously it was ~98%.

Thanks for the quick work! Regarding the fix: Well, yes but no?
Yes, the CPU load decreased significantly. However, it seems like it doesnt play any sound anymore? Maybe I'm just missing something because the API has slightly changed?
My code before:
// rg3d-sound = "0.18.0"
pub struct MyStruct {
    context: Arc<Mutex<Context>>,
    active_handles: Vec<(Instant, Handle<SoundSource>)>,
}

impl MyStruct {
pub fn play(&mut self, sound: MySound) {
    let data_source = sound.data_source();
    match SoundBuffer::new_generic(data_source) {
        Ok(sound_buffer) => {
            let source_builder = GenericSourceBuilder::new(sound_buffer)
                .with_status(Status::Playing)
                .with_play_once(true);
            match source_builder.build_source() {
                Ok(source) => {
                    match self.context.lock() {
                        Ok(mut context_lock) => {
                            let source_handle: Handle<SoundSource> = context_lock.add_source(source);

                            let expiry = Instant::now() + sound.duration();
                            self.active_handles.push((expiry, source_handle));
                        }
                        Err(err) => {
                            error!("Failed to play sound: {}", err);
                        }
                    }

                }
                Err(err) => {
                    error!("Failed to build source for {:?}: {}", sound, err);
                }
            }
        }
        Err(_) => {
            error!("Could not create SoundBuffer");
        }
    }
}
}

This correctly plays sounds but uses 100% CPU as described above.
My code afterwards:
// rg3d-sound = { git = "https://github.com/mrDIMAS/rg3d", rev = "34591e9" }
pub struct MyStruct {
    context: Context,
    active_handles: Vec<(Instant, Handle<SoundSource>)>,
}

impl MyStruct {
    pub fn play(&mut self, sound: MySound) {
        let data_source = sound.data_source();
        match SoundBuffer::new_generic(data_source) {
            Ok(sound_buffer) => {
                let source_builder = GenericSourceBuilder::new(sound_buffer)
                    .with_status(Status::Playing)
                    .with_play_once(true);
                match source_builder.build_source() {
                    Ok(source) => {
                        let mut state = self.context.state();
                        let source_handle = state.add_source(source);

                        let expiry = Instant::now() + sound.duration();
                        self.active_handles.push((expiry, source_handle));
                    }
                    Err(err) => {
                        error!("Failed to build source for {:?}: {}", sound, err);
                    }
                }
            }
            Err(_) => {
                error!("Could not create SoundBuffer");
            }
        }
    }
}

This consumes very few CPU but plays no sounds. Do I need to change something else as well (some other changes since 0.18.0)?

Note: I don't know if its a side effect or if it worked before already but (at least on my system) sounds also play flawlessly now in debug mode.

Cool. Yeah, I changed design a bit to support multiple contexts. I'm closing the issue then, it is good that you spotted this issue, thanks! I'll try to publish 0.19.0 on crates.io as soon as possible.


Do not require angle brackets in argument

From ryanartecona's comment on issue #4:

Having to specify the angle brackets in an argument lookup does feel like a mistake, though.

Thought so, too! That turned out to be trivial to fix.
If you merge this, I can open an issue so you don't forget to make a new release once you get back from your vacation. Have a good time!

For the sake of tying off loose ends:

I'm just curious: why do you think so?

By "bad CLI", I really meant "an inconsistent usage page". A usage page which either uses both ARG and <ARG> or both ARG and <arg> style of positional arguments could just as easily choose one of those styles to use throughout that usage page.
I personally prefer the <brackets> because it reads less ambiguously to me, but I know some others are more accustomed to UPCASE. Either way, there's no reason to mix them in these same usage page, so we shouldn't worry about supporting that in docopt.

Oh, I just misunderstood the meaning of "both cases" back then — I read it as if you meant "both arg and <arg> smell of bad CLI". Consider that particular loose end a false one :)


[Introduction] Clarify languages

Clarifies that languages are programming languages (ex: Java, Ruby, etc.) as opposed to spoken languages (English, Spanish, etc.)
Signed-off-by: Rob Dolin<EMAIL_ADDRESS>

Networks Convergence.


Width of editor is now more wider than before

Is the bug present when using the default Obsidian theme?
No
Is the bug present when snippets and plugins are disabled?
Yes
Minimal theme version
7.7.18
Describe the bug
Before the last updates, the editor size was less wider. I can't find the setting to fix that.
Is that related to a recent update?
Debug info
SYSTEM INFO:
Obsidian version: v1.7.1
Installer version: v1.6.3
Operating system: Darwin Kernel Version 23.5.0: Wed May  1 20:16:51 PDT 2024; root:xnu-10063.121.3~5/RELEASE_ARM64_T8103 23.5.0
Login status: logged in
Language: fr
Catalyst license: supporter
Insider build toggle: on
Live preview: on
Base theme: light
Community theme: Minimal v7.7.18
Snippets enabled: 0
Restricted mode: off
Plugins installed: 16
Plugins enabled: 14
1: Templater v2.4.2
2: Calendar v1.5.10
3: Smart Typography v1.0.18
4: Longform v2.0.7
5: Trash Explorer v1.2.2
6: Novel word count v3.10.2
7: Find orphaned files and broken links v1.10.1
8: Minimal Theme Settings v8.1.1
9: Rapid Notes v1.2.5
10: Hide Folders v1.5.1
11: Multi Properties v1.1.2
12: Share Note v0.8.15
13: Hider v1.5.1
14: Style Settings v1.0.9
RECOMMENDATIONS:
Custom theme and snippets: for cosmetic issues, please first try updating your theme and disabling your snippets. If still not fixed, please try to make the issue happen in the Sandbox Vault or disable community theme and snippets.
Community plugins: for bugs, please first try updating all your plugins to latest. If still not fixed, please try to make the issue happen in the Sandbox Vault or disable community plugins.

You can change this setting in Minimal Theme Settings → Typography → Normal line width
The issue is not a bug, but a fix related to Obsidian 1.7.1. This version of Obsidian fixes an issue that has been present for a long time: "REM units are now synced with the base font size."
Previously when you changed the main font size your line width setting no longer matched (approximately 40 characters by default).

Perfect. Thank you so much for your great work.


Add branch protection to project repositories

With the constant creation of new projects, especially those that are to be Open Source, branch protections are recommended to prevent code from slipping through.
Tasks

Replicate the branch rules within existing projects

Quite a few projects to get through at this point...oh joy.

All project repositories should have a RuleSet in place to manage Pull Requests.


Support Container Attachment

This PR contains the API changes to allow agents to request allocation of resources needed to attach a container to a swarm network.
Signed-off-by: Alessandro Boch<EMAIL_ADDRESS>

[Current coverage][cc-pull] is 54.47%

Merging [#1116][cc-pull] into [master][cc-base-branch] will decrease coverage by 0.36%

@@             master      #1116   diff @@
==========================================
  Files            77         76     -1
  Lines         11926      11845    -81
  Methods           0          0
  Messages          0          0
  Branches          0          0
==========================================
- Hits           6541       6453    -88
- Misses         4477       4481     +4
- Partials        908        911     +3

[][cc-pull]

Powered by Codecov. Last updated by [df20242...214e6d4][cc-compare]
[cc-base-branch]: https://codecov.io/gh/docker/swarmkit/branch/master?src=pr
[cc-compare]: https://codecov.io/gh/docker/swarmkit/compare/df202428aa690c44da202ab92396538ea692a3f6...214e6d40a04738b311ff2c85d0420f87e903034b
[cc-pull]: https://codecov.io/gh/docker/swarmkit/pull/1116?src=pr

Added the implemntation changes.
Tested these changes in addition to changes in docker/docker and libnetwork:
HOST:~$ docker swarm init
Swarm initialized: current node (egszn51lrykxh64u109efxfkh) is now a manager.
HOST1:~$ docker network create -d overlay swarm_nw1
14at7jybsjgosbbul0uxfamkt
HOST1:~$ docker network create -d overlay swarm_nw2
awzkaampy4glcvvssqzx4e4ge
HOST1:~$ docker run -d --name m0 --net swarm_nw1 alpine top
08a19de76aa6f369fcd66b5788a6502716fc94710bb45644a72df6fa9dd43053
HOST1:~$ docker service create --name svc  --network swarm_nw2 alpine top
05c1f6denhozog4hu91jgqnia
HOST1:~$ docker ps
CONTAINER ID        IMAGE               COMMAND             CREATED              STATUS              PORTS               NAMES
9407dfa06b94        alpine:latest       "top"               51 seconds ago       Up 50 seconds                           svc.1.9m6gxnh1csc05tkdimnllcyzi
08a19de76aa6        alpine              "top"               About a minute ago   Up About a minute                       m0
HOST1:~$

HOST2:~$ docker swarm join <IP_ADDRESS>:2377
This node joined a Swarm as a worker.
HOST2:~$ docker run -d --name w0 --net swarm_nw1 alpine top
230d446d545f2c15fcfa2f52eb7fde1cdb6108e3dc29df7b31e5cc36d13ccb2c
HOST2:~$ docker exec w0 ping -c 1 m0
PING m0 (<IP_ADDRESS>): 56 data bytes
64 bytes from <IP_ADDRESS>: seq=0 ttl=64 time=0.163 ms

--- m0 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 0.163/0.163/0.163 ms
HOST2:~$
HOST2:~$ docker network connect swarm_nw2 w0
HOST2:~$ docker exec w0 ping -c 1 svc.1.9m6gxnh1csc05tkdimnllcyzi
PING svc.1.9m6gxnh1csc05tkdimnllcyzi (<IP_ADDRESS>): 56 data bytes
64 bytes from <IP_ADDRESS>: seq=0 ttl=64 time=0.742 ms

--- svc.1.9m6gxnh1csc05tkdimnllcyzi ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 0.742/0.742/0.742 ms
HOST2:~$

@aboch thanks for getting this done.
one of the comments i heard from @aluzzardi & @stevvooe is to create a separate service (not to reuse dispatcher API).  ping @aluzzardi @stevvooe to confirm.
But I guess this PR can be reviewed for the functionality aspects.

I thought we had discussed breaking out network distribution into a Dispatcher method Networks. This would include a list for networks to create and a list of attachments that aren't bound to a task.
I'd prefer to make these changes correct, rather than trying to fit them in without the correct fixes. The current model is too messy and confusing.

@stevvooe
I set my initial goal to first  get the docker run and docker network connect work end to end on swarm networks.
With the notion that the further changes to have code comply with swarmkit flow and structures would be needed.
And that these would be additional, incremental changes, which can only be done right through the process of code review from maintainers.
Regarding your request, which I start to understand better now, of dispatcher listing any network attachment (for tasks or for user started containers), implies changes to the Task object I think, where now the agent side code would need to put together the task and the nw attachment before calling the engine/libnetwork code.
If the above is correct, this seems reasonable to me, so that meaningful objects have their indivual disatcher API.
But based of my current confidence around the swam code, this seems at first a big refactor change,  and  I am not that confident to have it part his PR.

But based of my current confidence around the swam code, this seems at first a big refactor change, and I am not that confident to have it part his PR.

My suggestions shouldn't result in a very large refactor. We just need to move a few fields around. The big change will be separating out the creation of networks from the main task flow. This will immensely simplify the object model of networks and tasks.
I don't think we can merge this functionality without changes here. Without doing so, we will leak networks under failure modes. Before this change, a networks life cycle was bound by the life cycle of the set of referencing tasks. With this change, we no longer have that lifecycle hook. Networks must have their own lifecycle.

@stevvooe
I dropped the ExecutorAttachment object and specs and reused the NetworkAttachment object and added a spec for it.
The dispatcher API now will give a stream of NetworkAttachment objects.
For now the stream will only include the attachments that were created from engine, to connect containers to swarm networks. But at least this should be inline with the future changes you had in mind where also the tasks' attachments will be dispatched via this stream.
Please take a look when you get a chance.
Thanks.

But at least this should be inline with the future changes you had in mind where also the tasks' attachments will be dispatched via this stream.

We need to make these fixes now. The current changes aren't the right model for GA and its clear the networking model is insufficient for our needs.

Before this change, a networks life cycle was bound by the life cycle of the set of referencing tasks. With this change, we no longer have that lifecycle hook. Networks must have their own lifecycle.

Networks today have their own lifecyle independent of Tasks.
I think you are referring to NetworkAttachment objects which lifecyle is in fact are bound to the Task.
With this change we are allowing engine to create its own NetworkAttachment objects for its own containers user wants to connect to the swarm networks. Yes, their lifecyle is bound to the user started containers. So now we would have NetworkAttachment objects managed by swarm (linked from Task lifecycle) and NetworkAttachment objects controlled by engine (bound to user container lifecyle).
But the only reason why we need swarm mgr to know about these attachments is just so that it can allocate the IP adresses, so that it won't give to tasks IPs which are being used by user started containers. We do not really need to pass this as NetworkAttachment, which is why initially I had a separate ContainerAttachment object.

@aboch Please rebase

@aluzzardi @icecrime  thanks to @aboch we now have a full e2e integrated PR : https://github.com/docker/docker/pull/25263

Is this PR superseded by #1373 ? @mrjana @aboch ?

@dperny Yes.


fix for nightly 2015-12-21

Now, I cannot build this library using nightly 2015-12-21.
build log followings:
% rustc --version
rustc 1.7.0-nightly (2b8e96dad 2015-12-21)
% cd diesel
% cargo build
   Compiling diesel v0.3.0 (file:///home/katsu/Program/Rust/diesel/diesel)
src/connection/mod.rs:214:26: 214:35 error: private type in public interface (error E0446), #[deny(private_in_public)] on by default
src/connection/mod.rs:214         PK: AsExpression<PkType<T>>,
                                                   ^~~~~~~~~
src/connection/mod.rs:211:30: 211:50 error: private type in public interface (error E0446), #[deny(private_in_public)] on by default
src/connection/mod.rs:211         T: Table + FilterDsl<FindPredicate<T, PK>>,
                                                       ^~~~~~~~~~~~~~~~~~~~
src/query_builder/select_statement/dsl_impls.rs:58:49: 58:54 error: private type in public interface (error E0446), #[deny(private_in_public)] on by default
src/query_builder/select_statement/dsl_impls.rs:58     SelectStatement<ST, S, F, W, O, LimitClause<Limit>, Of>: Query<SqlType=ST>,
                                                                                ^~~~~
src/query_builder/select_statement/dsl_impls.rs:60:63: 60:68 error: private type in public interface (error E0446), #[deny(private_in_public)] on by default
src/query_builder/select_statement/dsl_impls.rs:60     type Output = SelectStatement<ST, S, F, W, O, LimitClause<Limit>, Of>;
                                                                                ^~~~~
src/query_builder/select_statement/dsl_impls.rs:73:53: 73:59 error: private type in public interface (error E0446), #[deny(private_in_public)] on by default
src/query_builder/select_statement/dsl_impls.rs:73     SelectStatement<ST, S, F, W, O, L, OffsetClause<Offset>>: Query<SqlType=ST>,
                                                                                ^~~~~~
src/query_builder/select_statement/dsl_impls.rs:75:67: 75:73 error: private type in public interface (error E0446), #[deny(private_in_public)] on by default
src/query_builder/select_statement/dsl_impls.rs:75     type Output = SelectStatement<ST, S, F, W, O, L, OffsetClause<Offset>>;
                                                                                ^~~~~~
error: aborting due to 6 previous errors
Could not compile `diesel`.

I fix
% git diff origin/master
diff --git a/diesel/src/connection/mod.rs b/diesel/src/connection/mod.rs
index 274ecf6..1b9ada8 100644
--- a/diesel/src/connection/mod.rs
+++ b/diesel/src/connection/mod.rs
@@ -30,9 +30,9 @@ pub struct Connection {

 unsafe impl Send for Connection {}

-type PrimaryKey<T> = <T as Table>::PrimaryKey;
-type PkType<T> = <PrimaryKey<T> as Expression>::SqlType;
-type FindPredicate<T, PK> = Eq<PrimaryKey<T>, <PK as AsExpression<PkType<T>>>::Expression>;
+pub type PrimaryKey<T> = <T as Table>::PrimaryKey;
+pub type PkType<T> = <PrimaryKey<T> as Expression>::SqlType;
+pub type FindPredicate<T, PK> = Eq<PrimaryKey<T>, <PK as AsExpression<PkType<T>>>::Expression>;

 impl Connection {
     /// Establishes a new connection to the database at the given URL. The URL
diff --git a/diesel/src/query_builder/select_statement/dsl_impls.rs b/diesel/src/query_builder/select_statement/dsl_impls.rs
index 0522a62..7b7e74d 100644
--- a/diesel/src/query_builder/select_statement/dsl_impls.rs
+++ b/diesel/src/query_builder/select_statement/dsl_impls.rs
@@ -51,7 +51,7 @@ impl<ST, S, F, W, O, L, Of, Expr> OrderDsl<Expr>
     }
 }

-type Limit = <i64 as AsExpression<types::BigInt>>::Expression;
+pub type Limit = <i64 as AsExpression<types::BigInt>>::Expression;

 impl<ST, S, F, W, O, L, Of> LimitDsl for SelectStatement<ST, S, F, W, O, L, Of> where
     ST: NativeSqlType,
@@ -66,7 +66,7 @@ impl<ST, S, F, W, O, L, Of> LimitDsl for SelectStatement<ST, S, F, W, O, L, Of>
     }
 }

-type Offset = Limit;
+pub type Offset = Limit;

 impl<ST, S, F, W, O, L, Of> OffsetDsl for SelectStatement<ST, S, F, W, O, L, Of> where
     ST: NativeSqlType,

Can you mark all of these as #[doc(hidden)], please?

Also, can you bump this date https://github.com/sgrif/diesel/blob/master/.travis.yml#L6 to the latest nightly?


[Bug]: Identifiers not showing up from Radial Annotator

Steps to reproduce the bug
1. For a given Person where I know they have **Identifiers** I expect to see them when I open the **Radial Annotator.**
2. When I go this person's **Show** interface, I see the identifiers as expected.
3. When I click on the **Radial Annotator**, I see the expected number of identifiers in the Radial wedge for Identifiers.
4. When I click on the **Identifiers* wedge to open the pop-up, there are no identifiers there at all (of any type).
5. Examples: see record for Jennifier Giron (who has both an ORCID and a Q number), and see screenshots below using me as an example.
...

Screenshot

Expected behavior
When I click on the Identifer wedge in the Radial Annotator, I expect to see the identifiers there.
Additional Screenshots
No response
Environment
Production
Sandbox Used
No response
Version
v0.32.2
Browser Used
firefox

The similar bug is on Sources. If it has DOI, it is not visible in the radial (did not checked for other sectors).

@proceps We clearly need unit tests here.  If you could help that would be grant.  Perhaps in a new spec.


⚠ Bofrost.at has degraded performance

In 34e609e, Bofrost.at (https://www.bofrost.at) experienced degraded performance:

HTTP code: 200
Response time: 1767 ms

Resolved: Bofrost.at performance has improved in f751ba8 after 13 minutes.


Trace Flag 460 doesn't include SQL 2016 information

on the Trace Flag documentation trace flag 460 doesn't include information that it works on SQL 2016 but according to the KB article it does.
https://support.microsoft.com/en-us/topic/kb4468101-improvement-optional-replacement-for-string-or-binary-data-would-be-truncated-message-with-extended-information-in-sql-server-2016-and-2017-a4279ad6-1d3b-3960-77ef-c82a909f4b89

#8431 is PR. Don't know how to do the tie proper.

@josh-simar -- thank you for that PR.  It has been merged into our live documentation.
For future reference, just add a commit comment starting with Fixes to link a PR to an issue:
Fixes MicrosoftDocs/sql-docs/issues/8430


Separate the adopters logo wall from the carousel

Signed-off-by: Juozas<EMAIL_ADDRESS>

Yes, I like this -- we can have our logo wall and our user testimonials :-D

yes, this looks good to me and makes things a little simpler!
what do you think about moving the "who is using flux" section further up?

The logos seem to take up a lot of space on a MDPI screen (1280x800px, but screenshot is higher).

I think that "wow look how many logos" is more or less the impression we're going for, this LGTM
I think the carousel works fine further down the page on its own, and the wall of logos is a lot more impactful as a stand-alone thing. Great work!

I think that "wow look how many logos" is more or less the impression we're going for

I do not agree, as there is more valuable information below the logo fold, and at some point the number of logos starts to look like a desperate upsell instead of an acknowledgement of trust.

Or downscale them based on viewport (category)? I would keep the max height of the box of logos at 1/2 of the viewport at all times basically (no matter the number of rows). This also ensures the user knows the page continues.


Logging the deliverable ID

Description
It would be very useful to log the deliverable ID to identify the current deliverable. This is necessary when publishing multiple deliverables in a row, so when using a multi-deliverable DITA-OT project file.
Possible Solution
The most simple solution is probably to log it in log-arg.
Potential Alternatives
Even better it would be to prefix all error messages with the deliverable ID, this would help if the publishing does not run in verbose or debug mode.
Additional Context
–

My colleague @DunaMariusCosmin has a pull request for this: https://github.com/dita-ot/dita-ot/pull/4100
The DITA OT bundled with Oxygen 25.1 will contain this improvement so that every error reported in the Problems list when running a DITA OT project will contain as a prefix the deliverable ID.

@raducoravu, could you please merge #4100, so that the codes is shipped with the next point release? It would be very useful for us to have this functionality in the "normal" DITA-OT as well (not "only" in the Oxygen Publishing engine).

@xephon2 maybe you can comment on the pull request, I am not the one to decide if a PR should be merged or not.

I would also like to see #4100 merged.

Here is a testcase: 4100.zip
It has all four combinations of deliverable names and/or IDs:
$ grep '<del' project.xml
  <deliverable>
  <deliverable name="Deliverable description goes here">
  <deliverable id="del-id-1">
  <deliverable name="Deliverable description goes here" id="del-id-2">


LCFS - Restrict users from notionally transfer away more renewable fuel than they have reported in the Fuel supply summary (In progress)

Describe the task
Restrict BCeID users from notionally transferring away more renewable fuel than they have reported. They must report marketed fuel in the Fuel Supply section of the compliance report and can notionally transfer this away to another supplier. But users should not be allowed to input more volume transferred away in the Notional transfer schedule than they have reported in the Fuel supply schedule.
Purpose
You can't notionally transfer fuel that you have not reported in the Fuel supply input.
Acceptance Criteria

[ ] first
[ ] second
[ ] third

Additional context

Add any other context about the task here.
Or here

could they not export fuel that they reported in fuel supply the previous year?


Custom fields on menus don't get inserted on new menu items until after clicking "save menu"

This has something to do with the script that inserts the new menu items, I need to figure out if it's filterable in some way.

The template for these appears to be located at wp-includes/customize/class-wp-customize-nav-menu-item-control.php:60. There don't appear to be any hooks, but I need to look in to how the JS is called before I can know definitively.

Got this set up! Only thing I haven't done is enable the "Screen Options" settings, those get added after initial save. It's such a minor thing that I'm just leaving it as "todo" for now, but I may revisit it later.
Here's a description of what I did:

First, I had to rework the PHP so that the custom field templates weren't locked in to specific menu items. All the templates and logic where in place right in the function that constructed the menu item. I moved the templates in to their own function, and replaced the various things like $menu_item->ID with {{ item_id }}, and use regular expressions to replace the placeholder values.
I then set up a new script wp-admin.js, and a function which localized the PHP field templates in to JSON which gets fed in to wp-admin.js. All the field properties get replaced before being passed to the script, but the item properties are left in their placeholder form, so that they can be replaced as new items get inserted.
From there, I set up a MutationObserver to listen for changes to the DOM within the menu structure, and specifically identify when new items get added (a bit tricky, because as you drag things around, mutations fire). Using that, I then use grab the new item ID from the inserted item, and replace the {{ item_id }} placeholder with it.
Next, I check to see if at least one the menu locations assigned to the field (primary in this case) is assigned to the current menu that's being edited. If it is, then I change the custom field from display: none; to display: block;.
Finally, I then set up a simple listener for the location selectors to loop through all fields and hide/show them using the same method as the previous step.


Prefect 3.0 Support

Any plans on supporting Prefect 3.0 soon, seeing as it hit GA recently?

I just created PR https://github.com/datarootsio/prefect-dbt-flow/pull/53 that upgrades Prefect to 3.1.


Bug in photoBrowser index 0

No images are loaded when the index is 0.
myPhotoBrowserPage.open(0);
But
myPhotoBrowserPage.open();
works fine.
My temporary solution:
if(index > 0 ){      myPhotoBrowserPage.open(index);  } else {     myPhotoBrowserPage.open(); }
The temporary solution does not work if you clicked index > 0 before.
Use Version 3.4.2

Can't see such issue. Would be good to see JSFiddle or example with the issue

http://jsfiddle.net/9q623bgm/1/


is this repo supports multiple bin?

Hi Jerry,
is this repo supports multiple bin and item distribution now?

Not yet

I fixed this issue in the latest version, please refer to the Readme.

This issue has been resolved, please reopen if you have other questions.

@Westlife1002 Yes , it has been resolved. you can check it on example7.py, have a good day.

Hi Jerry,
it works well with the original example7.py. however, if I add the third bin, e.g.
init packing function
packer = Packer()
init bin
box = Bin('example7-Bin1', (5, 5, 5), 100,0,0)
box2 = Bin('example7-Bin2', (3, 3, 5), 100,0,0)
box3 = Bin('example7-Bin3', (5, 8, 10), 100,0,0)
add item
Item('item partno', (W,H,D), Weight, Packing Priority level, load bear, Upside down or not , 'item color')
packer.addBin(box)
packer.addBin(box2)
packer.addBin(box3)
I suddenly got below error:
Traceback (most recent call last):
File "C:\Users\Admin\Desktop\3D-bin-packing-master\example7.py", line 45, in
packer.pack(
File "C:\Users\Admin\Desktop\3D-bin-packing-master\py3dbp\main.py", line 585, in pack
self.bins[idx].gravity = self.gravityCenter(bin)
File "C:\Users\Admin\Desktop\3D-bin-packing-master\py3dbp\main.py", line 541, in gravityCenter
result.append(round(i / sum(r) * 100,2))
ZeroDivisionError: division by zero
what's the issue here? thanks for your help
Best,
Jerry

发件人: Shiu Ruei-Chang @.>
发送时间: 2023年9月19日 23:15
收件人: jerry800416/3D-bin-packing @.>
抄送: Westlife1002 @.>; Mention @.>
主题: Re: [jerry800416/3D-bin-packing] is this repo supports multiple bin? (Issue #11)
@Westlife1002https://github.com/Westlife1002 Yes , it has been resolved. you can check it on example7.py, have a good day.
―
Reply to this email directly, view it on GitHubhttps://github.com/jerry800416/3D-bin-packing/issues/11#issuecomment-1725868719, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AIJKO4W5W7G5PHYOXYIEG73X3GZJPANCNFSM6AAAAAATGJAFVE.
You are receiving this because you were mentioned.Message ID: @.***>

Hi Jerry,
it works well with the original example7.py. however, if I add the third bin, e.g.
init packing function
packer = Packer()
init bin
box = Bin('example7-Bin1', (5, 5, 5), 100,0,0)
box2 = Bin('example7-Bin2', (3, 3, 5), 100,0,0)
box3 = Bin('example7-Bin3', (5, 8, 10), 100,0,0)
add item
Item('item partno', (W,H,D), Weight, Packing Priority level, load bear, Upside down or not , 'item color')
packer.addBin(box)
packer.addBin(box2)
packer.addBin(box3)
I suddenly got below error:
Traceback (most recent call last):
File "C:\Users\Admin\Desktop\3D-bin-packing-master\example7.py", line 45, in
packer.pack(
File "C:\Users\Admin\Desktop\3D-bin-packing-master\py3dbp\main.py", line 585, in pack
self.bins[idx].gravity = self.gravityCenter(bin)
File "C:\Users\Admin\Desktop\3D-bin-packing-master\py3dbp\main.py", line 541, in gravityCenter
result.append(round(i / sum(r) * 100,2))
ZeroDivisionError: division by zero

@Westlife1002 In the current version, if the bin is empty, an error will be reported. This is a bug and I will correct it in the next version.

@Westlife1002 In the current version, if the bin is empty, an error will be reported. This is a bug and I will correct it in the next version.

ok, when will the new version be released?

You can add a check for zero. Replace the line 541 in the file "3D-bin-packing-master\py3dbp\main.py"
   result.append(round(i / sum(r) * 100,2))

on
        if sum(r) == 0 :
            result.append(0)
        else :
            result.append(round(i / sum(r) * 100,2))


Version <IP_ADDRESS> needs extra dep bounds

custom-setup/setup-depends needs a lower bound for Cabal. Cabal >= 1.24.1 && < 1.25 seems to work. It also needs a lower bound on directory. directory >= <IP_ADDRESS> seems to work. Specifically, withCurrentDirectory and libraryDynDirs are somewhat new.

Nice. Thanks! I'll make those changes.

Thanks! You should also update the metadata on Hackage for existing versions.


Support for pluggable customizer for Elasticsearch mappings creation.

Describe your changes :
Current OpenMetadata have static Elasticsearch(ES) mappings, but the search function doesn’t work correctly if the searching word contains Japanese characters.
A similar problem is reported on  #9158, on which language-base mapping is supported, and contains mappings for Japanese language.
The patch of #9158 uses Kuromoji-analyzer for supporting Japanese language, but Analyzers used for Japanese not only Kuromoji-analyzer, Ngram-Analyzer is used often too. In my experience, Ngram is used in many web systems, because both of the analyzer have it own profit and cons. Kuromoji is useful for Dictionary-based tokenization, but Ngram is useful for free-word searching.
So choice of analyzer is depend on usecase, not on language. For example, Ngram is useful not only for Japanese, but it will useful and be matched for searching on some asian languages like Chinese or Korean.
And this problem is not able to solve only with mapping files.
If we use Ngram-analyzer, we need 2 customizations on a query:

we need to add a {"type": "phrase" } option to query_string query of Elasticsearch for best matching
Wildcard query doesn't work correctly on fields tokenized by ngram-tokenizer. Matching to ngram-field is always partial matching (not exact matching), so query string *Word* doesn't match. It must be just Word. So the query string sent from OM must be (Word OR *Word*), not *Word*.

For supplying those customization onto query, we need functionality for customize QueryBuilder and query string.
I introduced a new ElasticSearchIndexResolver interface and made DefaultElasticSearchIndexResolver (which works same with current OM implementation) and NgramElasticSearchIndexResolver which uses Ngram-analyzer on description-fields and customize query for supporting ngram-matching. This interface is used for getting index-name, getting path of mapping file, customize query string, customize QueryBuilder and customize HighlightBuilder.
(HighlightBuilder customization might be required if new mapping files contain some new fields and is needed to add the field to highlight, but it is not used in this patch.)
Implementation-class is changeable by a setting elasticsearch.indexResolverClassName in openmetadata.yml.
The advantage of this patch is that users can supply their own mapping files and customization by creating a jar file which contains their own implementation of ElasticSearchIndexResolver and putting the jar in classpath. Users can customize query by their own usecase, not by language.
This patch is now used in our company as trial and currently works perfectly.
FYI: Before pull-request, I duscussed at Slack channel: https://openmetadata.slack.com/archives/C02DZK3QH8Q/p1671693745104339

Type of change :

[ ] Bug fix
[x] Improvement
[x] New feature
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] Documentation

Checklist:

[x] I have read the CONTRIBUTING document.
[x] I have commented on my code, particularly in hard-to-understand areas.
[ ] I have added tests that prove my fix is effective or that my feature works.
[x] All new and existing tests passed.

When I run all tests from maven, some tests for *Resource.class which uses TestContainer show errors, but it works successfully when I run it from IDE.

Reviewers
@open-metadata/backend

Thanks @tyano for the PR. Will review and merge it in.

@tyano do we want to maintain 2 separate index mappings. Why not merge it into one index with ngrams enabled?

@tyano can you please fix the styling check. Will merge it in

Fixed merge conflict with main branch and executed the stylers.

I found some mistakes in my patch. I will fix them soon. Please don't merge until I fix it.

This PR now conflict with #9158 , so I'm trying my code to the latest main branch.
I tried #9158 on my server and noticed that we can get search-results with the word '回答', but can not get results with '答', although '答' is a part of '回答'.
It is because #9158 uses Kuromoji analyzer for creating search-index, and kuromoji tokenized sentences by dictionary. it create a token '回答' because dictionary have the word, and never created a token '答'.
I think Ngram tokenizer will solve this problem...

@tyano trying to merge this patch . Can you run
 mvn googleformatter:format@reformat-sources to fix the styling check
Also I am thinking we should make ngram as the default behavior, wdyt?

@tyano can you fix the merge commit error that I did. I can't push changes into your branch

Indexed with Ngram but kept getting errors for sample data

Did you try testing  with sample data with ngram enabled?

I'll check them.

Found some faults.

We must install some plugins (analysis-icu and analysis-kuromoji) to Elasticsearch. I built my own Docker image which contains the plugins for testing. To fix this problem I will put a new Dockerfile for building the Elasticsearch.

The image described above must be pushed to your docker repository for making it publicly downloadable and change the image name in docker-compose.yaml and docker-compose-postgres.yaml.
I can not handle it, because the build process is a part of your company's release pipeline and you must supply the image name of the image which is built by your release pipeline.

Searching english word doesn't work correctly on Ngram index. I'm now working on this problem (maybe problems on mapping files). After solving this problem, I will push new code again.

I pushed new fixed code.
There are some notices:
I added a new Dockerfile: docker/local-metadata/Dockerfile_elasticsearch.
This dockerfile generates an Image of Elasticsearch containing analysis-icu and analysis-kuromoji, which are required by NGRAM mapping files.
For local use, you can just use docker/local-metadata/docker-compose.yaml or docker/local-metadata/docker-compose-postgres.yml. I changed those files as building the new Dockerfile_elasticsearch, instead of downloading the image from elastic.co.
After merging this PR, you must build the new ES image and push it to your public docker registry, then must change public docker-compose.yaml and docker-compose-postgres.yaml as using the new built ES image.
Without the new ES image, users will be faced errors like screenshots you put in above posts.
HighlightBuilder for NGRAM is changed
I noticed that searching by alphabet word didn't work correctly. When searching english word, the line like "Matches : n in Columns Description" on the result page showed wrong match count (like '70 in Column Description', despite of no such many columns exist).
It was caused by function_score query used from version 0.13.x. Using function_score query to NGRAMed fields breaks the highlights of the returned match-result.
Investigating this problem, I found this problem was easily avoidable by adding highlight_query onto HightlightBuilder. So I implemented customizeHighlight method of NgramElasticSearchIndexResolver as adding highlight_query.
This customization is needed only the time when table_search_index index is the search target. For distinguish indexes, I Added an indexType parameter in some part of code.
about Making Ngram as default
Making ngram as the default behavior seems good idea. Ngram works for both of basic space-separated language like english and no-separator-language like Chinese and Japanese. Ngram will be useful for many languages.
But I think changing index to ngram is a large change. Public testing is required.
If we use ngram as the default behavior in 0.13.x, we must describe the function is easily switchable by setting indexResolverClassName in openmetadata.yaml, so that users can change it to old default behavior if they face problems caused by the new analyzer.

@tyano apologies for the delay in reviewing the PR. Can you please fix the merge conflicts so we can push this for the 1.0 release.
Thanks

OK, I will try to merge the latest main branch. Please wait a few days. I will do it tomorrow or next tomorrow.

@harshach Now I'm trying to merge the latest main branch to my PR. After merging, an error occurs when I'm deploying a Metadata Ingestion after adding a database service.
I tested on main branch too, for checking if the error occurs for merge errors, but same error occurred on main branch too.
It says:
expert-db_metadata_Q1OSY4KX Failed to deploy Ingestion Pipeline due to airflow API returned Bad Request and response {"error": "Request Validation Error parsing payload. IngestionPipeline expected: 78 validation errors for IngestionPipeline

See an screen shot added on this post.
Could you guess what is happening? It might be an merge issue or

@ulixius9 any idea about the error above?

@tyano can you confirm if you have built the ingestion container after merging the main branch? from the error I can see that you are facing a json schema related issue related to processPIISensitive which we have recently introduced with this pr it seems like the server has picked up this schema change but the ingestion container has not picked this up, If you are testing your changes on docker you need to run the make generate command first from root dir and then build the docker image!

Ah, You right! I didn't run make generate.
I will try it.
Thank you.

merged.
Be careful that a new ES image is introduced.
The image will be automatically built if you use local-metada/docker-compose.yml or local-metada/docker-compose.yml   for local development. You need to build the image in you build pipeline while releasing it.
See: https://github.com/open-metadata/OpenMetadata/pull/9519#issuecomment-1396857867

@harshach I noticed new merge conflicts appear between my branch and main branch, so I resolved the conflict and merged it again.
Be careful you need to set an environment variable ELASTICSEARCH_INDEX_RESOLVER_CLASS_NAME to org.openmetadata.service.elasticsearch.indexresolver.NgramElasticSearchIndexResolver, or set an indexResolverClassName config parameter in openmetadata.yaml for using Ngram index.

we have introduce per language specific mappings. Closing this as resolved


Merge pull request #8 from ScientificThought/FolderRename

Fixing issue #9  - Folder rename.

I think you forgot to add the test fixes on this PR. Is it possible to do that?


feat: serialize sets to array for payloads

when requests go over the wire, sets should become arrays. https://github.com/OpenAPITools/openapi-generator/issues/11746 for reference

@mheap I think you're the only one with write access on this repo, sorry to ping


Contracts status

Pull Request Test Coverage Report for Build 226

62 of 62 (100.0%)  changed or added relevant lines in 7 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.07%) to 96.593%

Totals

Change from base Build 220:
0.07%

Covered Lines:
2977

Relevant Lines:
3082

💛  - Coveralls


Update composer.json

Added support for laravel 5.7

Any ETA on merging this PR? It's been more then 4 weeks now!

@georgeboot Yes, it's been for a while. The author doesn't have time for merge:)
I do hope for this merge as I can't update to 5.7 :(


Consider adding built-in change functions

Some common use cases are left to the user. It would be nice to have optional built-in ways to do basic stuff, most commonly adding/removing event listeners and classes. None of it should be included as part of the default build but explicitly required. It could be new public API.
Strawman:
const simulacra = require('simulacra')
const { bindEvents, animate, setDefault, chain } = simulacra

const animateOptions = [
  'fade-in' , // class to add on append
  'bounce', // class to add on mutate
  'fade-out', // class to add on remove
  200 // how long the element should be retained after removal, in ms
]

// Events get attached on append, and removed automatically.
const events = { click (event) { console.log('clicked!', event) } }

// Change function defined like so:
const change = chain(
  setDefault,
  bindEvents(events),
  animate(...animateOptions))

This should be done now.


Document how to install plugins

Is your feature request related to a problem?
I just started using Verdaccio on an AWS Amazon Linux 2 EC2 instance and I have no idea on how to install plugins.
I installed Verdaccio using npm install -g verdaccio and it runs fine, I can access the web interface.
Now I want to add the github oauth plugin but I have no idea on how to get this working. I tried
npm install -g verdaccio-github-oauth-ui
and modifying the config file for this plugin, but restarting Verdaccio only gives me:
Error: verdaccio-github-oauth-ui plugin not found. try "npm install verdaccio-github-oauth-ui"

I saw there is a plugins directory in the config file, but I have no idea how to install a plugin there, as running an npm install in that directory will just create a node_modules subdirectory containing the plugin and all its dependencies. (I tried this too btw and it also didn't work)
Describe the solution you'd like
I would like to see a few lines of documentation on how to install plugins, the same way the installation of Verdaccio itself is clearly explained. The only ones I find are about using Docker and extending the image to add plugins.
Thanks a lot!

Here one good example https://github.com/barolab/verdaccio-minio/blob/master/example/Dockerfile#L10

Actually, I was struggling with this as well. So I intended to create a PR for a docs update. Only your docs do not seem to be exposed on GItHub?
Here is what I came up with:

switch into a suitable working dir
create a path for the plugins volume:
mkdir volume
start the current stable release of Verdaccio with the plugins path mounted:
docker run --name verdaccio --volume $(pwd)/volume:/var/tmp/volume verdaccio/verdaccio:4
enter the running container as root user:
docker exec -it -u root verdaccio /bin/sh
change owner of the plugins volume path to the application user:
chown -R verdaccio /var/tmp/volume
switch to the application user:
su -s /bin/sh - verdaccio
switch to the plugins volume path:
cd /var/tmp/volume
install the plugin(s) you desire, in this case verdaccio-aws-s3-storage:
npm install --global-style --no-bin-links --no-optional --registry https://registry.verdaccio.org<EMAIL_ADDRESS>after successful plugin(s) install, stop the container (and remove any anonymous volumes: -v):
docker rm -fv verdaccio
ensure your plugins configuration is added to docker.yaml (from: https://github.com/verdaccio/verdaccio/blob/master/conf/docker.yaml), in this case verdaccio-aws-s3-storage config:store:
  aws-s3-storage:
    bucket: 'some-bucket'
    region: 'eu-central-1'

start a clean Verdaccio container, while mounting your customized config file and plugins volume:
docker run --name verdaccio --volume $(pwd)/docker.yaml:/verdaccio/conf/config.yaml:ro --volume $(pwd)/volume/node_modules:/verdaccio/plugins:ro verdaccio/verdaccio:4

I got started with the docker compose instructions from Using docker-compose, which worked well. I picked one config.yaml (from verdaccio / docker-examples) by chance.
Then I wanted to add the active directory auth plugin and stumbled across ir-fuel's problem.
My solution was

Copy locally installed plugin(s) from node_modules to plugins

Add the following line in config.yaml:
plugins: /verdaccio/plugins

(I suppose alternatively you may change docker-compose.yml to read "./node_modules:/verdaccio/plugins".)

Thanks to @martin-paulus for the key to success installing , which is the use of the --global-style switch with npm. I was able to shorten the procedure with help from @diamondarts and tips from the docs:

In an arbitrary folder on the docker host:npm install --global-style --no-bin-links --no-optional verdaccio-acl-plugin

This will produce a package-lock.json and node_modules/ folder. Critically, the global-style option ensures that the plugin's dependencies are nested instead of flattened as siblings, which won't work.
Set ownership to what will be the verdaccio user's ID inside the container and no-group ID:chown -R 10001:65533 ./node_modules/

Mount the docker container's plugins volume to the node_modules folder, e.g.:./node_modules:/verdaccio/plugins

Thanks @tommcc !
I managed to adapt your solution so I could use it in a Dockerfile instead
This is what I currently do
FROM verdaccio/verdaccio
USER root
ENV NODE_ENV=production
RUN npm install -g --no-bin-links --prefix /tmp/verdaccio-auth-memory-install verdaccio-auth-memory \
    && mv /tmp/verdaccio-auth-memory-install/lib/node_modules/verdaccio-auth-memory /verdaccio/plugins \
    && chown -R $VERDACCIO_USER_UID:root /verdaccio/plugins \
    && rm -rf /tmp/verdaccio-auth-memory-install
USER verdaccio

It's pretty ugly, but I think I prefer it over having to deal with setting up a plugin volume

adapting your adaption into a multistage build 😉
FROM verdaccio/verdaccio:5 as builder
USER root
ENV NODE_ENV=production

RUN apk add alpine-sdk python
RUN npm i --global-style verdaccio-bitbucket

FROM verdaccio/verdaccio:5
COPY config.yml /verdaccio/conf/config.yml
COPY --from=builder --chown=$VERDACCIO_USER_UID:root /opt/verdaccio/node_modules/verdaccio-bitbucket /verdaccio/plugins/verdaccio-bitbucket

Thanks guys for your comments. I think  would looks even better in the website, easy to find and SEO friendly. PR is welcome 🙏

To anyone else scratching their head, while @stephen-dahl's solution works great, note that you may also need to have explicitly set plugins: /verdaccio/plugins/ in your config file.

I might pr docs for this next week.

Docs were moved to this repository so you all are aware
https://github.com/verdaccio/verdaccio/tree/master/docs

This is helpful - are there any guides to help with actual plug-in development that includes the ability to set breakpoints?
I need to build an OKTA Authentication plugin as the one listed in the docs (https://github.com/hogarthww-labs/verdaccio-okta-auth) is unfinished and does not work

I've spent quite a bit of today trying to figure out how to actually get plugins to load when running verdaccio in docker compose. Figured I'd post my solution here since as it took piece mealing a bunch of info to finally get it to work.
For this, the simplest way seems to be extending the verdaccio docker image with your own. Remove image: verdaccion/verdaccio:latest from the docker-compose and add in build: ./. Place a Dockerfile in the same folder and use yarn to install the plugin.
# /var/docker/verdaccio/docker-compose.yml
storage: ./storage
auth:
  htpasswd:
    file: ./htpasswd
    max_users: -1
  azure-ad-login:
    tenant: "FILL THIS OUT"
    client_id: "FILL THIS OUT"
    client_secret: "FILL THIS OUT"
uplinks:
  npmjs:
    url: https://registry.npmjs.org/
packages:
  "@*/*":
    access: $all
    publish: $authenticated
    proxy: npmjs
  "**":
    proxy: npmjs
log: { type: stdout, format: pretty, level: http }

# /var/docker/verdaccio/Dockerfile
FROM verdaccio/verdaccio:latest
USER root
RUN yarn add verdaccio-azure-ad-login
USER verdaccio

I typically use NPM but I found that doing RUN npm install verdaccio-azure-ad-login would still give me the error that the plugin was not found. Not sure what the difference is between yarn add and npm install but this is what made it work for me.

I will tackle this topic on this month:
Starting with

Refactor and upgrade https://github.com/verdaccio/generator-verdaccio-plugin
Improve docs
Add API docs
Document using Docker and plugins

So all your knowledge and experience is vital for me, please if you haven't share example or open source plugins, techniques and so on please share here, I would need your help.

Thanks folks for all your examples, I'm creating some docs upgrades + docker examples ready to use here
https://github.com/verdaccio/verdaccio/pull/3217
please give me your feedback 🙏🏼 I will try to add more and then update the docs website.

Ok I consider this done.

Plugin generator has been improved
Examples how to install plugins with Docker
How to debug Docker

For that and other things, I think verdaccio shouldn't be planned to be installed globally. Why not use it as any other package? Seems simpler, more straight-forward, easier to keep track in git, see plugins installed, configs used, etc. Add plugin package to your project and it should be done.
(I've managed to do that locally in my company after some days expent on it - which was harder to do with pm2! If anyone finds helpful I've posted in https://stackoverflow.com/a/72522554/3645375)

Did you manage to get the azure login working? for me it always keep saying below
npm ERR! 409 Conflict - PUT http://localhost:8001/-/user/org.couchdb.user:ganesh.abcd..workingdomain.com - bad username/password, access denied
any idea what i might look at?