elasticdump hangs when using --offset option

Hi!
I have an issue when using the --offset option.
Here is the output of elasticdump in my case
starting dump
Warning: offseting 6382740 rows.
Using an offset doesn't guarantee that the offset rows have already been written, please refer to the HELP text.
These are only log lines, then the tool hangs. No items are added in ES output, elasticdump process is idle in the processes list.
I am using ES version 1.4 (AWS). ED version is 2.4.2
My command line is
./node_modules/elasticdump/bin/elasticdump --input=https://xxxxxx.us-west-2.es.amazonaws.com/goods --output=https://yyyyyy.us-west-2.es.amazonaws.com/goods --type=data --limit=4000 --offset=6382740
Access to ES domain is opened, so I do not need AWS keys as an options.

see https://github.com/taskrabbit/elasticsearch-dump/issues/234#issuecomment-228423196
Right now, the "offset" option only effects the writer... we are still reading in the first N (6382740 in your case) documents.  No one has updated this functionally yet to "skip" as part of the initial elasticsearch query.

Is there any update on this? Experiencing the same problem.

Hi!
Actually I do not remember how fixed it. As I remember I have used Amazon
embedded tools for this and amazon's console.
On Tue, May 16, 2017 at 4:04 PM, Roman Nehrulenko<EMAIL_ADDRESS>wrote:

Is there any update on this? Experiencing the same problem.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/taskrabbit/elasticsearch-dump/issues/265#issuecomment-301761064,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABfygbd460PulZojZEH3WnISgNImHN91ks5r6ZDXgaJpZM4K2aqt
.

@hamazaspavetisyan Thanks for the reply, but it's pretty clear now for me from the #234.

when working with interrupted import into ES (at offset 1231139) from file previously created via --output=index_data.json, I found it more helpful to use sed for generating offsetted input file, and continuing as usual
sed -n '1231139,$ p' index_data.json > index_data_next_part.json
elasticdump --input=index_data_next_part.json --output=...


Checkout as a context manager

Expected Behavior
Making checkout as a context manager let the users avoid writing explicit close() and we can make sure the lock is released. Keeping the current functionality along with this new ability should be the way to go.
with repo.checkout(write=True) as co:
    co.datasets['dset'] = array
    co.commit()

We could either raise a warning if context manager exits without commit or we can even do commit implicitly (by having another argument autocommit=True in the checkout() call)
Current Behavior
Currently, we need to explicitly call close() for all the write-enabled checkouts.
I could take it up if this doesn't raise any other concerns.

I like the idea, but this requires some real thought and testing before it could ever be implemented
My main concern is how this would affect lmdb transaction handling for the Dataset and Metadata classes. As noted In the Documentation we get massive performance increases by using context managers for reading/writing data. Without prototyping this, it's not clear to me how entering a CM at the checkout level (on level above datasets or metadata, as shown below) would work:
Hangar Object Hierarchy:

Repository
  └ Checkout
      ├ Metadata
      |    ├ MetadataWriter (CM)
      |    └ MetadataReader (CM)    # overridden by MetadataWriter CM if applicable
      └ Dataset
           ├ DatasetDataWriter (CM)
           └ DatasetDataReader (CM)  # overridden by DatasetDataWriter CM if applicable

Some of the problems I can see with moving the CM enter/exit logic up a level:

Transaction lifespan: would increase dramatically. The natural implementation would be to reimplement the defined behavior in the checkout object. This would start a single transaction when the __enter__ method is encountered, which would persist until __exit__. Long lived transactions are an antipattern in lmdb, as memory page reuse is not allowed while a txn is allive. In addition, fatal system crashes would all records added in a checkout (prior to the crash) to be lost, rather than the records related to the dataset/metadata under operation.
Thread Local Storage: lmdb transactions use TLS and can only ever exist in a single thread (the one they were created on). If we want to use multithreaded data loaders in the future, having to keep all record access in a single thread could potentially complicate the logic.
One Environment Per Process: An lmdb environment can only be opened once in a given process. Since all transactions are tied to a single environment, supporting multiprocess dataloaders/writers would get complicated (potentially impossible?). This can't even be solved by closing and re-opening the environment for any spawned processes, since this breaks flock() advisory locking on POSIX/UNIX OSs
Storage Backend Support: This will just be a mess if we attempt to move context management for storage backends up this far. This logic needs to be implemented independently for each backend. We could work around this (in a manner similar to how we do currently), but it would just be another thing which increases complexity of the underslying codebase.

These are just the ones i can come up with off the top of my head. there are probably more...
I think the question we really need to ask is: How important is this? would this feature be worth the pain right now? I personally don't think so, but I'd love to hear your opinion.

While I really like the user-friendliness of making CM here, you sort of convinced me that moving all the heavy lifting that happens in Dataset/Metadata level CM to repo level is not worth the pain right now. But I think, making repository also a CM while keeping the heavy lifting as it is (i.e. in Dataset/Metadata level) should be helpful. Plus we could keep the normal (naked) way of accessing repository as it is. This might make things weird for some, like calling two with statements, etc, but again, having an option doesn't harm.
with repo.checkout(write=True, autocommit=True) as co:
    with co.datasets['dset'] as dset:
        dset.add(arr, 'arr')

So essentially the entire purpose of the CM checkout block would be to ensure that the writer lock is closed?

Right (for now) and the user could avoid writing two more lines (for commit and close). Plus, it's up to the user whether they need to use the CM or the naked version.

Ok. To put this in writing, I'm good with this. I don't know when I'll have time, so I'm going to leave this as help wanted for a bit

Closed with #84


Driver for Amazon Athena

Hi team,
I'd like to contribute a driver for Amazon Athena. And before discussing specifics, I'd like to know if the community would be interested in the contribution.
Thank you.

Definitely! Always happy to expand database support.

Great news, thanks.
I'd like to discuss a change in the driver interface to include an optional method to cancel queries. Cancelling queries it's a very important feature since it stops customers being charged for queries that they no longer care about or queries run by mistake.
I propose to extend the driver data model to include a flag indicating that the driver implements a method to cancel queries. The client would render a "cancel query" button, disabled when the query is not running, if the driver has this flag set.
Thoughts?

That would be awesome! I'm in full support of that and I feel like its been requested in the past. Just never prioritized it since most drivers don't have support for it. Makes a lot of sense for these cloud platforms though.

I propose to extend the driver data model to include a flag indicating that the driver implements a method to cancel queries. The client would render a "cancel query" button, disabled when the query is not running, if the driver has this flag set.

This is perfect. Something similar is already done for the "connection client" support.

Very open for it! Could you please include the option to assume role and with mfa support?

Apologies for the delay on this. I expect to have a first version ready by end of next week.
That first version will use instance profile to obtain credentials. We can discuss other options for authentication when the pull request is open for review.

Any update on this? We are extremely interested on Amazon Athena support.

Hi! I've just created a new PR to add support for Athena in SQLPad: https://github.com/sqlpad/sqlpad/pull/1044


Add PHP 8.1 support

This package is installable for PHP 8.1, but there are some fixes needed, example testing output:
Deprecated: Return type of Behat\Testwork\Output\Node\EventListener\ChainEventListener::count() should either be compatible with Countable::count(): int, or the #[ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in /builds/mahalux/php-ci/test-project/vendor/behat/behat/src/Behat/Testwork/Output/Node/EventListener/ChainEventListener.php on line 54
Deprecated: Return type of Behat\Testwork\Output\Node\EventListener\ChainEventListener::getIterator() should either be compatible with IteratorAggregate::getIterator(): Traversable, or the #[ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in /builds/mahalux/php-ci/test-project/vendor/behat/behat/src/Behat/Testwork/Output/Node/EventListener/ChainEventListener.php on line 62
Deprecated: Return type of Behat\Behat\Gherkin\Specification\LazyFeatureIterator::current() should either be compatible with Iterator::current(): mixed, or the #[ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in /builds/mahalux/php-ci/test-project/vendor/behat/behat/src/Behat/Behat/Gherkin/Specification/LazyFeatureIterator.php on line 120
Deprecated: Return type of Behat\Behat\Gherkin\Specification\LazyFeatureIterator::next() should either be compatible with Iterator::next(): void, or the #[ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in /builds/mahalux/php-ci/test-project/vendor/behat/behat/src/Behat/Behat/Gherkin/Specification/LazyFeatureIterator.php on line 96
Deprecated: Return type of Behat\Behat\Gherkin\Specification\LazyFeatureIterator::key() should either be compatible with Iterator::key(): mixed, or the #[ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in /builds/mahalux/php-ci/test-project/vendor/behat/behat/src/Behat/Behat/Gherkin/Specification/LazyFeatureIterator.php on line 112
Deprecated: Return type of Behat\Behat\Gherkin\Specification\LazyFeatureIterator::valid() should either be compatible with Iterator::valid(): bool, or the #[ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in /builds/mahalux/php-ci/test-project/vendor/behat/behat/src/Behat/Behat/Gherkin/Specification/LazyFeatureIterator.php on line 104
Deprecated: Return type of Behat\Behat\Gherkin\Specification\LazyFeatureIterator::rewind() should either be compatible with Iterator::rewind(): void, or the #[ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in /builds/mahalux/php-ci/test-project/vendor/behat/behat/src/Behat/Behat/Gherkin/Specification/LazyFeatureIterator.php on line 87

Are you using the stable release or the dev version of Behat ? The dev version of Behat already contains some fixes for PHP 8.1 compat.

Are you using the stable release or the dev version of Behat ? The dev version of Behat already contains some fixes for PHP 8.1 compat.

stable release
I see there is https://github.com/Behat/Behat/commit/70b07e070373854b9b3c5e184b498f97d18322fb commit with CI testing, will it be possible to release it?

I've just released v4.9.0 Behat/Gherkin supporting PHP 8.1, all we need now is to fix the build for this repository.

Thanks!

Are you using the stable release or the dev version of Behat ? The dev version of Behat already contains some fixes for PHP 8.1 compat.

I can confirm the latest fixes in this repo are working with PHP 8.1. Please tag this repo as well.

Released in v3.9.0 🎉


Add more specific return types for querySelector with single element selector

Fixes #8114

Hi @jelbourn, I'm your friendly neighborhood Microsoft Pull Request Bot (You can call me MSBOT). Thanks for your contribution!

In order for us to evaluate and accept your PR, we ask that you sign a contribution license agreement. It's all electronic and will take just minutes. I promise there's no faxing. https://cla.microsoft.com.

TTYL, MSBOT;

I've emailed in a signed CLA (I'm a Google employee on the Angular team)

@jelbourn thanks for the PR. this file is actually an output of the build. the sources are in src\lib*. this change specifically needs o be in the dom.generated.d.ts, which is an output of another tool in https://github.com/Microsoft/TSJS-lib-generator
You can find the details of contributing lib.d.ts fixes in CONTRIBUTING.md at https://github.com/Microsoft/TypeScript/blob/master/CONTRIBUTING.md#contributing-libdts-fixes


action: download default release assets to sign

Summary

This PR resolves #44. We now do the following on releases if release-signing-artifacts is enabled:

Download the default release artifacts, i.e.{{ref}}.tar.gz and {{ref}}.zip;
Sign for them;
Upload them back to the release as {{ref}}-signed.{{extension}}, along with their signatures.

Release Note

The release-signing-artifacts setting was extended to re-upload and sign default assets. Previously, the stability of default asset hashes were not guaranteed, as they are generated on-the-fly by GitHub.
Documentation

https://docs.sigstore.dev does not seem to include documentation on the actions, but I will add a commit documenting the change to release-signing-artifacts in this repo's README.

Doing this also significantly increases the number of assets attached to a release by default: https://github.com/tnytown/gh-action-sigstore-python/releases/tag/TEST_RELEASE

3. Upload them back to the release as {{ref}}-signed.{{extension}}, along with their signatures.

What happens if we name it {{ref}}.{{extension}}, like the default artifacts? I'd partially expect that to fail, but if it works it might be slightly nicer from a consumer perspective (no need to understand why archives have "signed" in the name).

https://docs.sigstore.dev/ does not seem to include documentation on the actions, but I will add a commit documenting the change to release-signing-artifacts in this repo's README.

Yep, the README is good -- this action isn't documented at all on the main Sigstore website. That's something we should consider in the future, though!

Doing this also significantly increases the number of assets attached to a release by default: https://github.com/tnytown/gh-action-sigstore-python/releases/tag/TEST_RELEASE

Yeah -- the .crt and .sig outputs are technically redundant now that .sigstore is present, so I think we can add an option like bundle-only: bool with a default of false that'll control whether we emit those.
Then, with a 2.0 release, we can switch that setting to true.

Yeah -- the .crt and .sig outputs are technically redundant now that .sigstore is present, so I think we can add an option like bundle-only: bool with a default of false that'll control whether we emit those.

Then, with a 2.0 release, we can switch that setting to true.

Just to clarify: We can do this with a follow-up PR. I'll make an issue to track that.

What happens if we name it {{ref}}.{{extension}}, like the default artifacts? I'd partially expect that to fail, but if it works it might be slightly nicer from a consumer perspective (no need to understand why archives have "signed" in the name).

I swapped it to {{ref}}-signed.{{extension}} when trying to fix another issue with artifact upload. I'll try {{ref}}.{{extension}} again to see if it works!

Cool! If it doesn't, we could always do something generic like release.{{extension}} -- the stable URL contains the {{ref}} anyways 🙂

CI is broken because of third-party PRs not having access to OIDC, sigh. I'll tweak all-tests-pass so that it allows skippage.


(ready for review) 608 add threshold weighted scores

Please work through the following checklists. Delete anything that isn't relevant.
Development for new xarray-based metrics

[x] Works with n-dimensional data and includes reduce_dims, preserve_dims, and weights args.
[x] Typehints added
[x] Docstrings complete and followed Napoleon (google) style
[x] Reference to paper/webpage is in docstring
[x] Add error handling
[x] Imported into the API

Testing of new xarray-based metrics

[x] 100% unit test coverage
[x] Test that metric is compatible with dask.
[x] Test that metrics work with inputs that contain NaNs
[x] Test that broadcasting with xarray works
[x] Test both reduce and preserve dims arguments work
[x] Test that errors are raised as expected

Tutorial notebook

[x] Short introduction to why you would use that metric and what it tells you
[x] A link to a reference
[x] A "things to try next" section at the end
[x] Add notebook to Tutorial_Gallery.ipynb
[x] Optional - a detailed discussion of how the metric works at the end of the notebook

Documentation

[x] Add the score to the API documentation
[x] Add the score to the included list of metrics and tools

I haven't reviewed the tests yet, but so I don't neglect to mention it later, I will be looking to see whether there are some consistency checks against unweighted scores when the weights are set neutrally.

I haven't reviewed the tests yet, but so I don't neglect to mention it later, I will be looking to see whether there are some consistency checks against unweighted scores when the weights are set neutrally.

You will find these consistency tests in test_threshold_weighted_score1. Huber Loss isn't in scores yet, so I copied the Huber Loss output from an independent calculation.

Some minor feedback.
In included.md:

Capitalisation suggestions (for consistency with other table entries)
The four entries don't appear to be in alphabetical order

In the docstrings, the references don't appear to follow APA7.
Edit: Re. API docstrings rendering in readthedocs - I think you might need two back ticks (not just one) around function names. (You can check when it is put into branch 228, or if you build it locally in readthedocs). Further Edit: it looks like sometimes you need one back tick and sometimes you need two back ticks. It looks like @nikeethr used a mix (depending on use case) in FSS API docstrings, and the FSS API docstrings appear to be rendering nicely in readthedocs.

Thanks @Steph-Chong . I think I have addressed this now

This pull request is now ready for review.

Thanks for that feedback @tennlee - I'm still in the middle of the refactor, so please hold off on anymore feedback until this is ready for review again.

Thanks for that feedback @tennlee - I'm still in the middle of the refactor, so please hold off on anymore feedback until this is ready for review again.

Sorry @nicholasloveday - I didn't see this comment before I made my comments. If I had seen it first, I would not have commented. My apologies.

Thanks for that feedback @tennlee - I'm still in the middle of the refactor, so please hold off on anymore feedback until this is ready for review again.

Sorry @nicholasloveday - I didn't see this comment before I made my comments. If I had seen it first, I would not have commented. My apologies.

No problem! I just didn't want people using up their time reviewing something that is only partly done :)

Okay, I've finished the refactor and responded to that initial feedback. It's now ready for review again thanks @tennlee and @Steph-Chong

I should add that you may need to run the tutorials yourself rather than view them on GitHub so that the Plotly figures display

To get the nice plots in the notebooks to work, I had to install the "ipywidgets" package. I suggest we add it to the "maintainer" dependencies". I also had to instead of using "fig.show()" just end each cell with "fig". This triggers plotly to dynamically detect the right rendering backend for sphinx.

Thanks for that useful feedback @tennlee - I have pushed up the changes

Thanks for this. I have made a few fixes into the fork and pushed them up. The docs are now building locally so I have pushed the code up to https://scores.readthedocs.io/en/228-documentation-testing-branch/ so we can test it there.
A question - does it make sense to add mathjax for the new methods, or are they too complex to represent as a formula?

"228" on readthedocs looks good to me. Can you check it also and make sure I haven't missed anything and drop a comment here. Also let me know about the mathjax. I think those are the two remaining questions before this can be merged.

Thanks for this. I have made a few fixes into the fork and pushed them up. The docs are now building locally so I have pushed the code up to https://scores.readthedocs.io/en/228-documentation-testing-branch/ so we can test it there.
A question - does it make sense to add mathjax for the new methods, or are they too complex to represent as a formula?

I did think about it, but decided to leave them out. I think that the equations will be so long that anyone interested in them is better off going to the paper.

Here is the link to the list of things to check for on readthedocs: https://scores.readthedocs.io/en/stable/maintainer.html#this-section-covers-checking-the-documentation-renders-properly-in-readthedocs

I've gone through the readthedocs and have made several changes. I think that they addressed the issues that you raised. I've also made other changes (e.g., cut down whitespace on the figs in the tutorial)

I've gone through the readthedocs and have made several changes. I think that they addressed the issues that you raised. I've also made other changes (e.g., cut down whitespace on the figs in the tutorial)

Thanks. I haven't reviewed the changes yet, but I have pushed your latest work up to 228. It will take a couple of minutes to build and then you can take a look at how the changes have come through.

In the tutorial, it looks like the plots aren't rendering in nbviewer, as viewed here: https://nbviewer.org/urls/scores.readthedocs.io/en/228-documentation-testing-branch/tutorials/Threshold_Weighted_Scores.ipynb
Note: sometimes nbviewer when viewed from the link at the top of the readthedocs version (as linked above) of the tutorials renders differently compared to how nbviewer renders on the preview screen while binder is launching. So it is possible, the nbviewer preview on the binder launch screen will render differently. (I haven't tested nbviewr in the binder launch screen, if anyone wants to test it, I think you may need to change the URL to test it from 228 branch - off the top of my head the default binder link is linked to main).

There is maybe an issue with the plots again.
The plots don't appear to be rendering in:

readthedocs: https://scores.readthedocs.io/en/228-documentation-testing-branch/tutorials/Threshold_Weighted_Scores.html
nbviewer - from the link at the top of the readthedocs tutorial page): https://nbviewer.org/urls/scores.readthedocs.io/en/228-documentation-testing-branch/tutorials/Threshold_Weighted_Scores.ipynb
nbviewer - from the preview screen at the bottom of the binder launch page: https://mybinder.org/v2/gh/nci/scores/228-documentation-testing-branch?labpath=tutorials/Threshold_Weighted_Scores.ipynb

The plots do seem to be rendering in binder itself (once launched).
Note re. nbviewer: nbviewer can render differently in the two locations listed above.

Looks like all of this is linked to "whatever makes the notebooks work when they are built on my machine". I re-ran the notebook in my environment and pushed the change up to 228. I'm sure we'll get to the root cause eventually, but for now I will just have to re-run the notebooks each time I push to 228.

@tennlee I have made minor updates to the tutorial based on @rob-taggart feedback that he provided offline. I didn't regenerate the plots, so they should still display fine in the docs (they display for me when I generate the docs).

I couldn't work out how to leave these as code comments.
Some very minor feedback about the tutorial header levels (as they currently appear in branch 228 - I didn't rebuild the docs, so apologies if these have already been addressed):

Header "Other Threshold Weighted Scores." has a full stop at the end. I am guessing the full stop isn't meant to be there.
Some of the capitalisation of the headers is inconsistent. In particular,
2a. "Rectangular weighting", could consider capitalising "w" in "weighting"
2b. "Trapezoidal weighting", could consider capitalising "w" in "weighting"
2c. "Things to try next", could consider capitalising "To Try Next"

In the "Other Threshold Weighted Scores" section, the first sentence says: "In this tutorial, we demonstrated the threshold weighted square error (or twMSE) function."
I think there may be a missing "mean" - as in "threshold weighted mean square error (or twMSE) function."

Thanks @Steph-Chong for those suggestions. I've made some updates that address your feedback.

There is maybe an issue with the plots again.
The plots don't appear to be rendering in:

readthedocs: https://scores.readthedocs.io/en/228-documentation-testing-branch/tutorials/Threshold_Weighted_Scores.html
nbviewer - from the link at the top of the readthedocs tutorial page): https://nbviewer.org/urls/scores.readthedocs.io/en/228-documentation-testing-branch/tutorials/Threshold_Weighted_Scores.ipynb
nbviewer - from the preview screen at the bottom of the binder launch page: https://mybinder.org/v2/gh/nci/scores/228-documentation-testing-branch?labpath=tutorials/Threshold_Weighted_Scores.ipynb

The plots do seem to be rendering in binder itself (once launched).
Note re. nbviewer: nbviewer can render differently in the two locations listed above.

Looks like all of this is linked to "whatever makes the notebooks work when they are built on my machine". I re-ran the notebook in my environment and pushed the change up to 228. I'm sure we'll get to the root cause eventually, but for now I will just have to re-run the notebooks each time I push to 228.

I think that the issue isn't the environment. When I run the notebooks in VS code they don't render when I build the docs. When I run the notebooks in a a Jupyter notebook in a web browser, then they do render when I build the docs!

I am making comments about the tutorial here, since the file is too large for inline comments through github code review.
Regarding the tutorial, it notes that MSE can be interpreted as putting equal weight on all thresholds. Since it squares the error, it puts exponentially more weight on higher thresholds, so I think this isn't quite right to say. Also, the previous version noted the issue with MSE potentially obscuring certain things.

With rectangular weighting, it's unclear if multiple segments/intervals could be supplied, e.g. the user might want to weight anything <= 3 and also anything >= 3 as "1", and "0" between -3 and positive 3. I'm assuming this is currently not possible, but the tutorial could do a better job of explaining what an "interval" is here.

The trapezoidal weighting example given in the paragraph "Trapezoidal Weighting" isn't the same as the one presented in code in the example. Consider adding a plot to the "Trapezoidal Weighting" section to show what is described in the text, or consider adjusting the text description to match the later code and say "see below for an example"

With rectangular weighting, it's unclear if multiple segments/intervals could be supplied, e.g. the user might want to weight anything <= 3 and also anything >= 3 as "1", and "0" between -3 and positive 3. I'm assuming this is currently not possible, but the tutorial could do a better job of explaining what an "interval" is here.

Also, how does the user specify greater-than-or-equal to some threshold? i.e. from 3 to infinity.
One option here would be to replace the threshold_where_one with a callable which returned '1' when 1 and 0 when 0.
The nice thing about the current approach is that it's simpler for non-coders who might be less comfortable making up functions on the fly, and also to provide some guidance on the most common things to do. One way to support both approaches would be to be support some optional argument which would allow a function to be passed instead.
I also think we might need to explain why these weighting parameters are being passed in rather than using the already suitable "weights" argument to specify the weights. For example, why are we not using functions like weights_where_one or trap_weights to calculate a suitable weights array instead? Part of the answer is convenience and guidance for users, but I think it could be confusing to people who are thinking about the weights. Do the functions treat the weights in weights differently from the weightings which result frorm the various weighting parameters that are passed in?

It might be interesting to add an example for a variable which has both positive and negative values, either where the interval of interest is around the mean, or where it's on the extremes. It's not clear to me how this can be used on something like temperature, where both very hot and very cold extremes are both of similar (although potentially non-equal) interest. I really like the examples that are currently included, and I'm a bit worried about overcomplicating the tutorial, but I want to make sure this approach is suitable for that situation and an example would both put the implementation to the test and demonstrate appropriate practise.
Examples could include for example human comfort ranges for temperature or humidity, agriculture (where both heat and frost/ice could be problems), or operating equipment like trains where again there would be a temperature range.

I am making comments about the tutorial here, since the file is too large for inline comments through github code review.
Regarding the tutorial, it notes that MSE can be interpreted as putting equal weight on all thresholds. Since it squares the error, it puts exponentially more weight on higher thresholds (since multiplicative biases at higher thresholds will consequentially have higher absolute errors which are then squared), so I think this isn't quite right to say. Also, the previous version noted the issue with MSE potentially obscuring certain things.

@tennlee , they way you have put it is not quite correct.
Take the following example. My decision threshold is 40C. If tomorrow's forecast exceeds 40C, I will tell the workers on my construction site not to come into work tomorrow. Now consider the case where the forecast for tomorrow was 20C and the observations was 10C. That is a large error, and the squared error is even larger (100C^2). However, I still made the correct decision on the basis of the forecast to not close the construction site. So the size of the penalty can be independent of whether the forecast was good enough for people whose decision thresholds are 40C or higher. In this case, threshold-weighted squared error actually give a penalty of 0.
What Nick says is correct. MSE can be interpreted as putting equal weight on all thresholds. The Ehm 2016 paper explains why. Happy to talk about this in another forum rather than a gitlab comment

@nicholasloveday , one of @tennlee comments about weights makes me wonder whether:

there is a weight argument in the threshold weighted scores, which applies specified weights to each of the individual score prior to calculating the mean across some dimensions; and
how this kind of weight differs from threshold-weighting, and whether that is clear in the documentation.

I noticed in your docstrings that you talk about weights where one, etc, which is the second type of weight listed above, not the first.
To avoid confusion, you may need to talk consistently about threshold-weights (and explain that this is related to decision thresholds) to distinguish this from the other type of weighting. Both in the tutorial and in the docstrings for your functions

I am making comments about the tutorial here, since the file is too large for inline comments through github code review.
Regarding the tutorial, it notes that MSE can be interpreted as putting equal weight on all thresholds. Since it squares the error, it puts exponentially more weight on higher thresholds (since multiplicative biases at higher thresholds will consequentially have higher absolute errors which are then squared), so I think this isn't quite right to say. Also, the previous version noted the issue with MSE potentially obscuring certain things.

@tennlee , they way you have put it is not quite correct.
Take the following example. My decision threshold is 40C. If tomorrow's forecast exceeds 40C, I will tell the workers on my construction site not to come into work tomorrow. Now consider the case where the forecast for tomorrow was 20C and the observations was 10C. That is a large error, and the squared error is even larger (100C^2). However, I still made the correct decision on the basis of the forecast to not close the construction site. So the size of the penalty can be independent of whether the forecast was good enough for people whose decision thresholds are 40C or higher. In this case, threshold-weighted squared error actually give a penalty of 0.
What Nick says is correct. MSE can be interpreted as putting equal weight on all thresholds. The Ehm 2016 paper explains why. Happy to talk about this in another forum rather than a gitlab comment

Thanks, I've followed up offline.
My comments aren't about what's right and wrong, but about whether things are clear when introduced to a new reader with a general mathematical understanding of errors but without prior knowledge of decision thresholds. i.e. to help people avoid confusion about the exact disagreement being expressed in the comment.

@nicholasloveday , one of @tennlee comments about weights makes me wonder whether:
1. there is a weight argument in the threshold weighted scores, which applies specified weights to each of the individual score prior to calculating the mean across some dimensions; and

2. how this kind of weight differs from threshold-weighting, and whether that is clear in the documentation.

I noticed in your docstrings that you talk about weights where one, etc, which is the second type of weight listed above, not the first.
To avoid confusion, you may need to talk consistently about threshold-weights (and explain that this is related to decision thresholds) to distinguish this from the other type of weighting. Both in the tutorial and in the docstrings for your functions

Yes, spot on.

Take the following example. My decision threshold is 40C. If tomorrow's forecast exceeds 40C, I will tell the workers on my construction site not to come into work tomorrow. Now consider the case where the forecast for tomorrow was 20C and the observations was 10C. That is a large error, and the squared error is even larger (100C^2). However, I still made the correct decision on the basis of the forecast to not close the construction site. So the size of the penalty can be independent of whether the forecast was good enough for people whose decision thresholds are 40C or higher. In this case, threshold-weighted squared error actually give a penalty of 0

This would be a nice example to include.

The trapezoidal weighting example given in the paragraph "Trapezoidal Weighting" isn't the same as the one presented in code in the example. Consider adding a plot to the "Trapezoidal Weighting" section to show what is described in the text, or consider adjusting the text description to match the later code and say "see below for an example"

Added a text description in the trapezoidal weighting paragraph. I added rather than replaced as it is important to show how trapezoidal weights can be on either one side or both sides of the rectangular weights.

Take the following example. My decision threshold is 40C. If tomorrow's forecast exceeds 40C, I will tell the workers on my construction site not to come into work tomorrow. Now consider the case where the forecast for tomorrow was 20C and the observations was 10C. That is a large error, and the squared error is even larger (100C^2). However, I still made the correct decision on the basis of the forecast to not close the construction site. So the size of the penalty can be independent of whether the forecast was good enough for people whose decision thresholds are 40C or higher. In this case, threshold-weighted squared error actually give a penalty of 0
This would be a nice example to include.

I've added a modified example that lends itself more to a range of decision thresholds rather than a single decision threshold

I noticed in your docstrings that you talk about weights where one, etc, which is the second type of weight listed above, not the first.
To avoid confusion, you may need to talk consistently about threshold-weights (and explain that this is related to decision thresholds) to distinguish this from the other type of weighting. Both in the tutorial and in the docstrings for your functions

I have updated the tutorial and docstrings to clarify this.

With rectangular weighting, it's unclear if multiple segments/intervals could be supplied, e.g. the user might want to weight anything <= 3 and also anything >= 3 as "1", and "0" between -3 and positive 3. I'm assuming this is currently not possible, but the tutorial could do a better job of explaining what an "interval" is here.
It might be interesting to add an example for a variable which has both positive and negative values, either where the interval of interest is around the mean, or where it's on the extremes. It's not clear to me how this can be used on something like temperature, where both very hot and very cold extremes are both of similar (although potentially non-equal) interest. I really like the examples that are currently included, and I'm a bit worried about overcomplicating the tutorial, but I want to make sure this approach is suitable for that situation and an example would both put the implementation to the test and demonstrate appropriate practise.
Examples could include for example human comfort ranges for temperature or humidity, agriculture (where both heat and frost/ice could be problems), or operating equipment like trains where again there would be a temperature range.

Added an example in the tutorial. Also added an additional thing to try in the "things to try next" section.


hydrus-network: Update to version 502

Closes #XXXX

Relates to #XXXX

[x] I have read the Contributing Guide.

/verify

/verify


Can vector be used in smart contract code?

Hi,
I did some research on the smart contract example from https://github.com/pepyakin/substrate-contracts-adder.
This demo code cannot be deployed in substrate because it imports a non-existent function. By translating the wasm binary generated through this code to wat format, I found "(import "env" "llvm.wasm.grow.memory.i32" (func $env.llvm.wasm.grow.memory.i32 (type $t1)))" caused this issue. As far as I can see, this import declaration is related to the use of vector.
So here comes my question: Can vector really be used in smart contract code?
Thanks.

CC @pepyakin

Hello @deeper-zjin !
I am afraid the substrate-contracts-adder is a bit outdated. The problem you are seeing is adder uses old version of wee_alloc and apeears that you use newer version of nightly, where intrinsics to access memory.grow instructions has changed. In newer version of wee_alloc this problem should be resolved. This issue is not exclusively related to Vec, you will see it when you use allocator.
I also want to recommend you to check out pdsl, examples in particular. I also would recommend to subscribe to #1607 to see when the smart contract tutorial is out.


[BUG]'APNSwiftAlert' is deprecated: renamed to 'APNSwiftAlert'

I receive the following warning message:
'APNSwiftAlert' is deprecated: renamed to 'APNSwiftAlert'
Use 'APNSwiftAlert' instead
with code:
let alert = APNSwiftPayload.APNSwiftAlert(                 title: "",                 subtitle:"",                 body: "" )
Platform:

Xcode 13.1
Version apnswift 3.2.0

It might be a bit confusing, because the name is the same, but there is a difference:
APNSwiftPayload.APNSwiftAlert is deprecated.
APNSwift.APNSwiftAlert (or just APNSwiftAlert) is not.
So I don't see any bug here.


Custom styles for selected tabs and readme updates

Added a change to update the style for selected tabs. For example:
selectedTabStyle={{backgroundColor: '...'}}

Also updated the readme to reflect the above change as well as an option of how to hide the navigation on Android.

Thanks for the PR. We're doing some work that requires enhancements to the tabs in the coming days and will take your solution into consideration.

+1

<TabNavigator.Item tabStyle={...} /> is now supported in 0.3.1. If you want to configure the style when the tab is selected, pass in a different style when you pass in selected={true}.


Build compilation to dist folder seems incorrect

After running grunt build in uproxy-networking, the file layout in dist/ is unexpected. In particular, within the simple-socks-firefoxapp folder at dist/samples/simple-socks-firefoxapp, I notice
data/lib/freedom/freedom-for-chrome.js
data/lib/taskmanager/SpecRunner.html
data/lib/taskmanager/taskmanager.js

among others.

This is changing as part of require-ification (samples can be run from build/dev; build/dist is not built by default), so I think we can close this.


Enabled by default usage of internal/pool in internal/query.Client

Pull request type

Please check the type of change your PR introduces:

[ ] Bugfix
[ ] Feature
[ ] Code style update (formatting, renaming)
[x] Refactoring (no functional changes, no api changes)
[ ] Build related changes
[ ] Documentation content changes
[ ] Other (please describe):

What is the current behavior?

Issue Number: N/A
What is the new behavior?

Other information

Codecov Report
Attention: Patch coverage is 0% with 12 lines in your changes missing coverage. Please review.

Project coverage is 42.94%. Comparing base (e563693) to head (68f04d3).

Files with missing lines
Patch %
Lines

internal/query/client.go
0.00%
12 Missing :warning:

:exclamation:  There is a different number of reports uploaded between BASE (e563693) and HEAD (68f04d3). Click for more details.
HEAD has 107 uploads less than BASE

Flag
BASE (e563693)
HEAD (68f04d3)

unit
18
1

go-1.21.x
10
1

ubuntu
20
0

go-1.22.x
10
0

go-1.23.x
12
0

macOS
6
1

windows
6
0

integration
12
0

ydb-23.3
6
0

ydb-24.1
6
0

ydb-nightly
2
0

experiment
2
0

Additional details and impacted files
@@             Coverage Diff             @@
##           master    #1446       +/-   ##
===========================================
- Coverage   67.64%   42.94%   -24.71%
===========================================
  Files         350      295       -55
  Lines       36719    26941     -9778
===========================================
- Hits        24839    11570    -13269
- Misses      10790    14885     +4095
+ Partials     1090      486      -604

Flag
Coverage Δ

experiment
?

go-1.21.x
42.94% <0.00%> (-24.86%)
:arrow_down:

go-1.22.x
?

go-1.23.x
?

integration
?

macOS
42.94% <0.00%> (+1.39%)
:arrow_up:

ubuntu
?

unit
42.94% <0.00%> (+0.96%)
:arrow_up:

windows
?

ydb-23.3
?

ydb-24.1
?

ydb-nightly
?

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Mocking a class with parameters

Hello,
I am currently trying to test out mocking a class that has parameters in its constructor, how would i implement that using MOCK_BASE_CLASS?  In Visual Studio, there are no red underlines, however when I Compile the program, i get the following error:
no appropriate default constructor available
I am using visual studio 2013 and am developing on Windows Embedded Compact 2013

Hi,
The documentation has a few examples: http://turtle.sourceforge.net/turtle/reference.html#turtle.reference.creation.constructor
Does this answer your question? If not can you provide the code you're trying to compile?

The documentation helps, however the compiler is still giving me a hard time.
I'm mocking a class that takes two classes as parameters
I am using the MOCK_CONSTRUCTOR macro to mock my constructor and I am passing it like this:
"MOCK_CONSTRUCTOR(MockClass, 2, (ParameterClass1&, ParameterClass2*));"
Am I using the wrong class in the first parameter of MOCK_CONSTRUCTOR?

You need an «identifier» (see the documentation section I linked):
MOCK_CONSTRUCTOR(MockClass, 2, (ParameterClass1&, ParameterClass2*), MockClassConstructor)

To understand how to use an «identifier» later on you can read:

http://turtle.sourceforge.net/turtle/reference.html#turtle.reference.creation
http://turtle.sourceforge.net/turtle/reference.html#turtle.reference.expectation

That code really helped me out and I can't believe I didn't see that in my first pass through of the documentation.  Is there a way to mock a class that does not have a default constructor?  in the software I am testing, none of our classes have default constructors

Something like
MOCK_CLASS(MockClass, BaseClass)
{
    MOCK_STATIC_METHOD(MockClassConstructor, 2, (ParameterClass1&, ParameterClass2*))

    MockClass(ParameterClass1&, p1 ParameterClass2* p2)
        : BaseClass(p1, p2)
    {
        MockClassConstructor(p1, p2);
    }
};

?

That's helped tremendously.  last question i have for you is what exactly constitutes an identifier?  when i run the boost test where I am testing my mock class, it keeps saying I have an unknown identifier.  I've looked in the code, but can't seem to find anything

Can you show me the code which fails?

MOCK_CONSTRUCTOR(MockClassName, 2, BaseClass(ClassParam1&, ClassParam2*), MockClassName);
I was able to get mock constructor working by adding the class name to the third parameter (something i forgot to do before).  The following code compiles without error, but when I run the unit test that is using this mock class, i get the following error:
unknown location(0): error in "testName": unexpected call: MockCameraLight ( ? , 0050CF80)
I'm pretty sure the last number is a memory address and I am using this class is a pointer.  I would like to note that using the regular class with the same test suite works perfectly fine

You need to set an expectation to define how you expect the constructor to be called, see http://turtle.sourceforge.net/turtle/reference.html#turtle.reference.expectation


squid:S1319 - Declarations should use Java collection interfaces such as 'List' rather than specific implementation classes such as 'LinkedList'

This pull request is focused on resolving occurrences of Sonar rule
squid:S1319 - Declarations should use Java collection interfaces such as "List" rather than specific implementation classes such as "LinkedList".
You can find more information about the issue here:
https://dev.eclipse.org/sonar/rules/show/squid:S1319
Please let me know if you have any questions.
George Kankava

Automated message from Dropbox CLA bot
@georgekankava, thanks for the pull request! It looks like you haven't yet signed the Dropbox CLA. Please sign it here.

Automated message from Dropbox CLA bot*
@georgekankava, thanks for signing the CLA!

Automated message from Dropbox CLA bot*
@georgekankava, thanks for signing the CLA!

Looks good. Where possible, could you remove the import statements that reference the classes you replaced?

Yes I will go through and remove useless imports from the pull request.

@georgekankava looks like you posted an updated version but didn't address all the comments?

All comments have been applied. Please let me know if I have missed something.

Great, thanks for the comment. 👍


additionalProperties + enums

Hey @riedgar-ms, I ported my implementations of additionalProperties and enum to the rewrite.
I don't want to cause a delay to #651 being merged, so feel free to ignore this, and I'll be happy to submit this as a standalone PR to the main repo once the rewrite is merged. Just wanted this to be on your radar!

TODO: array prefixItems

Thanks for working on this - I'm aware that my implementation of the JSON schema was minimal. I'd tend towards letting my PR go through, and having this as a separate one, but we can see what Scott says on that.

I'm in full agreement. I want your PR merged as soon as possible!

@hudson-ai My changes have been merged, if you want to make this a PR against the main repo

@riedgar-ms awesome, great work! I'll rebase and submit a PR when I find some time :)

It was quicker than I expected. PR: https://github.com/guidance-ai/guidance/pull/685


Lolex is very slow

Lolex version : 3.0.0
Environment   : Ubuntu 16 inside VirtualBox
Other libraries you are using:
I'm running Node 10.5.0, and my test runner is Jest.

What actually happens
As in the title, Lolex is very slow, at least much slower than it could be. For a particular test in my project the current Lolex implementation takes 124s to complete on my machine. After some profiling and diagnosing the problem, I managed to trim that down to 22s.
The culprit is clock.timers: it's an object, but it's not an appropriate data structure for operations that Lolex performs on it. Profiling in Chrome showed that the slowest pieces of code in particular are the in loops with hasOwnProperty calls. Using Map for clock.timers resulted in a tremendous speed up.
How to reproduce
I have a piece of code that sets a new timeout a second after the function is called, something like this:
function doStuff() {
  // some stuff
  ...
  setTimeout(doStuff, 1000);
}

Then my test looks like so:
it('should count down', () => {
  const start = process.hrtime();
  server.respond();
  expect(document.querySelector('#clock').textContent).toBe('0h 30m');
  clock.tick(912000);
  expect(document.querySelector('#clock').textContent).toBe('0h 14m');
  clock.tick(828000);
  expect(document.querySelector('#clock').textContent).toBe('0h 01m');
  clock.tick(15000);
  expect(document.querySelector('#clock').textContent).toBe('45s');
});

Basically there's a clock component and I want to test what is displayed at different moments in time. While the test may not be perfect (it tests the whole integration instead of display and step separately), it highlighted the problem.
Next steps
I'll gladly prepare a PR with the changes. From what I've managed to find out, the lowest supported version of Node for Lolex is 6, and it supports Map, so there should be no problem.

Sorry for the lack of communication. I actually made 2 attempts at working on that, but every time I gave up because the way Lolex is structured makes it very uncomfortable for me to work on it (1 big file with everything in it). Also there were some other problems that would need to be solved anyway.
So I'll just close this issue, since at this point it's stale, as the bot said.

Is there any chance we could get the (lolex) code in the stress test above, a 10x change in lolex excution would change my life, even if I have to go bug-hunting. Any help extremely appreciated.
@fatfisz

Same answer as #159 . Just have a look at a suitable priority queue and see if you can make it work. Ask questions that are not already answered by these two issues.

@fatso83 I actually found https://github.com/akhilkedia/lolex-queue/blob/master/fast-lolex.js and got it to work for my needs (the tick() method wasn't implemented (only next() was), and a few other methods it depends on, I implemented it, and it's now over 10 times faster.

@arthurwolf I had a branch with fake-timers using a priority queue but https://github.com/sinonjs/fake-timers/issues/159#issuecomment-390140199

For the "it was never slow for me", it definitely was for me (used to run a
trading bot in "backtesting" mode instead of it's normal "live" mode, so
spending a lot of time within lolex).
About we don't have benchmarks, any timing will show a big difference
between the two methods. The code I forked from (lolex-fast.js) even
provides a testing script in their repo.
On Mon, Nov 29, 2021 at 6:52 AM Benjamin Gruenbaum @.***>
wrote:

@arthurwolf https://github.com/arthurwolf I had a branch with
fake-timers using a priority queue but #159 (comment)
https://github.com/sinonjs/fake-timers/issues/159#issuecomment-390140199
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/sinonjs/fake-timers/issues/229#issuecomment-981312408,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAA2SFO5Q5UKPM2NWS54PS3UOMIKLANCNFSM4GRO3Y6A
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

--
勇気とユーモア


No access to directory after setup in ubuntu 16.04

Cookbook version
Latest from git
Chef-client version
Chef: 12.17.44
Platform Details
Ubuntu 16.04 - AWS
Scenario:
I installed ldap using the the recipe but when I login with phpldapadmin I get the error "This base cannot be created with PLA." and can't edit anything.
If I install the package using apt-get ldap works fine and I can directory from phpldapadmin.
Steps to Reproduce:
Install openldap with:
    # configure ldap machine
    attribute ['openldap', 'tls_enabled'], false # TODO: need proper certificate for this
    attribute ['openldap', 'rootpw'], '{SSHA}XXXXXXXXXXXX'

    recipe 'openldap::server'

Expected Result:
phpldapadmin should be working but it is not
Am I missing something in configuration?

Finally understood that this cookbook is not creating initial base. Maybe a note could be added in README making it more clear what needs to be done.
I can make a PR to add this.
If anyone ends up in this issue and no PR is done to update README here's solution for CentOS:
http://www.idevelopment.info/data/LDAP/LDAP_Resources/OPENLDAP_Initialize_a_New_LDAP_Directory_CentOS5.shtml

This cookbook doesn't take care of populating a database and currently that is out of scope for it. If you'd like you can open up a PR but ideally we'd like to minimize the scope of this cookbook or put that in another cookbook. Closing this as it's not a bug.


Using tax_myPHAGE for viruses reconstructed from metagenomes

Thanks for the great tool! I am currently running tax_myPHAGE on a set of viral genomes reconstructed from metagenomic samples from the human gut. I was wondering if you have any recommendation when applying tax_myPHAGE for this kind of data. E.g. would you recommend keeping only the taxonomy for those viruses with at least medium/high completeness assigned by CheckV? How does the tool behave with viral genomes with lower/medium completeness?
Thank you!
Asier

Hi,
It assumes the genomes are complete, so incomplete genomes will cause the intergenomic similarity values to be inaccurate . The shorter the more inaccurate , as rule of thumb.  Cases around boundaries will become a problem  eg 94.9 v 95 for new or existing species.  But it would get you the correct genus.  I would nt run it with geomes predicted < 70% complete by checkV or be wary of the results
we have an update coming that substantially increase the speed for cases like this , where you want to run lots of samples very quickly

Thank you very much Andrew! I will then retrieve the taxonomy (probably genus-level) only for those viruses above certain quality threshold.
One extra question: I see sometimes the following message in the output file: "The number of expected genera is different from the predicted number of genus clusters. It will require more manual curation". Would you avoid using the taxonomy predicted for these cases or still use it at  certain taxonomic level?

the warning is because of the ICTV taxonomy, with the move to genomic cutoffs there are: 1) genera/species that already existed that don't confirm to these newer guidance and havent been updated 2) edge cases where other methods were used to calculate ANI other that might not take into account identity across the entire genome, leading to values just above (or below) the cutoff values of 70 and 95.  The warning is highlighting one of these - that the existing genus/species are not consistent with current standard cutoffs.  Details of why it might be inconsistent would be in the original taxonomy proposal - it not possible to know with manual checking. Hence the warning
I would report with a * may be inaccurate
You could always submit a taxonomy proposal for next year to resolve and/or correct the issue, for future versions of ICTV...... :-) not that is solves you current problem


Port Cyberdream to Helix editor

helix is one of more recent modal editors, based on kakoune. It has builtin support to treesitter and LSP.
I tried to port it but i think the theme config needs some improvements as it isn't complete match with the original. What do you think? Could anyone help me?

the complete theme creation documentation of helix can be found here: https://docs.helix-editor.com/themes.html

it seems that the cursor has the same color of the background, causing it to be "invisible"

Thanks for opening the PR! Looking great so far 🙂
I'll take a look at the theme when I next get a chance. Never used Helix but keen to take a look.
I encourage anyone who is actively using Helix to give it a spin too and provide feedback.

I've made a few changes and now it looks almost perfect (at least when looking at Golang code). Here's a screenshot of Helix at the top and nvim at the bottom:

Thanks again for the contribution!


Did you try to directly regress the x, y, z of the 3D bounding center?

Thanks for sharing the excellent work and the relevant code!
My question is:
After my own experiments and some slight changes, I found the performance (3D IOU) of the model was pretty bad if I directly regress the coordinate of the 3D bbox centre. However, the 2D Iou was better and reached 90% in the first few epochs. I am really confused about this result. Have you tried to directly regress these parameters? Do you have any idea about it?
These results are generated by resnet18 on Chen Split.
Evaluation result of regressing alpha
Car AP(Average<EMAIL_ADDRESS>0.70, 0.70:
bbox AP:88.38, 71.46, 64.18
bev AP:8.49, 5.02, 4.39
3d AP:2.49, 1.81, 1.51
aos AP:88.33, 71.40, 64.11
Car AP(Average<EMAIL_ADDRESS>0.50, 0.50:
bbox AP:88.38, 71.46, 64.18
bev AP:51.12, 37.06, 34.06
3d AP:31.00, 20.17, 19.07
aos AP:88.33, 71.40, 64.11
Evaluation result of regressing x, y, z and theta
Car AP(Average<EMAIL_ADDRESS>0.70, 0.70:
bbox AP:96.50, 80.83, 63.48
bev AP:0.75, 0.43, 0.38
3d AP:0.22, 0.21, 0.08
Car AP(Average<EMAIL_ADDRESS>0.50, 0.50:
bbox AP:96.50, 80.83, 63.48
bev AP:6.38, 4.43, 3.37
3d AP:4.52, 3.50, 2.40
Also, I noticed that the result on the validation set is pretty good using the full model. But it seems that there is a huge performance gap between the validation set and the test set on the KITTI server. So is it right that currently, the main problem is overfitting?

Could you elaborate on how you "directly regress the coordinate of the 3D bbox center."?  you mean the (x,y,z) coordinate of the 3D object in 3D world? That is basically impossible to directly regress. At least you need to regress (cx, cy, z), where (cx, cy) is the 3D object center projected on the image (regress them using based on the prior of the anchors' position).
Some code snippets would help.
Moreover, 2D bounding boxes are easier to trained.

However, you directly regress the z attribute of an anchor box. That is a 3D attribute that we lose in the 2D image. Then, why cann't we add two more attributes to the prediction? So the estimated attribute becomes delta_x2d, delta_y2d, delta_w, delta_h, x_3d, y_3d, z_3d, w, h, l, theta, which we can also recover a 3D bbox through this prediction.

Because the z_3d is related to the apparent size of an object, so the network is able to analyze it with local semantic information.  I also inject huge priors about the z_3d to each anchor (the mean and variance are collected in the training set, varying across 36 anchors) so that it becomes easier to learn the z_3d.
But as the object shift in x-direction, the appearance does not change and the network can only infer that by its "location", which is difficult for a network to learn with CNN,

Ok, I get it! Thanks for your patience!
Also, do you have any idea about the overfitting problem? The model works quite well in the validation set (Chen split). But the performance on KITTI's test server drops. So it seems that overfitting is the main problem of the performance gap.
However, the 2D IOU goes well in both two sets although the model has not seen the test data. In my opinion, the prediction of the 3D attribute causes severe overfitting on the training data. Maybe that is a potential improvement in the future. Do you agree with that?

We generally agree that images on the test set of the KITTI3D benchmark are more "different" from the training images and they are probably more difficult.
I agree that 2D predictions actually generalize quite well while 3D predictions are not that good. More researches/analysis would be really helpful :)

Thanks for your quick reply! It is really helpful!


Out of memory

Hello, the following problems occurred during the training. May I ask what caused them and what are the solutions?I'm using cuda10.1,python3.6.10.Waiting for your reply online.Thank you.
RuntimeError: cuda runtime error (2) : out of memory at /pytorch/torch/lib/THC/generic/THCStorage.cu:58

Try to reduce the batch size.

I changed the batch size, but this problem occurred again.RuntimeError: value cannot be converted to type double without overflow: inf-------- 原始邮件 --------发件人： Xiaohang Zhan<EMAIL_ADDRESS>2020年5月12日周二 11:45收件人： XiaohangZhan/face_recognition_framework<EMAIL_ADDRESS>TREASUREPPY<EMAIL_ADDRESS>Author<EMAIL_ADDRESS>  题： Re: [XiaohangZhan/face_recognition_framework] Out of memory (#6)
Try to reduce the batch size.
—You are receiving this because you authored the thread.Reply to this email directly, view it on GitHub, or unsubscribe.

@XiaohangZhan hello,I also encountered the same problem, the problem of reducing batchsize to 1 still exists.


[6.0.8] How to see the generated chunk-names?

I'm sorry to open up the issue here, but I tried it on Stack and Laracast and had no luck.
Is it possible to see the chunk names like in the earlier versions of Laravel Mix?
Currently when using npm run dev, I only see this result:
Laravel Mix v6.0.8

✔ Compiled Successfully in 20135ms
┌───────────────────────────────────┬───────────┐
│ File                              │ Size      │
├───────────────────────────────────┼───────────┤
│ /js/vendor.js                     │ 26.6 MiB  │
├───────────────────────────────────┼───────────┤
│ js/routes-sell-food.js            │ 2.15 MiB  │
├───────────────────────────────────┼───────────┤
│ js/resources_js_pages_display-of… │ 2.08 MiB  │
├───────────────────────────────────┼───────────┤

I need this because I'm getting warnings from Workbox like these.
WARNING in The chunk 'js/app' was provided in your Workbox chunks config, but was not found in the compilation.

WARNING in The chunk 'js/moment' was provided in your Workbox chunks config, but was not found in the compilation.

I asked for help in Workbox but I only got a tip that doesn't work in my case.

We hid them because they're generally not useful for most workflows.
You can use a custom plugin or callback to achieve what you want:

mix.after(stats => {
  const { assets } = stats.toJson({ assets: true });

  console.log("\n")
  console.table(assets.map(asset => ({
    name: asset.name,
    chunks: asset.chunkNames.join(", "),
  })))
})

Note that this table appears above the table output by Laravel Mix so you may need to scroll up in your terminal to see it.

We hid them because they're generally not useful for most workflows.
You can use a custom plugin or callback to achieve what you want:

mix.after(stats => {
  const { assets } = stats.toJson({ assets: true });

  console.log("\n")
  console.table(assets.map(asset => ({
    name: asset.name,
    chunks: asset.chunkNames.join(", "),
  })))
})

Note that this table appears above the table output by Laravel Mix so you may need to scroll up in your terminal to see it.


Improve Explain plan

For now when Explain plan is displayed, the eye is focusing on colorized operation params, not on operation name.

@Raubzeug add pic please

Operation
A-Cpu
A-Rows
E-Cost
E-Rows
E-Size
Limit (20)
TopSort (Limit: 20, TopSortBy: argument.Sum0)
Aggregate (GroupBy: , Aggregation: {_yql_agg_0: SUM(item.l_extendedprice * (1 - item.l_discount),state._yql_agg_0)})
InnerJoin (Grace) (o.o_custkey = c.c_custkey)
17 778 257
1 500 304
26 244 071
InnerJoin (Grace) (l.l_orderkey = o.o_orderkey)
12 377 278
1 500 304
26 244 071
Filter (l_returnflag == R)
0
6 001 215
1.0e+8
TableFullScan (Table: dt64/column/tpch/s1/lineitem, Scan: Parallel, ReadRanges: ["l_orderkey (-∞, +∞)","l_linenumber (-∞, +∞)"], ReadColumns: ["l_discount","l_extendedprice","l_orderkey","l_returnflag"])
0
6 001 215
1.0e+8
Filter (item.o_orderdate >= "8674" And item.o_orderdate < "8766")
0
375 000
7 410 701
TableFullScan (Table: dt64/column/tpch/s1/orders, Scan: Parallel, ReadRanges: ["o_orderkey (-∞, +∞)"], ReadColumns: ["o_custkey","o_orderdate","o_orderkey"])
0
1 500 000
29 642 803
InnerJoin (MapJoin) (c.c_nationkey = n.n_nationkey)
450 068
150 000
13 806 800
TableFullScan (Table: dt64/column/tpch/s1/customer, Scan: Parallel, ReadRanges: ["c_custkey (-∞, +∞)"], ReadColumns: ["c_acctbal","c_address","c_comment","c_custkey","c_name","c_nationkey","c_phone"])
0
150 000
13 806 800
TableFullScan (Table: dt64/column/tpch/s1/nation, Scan: Parallel, ReadRanges: ["n_nationkey (-∞, +∞)"], ReadColumns: ["n_name","n_nationkey"])
0
25
1 880


[move] Beginning of a Move source language

Motivation
The purpose of the Move source language is to make writing Move Modules and Scripts easier than through the Mover IR or Move bytecode.
The language is a fully expression based language that aims to making writing Move programs feel familiar to other languages, without hiding the core concepts in Move bytecode.
This code is still HIGHLY in development. There are probably countless bugs, and there are still a good handful of checks missing.
BUT, at this point, it can generate Move bytecode for all of the Core Modules. As such, I am moving it off my branch and switching development to the main Libra repo.
Do not rely on this crate yet! The code (and the syntax of the language) is still extremely volatile. The purpose of this PR is only to switch the development work flow.
See README for some high level architectural overview (should help navigating).
Test Plan
cargo check; cargo test
All testing has been manual up to this point.
Test framework is inbound

:umbrella: The latest upstream changes (presumably 25821e39f273853d5de68cf9a7b96be40584cee6) made this pull request unmergeable. Please resolve the merge conflicts.

:umbrella: The latest upstream changes (presumably 2b972fe9d38c190ad85764a33754eecff2c3e818) made this pull request unmergeable. Please resolve the merge conflicts.

@bors-libra r=tzakian

:pushpin: Commit 359fd60 has been approved by tzakian

:hourglass: Testing commit 359fd60a0c93807175df69a5e0d00aa81cbf059f with merge 782b31cb08eeb717ea2b6f3edbf616b13fd4cae8...

:sunny: Test successful - checks-circle_commit_workflow
Approved by: tzakian
Pushing 782b31cb08eeb717ea2b6f3edbf616b13fd4cae8 to master...


makejdk-any-platform.sh -s, -d parameters do not work as they are supposed to

Using -s which is expected to be given as an absolute PATH does not work - it gets appended to the workspace directory where makejdk-any-platform.sh is invoked. This example output was from trying to set them both to directories under /home/sxa/jdk11:
Didn't find any existing openjdk repository at /home/sxa/openjdk-build/workspace/home/sxa/jdk11/src//home/sxa/jdk11/src so cloning the source to openjdk
git clone --depth=1 https://github.com/adoptopenjdk/openjdk-jdk11u.git /home/sxa/openjdk-build/workspace//home/sxa/jdk11/src/src
Cloning into '/home/sxa/openjdk-build/workspace//home/sxa/jdk11/src/src'...
checkoutRequiredCodeToBuild:
  workspace = /home/sxa/openjdk-build/workspace//home/sxa/jdk11/src/src
  BUILD_VARIANT = hotspot
  TAG =
  BRANCH = dev

Same is true of -d rendering both more or less useless.

Also -T cannot be a full path to the location to put the resulting tarball - if you try you get a message like this:
Moving the artifact to /home/sxa/openjdk-build/workspace//home/sxa/jdk11/build
mv: cannot move `/home/sxa/openjdk-build/workspace/home/sxa/jdk11/src/src/build/linux-x86_64-normal-server-release/images/OpenJDK.tar.gz' to `/home/sxa/openjdk-build/workspace//home/sxa/jdk11/build//home/sxa/MyOpenJDK11.tar.gz': No such file or directory

@sxa Can you add some information how the error can be reproduces? If the error can be easily reproduced by a newcomer we can create a "good first issue" out of this one :)

BUILD_ARGS of
--destintion
--source
don't act as expected.
We need to check the behavior of using the makejdk-any-platform.sh with these options:
see: https://github.com/adoptium/temurin-build/blob/8e043e364ecac19f72fc592116de9386ed7ffd0d/sbin/common/config_init.sh#L263

Needs investigation as to whether these arguments work?

Hi @smlambert can I take on this?

Needs investigation as to whether these arguments work?

@andrew-m-leonard which command do we have to run with these parameters?

Needs investigation as to whether these arguments work?

@andrew-m-leonard @zdtsw which command do we have to run with these parameters?

makejdk-any-platform.sh  -s and -d  are the parameters for it.

makejdk-any-platform.sh  -s and -d  are the parameters for it.

@zdtsw Wondering :open_mouth:  why it isn't picking when I run it from the root directory of the project

makejdk-any-platform.sh  -s and -d  are the parameters for it.

@zdtsw Wondering 😮 why it isn't picking when I run it from the root directory of the project

Care to post your log? how you run it?

Care to post your log? how you run it?

@zdtsw there it is

Care to post your log? how you run it?

@zdtsw there it is

permisson on the file? change to executable and see how it goes without ./

permisson on the file? change to executable and see how it goes without

@zdtsw  Actually I had checked earlier and it has all the file rights read, write and execute

permisson on the file? change to executable and see how it goes without

@zdtsw  Actually I had checked earlier and it has all the file rights read, write and execute

permisson on the file? change to executable and see how it goes without

@zdtsw Actually I had checked earlier and it has all the file rights read, write and execute
Btw am running Ubuntu 20.04
@sxa @zdtsw I actaully tried to spin from another machine today and it didn't help. Could this be another issue?

I do not think this is an issue. At least to call a script, I always use the path or add sh (rarely)
The only case as far as I know is, the executable has been added into the PATH, or it is a linux built-in command, which can skip ./

@sxa can help us reproduce this bug possibly

makejdk-any-platform.sh requires a parameter to specify which JDK to build. If you run it without any parameters it will say:
[sxa@fedora openjdk-build]$ ./makejdk-any-platform.sh
Starting ./makejdk-any-platform.sh to configure, build (Adoptium)OpenJDK binary
Please provide a java version to build as an argument
[sxa@fedora openjdk-build]$

So the normal thing to do would be to run ./makejdk-any-platform.sh jdk.
To use -s (which is to specify a directory on your local machine to clone to) you need to tell it which directory to use for the -s option e.g.
[sxa@fedora openjdk-build]$ ./makejdk-any-platform.sh
Starting ./makejdk-any-platform.sh to configure, build (Adoptium)OpenJDK binary
Please provide a java version to build as an argument
[sxa@fedora openjdk-build]$

If you run ./makejdk-any-platform.sh -h it will give you more information on the parameters, or just look at the source of that help page which is in https://github.com/adoptium/temurin-build/blob/master/makejdk-any-platform.1


is there a way to run keybase on server without a gui?

Is there a way to access my repos in keybase on the server ? this is quite the blocker I have to move all the repos elsewhere otherwise

I saw the docs about running it headless on the server however I could not find the binary to install for headless regular keybase wants to install a bunch of GUI stuff

Hi, unfortunately we don't have a package to just install the CLI and KBFS. You could try building it yourself, or install the entire package and ignore the GUI binary.

I run Keybase on several machines without GUIs. Install the same package, the only difference is when you run run_keybase, add the -g option to not start the GUI.
This is documented here: https://keybase.io/docs/the_app/install_linux

@kg4zow  that would require installation of a full desktop environment tho since the package depends on it

My CentOS 7 server at home doesn't have any kind of GUI installed, and it installed (and upgraded) just fine. The package's dependencies are ...
# rpm -q keybase
keybase-<IP_ADDRESS>20610191041.a459abf326-1.x86_64
# rpm -qR keybase
/bin/bash
/sbin/service
at
fuse
libXss.so.1()(64bit)
lsof
procps
psmisc
rpmlib(CompressedFileNames) <= 3.0.4-1
rpmlib(PayloadFilesHavePrefix) <= 4.0-1

I don't use the Debian packages, but if you are, and it requires a GUI in order to install, you may want to talk to the Keybase devs about fixing the package's dependencies.

☝ that was me, didn't realize I was signed into my work account. Sorry for any confusion.

@kg4zow seems development of keybase slowed down a lot since zoom bought them. :( I am wrong hopefully

@NikosEfthias What I've noticed is that development has slowed down somewhat, and that whatever changes are being made, are being made by people who (1) are not the original Keybase devs, and (2) think that "WIP" is a useful commit message, or maybe that commit messages just aren't important on open-source projects.
My real hope is that, at some point, they'll decide to release the source code for the back end.

https://assetvest.org company is a changing situation in the world economy and financial markets. They ensure that the returns are satisfactory and 100% risk free. predominantly, Assetvest company has been the strong proponent investment strategies for natural resources, stock market and cryptocurrency investment program where investors make daily profits using the ROI (Return on investment) they have stabilized an efficient trading strategy to ensure maximum profit on real stocks, natural resources and Cryptocurrency investments.from what you've invested in the company and they also offer 2.25%-1.77% transfer commission with a minimum deposit of $ 100, The maximum deposit on their investment section (plan)depends on the investment category you are investing on. Their tier is unlimited as the company has multiple investment plans that work well and each of the investment plans has it own percentage offer. Their withdrawal are fast to any external wallet. meaning that after 3days of your investment, you can choose to withdraw both your capital and daily profits. You can even increase your investment by reloading your investment plan, making a new deposit or requesting a new withdrawal limit, you can choose to make a deposit as many times a day or at any time of your choosing as soon as you're liquid enough on your cash flow to make deposits. The same applies when prompted to do so, depending on the amount you have on your balance, you can withdraw at any time. The higher your investment, the faster the returns you will receive in return will grow. They're very stable and profitable. And also fast during the withdrawal on every withdrawal transaction.


Space Name does not accept underscores

Test Environment: https://openshift.io
While trying to create a space with a space name "sam_test_space", I got the following error -

Name must contain only letters (upper case or lower case), numbers, spaces, underscores (_) or hyphens(-). It cannot start or end with a space, an underscore or a hyphen.

As per the above message my space name should have been a valid one. Replacing the underscores in the space names with hyphens made the error message disappear.

@samuzzal-choudhury Fixed with https://github.com/fabric8-ui/ngx-fabric8-wit/pull/81
and new released<EMAIL_ADDRESS>


Error: Can not register multiple components with the same tag

Can some one help me to provide a help to get rid of this. I know in the current library there is no option to deregister the component, any help would be appreciated.
We are having single page where we are trying to load the different component at run time where tags are same for each component. We can't change the tag. Please let me know how to resolve this issue.

Can you elaborate on the use-case? Is there no way to make the tag name unique based on the use-case or even the url?


Use COLUMNS environment variable for console width

If this variable is defined (which is not always the case but at least
respecting it allows us to wrap things properly if it is), use it instead of
the hard-coded 80 character line width.
Closes #7.

Notice that I had to update catch.hpp as well because otherwise I was getting incomprehensible errors from main.cpp which used Tbc::TextAttributes from there and not from clara.h itself. In fact, I think this points out a serious problem with the current stitching approach: it can easily result in ODR violations because the same stuff is stitched into both catch.hpp and clara.h. To solve this you'd need to use unique namespaces for the internal stuff for the two headers.
BTW, why does Clara use .h extension when CATCH uses .hpp? Wouldn't it be better to use the latter for Clara too, if only for consistency?

Thanks for this. I'd like to use this PR at some point.
My concern at the moment is that I think you've hit on a problem with a half-finished implementation of an idea. I think I need to do a bit more work on the stitching script as the TextAttributes lib should be independently embedded in both Clara and Catch - thus avoiding ODR (and other) issues.
Unfortunately I haven't touched that for a while and I don't think it was completely finished. It's also a bit complex. So I'm going to need to do a big push to get that sorted - which I won't have time for for a while.


is_flash_attn_2_available() returns False

I installed the repo without CLI on virtualized instances from Vast.ai with A100 40GB and 80 GB.
is_flash_attn_2_available() is False. Does it mean flash-attn is not used by inference. does it adversely affect speed?
Using the Snippet on README
model_kwargs={"attn_implementation": "flash_attention_2"} if is_flash_attn_2_available() else {"attn_implementation": "sdpa"}

what is sdpa that is used instead? is it slower than flash-attn? If yes, what can I do to use the faster flash-attn?
It takes ~3s for a 10s audio.
See also #191
My Code:
import torch
from transformers import pipeline
from transformers.utils import is_flash_attn_2_available
import time
import numpy as np
import torch
from subprocess import CalledProcessError, run

def load_my_audio(file: str, sr: int = SAMPLE_RATE):
    #from whisper code base. loads audio into a numpy array
    ...

device = "cuda:0" if torch.cuda.is_available() else 'cpu'

pipe = pipeline(
    "automatic-speech-recognition",
    model="openai/whisper-large-v3", # select checkpoint from https://huggingface.co/openai/whisper-large-v3#model-details
    torch_dtype=torch.float16,
    device=device, # or mps for Mac devices
    model_kwargs={"attn_implementation": "flash_attention_2"} if is_flash_attn_2_available() else {"attn_implementation": "sdpa"},
)
np_array = load_my_audio("10s.wav")

generate_kwargs = {
    #'num_beams': 1,
    "language": 'it',
    "do_sample": False,
    #"temperature": 0,
    #"repetition_penalty": 3.0,
    # "condition_on_previous_text": True,
    #"task": "transcribe"
}

startt = time.time()
outputs = pipe(
    # "10s.wav",
    np_array,
    chunk_length_s=30,
    batch_size=1,
    return_timestamps=False,
    generate_kwargs = generate_kwargs
)
endt = time.time()
duration = endt - startt
print(duration)

Hey,


draft pack-repo add fails after upgrading OSX Git client to 2.16.1

When using git 2.15.1 I'm able to add pack repos just fine, after upgrading to git 2.16.1 I get the following error:
Error: Unable to update checked out version: exit status 128

draft version:
Client: &version.Version{SemVer:"v0.9.0", GitCommit:"6a6b387d84cc7530b45bfff704bd1d71e36082ed", GitTreeState:"clean"}

I'm not sure if this is a draft issue or not but figured I'd raise and track here as I'm sure others will be affected.
This is a full log of the steps to recreate:
➜   git version
git version 2.15.1

➜   draft pack-repo remove github.com/Azure/draft
Are you sure you want to do this? (y/N): y
removed pack repository github.com/Azure/draft

➜   draft pack-repo list
Error: no pack repositories to show
Error: exit status 1

➜   draft pack-repo add https://github.com/Azure/draft
Installing pack repo from https://github.com/Azure/draft
Installed pack repository github.com/Azure/draft

➜   brew upgrade git
Updating Homebrew...
==> Auto-updated Homebrew!
Updated 3 taps (caskroom/cask, homebrew/core, jenkins-x/jx).
==> New Formulae
ballerina                                coreos-ct                                qtkeychain                               shogun
==> Updated Formulae
jenkins-x/jx/jx ✔   chibi-scheme        harfbuzz            ledit               metricbeat          pyinvoke            svgo                zimg
abnfgen             conan               heartbeat           librealsense        packetbeat          python-markdown     terragrunt
apm-server          dbus-glib           heroku              libtrng             passenger           rabbitmq            udpxy
auditbeat           dwarfutils          html-xml-utils      log4cplus           picard-tools        rakudo-star         vagrant-completion
augeas              elasticsearch       i2p                 logentries          plantuml            rtags               vala
bit                 elixir              jhipster            logstash            presto              rustup-init         xrootd
carthage            filebeat            kibana              makensis            prips               sccache             zanata-client

==> Upgrading 1 outdated package, with result:
git 2.16.1
==> Upgrading git
==> Downloading https://homebrew.bintray.com/bottles/git-2.16.1.sierra.bottle.tar.gz
==> Downloading from https://akamai.bintray.com/6b/6b2d4930aabf8502bfe3fc654599deb8ee189a8ba3812a2ce04ee419365b9611?__gda__=exp=1517396029~hmac=bf6d4c41268134d3cea7
######################################################################## 100.0%
==> Pouring git-2.16.1.sierra.bottle.tar.gz
==> Caveats
Bash completion has been installed to:
  /usr/local/etc/bash_completion.d

zsh completions and functions have been installed to:
  /usr/local/share/zsh/site-functions

Emacs Lisp files have been installed to:
  /usr/local/share/emacs/site-lisp/git
==> Summary
🍺  /usr/local/Cellar/git/2.16.1: 1,495 files, 34.5MB

➜   draft pack-repo add https://github.com/Azure/draft
Installing pack repo from https://github.com/Azure/draft
Error: Unable to update checked out version: exit status 128
Error: exit status 1

I can confirm this on ubuntu as well. I'll see if I can figure this one out.

fwiw this appears to work:
$ draft pack-repo add https://github.com/Azure/draft --version v0.10.0

I submitted a patch upstream here. Once that's accepted, we can then update Masterminds/vcs in draft and draft-pack-repo to close this one out.

That's great - thanks for the updates and workaround.

Also bit by this. Nice to see a workaround 👍

Just ran into this issue. Thank you so much! 🙇


[Usage]: About '--chat-template' parameters for model google/paligemma2-3b-ft-docci-448

Your current environment
I use template_llava.jinja to launch the model google/paligemma2-3b-ft-docci-448. Despite working, I wonder 1) how to decide on a template for a specific model and 2) whether my setting is correct for the model google/paligemma2-3b-ft-docci-448?
vllm serve google/paligemma2-3b-ft-docci-448 --chat-template template_llava.jinja  --host <IP_ADDRESS>  --port 8001 --enforce-eager --dtype auto

on a
How would you like to use vllm
I want to run inference of a [specific model](put link here). I don't know how to integrate it with vllm.
Before submitting a new issue...

[X] Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

google/paligemma2-3b-ft-docci-448 doesn't have a real chat template, so I suggest you use the dummy one which does not format the text, similar to https://github.com/vllm-project/vllm/blob/main/examples/template_vlm2vec.jinja

@DarkLight1337 Thanks for your information. I checked with https://github.com/vllm-project/vllm/blob/b689ada91e381ac8a1b197b5a52d7f1fa32fac05/vllm/model_executor/models/paligemma.py#L137 and https://github.com/vllm-project/vllm/blob/b689ada91e381ac8a1b197b5a52d7f1fa32fac05/vllm/model_executor/models/qwen2_vl.py#L839. For paligemma, this is no PromptReplacement in place. so that's why you said dummy template can work?

No, that's unrelated to the chat template. The chat template is defined on HuggingFace repo (via chat_template.jinja or tokenizer_config.json) for most models.

@llv22
To clarify, the API server we provide will only work with instruct-format multimodal models. This is because we want to provide compatibility with OpenAI Vision API which's also based on /chat/completions (instead of /completions which has been deprecated by OpenAI themselves).
PaliGemma in particular was never instruct-finetuned or post trained, therefore using a chat template with it might result in unexpected behaviour. I suggest you use the model with llm.generate API or build a simple API server on top of it to support multimodal inference with /completions.


Reorganize tests and remove redundant integration tests

The tests are organized in a confusing structure and not as we usually organize them. There are a lot of redundant integration tests and some adapters lack unit tests.

[ ] 1. This test suite can be used to ensure we are spec compliant. It can either be used to generate test code or parsed and tested.
[x] 2. This folder has lots of integration tests for all operations (we should remove redundancy)
[x] 3. Add integration tests for "Test" operation
[ ] 4. Clean up test object model classes - simplify by adding them to test classes that use them
[x] 5. Ensure all test classes follow the Arrange, Act, Assert pattern; variable names are coherent/sensible (replace patchDoc with patchDocument,  obj with object/targetObject etc.), remove unnecessary serialization and formatting of error messages
[x] 6. ExpandoObjectAdapter and PocoAdapter do not have unit tests

This PR adds a new adapter called DynamicObjectAdapter with unit tests and integration tests in a folder called IntegrationTests.
cc @rynowak

ExpandoObjectAdapter was combined with DictionaryAdapter, so now it has unit tests. See this
PocoAdapter unit tests were added here

PR #117 cover 2., 3. and 5.

Shelving 1. for now. See #121


Failed to execute 'copyExternalImageToTexture' on 'GPUQueue': Copy rect is out of bounds of external image

Hi! Really nice project over here. Unfortunately it's not possible to preview it at the moment because of the error in the title.

Thank you for pointing it out, it should be fixed now! We are still working actively on this project and need to add/fix a lot of features, but happy for any feedback you have for us!


Support C#6 features

Expected

It should emit method definitions like int f() => 3 that have expression-bodies. (at the moment they're not emitted at all, so invocations of them result in a JS error that the method f doesn't exist).

It should likewise emit expression-bodied properties like int q => 3. (at the moment they're again not emitted at all).

It should handle the ?. operator. At the moment ?. is emitted identically to ., so the code string s=null; int? len = s?.Length causes a JS null reference exception.

It should handle string interpolation $"..." the same as String.Format, since that's what string interpolation compiles down to. (unless it's target-typed to something that is or implements IFormattableString). At the moment, interpolated strings are silently ignored.

Actual

Expression-bodied methods and functions aren't being emitted. I haven't tested expression-bodied operators but I bet they're the same.

The ?. operator isn't being handled.

All interpolated strings are silently ignored.

Steps To Reproduce
Install the VSIX. Do File>New>C#>Bridge.NET>ClassLibrary and use the following source code
namespace ClassLibrary1
{
    public class App
    {
        [Bridge.Html5.Ready]
        public static void Main()
        {
            string s0 = null;
            string s1 = "hello";
            int? len0 = s0?.Length;
            int? len1 = s1?.Length;
            string s2 = $"f={f()}";
            string s3 = string.Format("f={0} p={1} q={2} len0={3} len1={4}", f(), p, q,len0,len1);
            string s4 = f().ToString();
            System.Console.WriteLine(s1 + s2 + s3 + s4);
        }

        static int f() => 3;
        static int p { get; } = 4;
        static int q => 5;
    }
}

Build it.

Hi @ljw1004 - Thanks for the great sample. We have all this C#6 functionality working, and it will be released after Bridge 1.14.
We copied your sample into our demo project, and the following js code block shows the compiled output. The browsers Console output is below that.
Compiled JS
(function (globals) {
    "use strict";

    Bridge.define('Demo.App', {
        statics: {
            config: {
                properties: {
                    p: 4
                },
                init: function () {
                    Bridge.ready(this.main);
                }
            },
            getq: function () {
                return 5;
            },
            main: function () {
                var s0 = null;
                var s1 = "hello";
                var len0 = s0 != null ? s0.length : null;
                var len1 = s1 != null ? s1.length : null;
                var s2 = Bridge.String.format("f={0}", Demo.App.f());
                var s3 = Bridge.String.format("f={0} p={1} q={2} len0={3} len1={4}", Demo.App.f(), Demo.App.getp(), Demo.App.getq(), len0, len1);
                var s4 = Demo.App.f().toString();
                console.log(s1 + s2 + s3 + s4);
            },
            f: function () {
                return 3;
            }
        }
    });

    Bridge.init();
})(this);

Console Output
hellof=3f=3 p=4 q=5 len0= len1=53

We'll keep this thread open, and close once the functionality has been added to the master branch, after the 1.14 release.
Hope this helps.


Invalid amount passed

For card checkout, the amount passed through charge is not formatted correctly.
static String formatAmount(num amountInBase) {
    if (_currencyFormatter == null) throw "Currency formatter not initialized.";
    return _currencyFormatter!.format((amountInBase / 100));
  }

https://github.com/wilburt/flutter_paystack/blob/98d93d32df9a368a923bffd375e7b9479c85f85f/lib/src/common/utils.dart#L19
Amount is divided by 100, which alters the value entered by users.

This is a documented behaviour and is behaving as expected.
https://github.com/wilburt/flutter_paystack/blob/98d93d32df9a368a923bffd375e7b9479c85f85f/lib/src/models/charge.dart#L15

when creating a charge object if i set the amount to be 10,000 on the paystack ui it shows 100.0
Charge charge = Charge()
..amount = plan.amount
 ..reference = reference
..email =<EMAIL_ADDRESS>i think there is an issue with your currency formatter


[FR] Support for Additional Backends Compatible with OpenAI Protocol

It would be beneficial to add support for additional backends compatible with the OpenAI protocol, with the option to specify URLs through environment variables or a configuration file.

Support for arbitrary OpenAI API compatible backends just got added: https://github.com/tom-doerr/zsh_codex/pull/49
Is this what you are looking for?

Thanks for your response and hard work! I'm glad to see that #49 addresses my issue perfectly. This is precisely what I needed. Thank you again, and I'm excited to see how the project evolves in the future!


Unable to compile certain framework files

I used the files feature of barryvdh/laravel-debugbar on several pages to see what files were being loaded on most requests in order to add those files to the compiled.php that artisan optimize creates.
The following two files appeared on every request, so I figured that I could safely add them to the compiled list:

vendor/laravel/framework/src/Illuminate/Foundation/Auth/User.php
vendor/laravel/framework/src/Illuminate/Database/Connectors/Connector.php

However, when I try to load my app after compiling the file, I get the following errors (respectively for each of those files):
DATETIME [error] 975#0: *193 FastCGI sent in stderr: "PHP message: PHP Fatal error:  Cannot declare class Illuminate\Foundation\Auth\User, because the name is already in use in /home/vagrant/websites/example.com/bootstrap/cache/compiled.php on line 16873" while reading response header from upstream, client: <IP_ADDRESS>, server: example.com, request: "GET / HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm-example.com.sock:", host: "example.com"

DATETIME [error] 975#0: *193 FastCGI sent in stderr: "PHP message: PHP Fatal error:  Cannot declare class Illuminate\Database\Connectors\Connector, because the name is already in use in /home/vagrant/websites/example.com/bootstrap/cache/compiled.php on line 18346" while reading response header from upstream, client: <IP_ADDRESS>, server: example.com, request: "GET / HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm-example.com.sock:", host: "example.com"

Why would PHP be trying to load these files even though the classes have already been defined in compiled.php?  Is this a bug, is there some way around this, or do I have to allow it to load those files individually instead of compiling them into the big file?

Note that there may be other files affected by this as well, but these are just the only two I have discovered at the moment.

It appears that this happens when some classes are listed in a different order than how they are loaded.  Interestingly, some child classes can be defined before their parent classes, but others cannot.  It's really just trial and error with the arrangement to determine if this is the case for any particular class.


I want to know the difference between ACMP and ACMMP.

Are there any papers on ACMMP?
I want to know the difference between ACMP and ACMMP.

ACMM + Planar prior


Add core files.

I needed some signature testing software for bedrock-key and determined that there was actually no http-signature specific stuff in http-signature-crypto, so I think a more generic name is appropriate.  I checked NPM and signature-algorithm is available, but I'm open to other names.

Other than some renaming this is code just copied from http-signature-crypto?
signature-algorithm vs signature-algorithms?
This looks similar to the json-web-algorithms package api.

@davidlehn yes, this is just a copy.
took a peek at json-web-algorithms https://github.com/jhermsmeier/node-json-web-algorithms it is similar, but does not have ed25519 support.
No 's' because I'm saving disk space, I'll go with the consensus opinion on  a new name.

Yeah, I'm not saying to use that other package, was just pointing it out since I saw it while researching how others handle the tradeoff between disk space and a trailing 's'.

I found a few non-pluralized modules:
https://github.com/digitalbazaar/bedrock-key
https://github.com/digitalbazaar/bedrock-identity
https://github.com/digitalbazaar/bedrock-agreement
and the same people also came up with these names:
https://github.com/digitalbazaar/bedrock-messages
https://github.com/digitalbazaar/bedrock-docs
https://github.com/digitalbazaar/bedrock-jobs

They must be practicing the ancient wisdom that two standards are better than one.

I vote to add an s if we keep this generic module. But this module does seem a bit weird to me. How do we determine what gets added (or not) to it over time? Can it bloat to become unwieldy? Should we split the individual algorithms up -- or would that be something to wait to do in the future... but then this module just becomes a shell?
It seems like the future of this module would to provide just the base API for signing/verifying, much like node's crypto lib's createSigner/Signer/Verifier API. The individual algorithms would be their own packages that could be installed. It may be nice to just extend/wrap that existing API ... but that would only work on node.
A monolithic library will become a bigger problem more quickly when we want to use this on the client (which we do want to do). That's another argument to have a simple "plugin" API with individual modules providing the implementations for specific algorithms. Obviously there's additional overhead with more packages, but it seems like the right way to do it.

@dlongley you are suggesting that we split the alogithms into separate modules?  And would this module load some set of default modules as now?
Can we kick the can on these concerns in the interest of time?  We've added exactly one signature algorithm in the past 3 years.

@mattcollier,

Can we kick the can on these concerns in the interest of time? We've added exactly one signature algorithm in the past 3 years.

Yes, let's kick the can for now but please leave a TODO to split this module up into a main API with no signatures installed and plugins that people can install to add the signature algorithms they want.