[Subtask]: Store watermark in the snapshot level

Search before asking

[X] I have searched in the issues and found no similar issues.

Description
Currently, the watermark is stored in table properties, which makes it impossible to trace historical watermarks. We should store the watermark at the snapshot level, such as in the snapshot summary.
Parent issue
#1354
Are you willing to submit PR?

[ ] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

https://github.com/apache/iceberg/pull/8553 https://github.com/apache/iceberg/pull/9346

Closed as not planned.


php7 version doesn't work because php5 is missing...

What's the problem?
nginx won't start..  Gives this error:  php-cgi.exe System Error The program can't start because php5.dll is missing from your computer.
What do we need to reproduce the issue?
Windows 8.1
This is the file I downloaded: WPNXM-0.8.5-LiteRC-Setup-php70-w64.exe

Q
A

WPN-XM Version (wpn-xm.exe --version)
x.y.z

Windows Version (ver)
x.y.z

How to reproduce the issue?
Please add steps to reproduce the issue.

What should have happened?
Please describe the expected behavior.
Do you have an idea for a solution?
Please describe a possible solution.

Hey Thor!
I've downloaded and installed WPNXM-0.8.5-LiteRC-Setup-php70-w64.exe.
I can not reproduce the error. It works for me out of the box.
Also any PHP7 version includes php7.dll and not php5.dll.
I'm not sure what's going on there.
Would you be so nice to add more steps to reproduce the issue? Thank you.
Yours, Jens

Sorry, i just noticed that i tested the latest version 0.8.6.
WPNXM-0.8.5-LiteRC-Setup-php70-w64.exe:
Has several problems, which were adressed in the 0.8.6 version.

nginx was not extracted into bin\nginx, instead you get a versionized folder bin\nginx-1.9.2

workaround is to move all files from  bin\nginx-1.9.2 into bin\nginx

The PHP extension xdebug wasn't compatible with PHP7 version at that time. You need to download a matching version and replace php_xdebug.dll.

D:\wpnxm-085-literc70-64\bin\php>php -v
Failed loading d:\wpnxm-085-literc70-64\bin\php\ext\php_xdebug.dll
PHP 7.0.0beta1 (cli) (built: Jul 10 2015 10:56:50)
Copyright (c) 1997-2015 The PHP Group
Zend Engine v3.0.0-dev, Copyright (c) 1998-2015 Zend Technologies
    with Zend OPcache v7.0.6-dev, Copyright (c) 1999-2015, by Zend Technologies

just remove the inclusion of xdebug. You find it at the bottom of php.ini.


refine docs about loading stories dynamically for react-native

I refined the docs about loading stories dynamically, as it's maybe not that clear to everyone that this won't work for react-native.

Excellent! Thanks a bunch @formatlos !


Make BlockchainProvider2 transaction id methods aware of in-memory state

Right now these methods:
https://github.com/paradigmxyz/reth/blob/a7062066958f45665f41de58d94b1bc5c6d68c57/crates/storage/provider/src/providers/blockchain_provider.rs#L585-L598
just check the database for transactions. Previously this was fine because the database was always up to date with the canonical chain. Now, this is not the case, so these need to also check in-memory state.
This method also needs updating:
https://github.com/paradigmxyz/reth/blob/a7062066958f45665f41de58d94b1bc5c6d68c57/crates/storage/provider/src/providers/blockchain_provider.rs#L621-L623
As do these methods:
https://github.com/paradigmxyz/reth/blob/a7062066958f45665f41de58d94b1bc5c6d68c57/crates/storage/provider/src/providers/blockchain_provider.rs#L679-L695

hey @Rjected, I'm so excited to see this PR being closed due to their complexity. I want to learn about how works the handling of queries over in-memory state.

Well, I did my homework. I thought and reach out in this solution:

Iterates over all blocks
Search in transactions by ID provided
Returns the block

https://github.com/paradigmxyz/reth/blob/a7062066958f45665f41de58d94b1bc5c6d68c57/crates/storage/provider/src/providers/blockchain_provider.rs#L621-L623
cc @Rjected

Well, I did my homework. I thought and reach out in this solution:
1. Iterates over all in-memory blocks

2. Search in transactions by ID provided

3. Returns the block

https://github.com/paradigmxyz/reth/blob/a7062066958f45665f41de58d94b1bc5c6d68c57/crates/storage/provider/src/providers/blockchain_provider.rs#L621-L623
cc @Rjected

The fundamental problem with these txid methods, is that we don't really keep track of a txid index for in-memory blocks, unlike db blocks. So it would be good to do something like:
if let Some(num) = self.database.transaction_block(id)? {
    return Ok(Some(num))
} else {
    // what do we do here?
}

For the else branch, we should get the id for the latest block on disk, and iterate through in-memory blocks to determine which block the transaction is in. The id here is the tx number

Makes sense. Btw, why in-memory blocks don't works equal to blocks on disk?

Makes sense. Btw, why in-memory blocks don't works equal db blocks?

wdym? do you mean why do we not keep a txid index for in-memory blocks?

Makes sense. Btw, why in-memory blocks don't works equal db blocks?

wdym? do you mean why do we not keep a txid index for in-memory blocks?

yeah

Makes sense. Btw, why in-memory blocks don't works equal db blocks?

wdym? do you mean why do we not keep a txid index for in-memory blocks?

yeah

ah, we just don't do it yet, since it is extra complexity for maintaining the in-memory state. If it seems like a bottleneck after measurement, then we may think about introducing an index

hey @Rjected, what do you think about if I unassign this? Sincerely, I still can't make queries like this due to my capacity

I will wait by solution. I want to see how will someone resolve it...

Hey @Rjected, I've tried to give it try and I came up with the impl based on your comment https://github.com/paradigmxyz/reth/issues/10181#issuecomment-2276622516.
https://github.com/lakshya-sky/reth/blob/861c9773f6002e7099e82f4e871d7509c81e5707/crates/storage/provider/src/providers/blockchain_provider.rs#L625C4-L652C6
I wanted to confirm with you that should I continue?


Piwik become Matomo

Hello,
Yeap, you have to change name Piwik to Matomo ^^
Thanks

Thanks for your requests, I will see to find a solution for renaming the plugin.

Hi @MichelePalmieri !
Great to see you working at this issue! Do you plan to work on feature requests and another issues again? I've asked for renaming in this request, do you have an opinion to this? https://github.com/getgrav/grav/issues/2032
Kind regards,
Ralf


overriding the payload creator and new error is not automatically caught.

export const isLoading = createAction('IS_LOADING',(isLoadingFlag,message) => {

  return {
      isLoading:isLoadingFlag,
      message:message,
  }
})

/// creation action without payload creator

export const isLoadingWithDefault = createAction('IS_LOADING')

// Using the action creators

this.props.isLoading(new Error("Message!!!!!")) // => error doesn't exist is isn't populated and payload:object

this.props.isLoadingWithDefault(new Error("Message!!!")) // => error:true payload: is the error object

solution on custom creation action this will trigger so that action will contain an error:true
but requires that we must write odd code that will detect and error as first or last parameter ?
export const isLoading = createAction('IS_LOADING',(isLoadingFlag,message) => {
  // if it is an instance of an check
  if (isLoadingFlag instanceof Error) {
    return isLoadingFlag
  }
  else {
    return {
        isLoading:isLoadingFlag,
        message:message,
    }
  }
})

That issue has been resolved with https://github.com/acdlite/redux-actions/commit/b9d5eb7daec5e29cb02c34dfb855c51183051f71.


Added Feature -- Pagination for FormSets

Formset includes a paginator method which returns a django.core.utlis.Paginator object.
This will be a handy and useful shortcut to create paginator directly from the formset instance.
Added tests and documentation as well for formsets.paginator

All new feature require an accepted trac ticket.

All new feature require an accepted trac ticket.

Okay, I didn't know that. I will write a new ticket for this first and then rename this PR.

Closing as per ticket.


Qiskit 0.12 not tracking global phase properly

Information

Qiskit Terra version: 0.9.0, Qiskit 0.12.0
Python version: 3.7.4
Operating system: macOS 10.14.6

What is the current behavior?
A single qubit circuit of Rx(pi/2)X|0> produces different state vector vs Rx(pi/2)U3(pi,0,pi)|0>, which supposedly should be identical.
Qiskit 0.11.0 does not have this behavior.
Steps to reproduce the problem

Build a single qubit circuit in |0> state
Apply X gate then apply Rx(pi/2) gate
Simulate using Aer statevector_simulator
Build another single qubit circuit in |0> state
Apply U3(pi,0,pi) gate then apply Rx(pi/2) gate
Simulate using Aer statevector_simulator
Compare the two state vectors produced

What is the expected behavior?
The two state vectors produced should be identical.
Suggested solutions
Track global phase properly for all unitary gates.

This issue is described in #3083.


Feature: Algolia ISearchIndex Customization

Motivation
Fixes #15 by adding a new IAlgoliaIndexService and a default implementation of DefaultAlgoliaIndexService which reproduces the existing behavior.
Developers can use decoration to get access to the IAlgoliaIndexService, or use the ISearchClient directly to generate a new index.
The latter approach also allows developers to target a different index per-environment, which isn't currently possible because index names have to be const string values on the index model.
This PR also addresses the following issues:

Adds VS Code editing support with workspace settings (defines the solution file for Omnisharp)
Sets an explicit language version in Kentico.Xperience.AlgoliaSearch.csproj to C# 7.3, because this is what is compatible with netstandard2.0 which is what the library targets
Removes public modifier from all interface methods because this is not supported with C# 7.3

Also removes abstract interface method modifier which is only useful when other methods provide default implementations

Fixes XML doc comments
Fixes C# standard analyzer warnings for internal code (no public API change)

Checklist

[X] Code follows coding conventions held in this repo
[ ] Automated tests have been added
[X] Tests are passing
[ ] Docs have been updated (if applicable)
[ ] Temporary settings (e.g. variables used during development and testing) have been reverted to defaults

How to test
No tests were added since the new abstraction has the exact same behavior as the existing code.
dotnet test

I have a long-term vision to expose Rules defined in the application through a dashboard in the CMS.
This would enable marketers and administrators to enable/disable Rules as needed.
Rule toggling could support different marketing/promotion scenarios.

PR being reviewed here https://github.com/Kentico/xperience-algolia/pull/17


Generate shell completion scripts

This is in addition to the work done in PR#703. The status of this PR is WIP and pushed to get feedback from the maintainers.
I've made a few changes to the public API of ArgumentParser to get all the meta data required to generate the scripts, as well as the descriptions that zsh shows. For the SwiftPM tools, this means no duplication of the arguments anymore inside SwiftPM.
Things left to tackle:

[x] Support positional arguments
[x] Cleanup the code along with some documentation

Not handled in this PR, requires further discussion:

Compiler flags are still duplicated; should we add some sort of "discovery" based on the output of swift -h? -- What to do with flags we cannot handle (nicely), like -l<name>?
How to include this in the toolchain?
How to generate the full completion script, including setup code? (see _swift functions in the current scripts under Utilities/{bash,zsh}.

Ok I've dropped the script generation commands from this PR. I've included information in the generated output on how to use the shell scripts. I've also rebased the PR on top of master.

What's the status on this one?

I've rebased this PR on master

What's the status on this one?

LGTM!
Please resolve conflicts and rebase then we can run swiftci and merge.
Thank you for your work on this!

Great! I've rebased the PR and tests successfully on my machine.

@swift-ci please smoke test

@swift-ci please smoke test


fastjson version is too old

rocketmq use fastjson 1.2.69 which is too old, the newest version is 1.2.76

@nibiwodong Would you like to submit a PR to bump up this version?

@nibiwodong Would you like to submit a PR to bump up this version?

Sure.


Add retriable headers field for Retry Policy

Please describe the Improvement and/or Feature Request

For the retryOn field in retry policy, you can set it to various policies that Envoy has set https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/router_filter#x-envoy-retry-on
One of the policies is retriable_headers, which when set would have allowed users more control over what to retry on https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/router_filter#x-envoy-retriable-header-names
Currently, that policy isn't functional at the moment, since we don't have a retriable_headers field where the user would be able to specify what to retry on.
Scope (please mark with X where applicable)

New Functionality          [X ]

Possible use cases

I can work on this

Pls assign it to me


Failed: Update autorest version to prerelease 3.0.0-alpha.20240423.5.x

Triggered from pull request: https://github.com/Azure/autorest.csharp/pull/4629

API change check
APIView has identified API level changes in this PR and created following API reviews.
Azure.Health.Insights.RadiologyInsights

Close in favor of #43678


String identifier values in whereClauses for any type of WhereClause are not handled correctly

The ARS User Guide indicates that, if AnalysisSet, AnalysisGroup or DataGroup contain a compoundExpression, the whereClauses attribute should contain the identifier value of a referenced AnalysisSet, AnalysisGroup or DataGroup, represented as a string value.  However, data represented in this way fails validation according to the JSON-Schema representation of the model.

The model has been updated to include a new subClauseId slot attribute that is used to hold the identifier value of an AnalysisSet, DataSubset or Group referenced in a sub-clause referenced in the whereClauses attribute of a compound expression.  The model has also be updated to support generation of a JSON-Schema representation that correctly applies the following (updated) requirements:

A compound expression in any type of WhereClause may contain either a fully-defined WhereClause (including either a defined condition or another defined compoundExpression) or a referenced to an "identified WhereClause" (i.e., an AnalysisSet, a DataSubset or a Group).
When an "identified WhereClause" is referenced by identifier in the subClauseId attribute
of a compound expression:

The subClauseId attribute in a compound expression in an AnalysisSet may contain only the identifier value of another AnalysisSet.
The subClauseId attribute in a compound expression in a DataSubset may contain only the identifier value of another DataSubset.
The subClauseId attribute in a compound expression in a Group may contain only the identifier value of another Group.

This is similar to the issue described in #136


Add scheduled task for blacklight:allmaps harvest

https://github.com/UWM-Libraries/GeoDiscovery/commit/5627c19904c125df866f8041283091dbad811e96


[Bug] transformers latest 4.34.0, AttributeError: 'InternLMTokenizer' object has no attribute 'sp_model'

Describe the bug
>>> from transformers import AutoTokenizer, AutoModelForCausalLM
>>> tokenizer = AutoTokenizer.from_pretrained("internlm/internlm-chat-7b", trust_remote_code=True)

>>> tokenizer = AutoTokenizer.from_pretrained("/root/autodl-tmp/internlm-chat-7b", trust_remote_code=True)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/root/miniconda3/lib/python3.8/site-packages/transformers/models/auto/tokenization_auto.py", line 738, in from_pretrained
    return tokenizer_class.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs)
  File "/root/miniconda3/lib/python3.8/site-packages/transformers/tokenization_utils_base.py", line 2045, in from_pretrained
    return cls._from_pretrained(
  File "/root/miniconda3/lib/python3.8/site-packages/transformers/tokenization_utils_base.py", line 2256, in _from_pretrained
    tokenizer = cls(*init_inputs, **init_kwargs)
  File "/root/.cache/huggingface/modules/transformers_modules/internlm-chat-7b/tokenization_internlm.py", line 68, in __init__
    super().__init__(
  File "/root/miniconda3/lib/python3.8/site-packages/transformers/tokenization_utils.py", line 366, in __init__
    self._add_tokens(self.all_special_tokens_extended, special_tokens=True)
  File "/root/miniconda3/lib/python3.8/site-packages/transformers/tokenization_utils.py", line 462, in _add_tokens
    current_vocab = self.get_vocab().copy()
  File "/root/.cache/huggingface/modules/transformers_modules/internlm-chat-7b/tokenization_internlm.py", line 108, in get_vocab
    vocab = {self.convert_ids_to_tokens(i): i for i in range(self.vocab_size)}
  File "/root/.cache/huggingface/modules/transformers_modules/internlm-chat-7b/tokenization_internlm.py", line 96, in vocab_size
    return self.sp_model.get_piece_size()
AttributeError: 'InternLMTokenizer' object has no attribute 'sp_model'
>>> tokenizer = AutoTokenizer.from_pretrained("/root/autodl-tmp/internlm-chat-7b", trust_remote_code=True)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/root/miniconda3/lib/python3.8/site-packages/transformers/models/auto/tokenization_auto.py", line 738, in from_pretrained
    return tokenizer_class.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs)
  File "/root/miniconda3/lib/python3.8/site-packages/transformers/tokenization_utils_base.py", line 2045, in from_pretrained
    return cls._from_pretrained(
  File "/root/miniconda3/lib/python3.8/site-packages/transformers/tokenization_utils_base.py", line 2256, in _from_pretrained
    tokenizer = cls(*init_inputs, **init_kwargs)
  File "/root/.cache/huggingface/modules/transformers_modules/internlm-chat-7b/tokenization_internlm.py", line 68, in __init__
    super().__init__(
  File "/root/miniconda3/lib/python3.8/site-packages/transformers/tokenization_utils.py", line 366, in __init__
    self._add_tokens(self.all_special_tokens_extended, special_tokens=True)
  File "/root/miniconda3/lib/python3.8/site-packages/transformers/tokenization_utils.py", line 462, in _add_tokens
    current_vocab = self.get_vocab().copy()
  File "/root/.cache/huggingface/modules/transformers_modules/internlm-chat-7b/tokenization_internlm.py", line 108, in get_vocab
    vocab = {self.convert_ids_to_tokens(i): i for i in range(self.vocab_size)}
  File "/root/.cache/huggingface/modules/transformers_modules/internlm-chat-7b/tokenization_internlm.py", line 96, in vocab_size
    return self.sp_model.get_piece_size()
AttributeError: 'InternLMTokenizer' object has no attribute 'sp_model'

Environment
transformers==4.34.0

when version==4.33.3 is fine

Other information
No response

https://github.com/huggingface/transformers/issues/26340#issuecomment-1733371279

I can use the model and fine-tune it in autotrain-advanced with a modified tokenization_internlm.py. moved super().init() to a line after the creation of self.sp_model in tokenization_internlm.py
However, fine-tuning seems to finish early or the time estimate is just totally off.

Here is my adjusted file:
https://huggingface.co/KnutJaegersberg/deacon-20b-gptq/blob/main/tokenization_internlm.py

@KnutJaegersberg Moving super().init() is not supposed to influence anything.

Got it, I just share what I did :)

This bug has been already fixed by moving super.init() to the last line of InternlmLM.__init__
See: https://huggingface.co/internlm/internlm-chat-7b/blob/main/tokenization_internlm.py
Later I will create a PR to update our tokenizer file


Logs don't indicate they're coming from bugsnag

Looking at a deploy to Heroku and saw this:
remote:        W, [2018-03-08T00:33:52.460418 #1347]  WARN -- : No valid API key has been set, notifications will not be sent
Nothing in that message indicates it came from Bugsnag. I had to google for it which led me to this repo.
Version
gem 6.6.4

Hi @swrobel, thanks for the feedback.  We've had some discussion over logging clarity in some of the other notifiers recently, so will be looking in to making our logs better in general.

@cawllec @kattrali that commit doesn’t fully address this issue.

@jfelchner How so? Provided your log formatter does not exclude progname, every message from within bugsnag is marked with [Bugsnag].

It solves that issue. It doesn’t solve the fact that with a properly configured initializer, spurious log messages are printed stating that the API key is invalid.

@Cawllec @kattrali what is the plan here?  This is extremely annoying.

Ok great! Can we reopen this until then?

@jfelchner I'm with you here, but you're spamming my inbox when the issue I raised is actually fixed. The issue you want to track (and that I'm also tracking) is https://github.com/bugsnag/bugsnag-ruby/issues/391


[Enhancement] Use switch to replace the if-else.[MeshMessageProtocolAdaptor]

Search before asking

[X] I had searched in the issues and found no similar issues.

Enhancement Request

located at:
eventmesh-protocol-plugin/eventmesh-protocol-meshmessage/src/main/java/org/apache/eventmesh/protocol/meshmessage/MeshMessageProtocolAdaptor.java line 83-93,113-136
Describe the solution you'd like

Are you willing to submit PR?

[ ] Yes I am willing to submit a PR!

Hello, I would like to work on this. Please assign this issue to my name. Thank you!!


Update nuxt, @nuxt/vue-app, @nuxt/types dependencies

Fixes

Related to #3242 by @obulat
Description

Updated nuxt dependency in frontend/package.json and updated the lock file, and commited the changes.

[X] nuxt, @nuxt/vue-app, @nuxt/types: from 2.17.0 to ^2.17.2 [changes]

Testing Instructions

Checklist

[X] My pull request has a descriptive title (not a vague title likeUpdate index.md).
[X] My pull request targets the default branch of the repository (main) or a parent feature branch.
[X] My commit messages follow best practices.
[X] My code follows the established code style of the repository.
[ ] I added or updated tests for the changes I made (if applicable).
[ ] I added or updated documentation (if applicable).
[ ] I tried running the project locally and verified that there are no visible errors.
[ ] I ran the DAG documentation generator (if applicable).

Developer Certificate of Origin

Developer Certificate of Origin
Developer Certificate of Origin
Version 1.1

Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
1 Letterman Drive
Suite D4700
San Francisco, CA, 94129

Everyone is permitted to copy and distribute verbatim copies of this
license document, but changing it is not allowed.

Developer's Certificate of Origin 1.1

By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I
    have the right to submit it under the open source license
    indicated in the file; or

(b) The contribution is based upon previous work that, to the best
    of my knowledge, is covered under an appropriate open source
    license and I have the right under that license to submit that
    work with modifications, whether created in whole or in part
    by me, under the same open source license (unless I am
    permitted to submit under a different license), as indicated
    in the file; or

(c) The contribution was provided directly to me by some other
    person who certified (a), (b) or (c) and I have not modified
    it.

(d) I understand and agree that this project and the contribution
    are public and that a record of the contribution (including all
    personal information I submit with it, including my sign-off) is
    maintained indefinitely and may be redistributed consistent with
    this project or the open source license(s) involved.

just node-install

After running just node-install,  pnpm-lock.yaml was not updated. Can you point out what is the issue here?

@obulat After running just node-install, pnpm-lock.yaml was not updated. Can you point out what is the issue here?

I really don't understand why your lock file is updated, @khushi-kothari 😕
When I ran it locally on your branch, I got the following:
 j node-install                                                                                 ✔  8s  18:06:56
pnpm i
Scope: all 4 workspace projects
readPackage<EMAIL_ADDRESS>=><EMAIL_ADDRESS>in dependencies of @openverse/eslint-plugin
frontend                                 |  WARN  deprecated<EMAIL_ADDRESS>frontend                                 |  WARN  deprecated<EMAIL_ADDRESS>frontend                                 |  WARN  deprecated<EMAIL_ADDRESS>frontend                                 |  WARN  deprecated<EMAIL_ADDRESS>frontend                                 |  WARN  deprecated<EMAIL_ADDRESS>frontend                                 |  WARN  deprecated<EMAIL_ADDRESS>frontend                                 |  WARN  deprecated<EMAIL_ADDRESS>frontend                                 |  WARN  deprecated<EMAIL_ADDRESS>frontend                                 |  WARN  deprecated<EMAIL_ADDRESS>frontend                                 |  WARN  deprecated<EMAIL_ADDRESS>frontend                                 |  WARN  deprecated<EMAIL_ADDRESS>frontend                                 |  WARN  deprecated<EMAIL_ADDRESS>frontend                                 |  WARN  deprecated<EMAIL_ADDRESS>frontend                                 |  WARN  deprecated<EMAIL_ADDRESS>frontend                                 |  WARN  deprecated<EMAIL_ADDRESS>frontend                                 |  WARN  deprecated<EMAIL_ADDRESS>frontend                                 |  WARN  deprecated<EMAIL_ADDRESS>frontend                                 |  WARN  deprecated<EMAIL_ADDRESS>frontend                                 |  WARN  deprecated<EMAIL_ADDRESS>frontend                                 |  WARN  deprecated<EMAIL_ADDRESS>frontend                                 |  WARN  deprecated<EMAIL_ADDRESS>frontend                                 |  WARN  deprecated<EMAIL_ADDRESS>Packages: +258 -121
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++------------------------------------------------------------------------
Progress: resolved 2478, reused 2469, downloaded 0, added 1, done
<EMAIL_ADDRESS>Running postinstall script, done in 578ms
. postinstall$ pnpm --filter ./packages/* run build
│ No projects matched the filters in ".../openverse"
└─ Done in 338ms
Done in 20.1s
just frontend/run i18n:en
pnpm run i18n:en

><EMAIL_ADDRESS>i18n:en .../openverse/frontend
> pnpm i18n:get-translations --en-only

><EMAIL_ADDRESS>i18n:get-translations .../openverse/frontend
> node src/locales/scripts/get-translations "--en-only"

Successfully saved English translation to en.json.
just frontend/run i18n:copy-test-locales
pnpm run i18n:copy-test-locales

><EMAIL_ADDRESS>i18n:copy-test-locales .../openverse/frontend
> cp -r test/locales/* src/locales/

.../openverse  nuxt *66 !1  git status                                                                                 ✔  22s  18:07:56
On branch nuxt
Your branch is up to date with 'khushi-kothari/nuxt'.

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
        modified:   pnpm-lock.yaml

no changes added to commit (use "git add" and/or "git commit -a")

(I redacted the openverse file paths)
I committed the lock file and pushed it to your branch.
Other options to try are running pnpm install --filter=frontend or running just install from the root (openverse/), but I'm not sure they would work better.

@obulat After running just node-install, pnpm-lock.yaml was not updated. Can you point out what is the issue here?

I really don't understand why your lock file is updated, @khushi-kothari 😕
When I ran it locally on your branch, I got the following:
 j node-install                                                                                 ✔  8s  18:06:56
pnpm i
Scope: all 4 workspace projects
readPackage<EMAIL_ADDRESS>=><EMAIL_ADDRESS>in dependencies of @openverse/eslint-plugin
frontend                                 |  WARN  deprecated<EMAIL_ADDRESS>frontend                                 |  WARN  deprecated<EMAIL_ADDRESS>frontend                                 |  WARN  deprecated<EMAIL_ADDRESS>frontend                                 |  WARN  deprecated<EMAIL_ADDRESS>frontend                                 |  WARN  deprecated<EMAIL_ADDRESS>frontend                                 |  WARN  deprecated<EMAIL_ADDRESS>frontend                                 |  WARN  deprecated<EMAIL_ADDRESS>frontend                                 |  WARN  deprecated<EMAIL_ADDRESS>frontend                                 |  WARN  deprecated<EMAIL_ADDRESS>frontend                                 |  WARN  deprecated<EMAIL_ADDRESS>frontend                                 |  WARN  deprecated<EMAIL_ADDRESS>frontend                                 |  WARN  deprecated<EMAIL_ADDRESS>frontend                                 |  WARN  deprecated<EMAIL_ADDRESS>frontend                                 |  WARN  deprecated<EMAIL_ADDRESS>frontend                                 |  WARN  deprecated<EMAIL_ADDRESS>frontend                                 |  WARN  deprecated<EMAIL_ADDRESS>frontend                                 |  WARN  deprecated<EMAIL_ADDRESS>frontend                                 |  WARN  deprecated<EMAIL_ADDRESS>frontend                                 |  WARN  deprecated<EMAIL_ADDRESS>frontend                                 |  WARN  deprecated<EMAIL_ADDRESS>frontend                                 |  WARN  deprecated<EMAIL_ADDRESS>frontend                                 |  WARN  deprecated<EMAIL_ADDRESS>Packages: +258 -121
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++------------------------------------------------------------------------
Progress: resolved 2478, reused 2469, downloaded 0, added 1, done
<EMAIL_ADDRESS>Running postinstall script, done in 578ms
. postinstall$ pnpm --filter ./packages/* run build
│ No projects matched the filters in ".../openverse"
└─ Done in 338ms
Done in 20.1s
just frontend/run i18n:en
pnpm run i18n:en

><EMAIL_ADDRESS>i18n:en .../openverse/frontend
> pnpm i18n:get-translations --en-only

><EMAIL_ADDRESS>i18n:get-translations .../openverse/frontend
> node src/locales/scripts/get-translations "--en-only"

Successfully saved English translation to en.json.
just frontend/run i18n:copy-test-locales
pnpm run i18n:copy-test-locales

><EMAIL_ADDRESS>i18n:copy-test-locales .../openverse/frontend
> cp -r test/locales/* src/locales/

.../openverse  nuxt *66 !1  git status                                                                                 ✔  22s  18:07:56
On branch nuxt
Your branch is up to date with 'khushi-kothari/nuxt'.

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
        modified:   pnpm-lock.yaml

no changes added to commit (use "git add" and/or "git commit -a")

(I redacted the openverse file paths)
I committed the lock file and pushed it to your branch.
Other options to try are running pnpm install --filter=frontend or running just install from the root (openverse/), but I'm not sure they would work better.

Thanks for adding lock file here, I will look into what is causing this issue so next updates can be committed smoothly.

Ok @obulat that is the next task and am updating vue dependencies. Why are some of the CI tests failing in the current PR?

The CI tests are failing because the nuxt updates have updated one of the vue packages, and now there is a version mismatch and Nuxt cannot be built. So, the Vue dependencies should actually be updated in this PR instead of a separate PR.

Updated vue dependencies. Still 2 CI tests are failing.
Lock file updated this time and all changes added in the last commit that I added.

Weirdly, the lock file didn't fully match the updates in package.json, so the CI failed (you could see the messages about frozen lock file). I ran just install and committed the updated lock file. Let's see if the CI passes now :)

Hey @obulat I have added all tasks as individual PRs. Do you need any more help from my side on this issue #3242?

Hey @obulat I have added all tasks as individual PRs. Do you need any more help from my side on this issue #3242?

Hi, @khushi-kothari ! Thank you for creating the individual PRs. Could you please update them so that they only update the dependencies listed in the corresponding PR description?
For instance, Update focus-trap dependency should only update focus-trap, but if you look at the files changed, you can see that it updates all of these packages:

"@nuxtjs/i18n": "^7.3.1",
"@nuxtjs/sentry": "^7.5.0",
"@popperjs/core": "^2.11.8",
"@tailwindcss/typography": "^0.5.10",
"clipboard": "^2.0.11",    "uuid": "^9.0.1",
"vue": "^2.7.15",
"@types/uuid": "^9.0.6",
"autoprefixer": "^10.4.16",
"tailwindcss": "^3.3.3",
"vue-demi": "^0.14.6",
"vue-server-renderer": "^2.7.15",
"vue-template-compiler": "^2.7.15",

Hey @obulat I think as it is linting problem always there is some issue while tracking whitespace changes. I locally have the following change for some reason it is not tracked while commiting.

local ss:

But while viewing it remotely, it is like this :

I followed following steps in this PR :

Checked out the lock file from main to remove all unrelated changes.
git checkout main -- pnpm-lock.yaml
Ran just node-install to update the lock file.
Committed the updated lock file to the branch and pushed the changes.

Yet, linting error is thrown.

Hey @obulat I think as it is linting problem always there is some issue while tracking whitespace changes. I locally have the following change for some reason it is not tracked while commiting.

I am not sure what's happening with the whitespace changes and why are they not tracked...
The lockfile changes, on the other hand, are not simply whitespace changes. In several PRs you can see that the Vue packages are still touched in the lock files, even though you reverted the Vue changes to the package.json file.
I could update the lock files for you if the process I described doesn't work for you locally:

I followed following steps in this PR :
Checked out the lock file from main to remove all unrelated changes.
git checkout main -- pnpm-lock.yaml
Ran just node-install to update the lock file.
Committed the updated lock file to the branch and pushed the changes.

Hey @obulat I think as it is linting problem always there is some issue while tracking whitespace changes. I locally have the following change for some reason it is not tracked while commiting.

I am not sure what's happening with the whitespace changes and why are they not tracked... The lockfile changes, on the other hand, are not simply whitespace changes. In several PRs you can see that the Vue packages are still touched in the lock files, even though you reverted the Vue changes to the package.json file. I could update the lock files for you if the process I described doesn't work for you locally:

I followed following steps in this PR :
Checked out the lock file from main to remove all unrelated changes.
git checkout main -- pnpm-lock.yaml
Ran just node-install to update the lock file.
Committed the updated lock file to the branch and pushed the changes.

Ya that will be very helpful as the process you described isn't working for me locally.
I hope my contributions were helpful. As my first experience to open sourcing, I could say I will continue doing it and will grow in the process. Thankyou for being the kindest mentor and guiding me through this issue very calmly.

Ya that will be very helpful @obulat as the process you described isn't working for me locally.
I hope my contributions were helpful. As my first experience to open sourcing, I could say I will continue doing it and will grow in the process. Thankyou for being the kindest mentor and guiding me through this issue very calmly.

Thank you for your contributions, @khushi-kothari, they are very helpful! Hope you can keep contributing with Openverse ❤
It is often that dependency or local set up problems are more difficult than coding :)


[WIP] Add required anti-affinity for harvester network manager

related issue https://github.com/harvester/harvester/issues/1722

Does it affect upgrades?

Does it affect upgrades?

yes, added a workaround on https://github.com/harvester/docs/pull/67/commits/99529f3dac7a96d35c82db4980c34f2eace77c13, also better to fix the label issue early instead of later.


adding owasp dependency check

This is an simple and important way to add third partly libraries into Jenkins.
Let's add it.
-I'll update the Java cartridge to use it.

As i can see, we already met requirements as we have resolved dependencies for this plugin:

analysis-core
maven-plugin

And this plugin is pretty useful & popular. LGTM.


Play zulip sound if new message but user in different narrow

Automated message from Dropbox CLA bot
@borisyankov, it looks like you've already signed the Dropbox CLA. Thanks!

Looks great to me 👍 One super minor issue could be that we might not want to play sound in case of self @-mention and pm to self.

You are right. Though you will also be looking at the same narrow, if writing to yourself :)

I guess that makes sense :), I tested this by writing messages in the web client and see how the app responded. But in future once we support writing messages to a different narrow in the app, this might occur?


cKDTree is compiled twice on build and install

$ git clean -xdf
$ python setup.py build
... long clean compilation/build
$ python setup.py install
...
building 'scipy.spatial.ckdtree' extension
compiling C++ sources
C compiler: g++ -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall -fPIC
...

Other C++ extensions are not recompiled (e.g., sparsetools). While this is obviously minor and can be solved with ccache, I wonder if there's something in the build setup of scipy.spatial which can be fixed.

I was unable to reproduce this.
There have been 30+ changes to setup.py since Jul 2017.
I believe this was the one that fixed it: https://github.com/scipy/scipy/pull/7728


Choropleth Map of Population Density

What is the objective of the proposed tutorial?
This tutorial aims to demonstrate the computation of population density, together with interactive visualization of results in a choropleth map. It is useful for applications such as national risk assessment for disasters.
What is the scope of the proposed tutorial?
Population, Demographics
Please provide an outline of the structure of the proposed tutorial?

Population Density
a. Defining study region using geoBoundaries database
b. Computing population density from the Gridded Population of World (GPW)
Data Visualization
a. Creating a choropleth map using predefined color palette
b. Adding interactive widget for user to assess results

In what format will you be submitting the tutorial?
Select one: Markdown
This request will be reviewed by the Earth Engine community maintainers, who
will reply on this issue tracker with any questions or suggestions. Once
approved, this issue will be assigned to you and you can begin work on the
tutorial following instructions in
Writing a tutorial.
When creating your Pull Request, enter "Closes #issueno" in the description of
your Pull Request to link the tutorial to this issue.

This sounds great @pinkychow1010!
Please proceed with the next steps: https://developers.google.com/earth-engine/tutorials/community/write#first_time_setup

Thanks. I have submitted the pull request.

Your welcome
On Tue, May 7, 2024, 3:13 AM Ka Hei @.***> wrote:

Thanks. I have submitted the pull request.
—
Reply to this email directly, view it on GitHub
https://github.com/google/earthengine-community/issues/818#issuecomment-2097823694,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/BHTC2CM2DYCE4GVNPS2DD4LZBCLLRAVCNFSM6AAAAABHDYIZNGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAOJXHAZDGNRZGQ
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

Hi, just wondering how can I go further with the submitted tutorial? Any inputs required from my side?

Sorry for the slow reply. I've been away from work. I'll be able to review this pull request by the end of next week. Thanks for your patience.


test: Using module tempfile to avoid failed test on #15

Thanks for the PR!
Try to update your master branch, and create a new branch from it before submitting a PR (to avoid the 6 commits mentioned above)
Regarding the failed test, I'll comment directly in the review!

@blenzi Codecov seems to have an issue unrelated to this according to CI
The upload of coverage throws:
HTTP 400
Commit sha does not match Circle build.
Codecov upload failed

I am checking right now how to fix this!

Apparently, it is a known bug: https://github.com/codecov/codecov-bash/issues/134
I'll merge the commit for now!


[RFC] Proposal of a non git-based mixin

This proposal is a request for comments and a proposal for a non-git-based mixin. The current Mixin approach uses a git workflow in order to contribute files to the hub. This, unfortunately, has drawbacks with limited advantages. I will take the PyTorchModelHubMixin as an example. Here's a typical use-case with the mixin:
import torch
from huggingface_hub import PyTorchModelHubMixin

class MyModel(torch.nn.Module, PyTorchModelHubMixin):
    def __init__(self):
        super().__init__()
        self.transformer = torch.nn.Transformer()

model = MyModel()
model.push_to_hub(repo_path_or_name="<path_to_repo>")

This will:

Create the repo if it doesn't exist
Clone the repo in the <path_to_repo>
Call save_pretrained in the <path_to_repo>
Add, commit & push to the remote

Issues encountered
There are points of friction with this approach, which are detailed below.
save_pretrained and unrelated folders
The save_pretrained method doesn't need to be called before as push_to_hub already saves files in a local folder - but now calling it before calling push_to_hub will result in an error.
model = MyModel()
model.save_pretrained("<path_to_repo>")
model.push_to_hub(repo_path_or_name="<path_to_repo>"

This will not work as the folder <path_to_repo> is now a non-git folder in which the new <user>/<path_to_repo> repository would need to be cloned.

Why don't we allow cloning remote repositories in non-empty local folders?
We try to follow git to the letter (albeit with a few tweaks that make user experience more friendly). git does not support cloning repositories in non-empty folders as some complex questions arise:

Should the newly instantiated git folder pull the remote files, or should it simply have the remote set?
In case we want to pull the files, which files should overwrite which?
What happens if the folder is already a git repository? Should the clone behave differently according to what is contained in the folder?

Existing changes in the current directory
Since the entire folder in which the model is saved is now a git folder, any file saved in it will also be pushed to the remote.
model = MyModel()
model.push_to_hub(repo_path_or_name="<path_to_repo>")

with open("<path_to_repo>/README.md", "w+") as f:
    f.write("# My model")

model.push_to_hub(repo_path_or_name="<path_to_repo>")

This will also push the README.md even though the method seems to only push the model (the name being model.push_to_hub). This can be a problem, for example with transformers we need to have both the model and tokenizer in the same folder - but each of those objects has a push_to_hub method. Each push_to_hub method will add everything, commit and push
:warning: This could be potentially fixed by only doing git add <name_of_file_in_particular> (which isn't the case in any mixin right now)
No two consecutive push_to_hub
Piggy-backing on the issue above, calling push_to_hub twice in a row without any changes will result in an error:
OSError: On branch main
Your branch is up to date with 'origin/main'.

nothing to commit, working tree clean

This would happen in case users were misled into thinking
model.push_to_hub(...)
tokenizer.push_to_hub(...)

:warning: This too could be potentially fixed by only doing git add <name_of_file_in_particular> (which isn't the case in any mixin right now)
push_to_hub indicates something that is different than what is actually happening
The push_to_hub name implies that the files are pushed to the hub. However, what is happening under the hood is very different as it maintains a local clone of the remote repository. This method will apply changes to the local clone while its name implies it will only push files to the remote.
Therefore, I believe the following is misleading:
# Existing <path_to_repo> with a past model
model.push_to_hub(repo_path_or_name="<path_to_repo>")

Where a user may believe that by calling push_to_hub they have simply published the new model weights on the repo, the local clone of that repo is now full of remote and potentially unexpected (if working in collaboration with others) changes.
Proposal
IMO, the git-based approach is solid in cases where maintaining a local clone of the repository makes sense:

During training, in order to store different checkpoints
During training, In order to store tensorboard traces and other training artefacts
In order to have a fluid versioning system

I believe it is a valid approach for tools like the Trainer or when users want to keep a local versioned copy of their work before pushing everything remotely.
However, I believe it is ill-suited to the management of separate, independent items like a model. I believe that the push_to_hub method indicates that it will upload only the necessary files to the hub, and is, therefore, better suited to a simple HTTP post request rather than a full-blown local git workflow.
This PR offers an alternative approach leveraging the upload_file method instead of the Repository as the backbone for uploads.
This enables the following use-cases:
import torch
from huggingface_hub import PyTorchModelHubMixin

class MyModel(torch.nn.Module, PyTorchModelHubMixin):
    def __init__(self):
        super().__init__()

        self.transformer = torch.nn.Linear(2, 2)

model = MyModel()

model.save_pretrained("<local_path>")
# Already saved in non-git folder, but still works
model.push_to_hub(repo_path_or_name="<local_path>")

import torch
from huggingface_hub import PyTorchModelHubMixin

class MyModel(torch.nn.Module, PyTorchModelHubMixin):
    def __init__(self):
        super().__init__()

        self.transformer = torch.nn.Linear(2, 2)

model = MyModel()
model.push_to_hub(repo_path_or_name="<local_path>")
# Already pushed, but still works
model.push_to_hub(repo_path_or_name="<local_path>")

This implementation works with a temporary directory which means that the user doesn't need to be aware of a local directory to maintain -> only the object's serialization files will be pushed to the hub with no local side effects.
This is a POC with a few shortcomings that would need to be resolved before continuing if this is of interest:

The maximum file size is currently 5GB with HTTP post requests
There are no progress bars currently with upload_file
A list of files must be returned by the _save_pretrained method in order to know which files should be uploaded to the hub
Tests and doc :)

Looking forward to your comments!

As discussed offline, I think this is way less surprising like this. The name of the method and the object it is associated to indicates that only the model (or other objects) files are pushed to the Hub. Maintaining a local clone and properly dealing with it should be done with the (excellent) Repository API.
In terms of use-cases, when a user does for instance
config.push_to_hub(xxx)

they expect this command to take a few seconds to upload the config file in the proper repo. Since this actually clones the repo if needed, and/or pushes all other local changes in the same folder, the command can take a super long time and have adverse effects.
On the other hand, we should advertise more how to use a Repository to create a clean repo at the beginning of a training example, fill it with the model, tokenizer, metrics etc., then show how to use push_to_hub on this repo object. This clearly follows the usual git process, and is way more natural. I think we should take the time to showcase this on the Accelerate and Flax examples of the Transformers repo, for instance.

I think this proposal makes sense 👍  thanks for the detailed writeup and thoughts 🙏
I also entirely agree with both of @sgugger's points, including the last one.

Having experienced some of these issues firsthand, I am very much in favour of this proposal as it removes potentially confusing errors for the end-user (e.g. why am I seeing Git messages in the traceback?) and narrows the scope of what push_to_hub really means.
Thank you ❤!

I also really like the proposal!👍🏻 Providing a Mixin and/or an API for easy uploading to the files would be very helpful for a variety of use-case.
Especially as pointed out by @sgugger collaborating or working on a large scale could become harder. (Thinking of Enterprise level MLOPS, where DS and OPS engineer could collaborate on something)

they expect this command to take a few seconds to upload the config file in the proper repo. Since this actually clones the repo if needed, and/or pushes all other local changes in the same folder, the command can take a super long time and have adverse effects.

The maximum file size is currently 5GB with HTTP post requests
Can be solved by segmenting the file into multiple chunks and uploading them.

I think this is definitely the way to go. Uploading files directly fixes a ton of annoying issues. Good stuff 🚀
However, since we've realized that keras will require functional definitions of the save/push logic, it'll add a little rework in #284 . I opened up this discussion to talk about potential solutions to this code duplication problem.

behind the scene (server side) it's still using git (and you can specify a commit message etc as usual)
The trade off here is that we're trading some of the advantages of git (robustness to concurrent changes by multiple users, as you mention) for an increased ease of use for simple/basic use cases where one user just wants to "save stuff to the hub".
We'll need to make it clear that for the "full" git workflow we recommend using git or the git-wrapping Repository (which is going to be the case in transformers examples IIUC)

I am also in favour of this and at the same time pushing the Repository library for more sophisticated use-cases where a little more code is required but much more flexibility gained. I, for one didn't know that the hub is using git and git-lfs under the hood for a while starting working with the library.

Thank you for your comment @patrickvonplaten, you definitely raise a lot of valuable counter-arguments.
As @julien-c  says, we're still relying on git as that's the backend that will take care of everything. Using upload_file instead of the traditional git workflow means we're bypassing the handling of a local clone to favor a simple file "upload", which is behind the scenes a git commit with that single file modification.

With this new approach, when uploading files to an existing repo -> there should be a git commit message no? IMO every change to a repo should be tracked in the git history. This is still the case no?

This is indeed still the case. The commit message is currently not editable and will be shown as Upload <file_name> with git-lfs in the commit history, see here. However, I'm positive this could be implemented to also take a different commit message and I agree with you that this would be a welcome addition.

No two consecutive push_to_hub -> When I do this locally with git repositories it just says "Everything up-to-date" so no real error is thrown. Why is an error thrown here?

Right, the error shown here is just the result of different error handling - The real issue imo is that when using transformers and saving files in a directory, doing the following:
config.push_to_hub("xxx")

will push all edited files to the hub. As highlighted in the RFC however, tweaking the git_add could be another way to solve it while still relying on a git repo.

In general, I really like the git workflow since people are really used to it and it does catch errors quite nicely. E.g. if multiple people work on the same repo and are both pushing to the repo at the same time I think it's good behavior that git throws an error:

I entirely agree with you that git is the best way to collaborate, but we're not dropping git with this RFC.

We could solve this save_pretrained and unrelated folders rather easily by either creating this repo if it exists and moving all files from the "save_pretrained" repo to it and then follow the usual workstream and clone the repo if it exists and moving all files from the "save_pretrained" repo to it & follow the normal workflow no? I think there is a rather easy fix for this (the same holds true IMO about just using git add <pytorch_model.bin> for "Existing changes in the current directory" as you pointed out @LysandreJik

I agree that this is a solution!

The solution proposed here doesn't offer to replace the git workflow for use-cases where it is clearly superior as a collaboration means.  It does not remove any git features such as versioning and collaboration possibilities.
The solution proposed here offers to alleviate pain points that have been seen both during the Jax/Flax sprint due to errors that would crash mid-training, and experienced by both HF members and users.
The pain points being:

Needing to clone the entire repository locally to update a configuration or README file
The need to handle a local clone of the remote repository
Generally, a misleading API that indicates it will push a file to the hub, which may end up taking 10 minutes because it will first need to clone the repository first

Let me clarify that this proposal is meant only for mixins/config/tokenizer/single file uploads. It is not meant to be a replacement of the behavior of the Trainer.push_to_hub, which will still leverage Repository as a means to work on a model.
As @lvwerra, @sgugger and @julien-c mention, for use-cases where true git management is required, the Repository class should be used rather than push_to_hub as it offers way finer management of the repository while sticking to the traditional git workflow.

Okey this sounds much better then!
1.) Commit messages are very important though IMO, e.g. when uploading checkpoints every "n" steps during training with .push_to_hub(...), it's very important to see directly in the git commit history to what training step the latest commit corresponds to. But if it's trivial to add then this shouldn't be a problem.
2.) The other point, I'm feeling quite strongly about are "merge conflicts". For me it's a big drawback if this is not handled nicely anymore. Currently all Flax training scripts rely on .push_to_hub(...) and during the sprint every team worked concurrently on the same repo. A typical use case that I have in mind is that one person trained, noticed that some hyperparameters work better -> the next day the other person forgets to pull and changes the config as well and pushes -> this should result in an error. Can we implement this easily as well with the new design?
I'm still not seeing the huge advantages of this design that can not be solved by using git add config.json and git fetch --all && git checkout origin/master -- config.json.  fetch will locally download all the changes (but with GIT_LFS_SKIP_SMUDGE=1 this should be fast no?) and then just the file in question can be checked out. I guess another problem with the git-based solution is that a repo that has an already added & commited model in it can afterwards not anymore just push a config as it will necessarily have to push the model with it (or some complex branching could solve this...)
=> It would be great if we can solve 1.) & 2.) then I'm definitely in favor of this. Seems like there are already enough people in favor of this PR though ;-)

@patrickvonplaten The flax examples should not use the push to hub API, that is the main point. They use an API that is badly designed for this usecase and as you said yourself, most of the pain points seen in the Flax/JAX sprint would have been alleviated by properly creating/cloning the repo at the beginning of the script, which the push_to_hub API cannot do.
So the main point of the RFC is to see if everyone agrees to:

keep push_to_hub as a way to quickly push a file to the hub (typical usecase: fix a config file for instance) and only the files related to the object we call it with.
use Repository for anything that requires proper repo handling (like an example where you want to push every epochs all the objects for checkpointing). Trainer completely moves to use Repository in this PR and the code ends up being actually way easier, which is  a clear sign that it's a better API for that use case :-)

The push_to_hub name implies that the files are pushed to the hub. However, what is happening under the hood is very different as it maintains a local clone of the remote repository. This method will apply changes to the local clone while its name implies it will only push files to the remote.

Strong +1 to this. @nreimers and I extensively discussed about this when talking about the sentence-transformers integration. We ended up implementing a method called save_to_hub. Similar issues/comments came up with other 3rd party integrations.

I fully agree with the idea behind this proposal, but I would prefer if we went in a more general direction than just a method for mixin. For example, sentence transformers, spaCy or very likely StanfordNLP could benefit from something similar given their existing approaches.

Done as part of https://github.com/huggingface/huggingface_hub/pull/847.


Refactor some code in SqlQueryGenerator

Description
Pushed all code under each case into separate methods to make the code more readable and easier to navigate.
Related issues
Addresses [issue #].
Testing
No functional changes.
FHIR Team Checklist

[ ] Update the title of the PR to be succinct and less than 50 characters
[ ] Add a milestone to the PR for the sprint that it is merged (i.e. add S47)
[ ] Tag the PR with the type of update: Bug, Dependencies, Enhancement, or New-Feature
[ ] Tag the PR with Azure API for FHIR if this will release to the managed service
Review squash-merge requirements

Semver Change (docs)
Patch|Skip|Feature|Breaking (reason)

I've merged https://github.com/microsoft/fhir-server/pull/1764, so please update your branch (it's easier to force that update on you, than on author of that PR, sorry.)

Maybe it's worth to wait for this PR to be merged: https://github.com/microsoft/fhir-server/pull/1785 I assume merging this refactoring would be quite painful for @rotodd

Maybe it's worth to wait for this PR to be merged: #1785 I assume merging this refactoring would be quite painful for @rotodd

This PR is targeting my feature branch for now. It does make sense to merge it into main branch though. I will wait for @rotodd's PR to get merged in, pull latest changes from main and push an update to this branch

Oh, sorry, didn't notice target branch is different from main


update for new EC2 instances

Issue #, if available:
Description of changes:
We want to update and support i4g family.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Tests are failing because known upstream issue
<EMAIL_ADDRESS>unrecognized import path "gopkg.in/yaml.v3": reading https://gopkg.in/yaml.v3?go-get=1: 502 Bad Gateway
	server response: Cannot obtain refs from GitHub: cannot talk to GitHub: Get https://github.com/go-yaml/yaml.git/info/refs?service=git-upload-pack: net/http: request canceled


gcp_authn: Modify the token time constraint check

VerifyTimeConstraint() interface is correct for the token consumer. However, as the token producer here, we should instead include the clock skew as the part of the now time up front to account for the clock skew on the consumer side where the token will be consumed. Also, in this way, we can avoid changing the third-party library interface VerifyTimeConstraint() (which might requires longer turnaround time)
Signed-off-by: Tianyu Xia<EMAIL_ADDRESS>

One way of looking at this is that people/services expect a provided token to not expire immediately, so some positive time addition is better than none. And as these tokens generally have a lifetime of an hour, there's a big buffer to work with; no need to use a token right up to the last second.
What that extra amount should be is a judgment call; kClockSkewInSecond is probably as good as any. I think this would solve the issue in the Google environment (where, also, most systems are running NTP, so there's very little actual skew).

/retest

One way of looking at this is that people/services expect a provided token to not expire immediately, so some positive time addition is better than none. And as these tokens generally have a lifetime of an hour, there's a big buffer to work with; no need to use a token right up to the last second.
What that extra amount should be is a judgment call; kClockSkewInSecond is probably as good as any. I think this would solve the issue in the Google environment (where, also, most systems are running NTP, so there's very little actual skew).
(so, TLDR, LGTM)

Thanks for the review! Yes, as I commented in the code the clock skew here actually can be configurable by client in the future if it is needed.

/assign @yanavlasov
Could you take a look at this PR when you have free time? Thanks!

/assign @yanavlasov

/assign @yanavlasov

Just realized that we do not have unit tests for this code. The return statement on line 96 (token is valid) is not covered by tests.
Can you add unit tests for the TokenCacheImpl class please? I think it would be good to have tests that cover the skew.
/wait

Good point! I have verified the token cache locally before but because the token expiration is timing related we can not have deterministic behavior in test over time (i.e., token will be expired eventually). I will figure out the way of testing it (e.g., generating the token on each test run by leveraging time source etc). Per offline discussion, currently I left this as TODO so that we can check in this change unblock the customer side. The token related functionalities are verified by the test in jwt library itself.

You have control over system time with the TimeSource class. There are tests that do what you need to do in the test/extensions/filters/http/jwt_authn/jwt_cache_test.cc. You can use one of the token strings from the test and manipulate the tests system time object to set the time to what you need.
/wait-any

ping @yanavlasov  PTAL

cc @yanavlasov i think this is waiting on your review

/wait-any


What is the difference between this project and similar docker hyper project?

hyper official site
hyper github page

LXD is container based, we don't require any kind of virtualization technology, don't have any of that overhead and offer an API which is tailored specifically for containers, instead of attempting to make yet another libvirt which tries to abstract VMs and containers, having to compromise on features and ease of use by trying to abstract everything.


Allow the library to be bundled as a single file without seaprate worker js

Configuration:

Web browser and its version: Any / N/A
Operating system and its version: Any / N/A
PDF.js version: v2.13.216
Is a browser extension: No

Steps to reproduce the problem:

Attempt to bundle the library in a single, portable js file

Do const pdfjsLib = require('pdfjs-dist/webpack'); followed by some logic inside a main.js
Bundle main.js using webpack

Always needs configuration of the worker that points to a separate js file

Needs webpack config "pdf.worker": "pdfjs-dist/build/pdf.worker.entry" that builds a separate js file
Needs a line that points to the exact URL of the worker js file, e.g., pdfjsLib.GlobalWorkerOptions.workerSrc = './build/pdf.worker.bundle.js';
Both of the above requirements remove many of the benefits of bundling

What is the desired behavior?
It would be very useful if the PDF.js library were more portable and could be packaged inside another app. As it currently stands this does not seem to be possible, or is not documented. Following along the documentation at https://github.com/mozilla/pdf.js/tree/master/examples/webpack we see that even though webpack bundles most of the pdfjs library into a single file, there is always a separate worker .js file that needs to go along with it, requiring configuration (setting GlobalWorkerOptions.workerSrc). The webpack.js build does not appear to offer any benefit.

Please note that using the PDF.js library without Web Workers, i.e. with the pdf.worker.js file loaded on the main-thread, will lead to very poor overall performance and it'll hurt UI responsiveness which is why it's never been either recommended or even officially supported in browsers.[1]
Note that years ago we had a SINGLE_FILE build target, which sounds somewhat like what you're asking for here, however that was purposely removed in PR #9385 for the reasons outlined above.

[1] The ability to load the pdf.worker.js file on the main-thread only exists as a fallback (since older browsers had trouble with workers), for debugging purposes, and finally for Node.js environments.

@Snuffleupagus -- Is it possible to load the js for the worker at build time so that the faster worker functionality can remain intact, but the built js file can contain everything it needs? An implementation could look something like...
// No change in how we import the core functionality
import pdfjsLib from `pdfjs-dist/build/pdf';
// Bring in the worker code (built differently) now rather than at runtime
import pdfjsWorkerFunction from 'pdfjs-dist/build/pdf.worker-function';  // (new build file needed)

// Convert the function into a blob url to use workerSrc
const workerBlob = new Blob(
	[ '(', pdfjsWorkerFunction.toString(), ')()' ],
	{ type: 'application/javascript' },
);
const blobUrl = URL.createObjectURL(blob);
pdfjsLib.GlobalWorkerOptions.workerSrc = blobUrl; // ultimately ends up in new Worker(workerSrc)

// OR (better) just do the above in the library (_initialize method), and just have a new option like:
pdfjsLib.GlobalWorkerOptions.workerFunction = pdfjsWorkerFunction;

// Then continue as normal

This way when a webpack build is done, everthing would get bundled up together in a single file with no path-based configuration needed. And (I think) the worker should still work on a separate thread.
Regarding your [1] note: How can someone use the fallback to load the pdf.worker.js file on the main thread? I don't recall seeing that in the documentation.


Make this repo’s Community Profile meet the Community Profile Checklist

Description
Make this repo’s Community Profile meet the Community Profile Checklist. Adopting these standards helps people use and contribute to the project more effectively.

Fixed with #1304, #1305, #1306.


NodeJS SDK: Update codegen templates for docs references

Add @hidden (equivalent to @ignore in JSDoc) tag to remove symbol that should never appear in the documentation.

Export every classes otherwise they will not appear in documentation.
I've tried to use typedoc-plugin-missing-exports in docusaurus but gets some conflicts during the build.

Signed-off-by: jffarge<EMAIL_ADDRESS>

I like that last commit!
 Every one of them lowers the confidence in the test suite. Skip enough tests and the usefulness of the entire test suite will start getting questioned. Nice improvement!

I will trust the test suite more when these new errors will be addressed. Nice digging @slumbering 🚜

You need to update the expected results of the templating tests

Read the diff. Do you always want to hide the documentation of all the classes?

The reasoning behind skipping that test (and the issue for following up) is here: https://github.com/dagger/dagger/pull/3851#discussion_r1023164201
If we don't want to skip it we can just remove it as the codepaths are covered by the other test cases for now.


JDK25 will remove Java security manager. Code cleanup...

Since Java8 new gVisor sandbox, we removed a lot of code using the obsolete java security manager.
We still have some leftover that we might want to cleanup before testing with JDK25 later in 2025.
Also code using AccessController.doPrivileged(...) should be evaluated to complete deletion of java.security imports.

Other warnings I see is about Character class:
/appengine_standard/api_dev/src/main/java/com/google/appengine/api/search/dev/WordSeparatorAnalyzer.java:78: warning: [removal] Character(char) in Character has been deprecated and marked for removal
return !LuceneUtils.WORD_SEPARATORS.contains(new Character(c));

In general, we should try to remove as much as possible build warnings like "has been deprecated and marked for removal"
Some might not be possible yet


Configure Twilio to record voicemail and send it by email

When the software phone is closed on my computer, or when I am away and unable to answer a call, I wish to direct the caller to voicemail.
A notification of the call including the date/time, the phone number of the caller and a link to the recording should then be sent to me by email.

How I Failed to Configure a Voicemail Using TwiML Bins Alone
Starting from a script as simple as possible to forward calls to my SIP endpoint, I first added a message, read by a friendly female robot using <Say>, followed with audio recording for the voicemail, using <Record>:
<?xml version="1.0" encoding="UTF-8"?>
<Response>
  <Dial>
    <Sip>sip:me@1-202-555-0162.sip.us1.twilio.com;transport=tls</Sip>
  </Dial>
  <Say voice="woman" language="en">
    Hi, Jane Doe is not available at the moment.
    You may leave a message after the beep.
  </Say>
  <Record maxLength="60" />
</Response>

A few notes:

You can select from a short list of languages for the message with the voice man or woman or a much larger list of languages with the voice alice.
I have set the maximum duration of the recording to 60 seconds (the default is 1 hour).

This script handles correctly two out of three cases:

✓ voicemail triggers when software phone is closed or computer is unreachable
✓ voicemail triggers when software phone is open but I don't pick up the call
𐄂 voicemail also triggers, sadly, at the end of a successful call, if the caller stays on the line after I have hung up.

Twimlets to the Rescue!
After TwiML Bins, the next simplest option is Twimlets, a set of ready-made scripts developed and hosted by Twilio.
The Voicemail Twimlet answers my needs perfectly:

it plays a message (either using text-to-speech or by playing an audio file)
it records voicemail after the beep
it sends a link to the recording by email

Optionally, it can also transcribe the audio recording to text, but after a few tests, I did not find this option very useful because the transcription always assumed an English speaker even when I called from a French number, and the transcription was far from accurate, even in English.
For example:

Nature said that actually sheds some pebble ascension us and fish..
Just to learn you know I want one. To mull room so. He could come back and then we see the tax cheats..
It's not me it's age cheers..

I'll let you ponder what the original sentences where...
At the end of the page for the Twimlet, you find a form, the Twimlet Generator, which lets you build a URL with parameters to customize the behavior of the Twimlet:

Fill in your email and set the Transcribe option to false (it is enabled by default):

You get the resulting URL, in red, at the bottom of the generator:
http://twimlets.com/voicemail?Email=jane.doe%40bezen.org&Transcribe=false&

Before we can use this URL as a callback in a TwiML script, we need to replace every & with & because TwiML is an XML dialect, and & is part of the characters that have a special meaning and require escaping in XML. I also recommend changing the URL from http to https for safety.
With these two changes, the above URL becomes:
https://twimlets.com/voicemail?Email=jane.doe%40bezen.org&Transcribe=false&

If you leave the Message field empty, the default message will be played:

Please leave a message after the beep.

This is a bit dry. You can customize this message, adding your name or your phone number. Your message can span several lines, for example:

Hi, Jane Doe is not available at the moment.
You may leave her a message after the beep.

Type your text in the Message field and you get a more complicated URL:

Change it to https and replace every & with & and you get the corresponding callback URL for use in your TwiML scripts:
https://twimlets.com/voicemail?Email=jane.doe%40bezen.org&Message=Hi%2C%20Jane%20Doe%20is%20not%20available%20at%20the%20moment.%0AYou%20may%20leave%20her%20a%20message%20after%20the%20beep.&Transcribe=false&

Sadly, it is not possible to select the voice and language for the text-to-speech conversion. As a result, this method is only suitable for messages in English.
If you want to play a message in a different language or in your own voice, you will need to play an audio recording instead. I will describe how to record and setup a message in your own voice in the section below. You can skip the next section if you are using the text-to-speech conversion, and jump to the section after, which describes how to set the callback URL in the TwiML script which handles incoming calls.

Recording Your Voicemail Greeting
Here I am using Audacity to record the greeting message in WAV format. Alternatively, you can use any audio recorder capable of saving files in WAV or AIFF format. Twilio also supports playing files in MP3 format, but these are already compressed, and their conversion to the low bitrate format actually played on telephone networks is slower and may yield poor results. Thus Twilio recommends to record at 8kbps in a lossless format.
I do not think that they actually mean 8kbps uncompressed, but rather 8kHz with a depth of 8 bits which gives 64kbps uncompressed:

In telephony, a standard audio signal for a single phone call is encoded as 8,000 analog samples per second, of 8 bits each, giving a 64 kbit/s digital signal known as DS0.
— https://en.wikipedia.org/wiki/Pulse-code_modulation#Processing_and_coding

When you first open Audacity, there is no audio track yet:

In the top part of the window, you can select the Audio Host and Recording Device, e.g. the Built-In Microphone. If your computer does not have a built-in microphone, make sure that you plug one in before recording.

Next on the right on the same line, reduce the number of channels from 2 (Stereo) to 1 (Mono):

Then at the bottom-left of the window, change the Project Rate from 44100 Hz to 8000 Hz:

You can now press the  button to start the recording and  to stop it.
If you want to sound more natural, you can put the photograph of a person in front of you. Imagine a first sentence spoken by that person, then press the Record button and look at the person in the photograph while recording your greeting, as if you were answering them. Make sure that you do not leave long pauses before you announce that you are not available, however, or the callers might start speaking, thinking that you are actually on the line.
If you do not want to look at the buttons to start/end the recording, you can use keyboard shortcuts instead: R to record and space to stop the recording.
If you are not satisfied with a recording, press the  at the top-left of the Audio Track to remove it, then start a new recording.
Once you are satisfied with a recording, start by saving the project to keep a copy of the original:

then enter a name for your project:

and click the  button to save the project.
The project name is now displayed in the title bar at the top of the window.
Replay the recording using .

Hosting the Audio File as an Asset on Twilio
The Voicemail Twimlet can play audio files accessible from any URL. If you have your own Web server, you may host it there. I find it more convenient to keep everything in the same place though, and to let Twilio host the audio file for the greeting as well.

In the Twilio dashboard, open the  section.

Go to  subsection.

Click the button .

A file selection dialog opens. Navigate to the location of your audio file and select it for upload. Make sure to select the file with the wav extension, and not the file of the same name with the aup extension, which is the Audacity project.

A dialog opens to confirm the asset upload:

Do not check the box to make the asset private, which would prevent the Voicemail Twimlet from accessing it:

Private Assets are accessible to your Twilio Functions via code. We recommend using private assets for storing sensitive configuration you don't want hosted publicly. For example an XML or JSON file containing URLs for backend services

while

Public Assets are served over HTTPS from the Twilio CDN to ensure it is highly available and secure. Typically, Public Assets are used to store static files that support an application. For example, Public Assets are perfect for hosting .mp3 audio files used in TwiML or serve images sent through MMSes.
— https://www.twilio.com/docs/runtime/assets

Click the  button to confirm the upload.

The file is added to the list of assets, with a progress bar showing its deployment to Twilio's Content Delivery Network (CDN):

Once the deployment is complete, the progress bar is replaced with the path to access the file through Twilio Assets Gateway:

Click the  icon on the right of the path to copy the URL of the asset to the clipboard.

Using the Voicemail Twimlet with an Audio Greeting
We can now configure the Voicemail Twimlet to play our recorded greeting instead of a synthesized message.

Open the page of the Voicemail Twimlet:

https://www.twilio.com/labs/twimlets/voicemail

At the bottom of the page, fill in the Twimlet Generator form. Add your email and set the Transcribe option to false, as before, but this time, paste the URL of the recorded greeting into the Message box instead of typing a text message:

Copy the Resulting URL, e.g.

http://twimlets.com/voicemail?Email=jane.doe%40bezen.org&Message=https%3A%2F%2Fyour-runtime-domain.twil.io%2Fassets%2FVoicemail-Greeting-EDIT.wav&Transcribe=false&

Replace http with https at the start of the URL.

Replace every & in the URL with &.

https://twimlets.com/voicemail?Email=jane.doe%40bezen.org&Message=https%3A%2F%2Fyour-runtime-domain.twil.io%2Fassets%2FVoicemail-Greeting-EDIT.wav&Transcribe=false&

We will now set this URL as callback to handle the status of incoming calls.

Branching Out Incoming Calls into Twimlet Voicemail
At this point, you have a URL for the Voicemail Twimlet with your custom configuration, either using a text message in English or a link to an audio greeting that you recorded yourself.

Go back to the  section in the Twilio dashboard.

Go to  subsection.

In the list of TwiML Bins, click the link to access the script to handle incoming calls:

Add an action attribute to the <Dial> element, and set its value to the URL of the Voicemail Twimlet with your custom configuration, that we prepared previously:

If you forgot to replace all & in the URL with &, your TwiML will be displayed as invalid, without any further explanation, and you will be strongly discouraged, if not prevented, from saving it:

Once you have properly escaped all & as &, the TwiML becomes valid:

Click the  button to save it.

You can now test the proper activation of the voicemail. Close the VOIP software phone on your computer, then call your Twilio number from a regular phone.
After a few seconds, the call fails to connect and you are redirected to voicemail. You can leave a message to yourself, then check your emails. You should find an email from Twilio similar to this one:

From<EMAIL_ADDRESS>Subject: New Voicemail Message from 336[...]
To: Me<EMAIL_ADDRESS>You have a new voicemail from 336[...]
Click this link to listen to the message: https://api.twilio.com/2010-04-01/Accounts/[...]/Recordings/[...].mp3

Follow the link and you should hear the message that you left on your own voicemail.

Managing Voicemails
You can find the list of recorded voicemails in your Twilio dashboard:

Open the  section.

Go to  subsection.

Here you find a list of all your voicemail recordings, which you can filter by date:

Sadly, there is no easy option to delete all these recordings at once. Fortunately, Twilio will store up to 10,000 minutes of voicemail for free (that's more than 6 days!), and only charges $0.0005/minute per month beyond that point.
If you only wish to delete a few voicemail recordings, you can do it one by one:

Click the recording date to access the recording details:

At the bottom of the recording details, a fixed panel displays a link to delete the recording:

Click the link and a dialog opens to confirm the deletion:

Click the  button and the deletion is confirmed:

The recording is now marked as Deleted in the list of recordings:

Otherwise, you can use the Twilio API to list and delete voicemail recordings. Using the API, you can list up to 10,000 recordings in a single request, then delete them one by one.

You may leave your comments below.


Add eslint configuration

This is an idea from https://github.com/TypeFox/monaco-languageclient/issues/315
There currently is no eslint configuration. We should introduce one.

May I suggest using eslint-config-standard


HISTFILE and REDO_HISTORY_PATH not found

This one is weird. You can see that the ENV is set...but it doesn't load until I explicitly set it. I may be doing something wrong...but this is confusing to me.

This works too:

Ideas?

I think the problem is that HISTFILE is a default env variable which I'm not able to read correctly. By explicitly setting it, it becomes a variable locally to the current shell, which the process is able to read. This is clearly a bug, as a workaround you can set this in your zshrc:
export HISTFILE="$HOME/.zsh_history"

Absolutely. Thanks for making this and putting it in Hacker news. Really great idea.

Thank you! Currently the tool is a bit alpha, i've been using it myself for some days and it is expected to have some bugs :)

facing the same on arch linux + zsh

same problem here on mint + zsh
the provided fix worked but I had another error :
REDO_EDITOR or EDITOR env variable must be set

It was fixed by adding this line to ~/.zshrc
export REDO_EDITOR="<your favorite editor>"

So apparently any "default" variable from the environment can't be read from Go. Thanks for reporting, I will look into this and try to update the documentation accordingly

Looks like it apparently.
If it helps I am using oh-my-zsh, maybe that'll help.
Actually, Thank you for this tool.
I had a similar idea a while back but never got around to make it and you made something far better that I would have so :+1:

I think the only way to do this is to export the EDITOR and HISTFILE variable. Since they are hidden by default (not exported). This is slightly inconvenient but luckily a one time setup


Include a note that SDK generation is not supported with OpenAPI 3.0

Description:
Include a note to the document [1] that SDK generation is not supported with the APIs that are created using OpenAPI 3.0 support.
For example: generating server stubs and client SDKs are not disabled from the API Publisher and client SDK download is not supported in API Store.
When a user selects the option OPENAPI 3.0 there is the notification in the product mentioning that we do not support SDK generation with OpenAPI 3.0 but we need to get this documented in the official API docs, for the users who are evaluating the capabilities via the docs.
[1] https://docs.wso2.com/display/AM2xx/Generating+SDKs

Added the following note in [1] and [2].
SDK generation is not supported with the APIs that are created using OpenAPI 3.0 support.
[1] https://docs.wso2.com/display/AM2xx/Generating+SDKs
[2] https://docs.wso2.com/display/AM220/Generating+SDKs


Unable to build from source using README.md instructions

This might be a noob question but I am unable to build from source using the instructions. I am not very familiar with cmake or vcpkg so maybe this is the issue (it has been a while since i fought with c++).
I installed vcpkg from source and built it fine, then it seems to install pe-parse correctly, but when trying to build uthenticode it cant find it and i cant figure out how to tell it where it is installed.
mic@devbox:~/projects/uthenticode$ ~/projects/vcpkg/vcpkg install pe-parse
Computing installation plan...
The following packages are already installed:
    pe-parse[core]:x64-linux -> 1.2.0
Package pe-parse:x64-linux is already installed

Total elapsed time: 47.66 us

The package pe-parse:x64-linux provides CMake targets:

    find_package(pe-parse CONFIG REQUIRED)
    target_link_libraries(main PRIVATE pe-parse::pe-parser-library)

mic@devbox:~/projects/uthenticode$ mkdir build && cd build
mic@devbox:~/projects/uthenticode/build$ cmake ..
-- The C compiler identification is GNU 10.2.0
-- The CXX compiler identification is GNU 10.2.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMake Error at src/CMakeLists.txt:5 (find_package):
  By not providing "Findpe-parse.cmake" in CMAKE_MODULE_PATH this project has
  asked CMake to find a package configuration file provided by "pe-parse",
  but CMake did not find one.

  Could not find a package configuration file provided by "pe-parse" with any
  of the following names:

    pe-parseConfig.cmake
    pe-parse-config.cmake

  Add the installation prefix of "pe-parse" to CMAKE_PREFIX_PATH or set
  "pe-parse_DIR" to a directory containing one of the above files.  If
  "pe-parse" provides a separate development package or SDK, be sure it has
  been installed.

-- Configuring incomplete, errors occurred!
See also "/home/mic/projects/uthenticode/build/CMakeFiles/CMakeOutput.log".

Sorry for this - i figured it out. I first needed to install vcpkg properly:
mic@devbox:~/projects/uthenticode/build$ ~/projects/vcpkg/vcpkg integrate install
Applied user-wide integration for this vcpkg root.

CMake projects should use: "-DCMAKE_TOOLCHAIN_FILE=/home/mic/projects/vcpkg/scripts/buildsystems/vcpkg.cmake"
mic@devbox:~/projects/uthenticode/build$ cmake -DCMAKE_TOOLCHAIN_FILE=/home/mic/projects/vcpkg/scripts/buildsystems/vcpkg.cmake ..

From a clean build directory worked to properly configure the project and it built then.

Thanks for posting your solution! Yep, you need to configure CMake to get its packages from the vcpkg toolchain. It's a little  annoying, but you should only need to do it once per CMake build (or set it globally in your environment).
I'd be happy to accept a PR that tweaks the README slightly to include a reference to CMAKE_TOOLCHAIN_FILE 😄


Speed and accuracy test

I got one question about the test result after a scrutiny of this paper. Did you train these comparison models based on pytorch? If so, have you tuned hyperparameters in accord with official documents?
Thanks

I got one question about the test result after a scrutiny of this paper. Did you train these comparison models based on pytorch? If so, have you tuned hyperparameters in accord with official documents?
Thanks

Detailedly，ResNet101-SSD is re-trained and tested, cornernet and refinedet are tested based on their official provided models.
Calculating the ACC is with no controversy(mmAP), while computing the speed always exists divergence.
At the same time when we release the M2Det code, we will give a new speed comparison chart under fair conditions.


Badly labeled command?

When I read the Dev Center documentation, there are plenty of references to heroku ps - in particular around showing your dyno formation.  It's a very commonly used command, I guess?
I was demonstrating this to a friend, and we brought up the heroku CLI help, which showed:
VERSION
  heroku/7.7.1 darwin-x64 node-v10.8.0

USAGE
  $ heroku [COMMAND]

COMMANDS
  access          manage user access to apps
  addons          tools and services for developing, extending, and operating your app
  apps            manage apps on Heroku
  auth            check 2fa status
  authorizations  OAuth authorizations
  autocomplete    display autocomplete installation instructions
  buildpacks      scripts used to compile apps
  certs           a topic for the ssl plugin
  ci              run an application test suite on Heroku
  clients         OAuth clients on the platform
  config          environment variables of apps
  container       Use containers to build and deploy Heroku apps
  domains         custom domains for apps
  drains          forward logs to syslog or HTTPS
  features        add/remove app features
  git             manage local git repository for app
  help            display help for heroku
  keys            add/remove account ssh keys
  labs            add/remove experimental features
  local           run heroku app locally
  logs            display recent log output
  maintenance     enable/disable access to app
  members         manage organization members
  notifications   display notifications
  orgs            manage organizations
  pg              manage postgresql databases
  pipelines       groups of apps that share the same codebase
  plugins         list installed plugins
  ps              Client tools for Heroku Exec
...

I find it quite confusing that it's called "client tools for Heroku Exec", when clearly I can use it see which dynos are running, and that is probably its main function?
Also, some help commands start with a capital, and others do not :-)

@jdxcode i created #14 but maybe there's a better way to do this so that we don't override the ps topic at all?

@jkutner I think if you just remove the topic it should use the original one


URL Rewriting on backend service only

From what I can tell from Front Door behavior, setting a URL rewrite serves only configured front door routing rules.
Take the following example:

Front door configured with frontend host myfd.azurefd.net
A backend pool with one host, say https://myexample.com
A routing rule  matching /foo with URL rewrite set to /bar

Requesting https://myfd.azurefd.net/foo tries to serve https://myfd.azurefd.net/bar.  Is it possible to apply this behavior to the selected host from the backend pool (https://myexample.com/bar)'?
If I already have a separate routing rule matching /bar, that rule will be served when I request /foo.  In this scenario, how can I serve /bar from another backend pool?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 6b393c2f-c603-6287-743f-9c169673ce41
Version Independent ID: 475325a1-e953-2723-b626-80ea86a4d347
Content: Azure Front Door Service - URL Rewrite
Content Source: articles/frontdoor/front-door-url-rewrite.md
Service: frontdoor
GitHub Login: @sharad4u
Microsoft Alias: sharadag

@swhite24 , Thank you for your feedback. We will investigate and get back to you shortly.

@swhite24 , There is an option in the backend pool settings called "Backend host header".  The host header value sent to the backend for each request. So this will resolve your first question.
I am not able to fully understand your second questions. Can you explain with an example ?

Hi @msrini-MSFT, thanks for getting back to me.
I do currently have the backend host header set to match the backend host for each of the two backends, but am still seeing the behavior I described.
Here's some example relevant configuration:

Field
Backend1
Backend2

Host
foo.com
example.com

Host Header
foo.com
example.com

Field
Routing Rule 1
Routing Rule 2

Path
/app1
/app2

URL rewrite
/
/

Backend
Backend1
Backend2

And here's what I'm seeing:

Requested
Expected
Actual

myfd.azurefd.net/app1
foo.com/
myfd.azurefd.net/

myfd.azurefd.net/app2
example.com/
myfd.azurefd.net/

@swhite24 , According to your example,
myfd.azure.net/app1 will  rewrites to myfd.azure.net/app1/
The path will get appended not removed.
Reference: https://docs.microsoft.com/en-us/azure/frontdoor/front-door-url-rewrite
Even though the request , AFD rewrites the host header to azurewebsites.net. I made a request to AFD with afd FQDN and this is how I see the request hitting Web App:
2019-10-23 16:02:45 RYHILLDEMO **GET /**apple/ X-ARR-LOG-ID=b179ce2d-4c62-4a82-8af8-f31082531484 443 - <IP_ADDRESS> Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/77.0.3865.120+Safari/537.36 ASLBSA=029843cc081dae4fef36f2917f680c1a37a913e30ffbb25e372b523abf300c96 - XXXX.azurewebsites.net 404 0 2 424 1601 30
I am getting a 404 because I haven't configured the path with appropriate files.

Hmmm that's not how I understand those docs.  If a routing rule has a matched path of /app1 and custom forwarding path of /, an incoming request to /app1/foo should be rewritten as /foo. An incoming request of /app1 should be rewritten as /, etc.
The issue I'm seeing, though, seems to be that AFD is applying the URL rewrite to itself and not directly to the backend.  For example, a routing rule set to forward, not redirect, matching /foo to a custom forwarding path of /rules gives the following:
$  curl -i https://myfd.azurefd.net/foo
HTTP/1.1 200 Connection Established

HTTP/2 301
content-length: 41
content-type: text/html; charset=utf-8
location: /rules
set-cookie: ARRAffinity=518ba45d6184f6a3519235f21225af4816219d136bf638834cb4cefb1e692de7;Path=/;HttpOnly;Domain=example.com
x-azure-ref: 0d4iwXQAAAAB7j8+f1/ITRZs1+sHdHtw9Q0hHRURHRTE2MTAAMjYzNjg3ZmMtNTNlNi00Y2ViLTgzZDYtNDA3MjJlMWIxYTYx
date: Wed, 23 Oct 2019 17:05:59 GMT

Moved Permanently.

Here I'm receiving a 301 directly from AFD, saying to redirect from https://myfd.azurefd.net/foo to https://myfd.azurefd.net/rules.  My expectation is that https://myfd.azurefd.net/foo would not send a 301, especially for a non-redirecting routing rule, and would simply serve the backend while applying any path modifications.  Is that not the intended result?

@swhite24 , yes your understanding is correct. Only paths which is appended to the request will gets appended.
Regarding the redirection part of 301, I will do a test in my lab and tell you whether it is by design.

@swhite24 , Do you have any redirect config in your backend server or in AFD ?
When I tried to debug your URL which you mentioned, I see this:
Request URL: https://myfd.azurefd.net/foo
Request Method: GET
Status Code: 302
Remote Address: [2620:1ec:bdf::10]:443
Referrer Policy: no-referrer-when-downgrade
content-length: 0
date: Thu, 24 Oct 2019 05:29:33 GMT
location: /pages/404.html
server: Microsoft-IIS/10.0
When you only configure rewrite, you will not see re-directions. This is my AFD with a simple rewrite rule configured.
Request URL: http://vinofd.azurefd.net/apple/
Request Method: GET
Status Code: 200 OK
Remote Address: [2620:1ec:bdf::10]:80
Referrer Policy: no-referrer-when-downgrade
Accept-Ranges: bytes
Content-Encoding: gzip
Content-Length: 1378
Content-Type: text/html
Date: Thu, 24 Oct 2019 05:27:53 GMT
ETag: "80edb6315488d51:0"
Last-Modified: Mon, 21 Oct 2019 21:12:07 GMT
Set-Cookie: ARRAffinity=dd06b125264b13eb8d6fb6701baae4c57773365bd6be5765367aad0cd8521ed3;Path=/;HttpOnly;Domain=ryhilldemo.azurewebsites.net
Vary: Accept-Encoding
X-Azure-Ref: 0WTaxXQAAAADdyI++sZ4uR6gdpF57yos3TUFBMDFFREdFMDMxOABmNjJkMmVjNi1kNDBmLTQ2YTktOTY0Ni03NDBlMDQwZTllYzc=
X-Powered-By: ASP.NET

@swhite24 , I will go ahead and close this issue. Please tag me if you have any further questions.


update broken links

Hey @mcnaveen thanks for the PR, but these links were fixed already on another commit. So I'm closing it.
Appreciate you taking the time to create it though!!


Assertion failed in InteriorPointPoint

This statement crashes the debug version of the DuckDB Spatial extension:
SELECT ST_PointOnSurface(g)  FROM (SELECT ST_GeomFromText('MULTIPOINT((0 0), EMPTY)') As g);

The assertion failed here:
duckdb: /duckdb_spatial/build/debug/deps/projects/Source/GEOS/src/algorithm/InteriorPointPoint.cpp:68: void geos::algorithm::InteriorPointPoint::add(const geos::geom::CoordinateXY*): Assertion `point' failed.

Version:
┌─────────────────┬────────────┐
│ library_version │ source_id  │
│     varchar     │  varchar   │
├─────────────────┼────────────┤
│ v0.9.2-dev209   │ 6545a55cfe │
└─────────────────┴────────────┘

The assertion failure is triggered using the latest nightly version of the Spatial extension.

Hi! Thanks for reporting this issue! Maybe next time you could link me directly to where it was fixed or reference the GitHub discussion when you're the owner of the original GEOS issue so I don't accidentally open new issue in GEOS or spend time on my own fix.
I'll see if I can update our GEOS bundle to latest main later this week.

Sorry for missing the original issue link.
Next time I will attach it.

To follow up on this, we did not know in advance that both of these test cases triggered the same underlying bug. We are currently developing an automated testing tool for testing spatial databases, and independently found test cases that crash PostGIS and DuckDB. Sorry again!

I'm just checking whether an update is planned, so I could potentially report more issues. I would greatly appreciate this, but can understand that this might not be a high priority for you.

Im updating GEOS in #193 which should contain fixes for all the issues you found so far, but please feel free to report more issues (unless this is blocking) and I'll try fix them as soon as I can!