getLastNonConfigurationInstance() for retaining Activity Presenters

We could use onRetainNonConfigurationInstance() and  getLastNonConfigurationInstance() to make presenter for activities survive orientation changes.
Unfortunately, getLastNonConfigurationInstance() has been marked as @Deprecated. Imho this has been done overhasty because retaining fragments and loaders use LastNonConfigurationInstance internally.
On the other hand we shouldn't relay on @Deprecated API.
Probably this "workaround" could also be used to create retaining presenters for android.view.ViewGroup ...

@sockeqwe Fortunately, you can use onRetainCustomNonConfigurationInstance which is not deprecated

@Brian. That's a mouthful of a name :)
On Aug 6, 2015 2:47 PM, "Brian"<EMAIL_ADDRESS>wrote:

@sockeqwe https://github.com/sockeqwe Fortunately, you can use
onRetainCustomNonConfigurationInstance
http://developer.android.com/reference/android/support/v4/app/FragmentActivity.html#onRetainCustomNonConfigurationInstance()
which is not deprecated
—
Reply to this email directly or view it on GitHub
https://github.com/sockeqwe/mosby/issues/64#issuecomment-128471734.

Yes, that should work.  However, onRetainCustomNonConfigurationInstance() is a method defined in FragmentActivity. Therefore this method does not exists in plain old Activity. I wanted a solution that works universally for both. But I'm not sure if that will work since onRetainNonConfigurationInstance() is marked as final in FragmentActivity. Hence, this method cannot be overridden anyway.
I guess onRetainCustomNonConfigurationInstance() can be used and the plain old Activity (not FragmentActivity) has to override onRetainNonConfigurationInstance().
Thanks for the tip @brianguertin !

Implemented, will be part of Mosby 2.0

I know this issue is closed but @commonsware discovered and @adamwp confirmed that onRetainNonConfigurationInstance is being undeprecated in N: https://twitter.com/adamwp/status/720612665136009218

Implemented, will be part of Mosby 2.0

Is it? When I search for "getLastNonConfigurationInstance" in Mosby I get 0 results.

However, onRetainCustomNonConfigurationInstance() is a method defined in FragmentActivity. Therefore this method does not exists in plain old Activity.

So why not use FragmentActivity instead of Activity just for that reason?

It is implemented. You can have Activities with presenter that survive screen orientation changes since Mosby 2.0. simply extend from MvpActivity or  MvpViewStateActivity (or LCE variants) and call setRetainInstance(true) in onCreate()

Ah OK, it is implemented but not like the title suggested with onRetainNonConfigurationInstance but with setRetainInstance.

Under the hood Mosby 2.0 uses onRetainNonConfigurationInstance()

Is possible to see the code then, as I said

When I search for "getLastNonConfigurationInstance" in Mosby I get 0 results.

It actually uses getLastCustomNonConfigurationInstance/onRetainCustomNonConfigurationInstance.

@farmazon3000 you should search for onRetainCustomNonConfigurationInstance instead


SNOMED codes for edattendsource are incomplete/incorrect

The following codes have been listed in the Sheffield data specification.  However, LTH have a large number of rows where our codes do not match the ones in this list (see counts below this table).

SNOMED CODE
ECDS Description
Categories

507291000000100
Self-referral to accident and emergency department
Personal

1065391000000104
Referred by carer
Personal

315261000000101
Advised to attend accident and emergency department
Personal

276491000
Referred by member of Primary Health Care Team
Primary Care

166941000000106
Out of hours GP service
Primary Care

879591000000102
NHS telephone / internet advice e.g. NHS 111
Primary Care

1066431000000102
Emergency department
Hospital

1066441000000106
Urgent care service
Hospital

835091000000109
Outpatient service inc. ambulatory care
Hospital

835101000000101
Inpatient
Hospital

1079521000000104
Private specialist
Hospital

1077191000000103
Community nurse (not practice nurse)
Community

1052681000000105
Health visitor
Community

185363009
Midwife
Community

1065401000000101
School nurse
Community

1077201000000101
Community mental health nurse
Community

1065991000000100
Mental health assessment team
Community

877171000000103
Social services
Community

1077761000000105
Older persons day care centre
Community

1077211000000104
Homeless persons drop in centre
Community

1066011000000104
Custodial services : prison
Community

1066001000000101
Custodial services : detention centre
Community

185369008
Pharmacist (including community pharmacist)
Community

185366001
Dentist (including community dentist)
Community

185368000
Optician / optometrist
Community

1066021000000105
Advanced care practitioner
Emergency Services

198261000000104
Ambulance service - patient in transit
Emergency Services

889801000000100
Police service / forensic medical officer
Emergency Services

1066031000000107
Fire service
Emergency Services

1066061000000102
Search and rescue
Emergency Services

1066041000000103
Coastguard
Emergency Services

1066051000000100
Mountain rescue
Emergency Services

LTH counts that don't match with codes in specification.
1991000124105                42560
108161000000109               2180
1082331000000106                16
185360007                       13
198251000000102                 12
1052681000000100                 3

Close and track #27 instead.


Xnero: Watch apillz.com

Xnero requests the watch of the watch_keyword apillz\.com. See the MS search here and the Stack Exchange search in text, in URLs, and in code.

apillz\.com has been seen in 1 true positive, 0 false positives, and 0 NAAs.

Approved by cigien in Charcoal HQ


VueMultiSelect unit test

I based myself on fieldSelect and remove some test that made no sense for this fields (it doesn't have a value property).
I'm waiting for your comments.

Coverage increased (+1.5%) to 81.4% when pulling 4c7c71746c017054a59a434dfeace1e822cf4206 on lionel-bijaoui:LB_vueMultiSelect_unit_test into 4ab7a8ee26c0dc80b629719b26c47d7b57ee9432 on icebob:master.

Good PR, thank you. I wrote my comments, if you fix them, I can merge it :)

Coverage increased (+1.5%) to 81.4% when pulling 09a41f0ca519804262637a3962e7ab62b78dd55d on lionel-bijaoui:LB_vueMultiSelect_unit_test into 4ab7a8ee26c0dc80b629719b26c47d7b57ee9432 on icebob:master.

Are you ready? Can be merge?

Yes, it will certainly need more in the future, but this is honest.

I fixed the noUiSlider unit test


cap sync ios failing for version 0.10.0

Environment
I am running the latest Capacitor version (4.2.0), as well as the latest @sentry/capacitor version (0.10.0)
Steps to Reproduce

Pull this minimal reproduction repo: https://github.com/ekrapfl/sentry-test
npm ci
npm run build
npx cap sync

Expected Result
No errors when installing and syncing the latest version of @sentry/capacitor
Actual Result
You will see the following error in the console from the cap sync
✖ Updating iOS native dependencies with pod install - failed!
✖ update ios - failed!
[error] Analyzing dependencies
        [!] CocoaPods could not find compatible versions for pod "Sentry":
        In Podfile:
        SentryCapacitor (from `../../node_modules/@sentry/capacitor`) was resolved to 0.10.0, which depends on
        Sentry (~> 7.23.0)

        None of your spec sources contain a spec satisfying the dependency: `Sentry (~> 7.23.0)`.

        You have either:
        * out-of-date source repos which you can update with `pod repo update` or with `pod install --repo-update`.
        * mistyped the name or version.
        * not added the source repo that hosts the Podspec to your Podfile.

thanks for opening the issue with a sample :D
Have you tried to run pod install --repo-update?  I was able to reproduce it but I no longer can repro it after running the pod update.

No worries, I also faced the same problem on a Mac M1 (it always breaks something when the OS updates haha)
Glad it worked on your end, for the time being I will just update the troubleshooting page for cases like this :)


Use tox instead of nox

Example https://github.com/gaborbernat/python-build/actions/runs/293226462

pypy2 isolation is broken, can reproduce it locally. Not sure how/why it passes with nox, but it's a bug.

pypy does not handles/honor/cares about PYTHONHOME, see https://foss.heptapod.net/pypy/pypy/-/blob/branch/default/pypy/interpreter/app_main.py#L38, hence the issues here; prefixes remain under original executable path, which then pulls in the site-packages via https://foss.heptapod.net/pypy/pypy/-/blob/branch/default/lib-python/2.7/site.py#L295-297 and https://foss.heptapod.net/pypy/pypy/-/blob/branch/default/lib-python/2.7/distutils/sysconfig_pypy.py#L38-56 respectively

Also, please drop the PyPI publish action, I want to start signing the packages there, not just the git tag. We will have a release script for this anyway. It also decreases attack surface because we will not need to have a token with PyPI upload permissions dangling on github.

I personally consider the Github security teams solution more secure than our own machines. Because the github release machine has more security experts reviewing their solution, and we don't have to store security keys on our own machine. 🤔 Furthermore, I'd rather have release automated via CI, than end-user invoked, if possible.

I personally consider the Github security teams solution more secure than our own machines. Because the github release machine has more security experts reviewing their solution, and we don't have to store security keys on our own machine. thinking Furthermore, I'd rather have release automated via CI, than end-user invoked, if possible.

How so? Your machine has push permissions to Github, if it compromised it can create releases that look valid in the Github side and there is no security precautions Github can take to prevent that. And I do trust my machine more than Github.
Anyway, as I said, I want to sign the PyPI packages, so that requires me to do it locally anyway. I am sorry but I will have to put the foot down in this one :confused: I am happy to keep discussing if you want.

How so? Your machine has push permissions to Github, if is compromised it can create releases that look valid in the Github side and there is no security precautions Github can take to prevent that. And I do trust my machine more than Github.

I don't store security keys on my machine, if I can avoid. I create PyPi API keys that can publish to just 1 project, and store that under Github security tokens. Can't you store the security key for signing the package also in Github secrets store, if we want to sign it?

I don't store security keys on my machine, if I can avoid. I create PyPi API keys that can publish to just 1 project, and store that under Github security tokens. Can't you store the security key for signing the package also in Github secrets store, if we want to sign it?

Right, but don't you have a ssh key with push permissions to Github? With that an attacker could push malicious code and make a release, Github would then upload it to PyPI. Even if you have your ssh key encrypted or even if you use plain credentials, if an attacker has access to your machine, they could easily install a keylogger to get your password.

Right, but don't you have a ssh key with push permissions to Github?

No SSH, just HTTPS. However, my password is storred within macOs Keychain Access, that's password protected and fairly well securred.

For clarification, I have my ssh key as a gpg authenticaion key inside my yubikey. It's kind of a complicated setup, you can read about it here https://www.linode.com/docs/security/authentication/gpg-key-for-ssh-authentication/.

You want a local release script? That will be annoying, and if I understand correctly it means that either you'd be the only one authorized to do releases or we'd need to have a proliferation of people who have release privileges.
It's also a lot more of a PITA.
I think the only difference in threat models between "push signed tags to GH, verified on GH Actions" and "make signed releases from your local computer" is that a malicious actor with push permissions on GH could modify the action to not check the signature before publishing to PyPI, which would at least leave a public artifact. Otherwise you still need to be able to sign the git repository with the releaser's key.
Another option might be to have the GHA dump artifacts into a holding area and have a bot running on Heroku, GCP, AWS, a VPS or something check that they are properly signed, with the bot holding the token and doing the upload. I've got a project that lets you execute arbitrary bash scripts in response to Github webhooks: https://gitlab.com/pganssle/githooks
I use it to run my website and it's been working pretty well so far. Might be easy enough to use something like that to do the "release with intermediate verification" automation.

You want a local release script? That will be annoying, and if I understand correctly it means that either you'd be the only one authorized to do releases or we'd need to have a proliferation of people who have release privileges.

To publish release you should meet the following criteria:

Have push permission to the repo
Have push permission on PyPI
Have your GPG key listed in the documentation
3.1) Preferably have your GPG key signed by the original/main maintainer, in this case, me (I think we should probably make this a requirement, but I don't want to promote bad key signing practices)

Any maintainer, after given release permission, should be able to make a release.
The script could check for the requirements, at least the git and PyPI push permissions, before starting the process.

It's also a lot more of a PITA.

Which part exactly would be PITA? The only thing I'd imagine would be a bit of PITA is the PyPI upload, where you need to either provide the credentials or have a token. But I think that is a perfectly reasonable for a rrelease workflow. You won't be releasing

I think the only difference in threat models between "push signed tags to GH, verified on GH Actions" and "make signed releases from your local computer" is that a malicious actor with push permissions on GH could modify the action to not check the signature before publishing to PyPI, which would at least leave a public artifact. Otherwise you still need to be able to sign the git repository with the releaser's key.

A token with push permission to PyPI would be saved as a secret on Github. There can't be such thing as Github Actions verifying your git tag, if I have push access to Github I can disable the check and just use the token to upload anything I want. The verification can't happen somewhere that could be compromized.
Furthermore, gaining push access to Github is an extremely low bar. If I can access your computer and you have a Github session open, I have push access. If I acess your computer and you have an unlocked SSH key (I think most people use a keyring, and the key is generally unlocked on a timeout or even for each boot cycle).

Another option might be to have the GHA dump artifacts into a holding area and have a bot running on Heroku, GCP, AWS, a VPS or something check that they are properly signed, with the bot holding the token and doing the upload. I've got a project that lets you execute arbitrary bash scripts in response to Github webhooks: gitlab.com/pganssle/githooks

Basically the same attack surface as Github, could be a bit better depending on your provide security practices. Having a session open on the provide website (if they don't require reauthentication for sensitive operations) or an unlocked ssh key could compromise.
This also makes the setup more complex and can introduce financial cost.

I use it to run my website and it's been working pretty well so far. Might be easy enough to use something like that to do the "release with intermediate verification" automation.

That is fine for a personal website. This project had the possibility of compromising the supply chain.
Even ignoring all that, I want to sign the artifacts that get pushed to PyPI, some distros prefeer to download the sdist from there instead of fetching the source from Github. I don't see how to do this remotely.

Which part exactly would be PITA? The only thing I'd imagine would be a bit of PITA is the PyPI upload, where you need to either provide the credentials or have a token. But I think that is a perfectly reasonable for a release workflow. You won't be releasing very often.

I've been making releases manually on dateutil for years (and they are very infrequent), and I consider it a bit of a pain, because:

It happens in my local directory, which may have all kinds of weird build artifacts lying around.
There's no guarantee that the thing I'm uploading is the thing that just passed the tests on all platforms.
There's a lot of latency involved in pushing, waiting for tests, etc.
It's way harder to scale out and keep track of the people authorized to do releases.

By contrast, I can make sure that the builds all happen via a completely automated and reproducible process if I have CI do them, and it's very easy to scale out who has access. I just have to tag, wait a while, then check to make sure everything worked.

Basically the same attack surface as Github, could be a bit better depending on your provide security practices. Having a session open on the provide website (if they don't require reauthentication for sensitive operations) or an unlocked ssh key could compromise.

This is not the same attack surface as Github really, because very few people need admin access to the uploader bot, so the permissions to trigger a release are separated from the permissions to alter the checks required for a release, and most people don't need permissions to alter the checks.
So the workflow would be something like GHA triggers a release and a build then makes the artifacts available via the little deployment zip file they have. This triggers a webhook that causes ReleaserBot to:

Download the zip file containing the built artifacts.
Check that they were built against a commit signed by someone on the allowed releaser list.
Sign the artifacts with a dedicated key or subkey signed by the maintainers.
Use a token stored locally to upload to PyPI.
Publish logs somewhere public.

Now the threat models are:

Someone compromises the releaser bot.
Someone can MITM the releaser bot and pretend to be Github.
Someone compromises the GPG key and Github account of anyone authorized to make releases.

Number 2 is not a realitsic concern since it would use HTTPS. Number 3 is already the threat model for "local script to make releases", so we're no worse off in that situation. The most likely avenue for compromising the releaser bot would be getting access to the credentials of someone who can access it — which is almost certainly a subset of the set of people with permissions to upload, in which case we're again no worse off than we would be in a "local release script" situation. We'd actually be slightly better off in some respects, because the entire setup I'm describing is much more auditable than a local release script would be. If we set it up right, every event is automated and logged in public.

I've been making releases manually on dateutil for years (and they are very infrequent), and I consider it a bit of a pain, because:
1. It happens in my local directory, which may have all kinds of weird build artifacts lying around.

2. There's no guarantee that the thing I'm uploading is the thing that just passed the tests on all platforms.

3. There's a lot of latency involved in pushing, waiting for tests, etc.

4. It's way harder to scale out and keep track of the people authorized to do releases.

By contrast, I can make sure that the builds all happen via a completely automated and reproducible process if I have CI do them, and it's very easy to scale out who has access. I just have to tag, wait a while, then check to make sure everything worked.

All this can be fixed in a release script.

Now the threat models are:
1. Someone compromises the releaser bot.

2. Someone can MITM the releaser bot and pretend to be Github.

3. Someone compromises the GPG key and Github account of anyone authorized to make releases.

Number 2 is not a realitsic concern since it would use HTTPS. Number 3 is already the threat model for "local script to make releases", so we're no worse off in that situation. The most likely avenue for compromising the releaser bot would be getting access to the credentials of someone who can access it — which is almost certainly a subset of the set of people with permissions to upload, in which case we're again no worse off than we would be in a "local release script" situation. We'd actually be slightly better off in some respects, because the entire setup I'm describing is much more auditable than a local release script would be. If we set it up right, every event is automated and logged in public.

I am not comfortable with 1), which can be completely avoided with the release script. If I worked at the provider it could be fairly easy to simply fetch the PyPI token from the machine. Number 2) I agree with you. For number 3), what you said isn't necessarily true. If you simply have the keys on disk, yes, but you are not required to have the keys on the disk. For eg. in my setup, as I mentioned above, I use a yubikey, so my key is not on disk and it would be very difficult to steal the key, it would require a very advanced physical attack. For every operation, I am required to unlock the key with a pin and then physically touch it to sign-off, the key is never tranfered to the host computer, cryptographic operations happen on the key itself.
Having the release script locally simply has a smaller attack surface. And I will be making most, if not all, releases anyway, we can come back to this if that changes in the future. I think that is reasonable, no?

All this can be fixed in a release script.

I mean, almost none of it can be. I have release scripts. You could theoretically spin up a docker container every time (now your release script has a dependency on docker, logs are local only if you bother to save them). I guess you could have your release script poll CI and then block until it succeeds, but then of course you have to leave the computer doing the release open and running during the entire CI run. A release script does not help with permissions management (revoking permissions in one place rather than revoking them in several places).
In any case, the threat model for 1 that you propose is a little far-fetched if we were to host on a reputable hosting platform. I would think we would do this in one of only a few ways:

Ask the PSF to host it on their infrastructure — in which case compromise of our infra is roughly equivalent to compromise of PyPI anyway.
Host it on AWS / GCP / Azure — in which case it is astoundingly unlikely that some random employee of the huge cloud companies who are fiercely competing with each other would be able to just compromise a key.
One of us runs a local internet-connected server on a raspberry pi or something.

I think you're probably over-thinking things and doing a bit of security theater. Yes there's some possibility that if you hand this over to a bot it could get compromised. On the other hand, anything I've proposed here is head and shoulders more secure than many extremely popular Python libraries. If someone compromises my computer and GPG keys, they could make arbitrary releases of setuptools for example. If they compromise Bernat's computer they could presumably make releases of tox and virtualenv. I doubt they would spend much effort on it, and it would be reasonably simple to add in an automatic audit — configure a server to watch PyPI and send you an e-mail any time a new release is made.
It doesn't affect me much anyway. If there is a way to cut a release by sending you my public key and then pushing signed tags, there's a decent chance that I would cut a release from time to time. If it's a manual process using a local script, I won't.

I mean, almost none of it can be.

I agree it is not a perfect solution, but it fair IMO.

It happens in my local directory, which may have all kinds of weird build artifacts lying around.

The release script creates a temporary directory and git clones from out local .git to there, making sure it is a clean copy of the repo and everything is clean.

There's no guarantee that the thing I'm uploading is the thing that just passed the tests on all platforms.

The release script then query the CI for the check status. (more info in the next point)

There's a lot of latency involved in pushing, waiting for tests, etc.

You are tagging a release from a particular commit. There would be a requirement for this commit to have already been pushed and the Github CI finished. When tagging a release, we would query Github for the CI status and proceed if it succeeded.

It's way harder to scale out and keep track of the people authorized to do releases.

I don't agree much here, ultimately the people that have permission to push a release would be the people with access to the project on PyPI.

Host it on AWS / GCP / Azure — in which case it is astoundingly unlikely that some random employee of the huge cloud companies who are fiercely competing with each other would be able to just compromise a key.

I wouldn't be so sure, but yeah.

One of us runs a local internet-connected server on a raspberry pi or something.

I manage a server at my college, I could possibly host it there, although I don't know about the long-term reliability. I am gonna stay until next year to finish my bachelor's, not sure if I will stay for my master's, a lot of classes feel like a joke, we'll see.

I think you're probably over-thinking things and doing a bit of security theater.

Isn't security always a theater :stuck_out_tongue:? Everything is fine until it isn't, and when it isn't, it can have catastrophic consequences.
Every attack we are discussing here is unlikely, but unlikely still means that it can happen. An attack like this can totally happen, and if it does, it had the possibility to hurt people. So, for me it is totally worth, the release process being slightly more annoying is totally worth possibly, even if unlikely, prevent people from getting hurt. But I do understand and accept if you do not feel the same.

On the other hand, anything I've proposed here is head and shoulders more secure than what many extremely popular Python libraries use. If someone compromises my computer and GPG keys, they could make arbitrary releases of setuptools for example. If they compromise Bernat's computer they could presumably make releases of tox and virtualenv. I doubt they would spend much effort on it, and it would be reasonably simple to add in an automatic audit — configure a server to watch PyPI and send you an e-mail any time a new release is made.

And I am unhappy about that, but there is nothing I can do besides annoying the maintainers, and I don't think they would appreciate that :confused:
I guess that will probably change when something bad happens.

It doesn't affect me much anyway. If there is a way to cut a release by sending you my public key and then pushing signed tags, there's a decent chance that I would cut a release from time to time. If it's a manual process using a local script, I won't.

The point of the script is so that the process isn't manual...

But anyway, I think we can agree that for now there shouldn't be a release action :grin:
@pganssle if you want to keep discussing I am happy to, but please do not do it if I am annoying you. I know I am very opinionated about this, but I don't want to be annoying anyone! We can figure out a solution later.

@FFY00 this is also ready for merge, though you need to migrate the required CI checks.

Can you just update the CI badges in the README?
Also, could you keep the default CI names? I prefeer them as default, automatically generated with the matrix configuration.

Done.

@FFY00 I don't have rights to change the required checks, so in order to merge it you'll need to do it 👍

@FFY00 ok, now it's actually all good.

Thanks!


Crawl behavior

Trying to do a normal website recrawl.
It states that starting URL is unmodified. Then ends crawl. All references go to cached table with crawl state NEW and only starting URL remains in table reference with crawl state UNMODIFIED.
Start recrawl for second time  and again it stops after starting URL and cached table is cleared from all references.

You say if the start URL is unmodified it "stops" there?  It should not, unless maybe you changed the default behavior how "orphans" are handled (to be vefiried).
After a first crawl, all URLs will be cached.  On a second crawl, if a page if found not to be modified before its URL were extracted (like by relying on HTTP last-modified header), it will not go further with that page.  Since it needs to know if other pages on the site were modified, this is where the crawler will grab all URLs in the cache from the previous run and try them all again.  Once re-processed, they should find their way back in the cache for future runs.  Are you witnessing anything different and missing documents as a consequence?  If so please attach your config with steps to reproduce (e.g. which documents are affected).

Orphan strategy is IGNORE.
In first crawl it cached all references. Then finds starting URL to be unmodified and then stops recrawl and deletes all the other references.

What is the resulting effect of what you are describing?  In trying to reproduce, I would need the evidence you have that something is not working.  Is the crawler skipping recrawling some documents that were modified?   Having a copy of your config with how to reproduce and what to look for would help.
Not sure if it make a difference, but if you are honoring sitemaps (the default), the recrawlable resolver will honor the crawl delay specified in them.  It may be why you are not seeing documents being recrawled (if not enough time has gone by).  Assuming this is the problem you are having.

With the config you sent me via email I was able to reproduce.  It is a combination of configuration settings you have that causes this.
To find if there was a change, you are using the default <metadataChecksummer> implementation which relies on the Last-Modified date field found form the HTTP headers when making the HTTP call to get a document. By default, that date field is obtained with HTTP headers obtained when the actual document is being downloaded (by the <documentFetcher>).  Normally, URLs to be "followed" are extracted before the metadata checksummer is invoked to find out it was unmodified and then rejected (but the extracted URLs will be crawled).
In your case, you have configured the use of a <metadataFetcher> which will make an extra HTTP "HEAD" call to get the HTTP headers only and not the actual document.   This feature is not enabled by default and exists mostly to avoid downloading potentially large documents for no reason if we can find out using HTTP headers they were not modified.  So it costs an extra HTTP call to potentially save a few big downloads.   The net effect of this is, since the document was not downloaded, its URLs were not extracted and there is nothing to "follow" and it stops there.
Here are your easy options to fix:

If you want to keep using the last modified date in the HTTP header obtained from a <metadataFetcher> HEAD request to prevent some file downloads, keep things as is, and simply change the <orphansStrategy> to PROCESS.  This will ensure the crawler attempts to re-crawl all the "leftovers" from the cache.

If you are dealing mostly with small files and/or you would rather save the extra HTTP call for each document, you can comment/delete the <metadataFetcher> tag.  That way, the last modified date will still be used to find if a document has changed, but it will be obtained from HTTP headers obtained by downloading the file with a <documentFetcher> GET request.  URLs will be extracted and followed just fine, so you can keep the <orphansStrategy> to IGNORE.

If you go with the second option above, since you will have downloaded the file, feel free to disable the <metadataChecksummer> altogether if you would rather have the content compared instead, in case you find that approach more reliable to assess a change than relying on the HTTP header last modified date (not all sites are good with it).

I have to add... you may have to start with a fresh crawl to have some of these changes take effect.

Thank you for the explanation. The metadatafetcher made sense in another config, but can be removed from this one.


Issue in querying from emulator

I am having issue querying from Emulator. Basically what I am doing is:

connecting to Emulator on https://localhost:8081/
begin able to create collection on database
being able to add to collection
when I am trying to query what I have added to collection, it doesn't return any data. the code to query data is:
var client = await this.connectionHandler.InitialiseClient().ConfigureAwait(false); // create DocumentClient           var option = new FeedOptions() { MaxItemCount = maxItemCount, EnableCrossPartitionQuery = true };           var query = await client.CreateDocumentQuery<TEntity>(this.connectionHandler.CollectionLink, option).Where(expression).AsDocumentQuery().ExecuteNextAsync<TEntity>().ConfigureAwait(false);           var items = query.AsEnumerable();

Basically the item is empty. What I should emphasize is as soon as I switch to Azure DocumentDb the same code works.
Any idea?

Excellent support guys 👎


The frame rate is very low.

Overview
FPS is very low even in the basic example scenes on the screen. In UNITY we have the settings for the MIXEDREALITY development environment. What do I need to set more in the example scene to maintain 60 fps?....
Expected Behavior
.
Actual Behavior
.
Steps to reproduce
(Links to sample github project preferred)
.
Unity Editor Version
2017.2.1P2
Mixed Reality Toolkit Release Version
2<IP_ADDRESS>-RC2

@pwroff  Answer Thank you very much !! You saved my time!


ONNXRunTime support for Bool?

Hello!
I am trying to use ONNXRunTime to test a Julia implementation of an ONNX "And" layer in ONNX.jl. "And" takes two Boolean tensors as input then uses logical 'and' elementwise and returns a single Boolean tensor (source).
Unfortunately, unless I have coded my test set wrong, there is currently no implementation in ONNXRunTime.jl for tensor type bool (why is it commented out?).
This error was created during an ONNX test_set but might still provide relevant information:

Is there anything I can do to solve this issue? Will ONNXRunTime support the full range of datatypes at some point?
Best,
Duncan

Thanks! I forgot why exactly Bool is commented out, might be historic. Feel free to lift the comment and if you get stuck with errors ask for help.

Will ONNXRunTime support the full range of datatypes at some point?

Realistically it will support any use cases that somebody wants desperate enough to be willing to implement them.


Add links to the top of the FAQ

The FAQ would be more useful if we added a list of questions at the top with links to each question.
Great first pull request for someone new...

I'm not sure I understand how it should differ from the current FAQ. There is already a list of questions with links at the top.

Someone beat ya to it.  :) Thanks though!

That was fast 😄


Create 2021-05-04-BetaAnnouncement.markdown

Adding a draft of the launch announcement. Kept it short for now - looking to add more details across tech and product.
Couple of notes/questions;

Highlighted the artifacts but should we signal some of next steps?
Do we need the plugin list? Feels heavy as part of the announcement but might be helpful.
Do we have any documentation to pull next steps from?
Still needing 2 release notes for plugins (Data Prepper and Infra /DevOps) and release notes for Docker images and Linux tars
Do we want to add an update on branding?

Description
Create beta announcement file for initial review and collab.
Issues Resolved
Not applicable. New File.
By submitting this pull request, I confirm that my contribution is made under the terms of the BSD-3-Clause License.

Haven't read the text yet, but watch out for those camelCase filenames, would rename to x-y-z and be more descriptive for SEO.


Update whats new for preview 7; fix small bug

Fixes #13546
Fixes #13557

Different login at the mo, but I am mungojam honest 🙂


Flags 'js' and 'java' are mutually exclusive.

genjava.js will fail if foam.js flag 'js' is true. The 'js' and 'java' flags control which FooAware model is used: the class or interface.

I already (which is to say, 15 minutes after, without noticing your CL) fixed this in:
https://github.com/foam-framework/foam2/commit/1bf12b6085276a50f43ad0a691907edaafb63a51


Update modal from new header

https://github.com/open-cluster-management/backlog/issues/8775
https://github.com/open-cluster-management/backlog/issues/8737

/lgtm


Issue with darksky api?

This script has been running beautifully for the last years uploading the goodwe data. However, last saturday something seems to have changed: it's not uploading data anymore and also re-uploading historic data fails.
I'm trying to troubleshoot but I'm not a genius at these things. It seems to fail around the darksky api, which is then connecting apple.com: ?
DEBUG    Starting new HTTPS connection (1): api.darksky.net:443
DEBUG    https://api.darksky.net:443 "GET /forecast<SNIP>units=si&exclude=minutely,hourly,daily,alerts,flags HTTP/1.1" 301 134
DEBUG    Starting new HTTPS connection (1): support.apple.com:443
DEBUG    https://support.apple.com:443 "GET /en-us/HT213526 HTTP/1.1" 200 8885
ERROR    Expecting value: line 2 column 1 (char 1)
DEBUG    Starting new HTTPS connection (1): api.darksky.net:443

OK, it seems that indeed apple closed down the API, so removing the calls from the config file fixes the problem.

I see in pull request that somebody has pushed an alternative OpenWeather API.
Unfortunately, the repository is not maintained anymore hence it didn't make it to the last stable version afaik.


How do I use cuda with sherpa-onnx-node?

Hi!
I installed sherpa-onnx-node, cloned sherpa-onnx built it, what do I do next? I tried create offlineRecognizer with cuda set in config but it keep saying to compile it doesnt matter where I set $LD_LIBRARY_PATH to, wether to build or build/lib build/bin it will still give me same error.
Docs is missing info about this, where do I place files and which ones?

Are you using Linux x64?

yes sir

You don't need to re-build sherpa-onnx. You can do that, but it requires extra effort.
Please do the following steps
git clone https://github.com/k2-fsa/sherpa-onnx
cd nodejs-addon-examples
npm install

You will find it creates a new directory node_modules inside nodejs-addon-examples.
Inside node_modules, you can find a directory sherpa-onnx-linux-x64.
Inside sherpa-onnx-linux-x64, you can find several .so files.
Please download the new .so files from
https://github.com/k2-fsa/sherpa-onnx/releases/download/v1.10.16/sherpa-onnx-v1.10.16-linux-x64-gpu.tar.bz2
and replace those inside sherpa-onnx-linux-x64.
Note that there are several .so files in sherpa-onnx-v1.10.16-linux-x64-gpu.tar.bz2 that are not present in
the original sherpa-onnx-linux-x64. Please copy all of the .so files from sherpa-onnx-v1.10.16-linux-x64-gpu.tar.bz2 to
sherpa-onnx-linux-x64.
Then everything should work as expected, provided that you have installed CUDA toolkit 11.8.
If not, please follow
https://k2-fsa.github.io/k2/installation/cuda-cudnn.html#cuda-11-8
to install CUDA 11.8
Note again you don't need sudo permission to install CUDA 11.8. Just follow the above doc.

It worked! Much appreciated!


clickhouse merges slower than inserts

So the carbon-clickhouse server seems to be working ok, but I constantly run into the "Merges are processing significantly slower than inserts" errors in click house and I'm wondering if I'm tuning carbon-clickhouse wrong.  I mean I can keep bumping the merge_tree settings in clickhouse, but that feels wrong.  Currently using:
chunk-interval = "5s"
chunk-auto-interval = "10:20s,40:60s"
Which seems to buffer stuff ok, but I'm not even close to full load yet (14 metrics per host * every minute * currently ~100 hosts).  Eventually it will be thousands of hosts and probably more metrics.  Currently I have to keep it at least this in clickhouse:
<merge_tree>
    <parts_to_delay_insert>600</parts_to_delay_insert>
    <parts_to_throw_insert>120000</parts_to_throw_insert>
    <max_delay_to_insert>100</max_delay_to_insert>
    <max_suspicious_broken_parts>5</max_suspicious_broken_parts>
</merge_tree>

Any ideas would be great since I'm using the suggested table layout.  Thanks.

What I've been finding from other issues and docs (like https://clickhouse.yandex/docs/en/introduction/performance/#performance-when-inserting-data) is that my carbon-clickhouse doesn't seem to be creating big enough batches and is in stead doing one insert per client connection.  Is it expecting for me to have another layer in front of it to pool the inserts?  that kind of defeats the purpose of using this in the first place.

Check the data on the disk (by default in /data/carbon-clickhouse/). Each file is one insert. With your settings (chunk-auto-interval), if files accumulate, they should become larger and appear less often.
Check also the logs of the clickhouse-server. Perhaps there are merge errors

@wetnun My setup works fine on a single CH server with default merge settings and chunk-interval = 5s with 1.8 million metrics per minute (~35k per second) without any noticeable load on Clickhouse. I estimate it can take 10-20x more load or so.
Maybe you have weird hardware\software or heavy compression on columns. Post your CREATE TABLE here if they're not default.

I have a three node cluster for clickhouse (I'm planning on keeping metrics for a while).  And yes, it seems like maybe the issue is now clickhouse.  Came back this morning to clickhouse-server consuming all cores and not returning selects (although I think it's inserting).  What version of clickhouse server is considered "stable"?  I've seen references to the latest install not always being bug free.
As per tables:
-- create tables on all nodes
CREATE TABLE default.graphite_local (Path String, Value Float64, Time UInt32, Date Date, Timestamp UInt32) ENGINE = GraphiteMergeTree('graphite_rollup') ORDER BY (Path, Time) SETTINGS index_granularity = 8192
CREATE TABLE default.graphite_index_local (Date Date, Level UInt32, Path String, Version UInt32) ENGINE = ReplacingMergeTree(Version) PARTITION BY toYYYYMM(Date) ORDER BY (Level, Path, Date) SETTINGS index_granularity = 8192

-- create distributed on primary only
CREATE TABLE default.graphite_index (Date Date, Level UInt32, Path String, Version UInt32) ENGINE = Distributed(metrics_3shard_1rep, default, graphite_index_local, rand())
CREATE TABLE default.graphite (Path String, Value Float64, Time UInt32, Date Date, Timestamp UInt32) ENGINE = Distributed(metrics_3shard_1rep, default, graphite_local, rand())

When clickhouse-server starts freaking out I'm getting these errors, but only when the server is being weird:
2019.06.03 12:15:42.537878 [ 99 ] {c489ba25-b21b-4148-b64a-b43336010835} <Information> HTTPHandler: Done processing query
2019.06.03 12:15:42.540265 [ 74 ] {d33acae9-7354-4ca9-baae-ee7d236d37d6} <Information> default.graphite_local (Data): Delaying inserting block by 10.2588 ms. because there are 24744 parts
2019.06.03 12:15:42.551359 [ 74 ] {d33acae9-7354-4ca9-baae-ee7d236d37d6} <Information> executeQuery: Read 14 rows, 673.00 B in 0.014 sec., 1011 rows/sec., 47.49 KiB/sec.
2019.06.03 12:15:42.551488 [ 74 ] {d33acae9-7354-4ca9-baae-ee7d236d37d6} <Information> HTTPHandler: Done processing query
2019.06.03 12:15:42.590695 [ 18 ] {} <Error> void DB::BackgroundProcessingPool::threadFunction(): Code: 10, e.displayText() = DB::Exception: Not found column Version in block. There are only columns: Path, Value, Time, Date, Timestamp, Stack trace:

0. clickhouse-server(StackTrace::StackTrace()+0x16) [0x68d1bf6]
1. clickhouse-server(DB::Exception::Exception(std::string const&, int)+0x1f) [0x3174fbf]
2. clickhouse-server(DB::Block::getPositionByName(std::string const&) const+0xde) [0x5de13de]
3. clickhouse-server(DB::GraphiteRollupSortedBlockInputStream::GraphiteRollupSortedBlockInputStream(std::vector<std::shared_ptr<DB::IBlockInputStream>, std::allocator<std::shared_ptr<DB::IBlockInputStream> > > const&, std::vector<DB::SortColumnDescription, std::allocator<DB::SortColumnDescription> > const&, unsigned long, DB::Graphite::Params const&, long)+0x5ba) [0x63b48ca]
4. clickhouse-server(DB::MergeTreeDataMergerMutator::mergePartsToTemporaryPart(DB::FutureMergedMutatedPart const&, DB::MergeListEntry&, long, DB::DiskSpaceMonitor::Reservation*, bool)+0x26f9) [0x61fdd39]
5. clickhouse-server(DB::StorageMergeTree::merge(bool, std::string const&, bool, bool, std::string*)+0x35b) [0x614ecab]
6. clickhouse-server(DB::StorageMergeTree::backgroundTask()+0x168) [0x61506a8]
7. clickhouse-server(DB::BackgroundProcessingPool::threadFunction()+0x3e8) [0x61b1538]
8. clickhouse-server() [0x61b1f34]
9. clickhouse-server(ThreadPoolImpl<std::thread>::worker(std::_List_iterator<std::thread>)+0x187) [0x68d7947]
10. clickhouse-server() [0x745371f]
11. /lib64/libpthread.so.0(+0x7dd5) [0x7fa856d03dd5]
12. /lib64/libc.so.6(clone+0x6d) [0x7fa856005ead]
 (version <IP_ADDRESS>)

The rollup and parts still seems off.  Still feels wrong to have bumped the parts_to_throw_insert so high.  As per the temp files, it seems like it rotates them very fast.  Here is a log file I generated using watch -n 1 "ls -lh /opt/carbon-clickhouse/ | tee -a /tmp/carbon-clickhouse-tmp.log" over a minute or something.
https://gist.github.com/wetnun/9185ae7e05e0b515dfb84e2ef595adfa
Interesting point, if I have a graph (like carbon.agents.${main_box}.tcp.*), it always works, but my other graphs for that three node cluster fail to load because it can' get the data back (6+ metrics per graph).  The metrics all show up in the form of vm${vmId}.${thing}.${metric} which should be fine, I always do system ID first out of habit for hashing from older systems.
Is there maybe an example somewhere of multi-node graphite setup?  Found quite a few docs, but very few working examples of larger systems (honestly, this isn't large).  But short of maybe how carbon-clickhouse fetches data from clickhouse, it's starting to look like more of a clickhouse issue than anything.

What version of clickhouse server is considered "stable"? I've seen references to the latest install not always being bug free.

I'm using latest in 19.5 branch. Current 'stable' is of course not much stable, so don't run for bleeding edge.

When clickhouse-server starts freaking out I'm getting these errors, but only when the server is being weird

Well an error says it all. For some time Clickhouse docs were wrong about GraphiteMergeTree settings - they stated it requires Version column but the CH was actually using Timestamp. Probably in 19.6 they fixed this. So recreate your tables with Version instead of Timestamp or downgrade to 19.5

Hmm, I'll try changing versions of the server then.  I have been using the Altinity repo (running Centos7), but I see there is also https://repo.yandex.ru/clickhouse/rpm/stable/x86_64/.  Which only goes up to 19.4 under stable oddly enough.  But looks like the Altinity repo has the older versions, I'll try grabbing one of those.

You can also just change
<graphite_rollup>
 <version_column_name>Version</version_column_name>
</graphite_rollup>

Then the merges will succeed and you shouldn't get any errors.

Probably in 19.6 they fixed this.

As far as i see it unchanged in source code https://github.com/yandex/ClickHouse/blob/db6de83fa1e16e7a5062ee2c3e31d273c186f550/dbms/src/Storages/MergeTree/registerStorageMergeTree.cpp#L173
Most likely the version_column_name=Version is specified somewhere in the config.

So I rolled everything back to clickhouse-server-<IP_ADDRESS> and updated the config.xml rollup to use Timestamp instead of Version, so far it's running right now.  I guess time will tell if that was the main issue or not.  Thanks for the help.

Why wait, use SELECT * FROM system.merges to see current merges and SELECT * FROM system.parts WHERE table = 'graphite' to see if your parts are merged or not.

Totally forgot to follow up, sorry.  So it's been running super smooth since the downgrade.  Thanks for the help.

So it's been running super smooth since the downgrade

It would certainly run as well with a newer version and changed config, so don't hesitate to upgrade if needed :)

@wetnun rand() as a sharding key on a distributed table that sends inserts to a graphite table is going to “break” rollup. (I’m assuming that’s what you’re doing but maybe you are handling inserts some other way, in which case ignore me)
Rollup happens on the local graphite table in each shard, it has no awareness of the distributed table. Multiple shards (if not all) will have points for any given metric within a range when performing rollup. Usually this means you end up with as many shards as you have * the amount of data you expected the rollup rule to leave you with.
One way you can solve this is to shard on modulo(intHash64(Path), number_of_shards) so that a Path always inserts to the same shard.

Hmm, that seems logical.  I've had it running for a while now but I haven't dug into the disk space usage (of which it hasn't been much).  I was actually meaning to lookup how to do consistent hashing like that on insert.    But on Path I think it would work fine.
The puppet report collector I run on the same head (which I'm thinking of trying to integrate into Clickhouse somehow) is actually more of a system stress at the moment.


UNKNOWN: NewRandomAccessFile failed to Create/Open

Introduction
I am trying to save and load a scikeras model in a pipeline with standard scaling and principal component analysis. I can save the pipeline, but when I try to load it I get some strange errors. I am running Jupyter-lab on Windows 10 Enterprise with Python 3.9.16.
The error look like this:
WARNING:absl:Found untraced functions such as _update_step_xla while saving (showing 1 of 1). These functions will not be directly callable after loading.
INFO:tensorflow:Assets written to: C:\Users\...\AppData\Local\Temp\tmp1te5qppx\assets
INFO:tensorflow:Assets written to: C:\Users\...\AppData\Local\Temp\tmp1te5qppx\assets
---------------------------------------------------------------------------
OpError                                   Traceback (most recent call last)
~\AppData\Local\Temp\ipykernel_12448\1012496037.py in <module>
     60 pickle.dump(clf, open(".\model.sav", 'wb'))
     61
---> 62 clf2 = pickle.load(open(".\model.sav", 'rb'))
     63
     64 clf2.predict(X[:5])

~\Anaconda3\lib\site-packages\scikeras\_saving_utils.py in unpack_keras_model(packed_keras_model)
     50                     f.write(archive.extractfile(fname).read())
     51         model: keras.Model = load_model(temp_dir)
---> 52         model.load_weights(temp_dir)
     53         model.optimizer.build(model.trainable_variables)
     54         return model

~\Anaconda3\lib\site-packages\keras\utils\traceback_utils.py in error_handler(*args, **kwargs)
     68             # To get the full stack trace, call:
     69             # `tf.debugging.disable_traceback_filtering()`
---> 70             raise e.with_traceback(filtered_tb) from None
     71         finally:
     72             del filtered_tb

~\Anaconda3\lib\site-packages\tensorflow\python\training\py_checkpoint_reader.py in error_translator(e)
     43     raise errors_impl.InternalError(None, None, error_message)
     44   else:
---> 45     raise errors_impl.OpError(None, None, error_message, errors_impl.UNKNOWN)
     46
     47

OpError:

Also in the Anaconda Prompt window I get something mysterious message:
2023-03-01 16:24:12.033629: W tensorflow/core/util/tensor_slice_reader.cc:96] Could not open C:\Users\rgt0227\AppData\Local\Temp\tmppzfngdbl: UNKNOWN: NewRandomAccessFile failed to Create/Open: C:\Users\rgt0227\AppData\Local\Temp\tmppzfngdbl : Access denied.
; Input/output error

Code for reproduction
import numpy as np
from scikeras.wrappers import KerasClassifier
from sklearn.pipeline import Pipeline
from sklearn.decomposition import PCA
from sklearn.preprocessing import StandardScaler

from tensorflow import keras
from sklearn.datasets import make_classification
from typing import Dict, Iterable, Any
import pickle

X, y = make_classification(1000, 20, n_informative=10, random_state=0)
X.shape, y.shape, y.mean()

steps=[('std', StandardScaler()), ('pca', PCA()),  ('keras', MLPClassifier())]
clf = Pipeline(steps=steps)

print(clf.fit(X, y).score(X, y))

pickle.dump(clf, open(".\model.sav", 'wb'))

clf2 = pickle.load(open(".\model.sav", 'rb'))

clf2.predict(X[:5])

where MLPClassifier is taken from https://www.adriangb.com/scikeras/stable/notebooks/MLPClassifier_MLPRegressor.html
class MLPClassifier(KerasClassifier):

    def __init__(
        self,
        hidden_layer_sizes=(100, ),
        optimizer="adam",
        optimizer__learning_rate=0.001,
        epochs=200,
        verbose=0,
        **kwargs,
    ):
        super().__init__(**kwargs)
        self.hidden_layer_sizes = hidden_layer_sizes
        self.optimizer = optimizer
        self.epochs = epochs
        self.verbose = verbose

    def _keras_build_fn(self, compile_kwargs: Dict[str, Any]):
        model = keras.Sequential()
        inp = keras.layers.Input(shape=(self.n_features_in_))
        model.add(inp)
        for hidden_layer_size in self.hidden_layer_sizes:
            layer = keras.layers.Dense(hidden_layer_size, activation="relu")
            model.add(layer)
        if self.target_type_ == "binary":
            n_output_units = 1
            output_activation = "sigmoid"
            loss = "binary_crossentropy"
        elif self.target_type_ == "multiclass":
            n_output_units = self.n_classes_
            output_activation = "softmax"
            loss = "sparse_categorical_crossentropy"
        else:
            raise NotImplementedError(f"Unsupported task type: {self.target_type_}")
        out = keras.layers.Dense(n_output_units, activation=output_activation)
        model.add(out)
        model.compile(loss=loss, optimizer=compile_kwargs["optimizer"])
        return model

I don't think this is a SciKeras error. We test something very similar in CI. Is it possible something strange is going on with the permissions on your system?

Thanks @adriangb for your reply.  Yes, this is probably something related to permission on my system since I was recently unable to replicate it on another Windows machine. I have noticed something similar happens when I try use copy.deepcopy() on the SciKeras model.

I also get the same error. How did you fix it?

 E tensorflow/tsl/platform/windows/windows_file_system.cc:363] ERROR: GetSymbolicLinkTarget cannot open file for \\?\C:\Users\itsme\AppData\Local\Temp\tmpd4kdyj5q GetLastError: 5

 W tensorflow/core/util/tensor_slice_reader.cc:98] Could not open C:\Users\itsme\AppData\Local\Temp\tmpd4kdyj5q: UNKNOWN: NewRandomAccessFile failed to Create/Open: C:\Users\itsme\AppData\Local\Temp\tmpd4kdyj5q : Access is denied.

I have not managed to solve it. It has something to do with the permission settings on the particular machine, and our IT department have been looking into it. Without progress so far, I am afriad.
Från: bizso09 @.>
Skickat: den 21 oktober 2023 03:58
Till: adriangb/scikeras @.>
Kopia: Granat Robert @.>; State change @.>
Ämne: Re: [adriangb/scikeras] UNKNOWN: NewRandomAccessFile failed to Create/Open (Issue #291)
I also get the same error. How did you fix it?
E tensorflow/tsl/platform/windows/windows_file_system.cc:363] ERROR: GetSymbolicLinkTarget cannot open file for \?\C:\Users\itsme\AppData\Local\Temp\tmpd4kdyj5q<file://%3f/C:/Users/itsme/AppData/Local/Temp/tmpd4kdyj5q> GetLastError: 5
W tensorflow/core/util/tensor_slice_reader.cc:98] Could not open C:\Users\itsme\AppData\Local\Temp\tmpd4kdyj5q: UNKNOWN: NewRandomAccessFile failed to Create/Open: C:\Users\itsme\AppData\Local\Temp\tmpd4kdyj5q : Access is denied.
—
Reply to this email directly, view it on GitHubhttps://github.com/adriangb/scikeras/issues/291#issuecomment-1773600852, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AWFJZWV2C2VP7LU4IEWY5BTYAMT4JAVCNFSM6AAAAAAVMG2VLGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONZTGYYDAOBVGI.
You are receiving this because you modified the open/close state.Message ID<EMAIL_ADDRESS>


Support for additional domains

update
various inputs (select, etc)
light
fan

This currently works for most of the useful utility types, but all the oddball domains are being built out in a "whenever I feel like it" timeline.
If someone else is interested in leading the charge on this, moving it along faster, the effort relatively straightforward. Happy to pass off notes on how to do that


Language scopes go missing

Prerequisites

[x] Put an X between the brackets on this line if you have done all of the following:

Reproduced the problem in Safe Mode: https://flight-manual.atom.io/hacking-atom/sections/debugging/#using-safe-mode
Followed all applicable steps in the debugging guide: https://flight-manual.atom.io/hacking-atom/sections/debugging/
Checked the FAQs on the message board for common solutions: https://discuss.atom.io/c/faq
Checked that your issue isn't already filed: https://github.com/issues?utf8=✓&q=is%3Aissue+user%3Aatom
Checked that there is not already an Atom package that provides the described functionality: https://atom.io/packages

Description
I keep reaching this weird state where Atom will suddenly not recognise language scopes in the format of source.js.jsx. I noticed this because my jsx-scoped packages stopped working. I spent a lot of time uninstalling and reinstalling packages, resetting settings, using atom --clear-window-state etc. Sometimes I would end up in a state where suddenly the source.js.jsx scope is back! And then it shortly disappears again without me doing anything drastic (installing packages? restarting Atom??)
I use the shortcut cmd-alt-p (editor:log-cursor-scope) to check language scope. Broken state: In a .jsx file, language scope list does not include source.js.jsx. In this state my jsx packages don't work properly. Correct state: language scope list has source.js.jsx at the top of the list, jsx packages work.
Currently my editor is in a broken state, here's the editor:log-cursor-scope language scope list in a .jsx file:
javascript
program
lexical_declaration
variable_declarator
arrow_function
parenthesized_expression
jsx_element
jsx_element
jsx_element
jsx_opening_element
jsx_attribute
string

I noticed this list will include source.js.jsx sometimes, and my jsx packages will suddenly work again!
Sample with atom --safe mode:
javascript
program
lexical_declaration
variable_declarator
arrow_function
parenthesized_expression
jsx_element
jsx_element
jsx_expression
call_expression
arguments
arrow_function
statement_block
return_statement
parenthesized_expression
jsx_element
jsx_self_closing_element
jsx_attribute
string

In Atom's dev tools console:
> atom.workspace.getActiveTextEditor().getGrammar().scopeName
"javascript"

Here's my packages:
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
I feel like language-javascript-jsx or language-babel should provide the jsx scopes? Perhaps?
Steps to Reproduce
I can not figure out how to reproduce this. It appears at random, and sometimes disappears when I desperately toggle various settings or reload window after uninstall/reinstall/disable/enable packages almost at random. I know this makes it really hard to debug, but I thought I'd make a ticket in the hopes that someone else has run into this problem, finds the ticket, and might be able to provide better insight on how to reproduce.
Expected behavior: I find the Atom documentation a bit tricky to understand, but I THINK (from looking at the screen shots on this page) that the css selectors should look like . Mine look way different, here's a sample:

    <
        div
            className
...

Is this the legacy syntax?
Versions
Atom    : 1.27.2
Electron: 1.7.15
Chrome  : 58.0.3029.110
Node    : 7.9.0

omg I found the answer: https://github.com/dwyl/learn-react/issues/12#issuecomment-173223639
In short, the default language-javascript package should be disabled, as it will conflict with language-javascript-jsx and win the race condition to set .jsx files with syntax javascript. Argghhh.


Feat/add support to export image

What does this PR do?
Fixes Issue#168.
Why?
To improve user experience.
How do you solve this problem?
Adding a feature to download the collector image visualization.

I have read the CLA Document and I hereby sign the CLA

Merged manually to fix the formatting. See https://github.com/dash0hq/otelbin/commit/074bc093a1c1476750f5234ed448444cd3c32886 and others.
This is an awesome contribution @samueljrz, thank you – and well done!


Error:invalid_Client The  OAuth client was not found

Hi  everyone, I used an open source project about Email_Client which is based on  google API(Gmail API,GTL Core),when I want to run the project(Actually it was written for  IOS )  google page comes to screen and  this Error comes out "Error:invalid_Client The  OAuth client was not found ".
Can anyone help me to  solve this problem ??
Thanks in advance

Sorry, I'm not sure what the Email_Client project is.  You might be better off asking on that projects issues/discussion.  My guess is the app needs a client id, so some request is getting set without having given it a client id.  We have a little info about getting ids for our examples on the wiki.

Thanks for your reply, the picture below is  error i got

Sorry, without more info about what the app is doing, I really don't know what is happening.

That problem solved,in order to use Google API I  must sign  in this website [console.developers.google.com]  and get  a valid Client ID.

I set both correct GOOGLE_CLIENT_ID and GOOGLE_CLIENT_SECRET but still, it does not work.

@lightlinksolution

try reset keys
check if the values passed in setting.py file is in '' like 'somevalue'

i faced the same problem where i put the value with ' '(string format) in heroku. when i removed the ''.
run the command again heroku config:set GOOGLE_KEY=akdjshfkaalskdf
it worked

That error is displayed in case that under “APIs & auth” -> “Consent screen” you have empty field “PRODUCT NAME” – you need to select e-mail address as well.
You can find “Consent screen” under a link!, click on project name and then go to “APIs & auth” -> “Consent screen”
Find Full Source Code At Here

That error is displayed in case that under “APIs & auth” -> “Consent screen” you have empty field “PRODUCT NAME” – you need to select e-mail address as well.
You can find “Consent screen” under a link!, click on project name and then go to “APIs & auth” -> “Consent screen”
Full Source Available At Here

s,dcx sx

Where do I find GOOGLE_CLIENT_ID?

@borisjavier https://console.developers.google.com/apis/credentials/oauthclient/

This might be the problem with the url you have provided when creating the client id and secret key,
Because google block if you try to access is from unlisted url.
and for sure check your credential once again.....
For Example http://websitename.com

faced the same error, but my case was funny, copied the URL from terminal but it had newline characters coz the URL was too long

faced the same error, but my case was funny, copied the URL from terminal but it had newline characters coz the URL was too long

lol same


Fix sorting of datapoints, add test util functions

Fixes #266
Sorting data points in descending order allows getDatapoint() to pull the most recent one when it iterates over the slice.
Adds the test helper functions from https://github.com/benbjohnson/testing

Thanks for your PR :)


Saying I dont have perms.

My qbot is saying this

Im the owner of the group.

Oh, I know now what is the problem!


Multiline cells

@Evertras I did the same in the bubbletea.Table and it renders ok https://github.com/charmbracelet/bubbles/pull/433

It looks ok hence it fails on movement (probably because I do not know how to manipulate the yoffset)

Okay, I cannot do it that way since RowData is unaware of the column width.
What I am trying to do is to fill other cells with empty lines to align heigh of multiline

Some progress:

All we have to do is set the borderStyle.Height in renderRowData to the size of the highest cell

OMG I think I got this:

3 tests fails but I hope it will be straightforward to fix that

To test it on your own use option WithMultiline(true) while creating a new table

@Evertras if you will have some free time to help, I would be very grateful. The current state fails only 3 tests, but I think that not every case - something is noyes with styles for specific cells. I assume it is connected to the fact that I did not cover all logic in my loop

Do not bother with the look of the code - it is just POC that will need a refactor

For some reason, this is needed 🤔

And has to be particular Top - Botttom, Left, Center does not work

In a very dirty way, I checked that is not (probably) connected to overflow
func (m Model) renderRowData(row Row, rowStyle lipgloss.Style, last bool) string {
	numColumns := len(m.columns)

	columnStrings := []string{}
	totalRenderedWidth := 0
	tmpTotalRenderedWidth := 0
	rawColumnsStrings := make([]string, 3*len(m.columns))

	stylesInner, stylesLast := m.styleRows()

	maxCellHeight := 0
	columnIndex := 0
	for rawColumnIndex := 1; rawColumnIndex/3 < len(m.columns); rawColumnIndex += 3 {
		if rawColumnIndex != 1 {
			columnIndex = (rawColumnIndex - 1) / 3
			// 0 1
			// 1 4
			// 2 7
		}
		column := m.columns[columnIndex]
		emptyStyle := lipgloss.NewStyle()

		if m.horizontalScrollOffsetCol > 0 && columnIndex == m.horizontalScrollFreezeColumnsCount {
			rendered := m.renderRowColumnData(row, genOverflowColumnLeft(1), rowStyle, emptyStyle)

			rawColumnsStrings[rawColumnIndex-1] = rendered
		}

		if columnIndex >= m.horizontalScrollFreezeColumnsCount &&
			columnIndex < m.horizontalScrollOffsetCol+m.horizontalScrollFreezeColumnsCount {
			continue
		}

		//--
		cellStr := m.renderRowColumnData(row, column, rowStyle, emptyStyle)
		maxCellHeight = max(maxCellHeight, lipgloss.Height(cellStr))
		rawColumnsStrings[rawColumnIndex] = cellStr
		//--
		if m.maxTotalWidth != 0 {
			renderedWidth := lipgloss.Width(cellStr)

			const (
				borderAdjustment = 1
				overflowColWidth = 2
			)

			targetWidth := m.maxTotalWidth - overflowColWidth

			if columnIndex == len(m.columns)-1 {
				// If this is the last header, we don't need to account for the
				// overflow arrow column
				targetWidth = m.maxTotalWidth
			}

			if tmpTotalRenderedWidth+renderedWidth > targetWidth {
				overflowWidth := m.maxTotalWidth - tmpTotalRenderedWidth - borderAdjustment
				overflowColumn := genOverflowColumnRight(overflowWidth)
				overflowStr := m.renderRowColumnData(row, overflowColumn, rowStyle, emptyStyle)

				rawColumnsStrings[rawColumnIndex+1] = overflowStr

				break
			}

			tmpTotalRenderedWidth += renderedWidth
		}
	}
	// for i, column := range rawColumnsStrings {
	// 	fmt.Println(i, column)
	// }

	// fmt.Println("---")

	for columnIndex, column := range m.columns {
		var borderStyle lipgloss.Style
		var rowStyles borderStyleRow

		if !last {
			rowStyles = stylesInner
		} else {
			rowStyles = stylesLast
		}

		if m.horizontalScrollOffsetCol > 0 && columnIndex == m.horizontalScrollFreezeColumnsCount {
			var borderStyle lipgloss.Style

			if columnIndex == 0 {
				borderStyle = rowStyles.left.Copy()
			} else {
				borderStyle = rowStyles.inner.Copy()
			}

			cellStyle := rowStyle.Copy().Inherit(genOverflowColumnLeft(1).style).
				Inherit(m.baseStyle).Height(maxCellHeight)
			cellStyle = cellStyle.Inherit(borderStyle)
			idx := columnIndex*3 + 1 - 1
			if idx <= 0 {
				idx = 1
			}
			rendered := cellStyle.Render(rawColumnsStrings[idx])
			// rendered := cellStyle.Render(rawColumnsStrings[columnIndex*3-1])

			totalRenderedWidth += lipgloss.Width(rendered)

			columnStrings = append(columnStrings, rendered)
		}

		if columnIndex >= m.horizontalScrollFreezeColumnsCount &&
			columnIndex < m.horizontalScrollOffsetCol+m.horizontalScrollFreezeColumnsCount {
			continue
		}

		if len(columnStrings) == 0 {
			borderStyle = rowStyles.left
		} else if columnIndex < numColumns-1 {
			borderStyle = rowStyles.inner
		} else {
			borderStyle = rowStyles.right
		}

		cellStyle := rowStyle.Copy().Inherit(column.style).
			Inherit(m.baseStyle).Height(maxCellHeight)
		cellStyle = cellStyle.Inherit(borderStyle)
		idx := columnIndex*3 + 1
		// if idx <= 0 {
		// 	idx = 1
		// }
		// fmt.Println(idx, rawColumnsStrings[idx])
		cellStr := cellStyle.Render(rawColumnsStrings[idx])
		// cellStr := cellStyle.Render(rawColumnsStrings[columnIndex*3])

		if m.maxTotalWidth != 0 {
			renderedWidth := lipgloss.Width(cellStr)

			const (
				borderAdjustment = 1
				overflowColWidth = 2
			)

			targetWidth := m.maxTotalWidth - overflowColWidth

			if columnIndex == len(m.columns)-1 {
				// If this is the last header, we don't need to account for the
				// overflow arrow column
				targetWidth = m.maxTotalWidth
			}

			if totalRenderedWidth+renderedWidth > targetWidth {
				overflowWidth := m.maxTotalWidth - totalRenderedWidth - borderAdjustment
				cellStyle := rowStyle.Copy().Inherit(genOverflowStyle(rowStyles.right, overflowWidth)).
					Inherit(m.baseStyle).Height(maxCellHeight)
				cellStyle = cellStyle.Inherit(borderStyle)
				rendered := cellStyle.Render(rawColumnsStrings[columnIndex*3+1+1])

				totalRenderedWidth += lipgloss.Width(rendered)

				columnStrings = append(columnStrings, rendered)
				break
			}

			totalRenderedWidth += renderedWidth
		}

		columnStrings = append(columnStrings, cellStr)
	}

	return lipgloss.JoinHorizontal(lipgloss.Bottom, columnStrings...)
}

I managed to achieve multiline feature with clean tests

The only drawback is that it renders cell content twice if the multiline option is enabled. But I do not have any other idea to handle it without refactoring the current setup.

I'm okay with offering an additional feature at a small expense to performance if that feature is entirely opt-in, which it seems to be here.  We should add a small note about potential performance issues in the WithMultiline func comment for docs.
Also seems I need to update the linter settings, haven't looked at that in a while...

Looking at this a bit more, I feel like the perf shouldn't really be an issue.  So maybe we don't need the comment. 🤔

We'll also definitely want to add some tests around what multiline tables should look like with expected inputs, which should also bump our coverage up to where it needs to be.  Can you add some tests around expected outputs to https://github.com/Evertras/bubble-table/blob/main/table/view_test.go ?
Sorry for missing messages before, was away for a bit.

And as a final note, feel free to make the features example permanently show multiline as well as in your screenshot.

@Evertras I have been waiting to add tests and examples for your approval. As you agreed on that solution, I will add them soon :)

@Evertras done
No worries about the missed messages. I treated this PR as a dev log to maintain a history and share hints about the implementation for future contributors


Format and transition numbers

Formats the hit count. Adds some transition effects. Borrowed from MiMA

Btw, another question: could you use a media query to disable the animation with prefers-reduced motion?
This isn't strictly necessary, since this animation doesn't count as motion. However, some people (including myself) use the setting to just reduce the amount of animation on webpages.


Fix Unicode double-encode bug

Using response.content (where response is a reponse from a requests request) causes the data to be double-encoded in UTF-8 in Python 3. Changing this to response.text fixes this bug.

Continuous integration tests are failing; do you want to take a look at this and fix the pull request (or the tests, whatever is necessary)?

3.2 seems to be failing because of something going wrong in the coverage module. I'll take a look at the other tests and find the issues. The failed tests seem to be caused by expecting a bytes object and not a Unicode string. Using bytes objects was the source of the double-encoding bug, so likely the tests just need to be reconfigured to expect Unicode strings.

@Mego @ProgramFOX The tests seem positive now except 3.2 which fails due to an incompatibility of the pytest module (and therefore is ignored by now). What do you think?

@ByteCommander Given that the other tests pass and the Py3.2 test just fails because of something with pytest and not with ChatExchange, I'd say that you can merge.


NGO registration test (Rebase Completed One)

Issue Reference
This PR addresses the Issue : Fixes #593
Summarize
Just a rebase completed one.

This is a rebase completed one of #593
Please check and merge if required.
@syam3526 @vigneshhari @biswaz @vrthra @bobinson


nginx: do not run anything as root

Motivation for this change
Reopened and updated this PR https://github.com/NixOS/nixpkgs/pull/51551
Change location pidFile from /var/spool/nginx/log/nginx.pid to /run/nginx/nginx.pid
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Assured whether relevant documentation is up to date
[ ] Fits CONTRIBUTING.md.

сс @Infinisil cc @Mic92

Could someone else take care of this pr?

I'm only going to speak from an apache point of view because I really don't know much about nginx.
I want this to happen. It becomes even more apparent why this is a great idea after the latest apache CVE that allowed php scripts to escalate as root BUT... on the apache side I have concerns. I'm not aware of any other distro, companies, or large deploys of apache running apache as non root. There are unknowns to this that makes me feel this needs much more thorough testing. It would be nice if a test plan was put in place and people who could adequately test it volunteered to run the module in test then production fora while. Simply merging a change of this magnitude (again, speaking apache, not sure if nginx is the same) without real world testing is scary to me.

I now have nginx + unit (php) + roundcube working fine.

Updated capabilities - added CAP_SYS_RESOURCE
Need to fix this error:
nginx[13580]: 2019/05/30 21:24:22 [alert] 13582#13582: setrlimit(RLIMIT_NOFILE, 102400) failed (1: Operation not permitted)

Rebased PR.

Moved release notes to rl-2003.xml

Removed option CapabilityBoundingSet and capabilities - "CAP_SYS_RESOURCE"
Please recheck PR

Returned AmbientCapabilities = [ "CAP_NET_BIND_SERVICE" "CAP_SYS_RESOURCE" ];

@Izorkin: Does nginx still have permission to access letsencrypt certificates if we use services.nginx.virtualHosts.<name>.enableACME?
I didn't check yet, just asking.
Also, it seems the ACME test currently is broken on master so I can't easily check it in the test
cc @NinjaTrappeur @arianvp

@flokli yes, correct work:
ls -lah /var/lib/acme
итого 16K
drwxr-xr-x  4 root  root  4,0K дек 17 20:07 .
drwxr-xr-x 26 root  root  4,0K дек 17 20:10 ..
drwxr-xr-x  3 root  root  4,0K дек 17 20:07 acme-challenge
drwxr-xr-x  2 nginx nginx 4,0K дек 17 20:07 test1.example-stie

ls -lah /var/lib/acme/test1.example-stie
итого 32K
drwxr-xr-x 2 nginx nginx 4,0K дек 17 20:07 .
drwxr-xr-x 4 root  root  4,0K дек 17 20:07 ..
-rw-r--r-- 1 nginx nginx 3,1K дек 17 20:07 account_key.json
-rw-r--r-- 1 nginx nginx  912 дек 17 20:07 account_reg.json
-rwx------ 1 nginx nginx 3,9K дек 17 20:07 fullchain.pem
-rwx------ 1 nginx nginx 7,1K дек 17 20:07 full.pem
-rwx------ 1 nginx nginx 3,2K дек 17 20:07 key.pem

Acme tests still pass after merging this in, so should be good to go. Thanks!

@tbenst mentioned this broke nginx, as it's not able to open the log file:
Dec 22 03:54:01 hydra-server jdf8k5jc38ny3zfshcnxxiw52l9h8x4b-unit-script-nginx-pre-start[18157]: 2019/12/22 03:54:01 [emerg] 18158#18158: open() "/var/spool/nginx/logs/access.log" failed (13:>

When checking on a NixOS 19.09 machine however, I couldn't reproduce log files being owned by root. @NinjaTrappeur , @asymmetric, any thoughts?

Before update -
sudo ls -lah /var/spool/nginx/logs
итого 20K
drwxr-x--- 2 nginx nginx 4,0K дек 22 15:53 .
drwxr-x--- 8 nginx nginx 4,0K дек 22 15:53 ..
-rw-r--r-- 1 root  root  1,4K дек 22 15:53 access.log
-rw-r--r-- 1 root  root   134 дек 22 15:53 error.log
-rw-r--r-- 1 root  root     5 дек 22 15:53 nginx.pid

After this PR -
итого 16K
drwxr-x--- 2 nginx nginx 4,0K дек 22 18:08 .
drwxr-x--- 8 nginx nginx 4,0K дек 22 18:08 ..
-rw-r--r-- 1 nginx nginx 1,4K дек 22 18:08 access.log
-rw-r--r-- 1 nginx nginx  132 дек 22 18:08 error.log

Need remove log files or change owner to nginx

This should do the trick:
systemd.tmpfiles.rules = [
  "Z '${cfg.stateDir}/logs' 0750 ${cfg.user} ${cfg.group} - -"
];

If so, then it should probably be a z ${cfg.stateDir}/logs/* 6?? ...` - we don't want to set the executable bit on log files.
However, I also saw some ${cfg.stateDir}/*_temp directories and am not certain if we want to chown/chmod them too, and with which permissions.

This commit breaks services.nginx.virtualHosts.<vhost>.useACMEHost, even with security.acme.certs.<vhost>.allowKeysForGroup and group nginx. I haven't figured out why, yet.

@dhess, can you clarify exactly how you fixed this? I am suffering from the same exact problem, have a bunch of certs with restrictive permissions on them, and an nginx that can't start anymore.

I have the suspicion that this file broke running nginx as root:
When you manually configure services.nginx.user = "root";, file uploads fail like this:
Apr 02 18:24:33 myhostname nginx[11878]: 2020/04/02 18:24:33 [crit] 11885#11885: *71 open() "/var/spool/nginx/client_body_temp/0000000012" failed (13: Permission denied)

Here, the nginx master process 11878 runs as root (as configured) and the worker process 11885 runs as nobody.
The directory permissions are:
# ls -lah /var/spool/nginx/
total 32K
drwxr-x--- 8 root   root 4.0K Apr  2 18:59 .
drwxr-xr-x 3 root   root 4.0K Apr  2 18:04 ..
drwx------ 2 nobody root 4.0K Apr  2 18:22 client_body_temp
drwx------ 2 nobody root 4.0K Apr  2 18:22 fastcgi_temp
drwxr-x--- 2 root   root 4.0K Apr  2 18:22 logs
drwx------ 2 nobody root 4.0K Apr  2 18:22 proxy_temp
drwx------ 2 nobody root 4.0K Apr  2 18:22 scgi_temp
drwx------ 2 nobody root 4.0K Apr  2 18:22 uwsgi_temp

Because /var/spool/nginx/ is rwxr-x--- (lacks x for non-root user/group), the nobody user cannot traverse and thus write into the subdirectory it owns (client_body_temp).
In other words, it seems like the lines
https://github.com/NixOS/nixpkgs/commit/2a413da57efc4c2009c984c63def8e9060771269#diff-795fa65a7c41a479084de826e4e2e65cR672
    systemd.tmpfiles.rules = [
      "d '${cfg.stateDir}' 0750 ${cfg.user} ${cfg.group} - -"

do not have the desired effect: Yes, they change the parent directory, but if nginx then drops privileges as it did before, child dirs cannot be written to with the dropped perms.

@nh2 which also implied, nginx won't be able to write to logs directory, as it is not nobody writeable...
Looks like root requires exceptional handling here.

I'm having an issue very similar to this, but with services.ngnix.user set to the default. The issue is that the nginx user cannot write to client_body_temp or proxy_temp (or any of the other directories with group of nobody. This causes issues for Airsonic (when I download files) and Marix (when I send messages) because Nginx aggressively buffers large downloads and uploads using files in client_body_temp and proxy_temp.
My workaround for now has been to just SSH to my server and chown -R nginx:nginx /var/spool/nginx, but this is not sustainable. Should the Nginx module also have rules for:
      "d '${cfg.stateDir}/client_body_temp' 0750 ${cfg.user} ${cfg.group} - -"
      "d '${cfg.stateDir}/fastcgi_temp' 0750 ${cfg.user} ${cfg.group} - -"
      "d '${cfg.stateDir}/proxy_temp' 0750 ${cfg.user} ${cfg.group} - -"
      "d '${cfg.stateDir}/scgi_temp' 0750 ${cfg.user} ${cfg.group} - -"
      "d '${cfg.stateDir}/uwsgi_temp' 0750 ${cfg.user} ${cfg.group} - -"

in addition to the already existing:
      "d '${cfg.stateDir}/logs' 0750 ${cfg.user} ${cfg.group} - -"

?
I'm very new to this whole NixOS thing, so maybe there's a better way of achieving what I'm trying to accomplish.

@nh2 @sumnerevans  since 50d6e93dc8ce18c22f01989a92390ff21000191f,  there should be a Z line in tmpfiles that should chown /var/spool/nginx recursively to cfg.user / cfg.group - shouldn't nginx be able to create and update files in there with that?

@flokli I suspect that should help with @sumnerevans issue, but it seems that it wouldn't help with mine, since as described above, nobody is relevant for that.

I misspoke in my last comment. The user is nobody, the group was nginx (totally forgot which way ls output worked :man_facepalming:

Sorry, I'm now confused. Specifying root as a user likely requires special handling. Could both of you open issues with example configurations?

Specifying root as a user likely requires special handling.

That is what was there before:

Before, the systemd service ran as root, so the nginx master process ran as root, and then the worker processes ran as an unprivileged user (user nginx nginx here, or nobody by default if thta line is not given).
Now the systemd service runs as nginx, including the master process.

Could both of you open issues with example configurations?

Yes.


PEP 484 Forward References is causing an Error

We have released updated to SQLalchemy 2 and we are rewriting our schema to use the Mapped feature instead of Column. We are doing this so we can get better typing checks. https://docs.sqlalchemy.org/en/20/changelog/whatsnew_20.html#orm-declarative-models
For example, we have two tables:
class Material(Base):
    __tablename__ = "material"

    id: Mapped[uuid.UUID] = mapped_column(
        UUID(as_uuid=True), primary_key=True, server_default=text("gen_random_uuid()")
    )
    additional columns........
    queries: Mapped["Query"] = relationship("Query", cascade="all, delete")

class Query(Base):
    __tablename__ = "query"

    id: Mapped[uuid.UUID] = mapped_column(
        UUID(as_uuid=True), primary_key=True, server_default=text("gen_random_uuid()")
    )
    material_id: Mapped[uuid.UUID] = mapped_column(
        UUID(as_uuid=True), ForeignKey("material.id", ondelete="CASCADE")
    )
    additional columns........
    material: Mapped["Material"] = relationship("Material", back_populates="queries")

We have updated the relationships of queries: Mapped["Query"] and material: Mapped["Material"].
We have the strawberry types here:
@strawberry.type
class Query:
    id: uuid.UUID
    parts: list[QueryPart]

@strawberry.type
class Material:
    id: uuid.UUID
    queries: list[Query]

Describe the Bug
When I have double quotes,  queries: Mapped["Query"], we get this error:
 2023-02-13 19:42:13,065 ERROR: [strawberry.execution] Expected Iterable, but did not find one for field 'Material.queries'.
api       |
api       | GraphQL request:4:5
api       | 4 |     queries {
api       |   |     ^
api       | 5 |       parts {

When I remove the double quotes, queries: Mapped[Query], and move the class Query to the top of the file it works as expected.
How do I use the PEP 484 Forward References with Strawberry? https://peps.python.org/pep-0484/#forward-references
Or is there something else I am missing?
System Information

Operating system: Docker python:3.11.2-bullseye (On MacOS M1 PRO)
Strawberry version (if applicable): 0.156.4

Additional Context
Have made a post of Discord as well:
https://discord.com/channels/689806334337482765/1074780506413088939/1074780506413088939

I found the problem. It was me.
it needs to be     material: Mapped[list["Material"]]


Release 0.19.0-m0

Scheduled release date: June 10, 2024
This is a tracker to aggregate issues that should be part of 0.19.0-m0.
Please add specific Issues or PRs prefixed with the "Depends on" keyword
Make sure to follow the manual steps listed below before starting the release process.
Manual Steps
For detailed information regarding each step, please check: https://submariner.io/development/building-testing/ci-maintenance/

[ ] Custom GitHub Actions
[ ] GitHub Actions
[ ] Kubernetes Versions
[ ] Shipyard Linting Image Software

Released: https://github.com/submariner-io/releases/releases/tag/v0.19.0-m0


[ECS] Allow to skip fixing

I want to include checks for some Fixers, but not run the fixer, even when i pass --fix argument.
I propose something like this:
    $parameters->set(Option::SKIP_FIXER, [
… equal to Option::SKIP…
    ]);

Are you willing to contribute this feature?

Closing for on response.


Calculator: neumorphism css

Is your feature request related to a problem? Please describe.
Change background of the website/calculator, remove outline of the button. Currently it show square box on clicking the box.
Describe the solution you'd like
Change the background , I prefer neumorphism css for this project.

@Vishal-raj-1 @urvashi-code1255 Can I work on this issue? I would like to solve this as a part of GSSOC'21. Please assign it to me.

Sure go for it @Mukta-Sawant

@urvashi-code1255 @Vishal-raj-1 please check this out and do let me know the changes.

@urvashi-code1255 @Vishal-raj-1 please check this out and do let me know the changes.

Or should I make buttons neumorphic too? like given in the picture?

Yes @Mukta-Sawant make the buttons neumorphic too!

@Vishal-raj-1 @urvashi-code1255 Please, check this!

Its looking great @Mukta-Sawant . You can open a PR


caching api proposal

This is basically what I was thinking for all the necessary spock calls in the gov plugin, eager to get any feedback.  Still need to actually create the spock transformers and graphql endpoints.

Looks good to me. I believe there is still some discussion between timestamps and block numbers for poll start and end times, I think @jparklev will have clarified by end of day

Sounds good, regardless of whether its timestamps or block numbers in the polling events, spock would be able to get the timestamps from block.timestamp.

👍


Updated the calculator with some extra functions [GSSOC '23]

Description
This code includes the previous functions, f1, f2, f3, f4, f5, f6, f7, f8, negate, and backspace, as well as the additional functions binary_to_decimal and decimal_to_binary. The buttons bbinary and bdecimal allow you to convert between binary and decimal representations.
Fixes #https://github.com/avinashkranjan/Amazing-Python-Scripts/issues/1761

Type of change
Please delete options that are not relevant.

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update
[ ] Documentation Update

Checklist:

[x] My code follows the style guidelines(Clean Code) of this project
[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[x] I have created a helpful and easy to understand README.md
[x] My documentation follows Template for README.md
[x] I have added the project meta data in the PR template.
[x] I have created the requirements.txt file if needed.

Category:

[x] Calculators
[] AI/ML
[] Scrappers
[ ] Social_Media
[ ] PDF
[ ] Image_Processing
[ ] Video_Processing
[ ] Games
[ ] Networking
[ ] OS_Utilities
[ ] Automation
[ ] Cryptography
[ ] Computer_Vision
[ ] Fun
[ ] Others

Title:binary Calculator
Contributor: @slayerrr12

@avinashkranjan
pls review this pr

@avinashkranjan
pls review it

@avinashkranjan  it has been so long..... pls review this PR so that i may start contributing to other projects and work on other issues

@Yashbhadiyadra  pls review this PR


Bug: Framegear:  Getting Unexpected end of JSON input error with a button target and when there is no frame postUrl

Describe the bug and the steps to reproduce it
when a button has a target property AND when the frame does not have a postUrl property, Framegear will throw a  Uncaught (in promise) SyntaxError: Unexpected end of JSON input error in the browser console. In the server console you get TypeError: Cannot read properties of undefined (reading 'toString') and  POST /api/postFrame 500
to reproduce:

clone a frame in 100 lines
in app/page.tsx delete the  postUrl: `${NEXT_PUBLIC_URL}/api/frame`, and add a target to the first button like this: {    label: 'Story time',    target: `${NEXT_PUBLIC_URL}/api/frame`,  },
run framegear and the frame server, paste url in framegear, press the button
you should see the same errors

I believe  it has something to do framegear/app/api/postFrame/route.ts. If I hardcode the url in this line: const postUrl = frameData.url; so like this: const postUrl = "http://localhost:3000/api/frame"  there is no error. So frameData.url is undefined. In a previous version of framegear that I cloned months ago (I'm not sure which version) I did not have this issue. I updated everything the other day and I started getting this error.
Note: if you delete  the const { isValid, message } = await getFrameMessage() line and everything that uses isValid and message the frame will work in the frames.js debugger with the button target. So it seems like a framegear issue.
What's the expected behavior?
Framegear should not throw an error and a button click with a target should work.
What version of the libraries are you using?
onchainkit -  0.14.1
most recent framegear

I encountered the same issue with framegear

Frames have been deprecated.


Add missing 'apiKeyId' to APIGatewayRequestIdentity

Description of changes: Adding missing apiKeyId to APIGatewayRequestIdentity
Similar issue / changes - https://github.com/awslabs/aws-serverless-java-container/issues/136
Some documentation that makes reference to apiKeyId - api-gateway-mapping-template-reference
Also happy to add anything that's required to get this through
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Codecov Report

Merging #235 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #235   +/-   ##
=======================================
  Coverage   74.44%   74.44%
=======================================
  Files          20       20
  Lines         677      677
=======================================
  Hits          504      504
  Misses        128      128
  Partials       45       45

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 02a2e04...a52b111. Read the comment docs.

Thanks for the fix!


Rocky OS (CIQ) Zarf collaboration - Need Rocky OS Tech Blog

Is your feature request related to a problem? Please describe.

[ ] Deploy zarf on Rocky OS
[ ] Write technical blog for website

Describe the solution you'd like
Produce a Tech blog with the following purpose:
the value of open-source and here is an alternative open-source product that you can deploy and can be the answer to your problem - reference the Rocky OS tool
Describe alternatives you've considered
Questions

Is Rocky OS supported on Zarf currently?
Can Zarf be deployed on Rock OS?

Additional context
View Notes for more context
https://docs.google.com/document/d/1VtMA6VcR9pkvcbAz9ihIL557MadQwpbDq4FQRlBLgTY/edit?usp=sharing

@jeff-mccoy Am I correct that Rocky is being used in our pipeline this is a solvable problem now?

@Madeline-UX

It was when we were on self-hosted runners, but now that we're back to shared runners we are no longer using rocky.

FWIW DI2-ME's Build Harness uses Rocky

Thanks @RothAndrew, I'll check that out

@JasonvanBrackel There is some context here. This probably needs to be an internal ticket and a further conversation with @unicornbunny182 @jeff-mccoy


question on setup self-hosted models (vLLM)

Hi, thanks for your great work!
I'm new to DLN, and I'm trying to use LLaMa2 models to replace the GPT series reported in the paper as the forward/backward models. I followed the setup procedure in the README file, namely to export the huggingface path to the LLaMa2 tokenizer, and used the script for two-layer joint end-to-end training. Unfortunately, I got openai.error.APIConnectionError: Error communicating with OpenAI and the training procedure was broken.
I investigated the code a bit and found the "generate" function of the VLLM class involves querying the openai.Completion.acreate function/API. Is that the expected behavior when I switch to self-hosted models? I suspect there should be a dependency on vLLM, but I did not find one.
Thanks in advance if you could help me with this :)!

The script in this repo requires manually initiating a vLLM with the openai-compatible server beforehand! I will close this issue.

Hi @JingtongSu,
You are right. DLN does not manage the serving of models. The instructions describe how to connect to an LLM that is already hosted using vLLM. I apologize for any confusion; we are updating the documentation to better reflect this.
You can follow this instruction on how to set up an OpenAI compatible server using vLLM
Once your vLLM server is operational, you can refer to it as detailed in the DLN readme.

Hi @matheper, thanks for your quick reply! I have already figured that out and made my own DLN running. Thanks for taking the time of making the code public again :)


fix unhandled exception when trying to create a cluster on microOS

This should fix #441.

Thanks for the PR! I will take a look in the coming days and make a new release.


Selenium a strict requirement - but only required for one function

I am using ebay_rest without Selenium - I manage my own redirects as this is a web application. I would not install Selenium on our live server and it wouldn't work anyway in this configuration. All the Selenium imports in token.py are only used in a single function _get_authorization_code which I don't use (I supply the API object with all refresh token information directly).
Selenium is obviously very useful if you need it, but it is a very significant unnecessary requirement if you don't and a bit of a show-stopper for me.
The actual import is easy to deal with; either move the imports back into the relevant function or wrap in a try/except and throw an error if _get_authorization_code gets called. A user-friendly error ('Install Selenium to use the webdriver-based eBay user token authentication') could be displayed.
The trickier bit I guess is that Selenium is a pip required install; I'd prefer a recommended but not required install? ebay_rest isn't an end-user package; this is a developer package so I figure anyone working with it should be able to read the instructions and install Selenium if required (and add it to their requirements.txt/equivalent if packaging it).

Good to hear from you again, Andrew; I hope you have been well. Another user complained that the dependency was missing, so I added it again.
If Selenium became an Optional Dependency, would that work for you?

That would seem to be an excellent solution :)

I'll remedy this soon.

Andrew, I took a shot and achieved a partial success.
With the following, Selenium is no longer installed, and an instructive error is raised when there function that needs it is used.
pip install ebay_rest

But the following errors are out.
pip install ebay_rest[complete]

I reread the docs and googled it. No luck.
Any suggestions?
Within this commit these are the relevant files:
README.md
setup.cfg
src/ebay_rest/token.py

Note that Selenium was replaced with Playwright. remains.https://github.com/matecsaj/ebay_rest/commit/35c20f77e7a6f3e06e0ed5d36ec93d6c9875d569


Update snippet2501.fs

Add a member, showing that you can have such a member in a struct.
Ref: https://github.com/dotnet/docs/issues/5339 https://github.com/dotnet/docs/pull/5341

@rpetrusha can I get a sign off?