Faster cache retrival for large datasets

Description
When dealing with large datasets (>50 000 rows), its aparrent that the current mechanism for fetching cached data is not very performant. See this flamegraph (pictured below) for a query for 200 000 rows, where each row has a single child through a foreign key. The actual database query takes ~10% of the request time, while fetcing data from cache is ~60%. In this 60%, constructing the queryset.filter() alone is ~30%, while the compiler takes another ~15%, and the actual fetching from the cache is only ~10%.
The current mechanism for caching needs to run the compiler in order to create a unique cache key for a given model row based on the field selections in the GraphQL query, since the it's possible that the same row could appear in the same request with different field selections, and therefore we can't simply use its primary key as the cache keys.
One option could be to have some sort of cache merging stategy, where rows with the same primary keys would be merged to the cache, and then we could simply use the primary keys. How this would work with prefetched and related fields is a mystery though.
Or maybe there is some strategy for storing multiple rows for the same primary key in a way that wouldn't need to know so much about the query? Or maybe there is a way to rely on the order things are inserted to the cache?
This is a longer terms issue, but if you are reading this, suggestions are welcome!

Motivation
When fetching larger datasets without pagination, fetching from the query can become a bottleneck. Looking at the measurements, this could be made ~2x faster by faster cache retrieval.

After looking at this a little bit closer, I noticed that the entire caching mechanism is redudnant, and was there just for one case with related to-one fields. I was able to remove the mechanism, which of course removes this issues. Here is a link to the after-flamegraph (pictured below).

Released on 0.5.0.

BTW Thanks for all your work on the optimizer! This project is awesome :)


[DNM] player try again

Changes
testing e2e tests again, no need to worry about unittest atm will get them fixed until e2e gets green!

Submitter Checklist

[ ] 📝 A good commit message is important for other reviewers to understand the context of your change. Please refer to How to Write a Git Commit Message for more details how to write beautiful commit messages. We rather have the commit message in the PR body and the commit message instead of an external website.
[ ] ♽ Run make test before submitting a PR (ie: with pre-commit, no need to waste CPU cycle on CI. (or even better install pre-commit and do pre-commit install in the root of this repo).
[ ] ✨ We heavily rely on linters to get our code clean and consistent, please ensure that you have run make lint before submitting a PR. The markdownlint error can get usually fixed by running make fix-markdownlint (make sure it's installed first)
[ ] 📖 If you are adding a user facing feature or make a change of the behavior, please verify that you have documented it
[ ] 🧪 100% coverage is not a target but most of the time we would rather have a unit test if you make a code change.
[ ] 🎁 If that's something that is possible to do please ensure to check if we can add a e2e test.
[ ] 🔎 If there is a flakiness in the CI tests then don't necessary ignore it, better get the flakyness fixed before merging or if that's not possible there is a good reason to bypass it. (token rate limitation may be a good reason to skip).

this works! i moved it back to the original PR #1460


ℍ𝕠𝕨 𝕥𝕠 𝕣𝕖𝕔𝕠𝕧𝕖𝕣 𝕞𝕪 𝕝𝕠𝕤𝕥 𝕞𝕠𝕟𝕖𝕪 𝕥𝕠 ℂ𝕣𝕪𝕡𝕥𝕠 𝕔𝕦𝕣𝕣𝕖𝕟𝕔𝕪 Investment

I consider myself fortunate to have successfully distanced myself from JPY500. After five months of fragmented efforts, I made the decision to request a withdrawal to address some pressing matters. My attempt left me incredulous as, I perceived it as a potential heat despite making three additional inquiries within the span of a month, I received the same unhelpful response each time. Despite my numerous attempts to reach out to customer service, I never received any response. Untimely, I found assistance in processing my withdrawal from 𝕆𝕡𝕥𝕚𝕞 𝕀𝕟𝕧𝕖𝕤𝕥𝕚𝕘𝕒𝕥𝕠𝕣𝕤 . I strongly advise against any involvement with JPY500, or any tradinig company  as it could prove detrimental to one’s interests.
Just google search 𝕆𝕡𝕥𝕚𝕞 𝕀𝕟𝕧𝕖𝕤𝕥𝕚𝕘𝕒𝕥𝕠𝕣𝕤and submit your case on their website

I fell victim of crypto scam after searching on Google for information about a crypto asset brokerage firm. The top result was a sponsored link leading to a website with the web address mantao.network. Upon entering the website and clicking within the page, the website attempted to connect to my crypto wallet. After approving the request to connect to my wallet, I saw between 4.5 bitcoin,23 Ethereum and 89 Solona drained from my wallet. I was not been able to recover any of the stolen cryptocurrency until someone recommended RECOVERYCOINGROUP at gemale dot com. After contacting them and reaching to their terms, less than 5 days after, all my crypto assets were recovered. I strongly recommend this genius team.


.dib files are not handled as text in Azure DevOps repos

When I commit a .dib file to repo in Azure DevOps it cannot display the file on the web interface. It tries to handle it as a binary file.
Can I do anything about it?
Notebooks are great and I try to convince my teammates to use them, but my pull request looks like I'm trying to sneak in some binary bloat.

The only workaround I'm aware of at the moment is to use the .ipynb extension. It's not as friendly for diff tools.
We'll likely be adding additional supported file formats in the future.

DIB was already in use as a file extension which is why various things think it is a device independent bitmap.

Can someone reach out to the ADO team to get this fixed?  I'm starting to check in a lot of .dib Notebook files into ADO and none of it renders correctly

@gemolnar - can you confirm why the .dib is used when that extension is historically reserved for "device independent bitmap"?
We do not want to mess up handling of the extensions - either by handling the image files as non-binary.
We also do not want to allow users to control what extensions should be treated as non-binary or vice-versa.
Lastly - we do not want to get into space where the handling of the file will be driven by the contents rather than the extension as that cost more resource that base the handling on the file extension.

@jonsequitur - can you confirm why the .dib is used when that extension is historically reserved for "device independent bitmap"? I'd suggest using the ".dnb" or ".idnb".
On Azure DevOps side we:

do not want to mess up handling of the extensions (handle the image files as non-binary)
do not want to allow users to control what extensions should be treated as non-binary or vice-versa
do not want to get into space where the handling of the file will be driven by the contents rather than the extension as that may not work 100% and will consume more resource

So, we'll keep the current handling that is using hardcoded list of extensions that are treated as binary files. We'll not remove the ".dib" extension from that list as it's "device independent bitmap" as already explained above.

We could consider adding support for an alternative file extension for this format.
The decision to use .dib was made early on and we realized there was a file extension conflict only later. We haven't wanted to break backwards compatibility.

ADO will not fix the rendering of 'dib' files https://developercommunity.visualstudio.com/t/dotnet-notebook-dib-files-not-shown/10106469?viewtype=all
Have you/Could you (@jonsequitur @bjanousek & other relevant team members) discuss what extension we could use where the ".dib" could be renamed to (and eventually be rendered as Notebooks in ADO)?  suggestions *.pnb, *.pgnb (for PolyGlot NoteBook)
Thank you!

As of now, this issue still exists.


🚸 Sequence of writing a record vs completing the upload vs. restart an upload

Currently, we write the SQL record without a check that the upload has actually completed.
https://github.com/laminlabs/lamindb/blob/823a682f77eb44395fa0588e742556bbb996f2d9/lamindb/_artifact.py#L905-L918
For long uploads, this can be confusing.
We could switch the order and only write the SQL record if the upload is successful.

The problem is that if the upload fails in some way we'll have dangling files in storage, without a reference to the metadata record.
We could handle this case like we handle it for the metadata record right now: run a clean up operation for files.
But ideally we enable uploads that can be restarted. Hence, I'd like to work in this together with thinking through restarted uploads.

We could also do what some products do: show an “uploading” or “downloading” symbol while a file is still in progress. There is a trivial solution that makes another 0.1s network request and marks the artifact as "complete".

Background: https://laminlabs.slack.com/archives/C03P6D8U1PC/p1712153431934569

Not planned right now as, for instance, it's helpful to see when somebody first registers a transform.
I'd rather indicate on the UI that an artifact upload isn't yet completed or a transform doesn't yet have a notebook.


Slider should have an editable range

It would be sweet if a user could specifiy a certain range of values for the slider in the settings-meny (edit box). So if a user wanted a slider that ranges from something like 20<->40 instead of the standard 0<->100 the could have that!
There is support in the MUI-slider api for this https://mui.com/material-ui/api/slider/ so the implementation should be quite straight forward!

Fixed in commit e847eb9


Add details on subhourly clipping loss correction

Two items that could be helpful:

Clarify in the UI that the subhourly clipping correction checkbox is for the "Allen method"

this is to avoid confusion with the "Walker method" - I'm not sure of the status of the Walker method in SAM/pysam, but it's been discussed publicly, e.g., https://www.nrel.gov/docs/fy24osti/90054.pdf.

Consider adding a reference to [1] in the help docs under "Enable subhourly clipping loss correction" (note: I'm a co-author on [1], so I have a potential COI with this suggestion...)

[1] J. O. Allen, R. Dhakal, W. B. Hobbs, and W. Li, “Allen Method for PV Subhourly Clipping Correction,” 2024. [Online]. Available: https://github.com/epri-dev/allen-subhourly-clipping-correction

Thanks @williamhobbs ! We're always happy to link to more documentation, so we'll accept the COI. 😛 And that is a helpful suggestion for the UI, as you're right that we've talked about the Walker method publicly. We'll add this for our fall release!
FYI @cpaulgilman  on the documentation addition and @mjprilliman on the UI change.


NaN appears as cell in downloaded text file from heatmap

I'm not sure what NaN is doing as a value for the number of proteins available in a particular protein family for a specific genome.  Look at attached spreadsheet
ROS genes PGFams.txt

how can i reproduce this problem. how did you get the page before you click download button?

@ARWattam NaN is typical javascript issue when it expect numbers but the value is not. I need more info, where were you, which dataset you were loading, etc.

the fix is deployed on production.


Add keycap emoji numeral parsing! closes #28

This commit fixes issue #28 by adding a parser to convert numbers to keycap emoji numerals.

Looks good to me, I'll merge later tonight, thanks!

Yeah this is silly but cool, thanks :)
http://www.saythisnumber.in/emoji/1997


Update .gitignore

Keep them in sync 🎉
https://github.com/spatie/package-skeleton-laravel/pull/64

Thanks!


Fix travis compile with arguments

We noticed that travis compile N and travis compile M.N (as described here) do not work.
// repository is HPI-BP2015H/test, but we also tried this with HPI-BP2015H/SWT-Demo
user@host:~/test$ travis compile 8
undefined method `delete' for nil:NilClass
for a full error report, run travis report
user@host:~/test$ travis report
System
Ruby:                     Ruby 1.9.3-p484
Operating System:         Ubuntu 14.04
RubyGems:                 RubyGems 1.8.23

CLI
Version:                  1.8.2
Plugins:                  "travis-build"
Auto-Completion:          yes
Last Version Check:       2016-03-31 15:25:01 +0200

Session
API Endpoint:             https://api.travis-ci.org/
Logged In:                no
Verify SSL:               yes
Enterprise:               no

Endpoints
org:                      https://api.travis-ci.org/ (current)

Last Exception
An error occurred running `travis compile`:
    NoMethodError: undefined method `delete' for nil:NilClass
        from /home/user/.travis/travis-build/init.rb:102:in `push_down_deploy'
        from /home/user/.travis/travis-build/init.rb:47:in `run'
        from /var/lib/gems/1.9.1/gems/travis-1.8.2/lib/travis/cli/command.rb:198:in `execute'
        from /var/lib/gems/1.9.1/gems/travis-1.8.2/lib/travis/cli.rb:64:in `run'
        from /var/lib/gems/1.9.1/gems/travis-1.8.2/bin/travis:18:in `<top (required)>'
        from /usr/local/bin/travis:23:in `load'
        from /usr/local/bin/travis:23:in `<main>'

Adding a @ fixed the problem for us.

@BanzaiMan could you have a look?


Updates to make top level project

Removes all instances of "incubating"
Removes the DISCLAIMER file and dialog in windows installer
Updates README to match doap description
Rewrap daffodil-cli readme so it's more readable

DAFFODIL-2470

List of things that must be updated before this should be merged to allow things to work:

Daffodil accepted by the ASF board as a TLP
Repos renamed to https://github.com/apache/daffodil and https://github.com/apache/daffodil-site
Codecov updated to work with the new repo URLs (that might be automatic with the rename)
Sonarcloud configuration
daffodil directory created in dist.apache.org/repos/

Might need to test Windows installer before doing a real release if not already tested.

Definitely. The release candidate builder that builds the windows exe currently works by cloning the repo, so it's a bit difficult to test things that haven't been committed, without hacking a bunch of stuff together. I'm thinking that before merging, I can make some tweaks so it clones my fork and that should give some confidence that it works before merging (haven't done that). But to be 100% sure it needs to be tested after merged. Fortunately, there's a --dry-run option so that should make it pretty easy to build a test release.

I spent some time this morning building/testing the windows MSI. And I did find fix a couple issues that are now fixed in the latest fixup commit. I am much more confident that the windows MSI installer works and has no references to incubation anymore.
We'll still want to test the release candiate script once repos are renamed and this commit is merged to make sure that all works, but I think the windows aspect of that should be good.

Note to self, we also need to update the following wiki pages once the transition is complete:

Code Contributor Workflow
Release Workflow

Checklist of external places that refer to Daffodil as incubator that will need updating:
I did a google web search for "apache daffodil (incubating)" and ran through the various returns there.
These reference the github repo, so can't be updated until the repo location/name is updated:

scaladex - https://index.scala-lang.org/apache/incubator-daffodil/daffodil-core/3.0.0?target=_2.12
openhub - https://www.openhub.net/p/dfdl-daffodil

This also has mentions explicitly of "apache daffodil (incubating)"

daffodil nifi - https://opensource.ncsa.illinois.edu/bitbucket/projects/DFDL/repos/daffodil-nifi/browse

I already updated github DFDLSchemas site, and references to daffodil (incubating) within individual schemas I searched and they're not there, or if they are, they're in a historic context where I don't think we need to update them.
I verified the wikipedia page on DFDL does not mention "(incubating)" w.r.t. daffodil.

I'm not sure we have access to the NCSA repo anymore. The maintained NiFi Daffodil repo has moved to https://github.com/TresysTechnology/nifi-daffodil. I have just now updated that repo to remove the incubating.

I definitely do not have write access to the ncsa repo, though I can login. Probably would be good to put a forwarding link in the README file. But if nobody has write access, then we can just let this fade away in time.

I did some testing with the release candidate script this morning. I renamed my forked github repos to "daffodil" and "daffodil-site" (I reccomend others do this too), and temporarily tweaked the release candidate script to build from my fork instead of from apache and ran a dry run. Only issue was the missing dist/{dev,release}/daffodil directories. I've created those and now the release candidate scripts works with no issues, and I didn't find any instances of "incubat*". All that's left I think is for INFRA to rename the apache github repos and I'm pretty confident this change can be safely merged.

OpenHub has been updated.
Scaladex - according to https://github.com/scalacenter/scaladex-contrib it seems this site updates itself by pulling from maven central, so long as what is pushed there has an scm tag indicating where the source lives.

I believe this means this won't update until a release is pushed to maven central that has source scm tag that does not specify "incubating-" in it.

Yeah, I think you're right. I have confirmed the new poms do have the new github URL, so once we do a release this should be fixed.
Though github.com/apache/incubator-daffodil does redirect to github.com/apache/daffodil, so if scaladex can follow redirects, it might auto update?

Furthermore, unfortunately, at Maven central, the link for source code for daffodil specifies the gitbox URL
https://gitbox.apache.org/repos/asf/incubator-daffodil.git
And this fails.
Moving this to a JIRA ticket https://issues.apache.org/jira/browse/DAFFODIL-2471

Non code update, the about line on the github page

Ah, good find. We could have infra fix this, but the better more future proof method is probably to add a .asf.yml file. That let's us change the description, but also configure other aspects of github.

Just created PR #492 to hopefuly fix the description and allow us to configure gitbox/github more easily in the future.


Validate OWL

Input ontologies should be validated to make sure they are valid OWL

Linked to:

https://github.com/SD2E/paml/pull/41
https://github.com/SD2E/paml/issues/30

... and this isn't actually an issue for sbol_factory, since by the time it encounters the file, it's reading it in and will discover validation problems the hard way.


No longer working via Service

[Enter feedback here]

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: d5505062-94f3-e15d-a52f-ce1ac7f8cb69
Version Independent ID: 34076b73-0ded-2111-ed44-2a31e07deb53
Content: Row-level security (RLS) with Power BI - Power BI
Content Source: powerbi-docs/enterprise/service-admin-rls.md
Service: powerbi
Sub-service: powerbi-admin
GitHub Login: @paulinbar
Microsoft Alias: painbar

Hi @justin-m-schmidt, I'm closing this issue for the time being. Please reopen if you want to provide more detail regarding the issue you faced. Thanks.


update KubeVirt CDI version

1.8 is intended to be the stable release for the demo.  Also, 1.6 has a pretty bad bug causing the controller to crash.

@karmab want to take a look at this one?

while cdi deployed fine with this change, i hit https://github.com/kubevirt/containerized-data-importer/issues/666#issuecomment-483730215 preventing kubevirt to deploy

@karmab I assume you have the same issue with the older version of CDI?  How is this preventing KubeVirt from deploying?


Bug: Cannot create new task or method in new submission

Attempting to click on the "New" button for either "Task" or "Method" when adding a submission does not yield the dialog box.
This only seems to happen if the paper is present, but no message to notify the user of this is displayed.

I'm going to agree with you in not considering this a bug. I'm going to go ahead and close this one. Thanks for the response, and of course, reopen if you disagree with the closure. Thanks again for the insights!


Migrate Controller Specs to Request Specs

We have dozens (literally dozens!) of Controller specs. The current attitudes are to move away from Controller specs in favor of Request specs. We have some Request specs started for our API.
See here for some more discussion: https://github.com/rspec/rspec-rails/issues/1838
This is also an opportunity to review our specs and ensure:

we have good coverage for our controller actions
we are not testing things unnecessarily

Controller Specs Needing Conversion:

[x] adjustments_controller_spec
[x] audits_controller_spec
[x] barcode_items_controller_spec
[x] dashboard_controller_spec
[x] data_exports_controller_spec
[x] diaper_drive_participants_controller_spec
[x] diaper_drives_controller_spec
[x] distributions_controller_spec
[ ] donation_sites_controller_spec
[ ] donations_controller_spec
[ ] items_controller_spec
[ ] organizations_controller_spec
[ ] partners_controller_spec
[ ] purchases_controller_spec
[ ] requests_controller_spec
[ ] static_controller_spec
[ ] transfers_controller_spec
[ ] users_controller_spec
[ ] vendors_controller_spec

Reference

https://relishapp.com/rspec/rspec-rails/docs/request-specs/request-spec
https://everydayrails.com/2016/08/29/replace-rspec-controller-tests.html

Hello again @seanmarcia, can i take data_exports_controller_spec, diaper_drive_participants_controller_spec, diaper_drives_controller_spec and distributions_controller_spec ?

Yes! Thank you!

@seanmarcia How many of specs are still in need of conversion? I'd like to help out if some are still available

Hey @mcshakes the completed list has been updated to show the ones that are finished or there are PRs for!

Can I grab these?

transfers_controller_spec
users_controller_spec
vendors_controller_spec

Definitely! I just marked them off :) Thanks so much!

@seanmarcia can I grab:

donation_sites_controller_spec
donations_controller_spec
items_controller_spec
organizations_controller_spec

They're yours! Just marked them off :)

@seanmarcia I'll take partners_controller_spec.rb

Hi @seanmarcia, i'll take static_controller_spec

👍

requests_controller_spec - #1501

I will take purchases_controller_spec

@seanmarcia Can I take these?

admin_controller_spec
admins_controller_spec
storage_locations_controller_spec
users_controller_spec

i will take partners_controller_spec (admin)

I'll take base_items_controller_spec (admin)

@seanmarcia admins_controllers_spec file is multiple-line comment and I can't find with rails routes routes that match with admins. Also users_controller_spec is already done, I didn't notice. Can I grab organizations_controller_spec (admin) instead?


[Bug] GetDiverging(pull) can lock pull in a 500 state

if zwo threads add remote repo to pullBase repo they can lock it
2020/02/03 02:47:28 routers/repo/pull.go:392:PrepareViewPullInfo() [E] GetDiverging: headGitRepo.AddRemote: exit status 128 - fatal: remote tmp-pull-4206-base already exists.
https://gitea.com/gitea/go-sdk/pulls/256

solution: use a tempRepo to compare base & head repo ...
-> but to save resorces we need a cahcke or precalc the values on each push to pull
move this "task" to checkPullTask?


git ignore tree viewer

[QUESTION]
I added my test directory test to .gitignore in order to not track changes in those files. Atom tree viewer also hides the test directory, apparently due to Git Plus. Could you please add this option that all folders and files are shown even though there are not tracked by Git ?

It might be due to Atom hiding files in Tree Viewer. Head to the settings of Tree Viewer and make sure these are unchecked.

Source


Issue #SB-25253: Cassandra Migrate job to handle table name from jobconfig

SB-25253  Cassandra migrate script to handle report table name

https://project-sunbird.atlassian.net/browse/SB-25253
The PR contains changes for cassandra-migration job to handle the table name from job config
Type of change
Please choose appropriate options.

[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update

How Has This Been Tested?
Please describe the tests that you ran to verify your changes in the below checkboxes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

[ ] Ran Test A
[ ] Ran Test B

Test Configuration:

Software versions:
Hardware versions:

Checklist:

[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[x] My changes generate no new warnings
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] New and existing unit tests pass locally with my changes
[ ] Any dependent changes have been merged and published in downstream modules

PR looks fine.


Admin quote change!

Please change your Checkoutcom_Ckopayment_Model_CheckoutcomUtils::getQuote() from Mage::getModel('sales/quote')->load($quoteId); to the proper Mage::getSingleton('adminhtml/session_quote')->getQuote();
Loading a quote from the database is crazy as all the information recently added are in the session, that's the reason why you got the code like if (empty($billStreet1) && empty($shipStreet1)) because nothing is coming out of that newly fetched quote.
It has been 3 month that I've gave a list of bugs in your code, so far you haven't fixed a single!

@hwebb Thank you for your suggestion. Can you confirm which version of the plugin are you on? We release version 3.0.4 recently with major improvements and bug fixes. Did you try the new version or still using the older version? Can you list the issues you are having with the new version?

I don't think you read what I wrote, can you read and take a look before asking this question?

@hwebb I did read what you wrote and I also said thank you. We are looking into that issue related to the getQuote function. My questions were mainly related to the list of bugs that you said we haven't fixed. If you are not using the the new release, I would strongly advise that you update.


Can't exit espflash if firmware is stuck

The exact cause is unknown at this point, but here is the setup:

using windows 11 and windows terminal
using an ESP32-S3 with the internal USB/JTAG interface
using espflash 2.0.0 with the #399 patch
trying to run the code from https://github.com/card-io-rs/card-io-fw/tree/68d58393aa12d0ed56ff9de52428f8fab9de7c12

The serial output includes the bootloader messages and the "Hello, world!" printed from the main task, but does not include the state printing a few lines below. In this state (i.e. probably while stuck in Board::initialize, which is trying to configure a display over SPI, but disregarding the bug in my code), espflash doesn't react to Ctrl+C or Ctrl+R and does not exit.
Manually resetting the MCU exits espflash and then the terminal prints the control characters:

And just after posting this, the problem goes away by itself, of course...

I have the same issue on Alacritty and the default terminal of RaspberryPiOS. Only way to stop espflash is to kill it in another terminal.
It doesn't even stop on SIGQUIT Ctrl+\.
Anyone knows a fix?


[BUG/QUESTION] Error connecting to APIM endpoints

Describe the bug
Hello there, I cannot reach APIM endpoints. The development portal is online.
https://<redacted>.azure-api.net/manpage/openai.json
returns:
{ "statusCode": 404, "message": "Resource not found" }

https://<redacted>.management.azure-api.net/
returns:
{"error":{"code":"MissingOrIncorrectVersionParameter","message":"API version query parameter is not specified or was specified incorrectly.  Supported versions: 2014-02-14-preview,2014-02-14,2015-09-15,2016-07-07,2016-10-10,2017-03-01,2018-01-01,2018-06-01-preview,2019-01-01,2019-12-01-preview,2019-12-01,2020-06-01-preview,2020-12-01,2021-01-01-preview,2021-04-01-preview,2021-08-01,2021-12-01-preview,2022-04-01-preview,2022-08-01,2022-09-01-preview,2023-03-01-preview,2023-05-01-preview,2023-09-01-preview  Example query param format: api-version=SupportedVersion","details":null}}

To Reproduce
Steps to reproduce the behavior:

setup following params:

{
  "GRAPHRAG_API_BASE": "https://graphrag-azure-openai.openai.azure.com/",
  "GRAPHRAG_API_VERSION": "2023-09-01-preview",
  "GRAPHRAG_EMBEDDING_DEPLOYMENT_NAME": "Embeddings",
  "GRAPHRAG_EMBEDDING_MODEL": "text-embedding-3-large",
  "GRAPHRAG_LLM_DEPLOYMENT_NAME": "GraphRag-Azure-OpenAI",
  "GRAPHRAG_LLM_MODEL": "GPT-4o",
  "LOCATION": "francecentral",
  "RESOURCE_GROUP": "GraphRAG-north-europe"
}

other optional params are not provided.

run deploy script

bash deploy.sh -p deploy.parameters.json
Error while deploying in waitForGraphrag() function:
+ az rest --method get --url https://<redacted>.azure-api.net/manpage/openapi.json
+ '[' 1 -eq 0 ']'
+ sleep 40
+ printf .
.+ (( i++ ))
+ (( i < 40 ))
+ '[' false == true ']'
+ printf ' Failed.\n'
 Failed.
+ exit 1

Expected behavior
The endpoints should be available, but I do not know why they are not.
Screenshots
If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):

OS: Docker Devcontainer
Version: Python 3.12

Additional context

I've not encountered this in my testing but will keep an eye out and try to reproduce. Thanks for bringing to our attention @Arvoid00

@Arvoid00 in some organizations, you may have an Azure Policy in place that applies additional NSG rules on top of APIM at deployment time. In my experience, this is what impacts the deploy.sh script running to final completion, specifically at the rest call that you reference. If your organization has a VPN, ensure you are connected to it before running the deploy.sh script.

@Arvoid00
Please update your GRAPHRAG_API_VERSION to use 2024-02-15-preview instead of 2023-09-01-preview.

any luck @Arvoid00?

Hi all, thanks for the responses. I have not been able to test said proposals due to a shift in priorities. I will provide an update when I can. For now this issue can be closed.

Did you manage to fix this? @Arvoid00  I tried changing the Graph_API_Version in the params and still get the same issue as you. If I navigate to the APIM Gateway URL directly (without swagger link) I get a 404 error. When I add the swagger link I get a 503 error.


create an in-memory wasmer compiled modules cache

duplicate issue:
https://github.com/spacemeshos/svm/issues/4


.card-outline-* classes no longer work

JSBin example
I'm pretty sure that the reason why it broke is with the recent change to cards

Hi @Starsam80!
You appear to have posted a live example (http://jsbin.com/cupegocuce/edit), which is always a good first step. However, according to Bootlint, your example has some Bootstrap usage errors, which might potentially be causing your issue:

W005: Unable to locate jQuery, which is required for Bootstrap's JavaScript plugins to work

You'll need to fix these errors and post a revised example before we can proceed further.
Thanks!
(Please note that this is a fully automated comment.)

whoops, even though this shouldn't be related to javascript. revised example

I was able to fix it by adding border: 1px solid. new example, with and without fix

Confirmed. We're setting a border-color but not any border-width.

Bisection showed that 955e0405ea04e8bce3fa8395cd5e1c4030c4947a was what broke this.

CC: @mdo

Any update on this? I reverted 955e0405ea04e8bce3fa8395cd5e1c4030c4947a for now on my branch, but it gets rid of header navs. I could do a pull request, but I have no idea if header navs are here to stay.

@Starsam80 If there was an update, you'd see it here.
@mdo Ping.

in mixins/_cards.scss I added
// Card variants
@mixin card-variant($background, $border) {
background-color: $background;
border-color: $border;
border-style: $card-border-style;
border-width: $card-border-width;
}
@mixin card-outline-variant($color) {
background-color: transparent;
border-color: $color;
border-style: $card-border-style;
border-width: $card-border-width;
}
added
$card-border-style:         solid !default;
to _variables.scss also.
and it works as expected.

@mdo Ping.

Change coming in #19861.


tree is missing on ubuntu-24.04

Description
tree command is missing on ubuntu-24.04, but it is present on ubuntu-22.04 and ubuntu-20.04.
Is this an intentional change?

tree is not listed in Installed Software.
I agree, "tree" is not a vital package.

related issue: https://github.com/actions/runner-images/issues/9848
Workaround
Adding this step resolved my problem but added a few seconds to complete runs.
      - name: Install tree command
        run: sudo apt-get install --no-install-recommends --assume-yes tree

Platforms affected

[ ] Azure DevOps
[X] GitHub Actions - Standard Runners
[ ] GitHub Actions - Larger Runners

Runner images affected

[ ] Ubuntu 20.04
[ ] Ubuntu 22.04
[X] Ubuntu 24.04
[ ] macOS 11
[ ] macOS 12
[ ] macOS 13
[ ] macOS 13 Arm64
[ ] macOS 14
[ ] macOS 14 Arm64
[ ] Windows Server 2019
[ ] Windows Server 2022

Image version and build link
  Image: ubuntu-24.04
  Version: 20240516.4.0
  Included Software: https://github.com/actions/runner-images/blob/ubuntu24/20240516.4/images/ubuntu/Ubuntu2404-Readme.md
  Image Release: https://github.com/actions/runner-images/releases/tag/ubuntu24%2F20240516.4

https://github.com/kachick/example-tree-ubuntu-24.04-runner-image/blob/30b9f514581467a58d0ac2afd9ff5d59c7c2d000/.github/workflows/tree.yml
https://github.com/kachick/example-tree-ubuntu-24.04-runner-image/actions/runs/9249889143/job/25442442519

Is it regression?
https://github.com/kachick/example-tree-ubuntu-24.04-runner-image/actions/runs/9249889143/job/25442442678
Expected behavior
I can use tree command in ubuntu runner by default.
Actual behavior
tree command is missing. I need to install it with apt-get.
Repro steps
  - name: Inspect depednency
    run: |
      command -v tree
      tree --version
  - run: tree

Hello @kachick, the tree is added. Will be deployed soon.

Thank you! 🙏


Update 11/28/2015

[x] The utils/ and models/ folders have been added to Coveralls and tests now have their own separate folder so coverage is now more accurately calculated (before, only our data/ folder was being tracked).

[x] All necessary functionality with the exception of graphing functions have been moved from stimuli.py, smoothing.py, and utils_functions.py to the new run class, which is complete with tests. Please make the transition soon so we can get rid of these files.

[x] Euclidean distance has been added to .behav as well instead of only .design_matrix(). Scripts that use .behav but not .design_matrix() now have the extra option of using euclidean distance as an input.

[x] Single-variable correlation coefficient estimation has been added to the run class as the method .correlation().

[ ] Task: Last chance to decide which data set we want to use: raw or filtered.

[ ] Task: Please list functions in order of priority to add to run class and/or write tests for. Please also specify which should be done for each function (e.g., Write test for ___ AND add to run class).

[ ] Question: Do we need pearson.py?

[ ] Question: In what situation would we use neural_highres() instead of events2neural()?

Regarding to the last two questions, first, we have not used the pearson.py so far, but we can keep it since I think it was already there when the repo was built.  neural_highres() handles more complicated conditions while events2neural() is for simpler cases. In our case,  neural_highres() is more suitable.
One more thing, would you mind adding four conditions files (cond001, cond002, cond003, cond004 from ds005) to the run class as attributes, since I need to rewrite run_conv_script.py using these four condition files instead of generating our own 4 conditions from behavdata? I will upload the new run_conv_script.py tonight. I think the time_course method in run class need to be adjusted according to the new run_conv_script.py. Thanks!

I've added cond002.txt, cond003.txt, cond004.txt files, but I didn't add cond001.txt because there's only one task for this study.

Last set of midterms and first finals; will be MIA until Thursday night.

@VictorKong94 If I decide to use filtered data, but I notice that filtered data do not have behavedata.txt but only neural data, how does make_class deal with this situation?
Also, on my computer by default, filtered data also has the same name as 'ds005' , did you rename it on your computer?


Collaboration with rules_dotnet?

Have there been any previous conversations about merging the rules_dotnet effort and this one? I think it would be a tremendous gain for the already tiny . NET/Bazel community to combine these 2 rulesets.
There are features that both rulesets can gain from each other and users of both rulesets could benefit from buildfile generators for example.
I'm not a maintainer of rules_dotnet so I can't speak with any authority but maybe @tomaszstrejczek can chime in?

There haven't been any conversation so far. These rules started as an internal (to D2L) fork of rules_dotnet before it was rewritten. As it stands their APIs are quite different. I'll try to summarize things at a high-level as I see it:
The main things we need that I think were gaps with rules_dotnet:

Correct support for target frameworks: It should behave like MSBuild, e.g. you should be able to build a DLL for multiple target frameworks, and when choosing dependencies to compile/run against it should choose the same ones that MSBuild would. It looks like this. rules_dotnet by contrast (if I remember correctly) expects the user to wire things up manually in BUILD files like the example here.
.NET Framework support, using the .NET-core SDK for compilation on Linux/Mac (we support builds of .NET Framework on Linux/Mac, even if the output can only be run on Windows.)
Usage of reference-only assemblies for incremental build performance: we support this in the usual way (we use -refout: to the compiler), but we also have first-class support with an internals_visible_to attribute on e.g. csharp_library (we will generate 3 DLLs: the real one (changes most frequently) the ref-only version for friend assemblies (changes less frequently) and the general ref-only version (changes much less frequently.) See this example.

The things that are a "priority" (we haven't worked on this much lately, but when we pick it up its our next steps) for us, some of this rules_dotnet has and some it doesn't:

Get bazel test/bazel run working (#9 , #71 , #128 , #45)
Get packaging/publishing rules (#30 , #148)
More flexible resource specification (#77)
Persistent workers using a shim on top of vbcscompiler (#12)
Aspect-based csproj generation

Things that are not a priority for D2L (the owners of this repo) but we have no objections to/think are cool:

F#, VB support (we don't use these)
Mono support (we don't use Mono)

So -- we would be happy to collaborate in any way that works. We've made this code open source on GitHub (under a permissive license) partly so that it's available to be copied from. My impression is that both sets of rules are a bit far from feature complete (especially this repo).
I would recommend people use rules_dotnet if it works for them. If, in the future, we could delete this repo and use rules_dotnet for our codebase we will. Our current plans are to expand these rules until they are relatively feature complete, in step with usage of the rules in our internal code base. We have specific (if not terribly unique) needs and at this point we think having these separate rules is working best for us.
We're also happy to take external contributions if people find these rules fit their use-cases better.

Ok, it sound then like these rules are still very much a work in progress? It might then be premature to combine the efforts but I think once these rules have run and test working properly it might be a good time to look into this.
E.g. I contributed the F# support to rules_dotnet and it could pretty much use 95% of the .Net plumbing that existed there since the only step that is different is the compilation task.
It's also on my radar to look into .fsproj/.csproj generation from the Bazel targets and generation of 3rdparty Bazel targets via Paket(https://fsprojects.github.io/Paket/) since it can generate a lockfile which fits well with Bazel's determinism.
I would love for these efforts to benefit as many .Net/Bazel users as possible since we are very few at the moment. But I really do think that Bazel can be a huge benefit for shops with large .Net codebases.

Ok, it sounds then like these rules are still very much a work in progress? It might then be premature to combine the efforts but I think once these rules have run and test working properly it might be a good time to look into this.

Absolutely. We (D2L) use them, and I think at least one other org uses them (Selenium), but only in a limited capacity.
I'll update our README today to make sure that it's clear that people should check rules_dotnet first.

E.g. I contributed the F# support to rules_dotnet and it could pretty much use 95% of the .Net plumbing that existed there since the only step that is different is the compilation task.

Neat!

It's also on my radar to look into .fsproj/.csproj generation from the Bazel targets and generation of 3rdparty Bazel targets via Paket(https://fsprojects.github.io/Paket/) since it can generate a lockfile which fits well with Bazel's determinism.

Cool! Check out the docs for aspects. I haven't used it personally, but the idea is to teach Bazel how to generate things like IDE files from targets (aside: this is partly why having csharp_library support multi-targeting is important to us; it makes the targets one-to-one which probably makes this simpler).
A related thing is going the other way: generating BUILD files from csproj. We do this because we're not (even close to) ready to have our BUILD files be "the source of truth". We have a .NET library for constructing and serializing Starlark (with idiomatic formatting) but it's currently very coupled to our internal build system. Having the core part of that (the Starlark library and the conversion from csproj to Starlark) could be really helpful for people. I think rules_dotnet has this but only for NuGet/WORKSPACE.

I would love for these efforts to benefit as many .Net/Bazel users as possible since we are very few at the moment. But I really do think that Bazel can be a huge benefit for shops with large .Net codebases.

Yeah, I agree. 😄


The need for a CI

This repository should be taking advantage of a CI like Travis CI. Such a CI would be able to automatically release code to npm, compile scss and add prefixes.
My suggestion here is to integrate this project with Travis Ci, give Travis a gituhb token and then configuring it to compile scss, add prefixes, minify it then release it to npm. Therefore, people can use services like jsdelivr to use your code instead of having to download the boilerplate code every single time.

I might make it however I don't have much time for it right now. I'll leave this issue opened, until I make it

I, however, do have a bit of time on my hands. Therefore I suggest you should connect it to travis-ci.com, set a repository token and I'll see what I can do.

Well okay, I'll do it tomorrow

Sure.


refactor: structure code for ease of use as module

I wonder if it's in scope and you're willing to refactor the CMake support for cppfront so that it could be used as a module.
My objective is for this project to be able to be used by https://github.com/JohelEGP/jegp.cmake_modules/#jegpcpp2. As a consequence, it'd be easy to integrate into Compiler Explorer: https://godbolt.org/z/Pq48G6T7f. Even if CE starts supporting the compilation of the generated C++ source, it won't integrate with its CMake project feature due to CMake's lack of support for Cpp2 in CMake.
As for the implementation, the most obvious place seems to be https://github.com/modern-cmake/cppfront/blob/main/cmake/CppfrontHelpers.cmake, which contains the important helpers to make this work. The most obvious challenge is the use of the target cppfront::cppfront. This suggests that an indirection to make it an IMPORTED target instead might viable. That could be done by detecting preexisting support for cppfront, as opposed to compiling it like the main CMakeLists.txt, in a similar fashion to the Cpp2 and cppfront variables used to detect support at https://github.com/JohelEGP/jegp.cmake_modules/#variables.
I'd have loved for this project to replace https://github.com/JohelEGP/jegp.cmake_modules/#jegpcpp2, but it's scope also extends to Cpp2 rather than just cppfront.

The most obvious challenge is the use of the target cppfront::cppfront. This suggests that an indirection to make it an IMPORTED target instead might viable. That could be done by detecting preexisting support for cppfront, as opposed to compiling it like the main CMakeLists.txt

I think this is reasonable... it could be restructured as FindCpp2 (or something that doesn't conflict with the existing cppfront name) which tries to find_package(cppfront) (and any number of fallback strategies) if cppfront::cppfront does not already exist as a target.

I'd have loved for this project to replace https://github.com/JohelEGP/jegp.cmake_modules/#jegpcpp2,

jegp_cpp2_target(<target>) seems equivalent to cppfront_enable(TARGETS <targets>...), here.

but it's scope also extends to Cpp2 rather than just cppfront.

I'm not sure there's much of a distinction until competing implementations arise?

I'm not sure there's much of a distinction until competing implementations arise?

That's right.

Actually, I tried hacking on this a little bit and I'm not sure anymore what problem you're trying to solving, exactly. It seems to me that the cmake-only cppfront-config.cmake package (which takes cppfront-exe as a dependency) already suffices. Can you illustrate (preferably with an example) what the problem is?

cppfront-config.cmake is generated. I can't do that in CE.

Maybe we could talk to Matt about adding this project as an optional "library" on CE. Then the IDE/CMake projects could just find_package(cppfront).

That's a great idea!

That's a great idea!

I just pinged him on Slack. Happy to make changes here to facilitate CE integration :)

Success: https://godbolt.org/z/KP6c6WvGT. I replaced my module with CppfrontHelpers.cmake, and in CMakeLists.txt added:
add_library(cppfront_cpp2util INTERFACE)
add_library(cppfront::cpp2util ALIAS cppfront_cpp2util)
target_include_directories(cppfront_cpp2util INTERFACE "${JEGP_CPPFRONT_INCLUDE_DIRECTORIES}")

add_executable(cppfront_cppfront IMPORTED)
add_executable(cppfront::cppfront ALIAS cppfront_cppfront)
set_target_properties(cppfront_cppfront PROPERTIES IMPORTED_LOCATION "${JEGP_CXX2_COMPILER}")

Great! One little note, this will suffice:
add_library(cppfront::cpp2util INTERFACE IMPORTED)
target_include_directories(cppfront::cpp2util INTERFACE "${JEGP_CPPFRONT_INCLUDE_DIRECTORIES}")

add_executable(cppfront::cpp2util IMPORTED)
set_target_properties(cppfront::cpp2util PROPERTIES IMPORTED_LOCATION "${JEGP_CXX2_COMPILER}")

IMPORTED targets are allowed to have :: directly in their names :slightly_smiling_face:

I have switched to using jegp_cpp2_target_sources.
That supports C++ modules: https://cpp2.godbolt.org/z/of6zr8P66.


uktv play  not working

What is the purpose of your issue?

[x] Bug report

Bug report
If the problem occurs when downloading a URL please provide the full verbose output as follows:

Restart youtube-dl-gui
Go to Options > Extra tab
Enable Debug youtube-dl
Go to Options > Advanced tab and Clear your log content
Try to download the URL
Copy the whole log content and insert it between the ``` part below

[05/25/18 09:39:47] WARNING: Falling back on generic information extractor.
[05/25/18 09:39:47] ERROR: Unsupported URL: https://uktvplay.uktv.co.uk/shows/the-horne-section/watch-online/

What operating system do you use ?
windows 10
List of actions to perform to reproduce the problem:

..paste in link in error log selct add
..press download
..EERRROORRR

What is the expected behaviour ?
it to download the show
What happens instead ?
its failed to download

Feature request (request for a new functionality)
Please make sure that the requested feature is NOT already in the TODO list

[x] I've verified and i assure that my requested feature is NOT in the TODO list

What operating system do you use ?

@5kedone this is a problem with youtube-dl which is what youtube-dl-gui uses for the actual download, it seems to have been reported here. I encourage you to follow this ticket instead.


using setState method make crash camera scanner (white screen or freeze)

Hey !
I currently facing a issue with your package. When I try to use the function setState the state change well but just after the scanner doesn't work anymore... The same code without setState is working correctly (for exemple a toast with the barcode result works well). I also try setState outside the onDetect (icon onPressed) and change a state without barcode info (change the backgroud color onDetect and outside onDetect).
Code to reproduce problem :

import 'package:flutter/material.dart';
import 'package:mobile_scanner/mobile_scanner.dart';

class QRCodeScan extends StatefulWidget {
  const QRCodeScan({Key? key}) : super(key: key);

  @override
  State<QRCodeScan> createState() => _QRCodeScanState();
}

class _QRCodeScanState extends State<QRCodeScan>
    with SingleTickerProviderStateMixin {
  final MobileScannerController _controllerQrCode = MobileScannerController(
      torchEnabled: false, formats: [BarcodeFormat.qrCode]);
  String? _barcode;

  @override
  Widget build(BuildContext context) {
    final GlobalKey qrKey = GlobalKey(debugLabel: 'QR Code scan');
    return Scaffold(
      appBar: AppBar(),
      body: Column(
        children: <Widget>[
          const Text('Title'),
          Padding(
            padding: const EdgeInsets.fromLTRB(20, 10, 20, 0),
            child: Text(_barcode ?? 'Some text'),
          ),
          Expanded(
            child: Container(
              padding: const EdgeInsets.only(top: 30),
              child:
                MobileScanner(
                  key: qrKey,
                  controller: _controllerQrCode,
                  allowDuplicates: true,
                  onDetect: (barcode, args) {
                    setState(() {
                      _barcode = barcode.rawValue!;
                    });
                  }
                ),
              ),
          ),
        ],
      ),
    );
  }

  @override
  void dispose() {
    super.dispose();
    if (_controllerQrCode.isStarting) {
      _controllerQrCode.stop();
    }
    _controllerQrCode.dispose();
  }
}

Console log just after scan :
D/DeferrableSurface(31878): surface closed,  useCount=1 closed=true androidx.camera.core.SurfaceRequest$2@17201a2
D/Camera2CameraImpl(31878): {Camera@9c78a7e[id=0]} Use cases [androidx.camera.core.Preview-e4a61e9f-faf2-42b5-b483-c4c2be8ed6e3225272625, androidx.camera.core.ImageAnalysis-e90d6560-811c-4518-9038-9fd9b81e596c78797334] now DETACHED for camera
D/DeferrableSurface(31878): surface closed,  useCount=1 closed=true androidx.camera.core.impl.ImmediateSurface@1ec4769
D/UseCaseAttachState(31878): All use case: [] for camera: 0
D/Camera2CameraImpl(31878): {Camera@9c78a7e[id=0]} Resetting Capture Session
D/Camera2CameraImpl(31878): {Camera@9c78a7e[id=0]} Releasing session in state OPENED
D/UseCaseAttachState(31878): Active and attached use case: [] for camera: 0
D/UseCaseAttachState(31878): Active and attached use case: [] for camera: 0
D/Camera2CameraImpl(31878): {Camera@9c78a7e[id=0]} Closing camera.
D/Camera2CameraImpl(31878): {Camera@9c78a7e[id=0]} Transitioning camera internal state: OPENED --> CLOSING
D/CameraStateRegistry(31878): Recalculating open cameras:
D/CameraStateRegistry(31878): Camera                                       State
D/CameraStateRegistry(31878): -------------------------------------------------------------------
D/CameraStateRegistry(31878): Camera@38da318[id=1]                         UNKNOWN
D/CameraStateRegistry(31878): Camera@9c78a7e[id=0]                         CLOSING
D/CameraStateRegistry(31878): -------------------------------------------------------------------
D/CameraStateRegistry(31878): Open count: 1 (Max allowed: 1)
D/CameraStateMachine(31878): New public camera state CameraState{type=CLOSING, error=null} from CLOSING and null
D/CameraStateMachine(31878): Publishing new public camera state CameraState{type=CLOSING, error=null}
D/Camera2CameraImpl(31878): {Camera@9c78a7e[id=0]} Resetting Capture Session
D/Camera2CameraImpl(31878): {Camera@9c78a7e[id=0]} Releasing session in state CLOSING
E/BufferQueueProducer(31878): [SurfaceTexture-0-31878-25](id:7c8600000063,api:4,p:947,c:31878) queueBuffer: BufferQueue has been abandoned
D/CaptureSession(31878): onSessionFinished()
E/BufferQueueProducer(31878): [SurfaceTexture-0-31878-25](id:7c8600000063,api:4,p:947,c:31878) cancelBuffer: BufferQueue has been abandoned
D/UseCaseAttachState(31878): Active and attached use case: [] for camera: 0
D/DeferrableSurface(31878): use count-1,  useCount=0 closed=true androidx.camera.core.SurfaceRequest$2@17201a2
D/DeferrableSurface(31878): Surface no longer in use[total_surfaces=2, used_surfaces=1](androidx.camera.core.SurfaceRequest$2@17201a2}
D/DeferrableSurface(31878): Surface terminated[total_surfaces=1, used_surfaces=1](androidx.camera.core.SurfaceRequest$2@17201a2}
D/DeferrableSurface(31878): use count-1,  useCount=0 closed=true androidx.camera.core.impl.ImmediateSurface@1ec4769
D/DeferrableSurface(31878): Surface no longer in use[total_surfaces=1, used_surfaces=0](androidx.camera.core.impl.ImmediateSurface@1ec4769}
D/DeferrableSurface(31878): Surface terminated[total_surfaces=0, used_surfaces=0](androidx.camera.core.impl.ImmediateSurface@1ec4769}
D/Camera2CameraImpl(31878): {Camera@9c78a7e[id=0]} CameraDevice.onClosed()
D/Camera2CameraImpl(31878): {Camera@9c78a7e[id=0]} Transitioning camera internal state: CLOSING --> INITIALIZED
D/CameraStateRegistry(31878): Recalculating open cameras:
D/CameraStateRegistry(31878): Camera                                       State
D/CameraStateRegistry(31878): -------------------------------------------------------------------
D/CameraStateRegistry(31878): Camera@38da318[id=1]                         UNKNOWN
D/CameraStateRegistry(31878): Camera@9c78a7e[id=0]                         CLOSED
D/CameraStateRegistry(31878): -------------------------------------------------------------------
D/CameraStateRegistry(31878): Open count: 0 (Max allowed: 1)
D/CameraStateMachine(31878): New public camera state CameraState{type=CLOSED, error=null} from CLOSED and null
D/CameraStateMachine(31878): Publishing new public camera state CameraState{type=CLOSED, error=null}

Thank in advance for the help. Don't hesitated to ask me some information if needed !

Any updates?

I faced this exact issue, issue that was not present in a previous version of my application.
Cause.
I pinpointed it to the addition of a GlobalKey used to manage a Scaffold's drawer in my widget tree.
I tried adding a GlobalKey to the MobileScanner widget exactly like in your code, @launay12u, and it caused the same issue.
I haven't thoroughly tested it, but I suspect the presence of a GlobalKey on the MobileScanner widget, or any of its parent in the widget tree, causes the issue when setState() is called.
Workaround: not using setState().
You can just avoid using setState() but in my case it was not an option since I have UI updates to display after each scan.
Workaround: removing every GlobalKey from the widget tree.
I fixed the issue by conditionning my Scaffold's GlobalKey to not be used on my scanner screen, since the drawer is not used on the screen it was not an issue to remove the key entirely.
So, if you don't use your GlobalKey(s) on your scanner screen, you can juste remove them or have them behind a condition. There might be other workarounds if you need the key, one of them is to build a new MobileScannerController after each scan, this solution is not ideal since it still causes the screen to freeze for a split second after each scan.

Since there has been lots of updates, please check with latest version 3.0.0-beta.4 and comment if this issue still exists.


[fix] Container.add() 'filters' and 'rewriters' option passing to logger

In "Container.add(id, options)" every key in 'options' is tried to be resolved to an existing transports id and an error is thrown if this was not possible (line 68).
These options are directly passed as constructor parameter to the created Logger instance. Therefore, to be able to support setting 'filters' and 'rewriters' options for the created Logger these settings have to be skipped (similar to the 'transports' option) during the namedOptions resolution.

Thanks for your contribution!


Alternative keypoints

Hello,
I have a few questions regarding the project. I'm struggling to find how model finds the keypoint values. I found with some code tinkering that you get 10 keypoints as per config that you then pass to generator along with jacobian values of those 10 keypoints if I'm not mistaken.
Do you have pretrained model for keypoint extraction or is that learned behavior in training?
Also, if I wanted to plug in my own 'keypoints' obtained from pretrained alternative keypoint extractor and train on those, would something like that be easily doable? And how long did training( (for 100 epochs?) take you on 4 GPUs approximately?
Finally, a bit unrelated, using your video preprocessing steps, I prepared vox celeb mp4s and even tried a few videos in png format, but cannot find anywhere the pairs.csv mentioned in config file and in code, how do you obtain that file and is it necessary?
Thank you!

Keypoint detector is unsupervised, and learned end to end with an entire system. You can replace keypoint detector with pretrained one if you want.
Pairs.csv is not needed. I used them to specify files for evaluation.

Thank you for a quick reply!
Could you also share how long did training for 100 epochs last on VoxCeleb dataset approximately and what GPU config did you use? Also, did you use png or mp4 format?

@fpandza Did you able to train the model with custom keypoints detected from a separate model?
@AliaksandrSiarohin I want to replace the keypoints with some third-party model, like dlib. Can you suggest the necessary steps?
Thanks

Do you plan to estimate keypoints during training or prior to it?

I am planning to use dlib for estimating the keypoints and use them to train a motion model.
For every image, instead of training a unsupervised model for getting keypoints, I am thinking to use dlib.

Hi @AliaksandrSiarohin,
In krypoint_detector.py, I made some changes to create a key-point map of [1,68,58,58] where each 58x58 matrix has all zeros except one. The location where dlib keypoint is detected, is set to 1 in the above tesnor. Now when I try to train it, I got the error:
Traceback (most recent call last):
  File "run.py", line 81, in <module>
    train(config, generator, discriminator, kp_detector, opt.checkpoint, log_dir, dataset, opt.device_ids)
  File "/home/madhav3101/gan_codes/first-order-model-dlib/train.py", line 52, in train
    losses_generator, generated = generator_full(x)
  File "/home/madhav3101/env_gan/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/madhav3101/env_gan/lib/python3.7/site-packages/torch/nn/parallel/data_parallel.py", line 159, in forward
    return self.module(*inputs[0], **kwargs[0])
  File "/home/madhav3101/env_gan/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/madhav3101/gan_codes/first-order-model-dlib/modules/model.py", line 210, in forward
    jacobian_transformed = torch.matmul(transform.jacobian(transformed_kp['value']),
  File "/home/madhav3101/gan_codes/first-order-model-dlib/modules/model.py", line 117, in jacobian
    grad_x = grad(new_coordinates[..., 0].sum(), coordinates, create_graph=True)
  File "/home/madhav3101/env_gan/lib/python3.7/site-packages/torch/autograd/__init__.py", line 204, in grad
    inputs, allow_unused)
RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn

I guess, the error is occurring because keypoints detected by dlib are static and don't have anything to backpropogate. The jacobian is trying to get the gradient of it.
Any suggestion on how to solve this or any other approach to use dlib keypoints?

Set equivariance loss to zero.

Hi @AliaksandrSiarohin ,
Thanks for your suggestion. I set equivariance_value: 0 and equivariance_jacobian: 0. The code starts working. The perceptual loss also starts reducing.
I checked the visualization and it shows some strange output (image below). Can you suggest, what has gone wrong? Is it because of the horizontal_flip parameter or something else?

It is because x and y axis is inverted. You should invert your keypoints axis. Also you should not use horizontal flip, unless you use also flipping the keypoint.

@mdv3101 Can you share details or code how you implement it?  Thanks a lot.

Keypoint detector is unsupervised, and learned end to end with an entire system. You can replace keypoint detector with pretrained one if you want. Pairs.csv is not needed. I used them to specify files for evaluation.

@AliaksandrSiarohin  Can you explain how can I replace keypoint detecter with my own pretrained detector and use it with the remaining model. Basically I couldn't figure out how to estimate jacobian in that case.

You may not use jacobians.

@AliaksandrSiarohin wouldn't that decrease performance, if I don't estimate jacobians

Supervised keypoints will decrease it anyway.


Names of peers (currently only have IDs, hard to distinguish which is which)

Peer name exists in the client config which we should be able to fetch

Indeed, please close!


PowerToys run plugin for bing chat

Description of the new feature / enhancement
Be able to write questions directly to bing chat, like b:  and then it opens edge with https://edgeservices.bing.com/edgesvc/chat and searching for the question.
I do not understand why this is not just a standard feature.
Scenario when this would be used?
When i want to use the bing chat search instead of normal search engine.
Supporting information
No response

Hello and thanks for your message.

I do not understand why this is not just a standard feature.

Expectations? See, PowerToys is made by/with a lot of volounteers and Bing Chat is pretty new (at least for the publc). But, are you using it in the OS already?

/dup #24445


NetworkOperation / NSURLTaskOperation crashes if task.state is set to .Finished more than once

Some libraries (namely: AFNetworking) set the state for the NSURLSessionTask to Finished more than once.  NSURLTaskOperation attempts to unregister the KVO observer each time--crashing on the 2nd attempt.
I ended up using technique similar to the link below with a boolean flag and an NSLock.
https://github.com/pluralsight/PSOperations/blob/master/PSOperations/URLSessionTaskOperation.swift
I'd submit a PR, but I'm wondering if there's something I'm missing?
Side note: NetworkOperation.swift needs to be renamed to NSURLTaskOperation.swift

Hi @mooshee - thanks for the bug report! I don't use AFNetworking/Alamofire so, didn't know about this.
If you submit a pull request for the bug fix, can you branch from swift/2.1 and send the PR back to that branch? (And only use Swift 2.1 code). Also please do not change the name. This can then be released in version 2.8.3.
I can then pull the changes into development and will consider changing the name in a separate issue for the next release, 2.10.0.

This has been fixed, it will be released with v2.10, which can be tested using the branch at #298.

I apologize for not being to get this fix in.  I’ve been in a development frenzy.  This is the first time I’ve used this framework (for two brand new apps) and it’s been fantastic.  Thank you @danthorpe!

On Apr 26, 2016, at 3:30 AM, Daniel Thorpe<EMAIL_ADDRESS>wrote:
This has been fixed, it will be released with v2.10, which can be tested using the branch at #298 https://github.com/danthorpe/Operations/pull/298.
—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub https://github.com/danthorpe/Operations/issues/262#issuecomment-214653602


Kapacitor receives data from InfluxDB, but task doesn't

I've been trying to get Kapacitor alerts working, starting with the sample cpu_alert task, but I've run into some problems.
I already have data being written by Telegraf to an InfluxDB database called telegraf:
> use telegraf
Using database telegraf

> select count("usage_idle") from cpu;
name: cpu
---------
time	count
0	263790

> select count("usage_idle") from cpu;
name: cpu
---------
time	count
0	263799

After starting kapacitord, Influx tells me:
> show subscriptions
name: telegraf
--------------
retention_policy	name						mode	destinations
autogen			kapacitor-0a5c0b88-3ea0-466b-b54b-f65ac5d6f2ba	ANY	[http://localhost:9092]

And Kapacitor seems to be receiving the data from InfluxDB:
$ kapacitor stats ingress
Database                      Retention Policy              Measurement                   Points Received
...
telegraf                      autogen                       cpu                           72
...

However:
$ kapacitor show cpu_alert
ID: cpu_alert
Error:
Template:
Type: stream
Status: enabled
Executing: true
Created: 13 Sep 16 17:28 CEST
Modified: 14 Sep 16 09:32 CEST
LastEnabled: 14 Sep 16 09:32 CEST
Databases Retention Policies: ["telegraf"."default"]
TICKscript:
stream
    |from()
        .measurement('cpu')
    |alert()
        .crit(lambda: "usage_idle" < 99)
        .log('/tmp/alerts.log')

DOT:
digraph cpu_alert {
graph [throughput="0.00 points/s"];

stream0 [avg_exec_time_ns="0" ];
stream0 -> from1 [processed="0"];

from1 [avg_exec_time_ns="0" ];
from1 -> alert2 [processed="0"];

alert2 [alerts_triggered="0" avg_exec_time_ns="0" crits_triggered="0" infos_triggered="0" oks_triggered="0" warns_triggered="0" ];
}

Running on Debian 8, on the same machine, Kapacitor and InfluxDB are both at the currently latest available versions from the Debian repository.

The retention policy is different. The task is expecting telegraf.default while Kapacitor is receiving telegraf.autogen.

Aha. Somehow I assumed that default would mean the database's configured default RP.  Working perfectly now – thanks so much!

I found it today .


Missing tokenizer for RedPajama-INCITE?

When I was trying to run the evaluation of RedPajama-INCITE models from the Huggingface integration, I got an error of could not find the tokenizer.
helm.benchmark.executor.ExecutorError: Could not find tokenizer for model: togethercomputer Request: Request(model='togethercomputer/RedPajama-INCITE-Instruct-3B-v1', embedding=False,
...

The command to reproduce this is helm-run  --enable-huggingface-models togethercomputer/RedPajama-INCITE-Instruct-3B-v1 --run-specs "mmlu:subject=anatomy,model=togethercomputer/RedPajama-INCITE-Instruct-3B-v1" --suite v1 --max-eval-instances 10
I also tried evaluating a local version of RedPajama-INCITE models, but the tokenizer issue persists, showing a slightly different breakdown in initializing the HF model (although I do have tokenizer_config.json and tokenizer.json in my model path).
helm.benchmark.executor.ExecutorError: __init__() missing 1 required positional argument: 'tokenizer'

Any suggestion on how to overcome it?

Thanks for the bug report; hopefully we can have a fix out this week. cc @JosselinSomervilleRoberts

@KaiserWhoLearns I am currently looking at this but just so you know the model you are mentioning is supported in our config. You can directly use: helm-run --run-specs "mmlu:subject=anatomy,model_deployment=together/redpajama-incite-instruct-3b-v1" --suite v1 --max-eval-instances 10

Regarding the huggingace error, I was not able to run it due to my limited GPU, but it worked on smaller models helm-run  --enable-huggingface-models EleutherAI/pythia-70m-deduped --run-specs "mmlu:subject=anatomy,model=EleutherAI/pythia-70m-deduped" --suite v1 --max-eval-instances 20   so I am not sure what is going on here. COuld you provide more context maybe?

@JosselinSomervilleRoberts Thanks. I tried running with the Pythia model on my end and obtained the same issue (TypeError: __init__() missing 1 required positional argument: 'tokenizer'). Perhaps I am running with an old version (crfm-helm==0.3.0)?

@KaiserWhoLearns thanks for this. I checked and the bug is indeed present on 0.3.0 but not main. This will be fixed in 0.4.0 which should be released soon (hopefully in the next few days). If you cannot wait, you an clone the repo and use HELM this way.

I will close this when 0.4.0 comes out (https://github.com/stanford-crfm/helm/pull/2023).


consul port is 0 and traefik.port is ignored

version: docker  2.2.0-rc4

"traefik.enable=true",
"traefik.frontend.rule=Host:api.something",
"traefik.traefik.backend.healthcheck.port=7081",
"traefik.port=7081",
"traefik.loadbalancer.server.port=7081"

the host rule is picked up fine, but port is ignored, leaving backend listed as "http://<IP_ADDRESS>:0"
i.e. port zero
logs:
msg="Configuration received from provider consulcatalog: {\"http\":{\"routers\":{\"odb-liberator\":{\"service\":\"odb-liberator\",\"rule\":\"Host(`odb-liberator`)\"}},\"services\":{\"odb-liberator\":{\"loadBalancer\":{\"servers\":[{\"url\":\"http://<IP_ADDRESS>:0\"}],\"passHostHeader\":true}}}},\"tcp\":{},\"udp\":{}}"

Hi! I'm Træfiker :robot: the bot in charge of communication regulation.
Thanks for your interest in Traefik!
Issue templates help us help you by providing all necessary information.
Please edit your issue and use the available templates:

Feature request
Bug report

And remember: each time someone ignores the template, a cute little bunny dies.

#SaveTheCuteBunny :heart: :rabbit: :heart:

Hello,
You are mixing Traefik v1 and Traefik v2 labels.
Could you go the community forum, thank you.


Split date and time over multiple Input-Fields

I have a form which asks for Hours and Minutes and Month and Day in separate fields. Is it possible to setup pickadate to split the output of pickadate / time over two fields?

correction

Anybody have a solution yet?


Malformed .in results in uncaught exception

Lets see how pip-compile performs given a purposefully malformed .in file. Normally would expect a pip-compile specific Exception and a non-zero exit code.
malformed-pip.in
>=24pip\n
Which is obviously malformed. Since a .in is user input, it can contain errors.
pip has vendored package, packaging. When packaging fails to parse a requirement, it produces this Exception chain
pip._vendor.packaging._tokenizer.ParserSyntaxError
pip._vendor.packaging.requirements.InvalidRequirement
pip._internal.exceptions.InstallationError

pip-compile does not gracefully catch the InstallationError. Instead just bombs with exit code 1 and the below traceback.
Environment Versions
OS: Linux distro Void Linux LXDE
Python version: Python 3.9.16
pip version: pip 24.3.1
pip-tools version: pip-compile, version 7.4.1
Steps to replicate
echo ">=24pip\n" > malformed-pip.in
pip-compile --allow-unsafe --no-header -o malformed-pip.txt malformed-pip.in
Expected result
pip-compile should catch the Exception in a try-except block. Exception is reraised with a pip-compile specific Exception. The entire traceback does not get shown.
Taken from the traceback, handling of the Exception doesn't happen here.
piptools/scripts/compile.py", line 383, in cli
constraints.extend(

Actual result
Traceback (most recent call last):
File "[venv base path]/lib/python3.9/site-packages/pip/_vendor/packaging/requirements.py", line 36, in init
parsed = _parse_requirement(requirement_string)
File "[venv base path]/lib/python3.9/site-packages/pip/_vendor/packaging/_parser.py", line 62, in parse_requirement
return _parse_requirement(Tokenizer(source, rules=DEFAULT_RULES))
File "[venv base path]/lib/python3.9/site-packages/pip/_vendor/packaging/_parser.py", line 71, in _parse_requirement
name_token = tokenizer.expect(
File "[venv base path]/lib/python3.9/site-packages/pip/_vendor/packaging/_tokenizer.py", line 142, in expect
raise self.raise_syntax_error(f"Expected {expected}")
File "[venv base path]/lib/python3.9/site-packages/pip/_vendor/packaging/_tokenizer.py", line 167, in raise_syntax_error
raise ParserSyntaxError(
pip._vendor.packaging._tokenizer.ParserSyntaxError: Expected package name at the start of dependency specifier
>=24pip
^
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "[venv base path]/lib/python3.9/site-packages/pip/_internal/req/constructors.py", line 362, in _parse_req_string
return get_requirement(req_as_string)
File "[venv base path]/lib/python3.9/site-packages/pip/_internal/utils/packaging.py", line 45, in get_requirement
return Requirement(req_string)
File "[venv base path]/lib/python3.9/site-packages/pip/_vendor/packaging/requirements.py", line 38, in init
raise InvalidRequirement(str(e)) from e
pip._vendor.packaging.requirements.InvalidRequirement: Expected package name at the start of dependency specifier
>=24pip
^
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "[venv base path]/bin/pip-compile", line 8, in
sys.exit(cli())
File "[venv base path]/lib/python3.9/site-packages/click/core.py", line 1157, in call
return self.main(*args, **kwargs)
File "[venv base path]/lib/python3.9/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
File "[venv base path]/lib/python3.9/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "[venv base path]/lib/python3.9/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
File "[venv base path]/lib/python3.9/site-packages/click/decorators.py", line 33, in new_func
return f(get_current_context(), *args, **kwargs)
File "[venv base path]/lib/python3.9/site-packages/piptools/scripts/compile.py", line 383, in cli
constraints.extend(
File "[venv base path]/lib/python3.9/site-packages/piptools/_compat/pip_compat.py", line 77, in parse_requirements
yield install_req_from_parsed_requirement(parsed_req, isolated=isolated)
File "[venv base path]/lib/python3.9/site-packages/pip/_internal/req/constructors.py", line 480, in install_req_from_parsed_requirement
req = install_req_from_line(
File "[venv base path]/lib/python3.9/site-packages/pip/_internal/req/constructors.py", line 405, in install_req_from_line
parts = parse_req_from_line(name, line_source)
File "[venv base path]/lib/python3.9/site-packages/pip/_internal/req/constructors.py", line 379, in parse_req_from_line
req: Optional[Requirement] = _parse_req_string(req_as_string)
File "[venv base path]/lib/python3.9/site-packages/pip/_internal/req/constructors.py", line 376, in _parse_req_string
raise InstallationError(msg)
pip._internal.exceptions.InstallationError: Invalid requirement: '>=24pip': Expected package name at the start of dependency specifier
>=24pip
^ (from line 1 of /tmp/pytest-of-chaosmonkey/pytest-24/test_compile_malformed_in_mix_0/requirements/malformed-pip.in)

Here is the code excerpt. Since the unexpected Exception is not caught, the exit code is not 2.
The return variable name is metadata. The try-except block doesn't catch the exception, so it looks like metadata gets assigned!
When metadata.requirements is accessed, somewhere, a Generator is executed and raises the Exception, resulting in a nasty repetitive traceback.
elif is_setup_file:
            setup_file_found = True
            try:
                metadata = build_project_metadata(
                    src_file=Path(src_file),
                    build_targets=build_deps_targets,
                    attempt_static_parse=not bool(build_deps_targets),
                    isolated=build_isolation,
                    quiet=log.verbosity <= 0,
                )
            except BuildBackendException as e:
                log.error(str(e))
                log.error(f"Failed to parse {os.path.abspath(src_file)}")
                sys.exit(2)

            if not only_build_deps:
                constraints.extend(metadata.requirements)

Generator is a curse word!
There are three cases in the if-elif-else block calling the Generator that can produce an Exception. The Exception can occur on every constraints.extend(gen_reqs) call.
piptools/scripts/compile.py lines: 366, 383, and 390
The solution is to not do that.
parse_requirements and metadata.requirements must be brought into ONE try-except block. Rather than three.
I get it. The code is easy to read, but that comes at the expense of not being DRY.


Use a Supplier to construct objects from id.

Provide lambda for default constructing frequently used objects to avoid reflection.

@bjorncs PR

or @bratseth


access model via sputnik

I'm a little bit concerned about the API difference we're getting here.
Before there was:
Language.from_dir
We never really made this convenient, because of the awkwardness of installing the data in different places.
But now we have:
Language.from_package
So the user has to instantiate this package object.
Given that this is the library entry-point, we need to have something a bit easier. Is it possible to have something like this:
nlp = spacy.en.English('<some resource descriptor>')
nlp = spacy.Language('en', '<some resource descriptor>')

Inside English.__init__, we would try to create a Package object from the string, if a string is what the user gave us. If the user gives us a Package object we continue on our way. Fallback to default if nothing specified.

a model can now be specified:

by a path to the model directory (DEPRECATED)

Language(data_dir='path/to/data')

by a language identifier (and optionally a package root dir)

Language(lang='en')
Language(lang='en', data_dir='spacy/data')

by a model name/version (and optionally a package root dir)

Language(model='en_default')
Language(model='en_default', version='1.0.0')
Language(model='en_default', version='1.0.0', data_dir='spacy/data')

English() works the same, but with lang='en' preset.

Btw: is templates.json still un use?

Let me check again that I've understood how this works properly.
Let's say our back-end was a DB. Would this data model work?

Root resource is the table
Each model is a row
Model version is the primary key
Model language is a column
Boolean column package_default
Boolean columns for en_default, it_default, fr_default etc

There's a global default root resource: a directory data/ that's a sibling to spacy.__init__.__file__. But you can point to a different root resource instead. In future functionality, the root resource need not be a file-system.
Within the root resource, models are uniquely identified by version string. You can also ask the resource to supply its default model, or the model that's default for a given language.
You can't ask for version X of language Y, because there's only one model with version X, and it has exactly one language.
Question: Does the same root resource know about models for different spaCy versions? If so, does spaCy (silently) send its version number, as part of the "SELECT" statement it's making for the model?

Yes, your database design would almost work, small correction: you forgot the model name and we assume that all models you can view are ones that are compatible with your current spaCy installation (does this answer already your question?).
Hence, within the root resource, models are uniquely identified by name and version string. This means there could be also models that are not default model in any language, but still addressable by name/version.
In future functionality root resource could be also outside the filesystem (e.g., REST API), but we currently lack a mechanism to flexibly specify such resources. I overloaded the semantics of data_dir specifically not to invent such a scheme, but to reuse the path-spec that already existed.

Okay so it's more like:

The root resource is the DB connection
The spaCy version currently running specifies the table
The model name is a string that serves as the primary key

Is that more like it?

Yep, that works.

Okay, so does this make sense?
Language(model='en_default', version='1.0.0')

The version is specified by the spaCy code currently executing, isn't it?

Yes, it does. There could be still multiple compatible versions of the same model. Example: you created a model that contained a "bug". The bugfix doesn't require to release a new spaCy version, but only a new model version that ships as new default. At the same time you might want to pin your spaCy instance to a particular model version to avoid changes in your application even if a new model version is released. Hence, there is still the need for selecting a model by version. The check for the spaCy version compatibility is there to avoid mistakes and offering functioning upgrade paths.

Hmm. It'd be pretty nice to drop the extra parameter, though!
Why not just concatenate the version onto the model string, if a user wants to specify a version constraint?
Structured strings aren't a great solution, but this is how it works for semver, and there might be a use-case for specifying >= , <, etc.
Scenario: Let's say you're building something, and you have two teams. One team develops new models for spaCy, e.g. with new word vectors, new named entity features for your domain, etc. The other team consumes those models. The first team wants to just crank out models, but the second team might not be ready to consume them right away, e.g. a new model might introduce new entity types, or it might cause you to retrain your models.

Agree, this is much cleaner (see new commit). Sputnik already works this way internally, hence it was easy to change.


remove guava

I used openrewrite to remove most usages of guava.  The loadbalancer project uses a loading cache which we need to keep in place but I reduced the transitive graph by only pulling in guava in that subproject.

Is there a process for a review and/or merge for this project?


Pass directory when launching win32 app; Allow launching win32 apps with admin permissions

This PR contains two fixes.

Pass directory when launching win32 apps.

I have a game which crashes if the working directory isn't the same as the directory the executable is in. This change always passes that directory as the working directory when launching win32 apps.

Allow launching win32 apps with admin permissions.

I have a game which requires admin permissions. If I manually add it to steam it works (windows pops up the UAC prompt for me to accept, then launches the game). But when launched through GlosSI, it fails to start up.
It turns out CreateProcessW was failing with ERROR_ELEVATION_REQUIRED (error code 740).
If CreateProcessW fails with that error, I use ShellExecuteExW with the runas verb to launch the game. This way windows shows the UAC prompt and the game can run with admin permissions.
With this change, games that don't require admin permissions are still launched with CreateProcessW. It's only if a game fails to launch that I fall back on ShellExecuteExW.
This fixes #205

Thank you very much for your contribution!
Fix 2 looks good to me.

I have issues with "Fix" 1, however.
As you may or may not have noticed, Passing the path to "createProcess" was in GlosSI once, but I took it out to allow user customization in Steams shortcut properties.
See: https://github.com/Alia5/GlosSI/commit/1a47ef8c351570681516485d63039d6f580fd389
It's been like that for ~15 months and never caused issue to my knowledge.
You could have got that from the git-blame btw ;P
So please take this patch out of this PR.
If you have a fringe case where this does cause issue, though, please open a separate PR for this and provide a json config-parameter for this.

Additionally, please rebase this PR onto develop

Thanks for such a quick review! Oh cool, I didn't realize that if I set "Start In" in the steam shortcut that gets used.
I removed the launch directory stuff.

I'm confused on how to apply this fix

Download the latest snapshot build...


add sitemap generation

Add sitemap generation, which is used by search engines (google, yandex, bing, etc) to find information on the site faster.
This allows you to quickly send information about a new article, so that it would be indexed.

Like this

Like this


feat: add timewindow to calculation

According to the status page (https://www.githubstatus.com/) , there are problems with workflows. Perfect.


653 exporting save name

Describe your changes

Added --persist_snippets config to SqlMagic
if args.save and --persist_snippets is True, snippet is saved in a new directory jupysql-snippets/ for the users to access.
Refactored such that only relevant part of the snippet is stored in .sql file.
Add a user-warning-message to not edit the sql files directly but only through the notebooks
If jupysql-snippets exists, we automatically load them as snippets

Issue number
Closes #653
Checklist before requesting a review

[x] Performed a self-review of my code
[x] Formatted my code with pkgmt format
[x] Added tests (when necessary).
[x] Added docstring documentation and update the changelog (when needed)

#653-demo.ipynb.zip

:books: Documentation preview :books:: https://jupysql--741.org.readthedocs.build/en/741/

@edublancas I have added the functionalities required in the issue.
I will add the test cases and make changes to the doc if it is approved.
I have added the demo notebook for testing

please review @yafimvo @neelasha23

What's the expected behaviour for this usecase:

User sets %config SqlMagic.persist_snippets = True and saves a snippet say cte_one.
User opens another notebook where persist_snippets is not set or is set to False, and user saves another snippet with same name cte_one.

Do we ignore this case / modify the original snippet / warn user that there's a snippet named cte_one saved as sql file
@AnirudhVIyer @edublancas

What's the expected behaviour for this usecase:

User sets %config SqlMagic.persist_snippets = True and saves a snippet say cte_one.
User opens another notebook where persist_snippets is not set or is set to False, and user saves another snippet with same name cte_one.

Do we ignore this case / modify the original snippet / warn user that there's a snippet named cte_one saved as sql file
@AnirudhVIyer @edublancas

Hey,
Right now, the behaviour is as follows:

user sets %config SqlMagic.persist_snippets = True and saves a snippet as cte_one

In a new notebook or in the same notebook, user saves another snippet as cte_one with %config SqlMagic.persist_snippets = False , then cte_one will be overwritten in SQLstore for that session . This will not overwrite the snippet stored as SQL

No change will be made to the   jupysql-snippets/cte_one.sql file, change will only be made if %config SqlMagic.persist_snippets = True.

If the user goes back to the initial notebook, or creates a new notebook, then the snippet cte_one loaded automatically will be the original one from the sql file.

I think we can add these tests also

The snippet was saved and loaded but the table/data doesn't exist - we should expect an error here
.sql file exists and SqlMagic.persist_snippets = False - we should make sure we don't override the file and the relevant snippets are being used (I think it will be easier if we use parameters here)
Check this message "Manual editing of .sql files may not be..." we should make sure we display it only in certain cases

Sure! Adding the tests

@neelasha23 @yafimvo please review again and ensure you close conversations if your observations have been addressed

I think we can add these tests also

The snippet was saved and loaded but the table/data doesn't exist - we should expect an error here
.sql file exists and SqlMagic.persist_snippets = False - we should make sure we don't override the file and the relevant snippets are being used (I think it will be easier if we use parameters here)
Check this message "Manual editing of .sql files may not be..." we should make sure we display it only in certain cases

Hey @yafimvo added the tests related to these cases:
test_save_snippet_as_sql
test_snippet_not_saved
test_load_snippet_from_sql
test_load_snippet_no_table
test_snippet_sql_not_overriden
test_snippet_sql_message

@AnirudhVIyer please resolve merge conflicts

great work! I tested it and found that the second notebook now runs without issues. however, I encountered a caveat. these are the notebooks for testing: testing-persist-sql.zip
if you run first.ipynb (to store the snippets), you'll see that the latest snippet is rendered as:
WITH `penguins_all` AS (
select * from penguins)
select * from penguins_all
where sex = 'MALE'

which is right. then, if you run second.ipynb, you'll see that the snippet contents are different:
WITH `penguins_male` AS (WITH `penguins_all` AS (
select * from penguins)
select * from penguins_all
where sex = 'MALE')
select * from penguins_male
where body_mass_g > 4200

there is a nested WITH. and even though this runs on DuckDB, it might not run in all databases. so the objective is that in second.ipynb, the contents of the snippet must be the same (no nested WITH).
the problem is that you're storing the full snippet, and then retrieving it as is. to fix it, you can write a parser that writes the snippet along with some markers so you can retrieve both the original contents and its dependencies.
e.g.,
/* This SQL file was produced by JupySQL, do not edit directly. Edit the notebook where this snippet was defined */
/* {"with_": ["first", "second"]} */

with first as (
    select * from table
), second as (
    select * from first where x > 10
),

/* SNIPPET BEGINS */
select * from second where y > 100
/* SNIPPET ENDS */

when reading the snippet, you can parse it so you now that the with_ argument should be with_=["first", "second"] and the content of the snippet should be
select * from second where y > 100

note that the {"with_": ["first", "second"]} string is valid JSON, you can serialize and deserialize with the built-in jsonlibary
since this will require a lot of new logic, please create separate functions so you can test them independently of the rest of the logic

@edublancas changed the parsing logic, now we store the snippets in .sql files differently.
Now there is no nesting.
Before:

After:


Support for Multi-column imputation

@JuliaRegistrator register()

Registration pull request created: JuliaRegistries/General/5292
After the above pull request is merged, it is recommended that a tag is created on this repository for the registered package version.
This will be done automatically if Julia TagBot is installed, or can be done manually through the github interface, or via:
git tag -a v2.4.0 -m "<description of version>" 830cfcbb8884e3bf983e14c04269ef0ae3bbb36b
git push origin v2.4.0

PR #98


Enhance Object creation

This is a ~WIP~ PR to address #253.
On my TODO list I've still got the following to get through:
~* Update the AI to handle the new object creation ops~
~* More test for lifting, splicing and AI type tracking~
~* Remove the legacy object generators~
~* Potentially have additional generators for props, getters and setters.~ (Not possible with the current model)
Feel free to add your thoughts to these initial changes and I'll incorporate them as we progress with this PR.

@saelo this PR is ready for review, let me know your thoughts on the modelling and the handling of splicing in restricted contexts please. If you're happy with this approach, I'll go ahead and get started with implementing #8 using the same approach.

This is now implemented with https://github.com/googleprojectzero/fuzzilli/commit/840c09e07f342c2210694ececd23ef8ac3d669d6 so I'm closing this PR. Let me know if you see any features still missing! Thanks!


Databricks Workflows GA

Hi,
I build a data vault model for a customer using dbt and want to use the Databricks workflow integration as mentioned here: https://github.com/databricks/dbt-databricks/blob/main/docs/databricks-workflows.md
Is there a way to apply for the private preview, or can you estimate, when the feature becomes public preview or GA?
Best,
Nicolai

Hi @nicolai92. It's going into public preview this week for workspaces which have access to serverless warehouses.

+1 on what @KoningJasper said. Closing this issue out


Project Name Collides with Pre-existing "Depthcharge" Coreboot Payload for Chromebooks

Overview
It quickly came to my attention that in my attempt to be clever with the whole "Sinking U-Boots with Depthcharge" pun, I've inadvertently selected a project name that collides with the depthcharge Coreboot payload that's the successor to U-Boot on Chromebooks.
Personally, I am really attached to the name and part of me is really hoping the two identically named projects can coexist separately.  However, I understand that this can definitely lead to some confusion having two projects in the world that use the term "payload" and related to (but not affiliated with) U-Boot.
As such, I just wanted to create a tracker item to publicly recognize this, gather my thoughts, mull over things a bit, and provide a single thread where any relevant stakeholders can easily make their thoughts known... if it's even remotely an issue to them.
Was this intentional?
No.  I went through a bunch of really terrible names over the course of months, and this was the one that I finally was excited with.
I would like to be a good FOSS citizen and certainly do not wish to cause confusion for aforementioned projects.
Post Mortem - Jon! How did you manage to miss this?
When going through half a dozen (really terrible) project name candidates, I did honestly try to avoid collisions.  Before finalizing on "Depthcharge" I was looking for other Python modules in PyPi and Linux distributions' package repositories, as well as subdomains on readthedocs.io.
My oversight appears to have been being so focused on the Python-centric and name availability in packaging/doc/distribution aspects that I failed to actually look for " U-Boot"  In retrospect -- duh, Jon.
Who had the name first?
The other "depthcharge".
For now, I'm keeping the name as-is.  It's a small project and an initial body of work that I plain to maintain and continue developing.  In comparison to an 8 year-old codebase in a major FOSS project -- this is small potatoes and may not even be a blip on their radar.  I certainly would like my work to grow and evolve, and thus feel it's important to at least have some sort of a plan.
At this time, I don't think the disambiguation is too hard...
Are you a security researcher analyzing and attacking U-Boot? You want this code.
Are you doing something with a Chromebook or Chromium project? This is not the code you're looking for.
If users of begin expressing actual confusion beyond "lol smooth move Jon" - I could together some disambiguation blurbs in the README and front page that redirects people looking for the Coreboot payload to the correct pages.  (However, this could backfire and allow search engines to pick up more in bound links, only exacerbating the problem I've created.)
If it becomes painfully apparent that I am indeed creating significant consternation for the Chromium / Coreboot folks -- I will plan to transition the project to a new name before creating a 1.0.0 stable API release.
Suggestions are welcome. Amusing nautical puns are strongly encouraged.

Has not been an issue; closing this.


Release version0.38.0 of sqlparser-rs

Follow on to https://github.com/sqlparser-rs/sqlparser-rs/issues/925
Filing a ticket here to track the work of a release (mostly so others can follow along)
Target Release Date: Approximately 2023-09-23
Release instructions: https://github.com/sqlparser-rs/sqlparser-rs/blob/main/docs/releasing.md
Steps:

[x] Make a PR to update CHANGELOG.md: https://github.com/sqlparser-rs/sqlparser-rs/pull/973
[x] run cargo release
[x] verify the new release is on crates.io https://crates.io/crates/sqlparser/0.38.0
[x] Make ticket for next release: https://github.com/sqlparser-rs/sqlparser-rs/issues/974

The release is on crates.ip: https://crates.io/crates/sqlparser/0.38.0 🎉


package aiohttp got an error

ERROR: Package 'aiohttp' requires a different Python: 3.5.2 not in '>=3.5.3'
This shouldn't happen, please check the version dependency.
And
Do you have moneyflow data from tencent?
Thanks.
/Patrick

close by d2930a035e2d5ea1c2fcbaf1a4cb7a0e94c935ae


preview - Previews the video over the specified range.

Only turned this into an issue to test the GitHub feature, this doesn't need to be its own issue.