Retaining original filename

There are cases when it's important to retain the original filename rather than a UUID. I see there is a pretty_location plugin that uses the UID, but no way of using the original filename?
require 'image_processing/mini_magick'

class ImageUploader < Shrine
  include ImageProcessing::MiniMagick
  plugin :store_dimensions
  plugin :processing
  plugin :versions
  plugin :delete_raw

  process(:store) do |io, context|
    original = io.download

    large = resize_to_limit!(original, 800, 800)
    medium = resize_to_limit(original,  500, 500)
    small = resize_to_limit(original,  300, 300)

    { original: io, large: large, medium: medium, small: small }
  end
end

Shrine stores metadata for each uploaded file, so since the original file is not kept it has nowhere to store the original filename (it wouldn't make sense to store it one of the versions because it's not their original filename).
If you would like to retain the original filename in this case, I would recommend using the metadata_attributes plugin to store the filename of the original file in a separate column.
class ImageUploader < Shrine
  plugin :metadata_attributes, filename: :original_filename
end

With the above configuration you just have to create the <attachment>_original_filename column on the model, and Shrine will automatically synchronize it whenever a new file is assigned or removed.


HP Pavilion Slimline S5520BR hangs when executing _PTS(5)

It seems that the SMI handler somehow gets confused and hangs (ACPI method is named ESMI, and IO port 0xb2 is often used to initiate a SMI.

Can you share a dump of your ACPI tables?
I wonder why opcode 0x70 (Store) produces both a read and a write.

Could you check that #146 fixes this?


Remove mobile client specific changes

Starting from March/April store release and client SDK v 1.4.1 mobile fully supports auth. As discussed in a thread, we can remove this reference and capture the same in a blog post.

Docs Build status updates of commit 81162c7:
:warning: Validation status: warnings

File
Status
Preview URL
Details

msteams-platform/concepts/authentication/auth-flow-bot.md
:warning:Warning

Details

msteams-platform/concepts/authentication/auth-flow-bot.md

Line 46: [Warning] Invalid file link:(~/concepts/authentication/auth-bot-AAD).
Line 30: [Suggestion] Illegal link: as described below -- missing bookmark. The file concepts/authentication/auth-flow-bot.md doesn't contain a bookmark named Mobile-clients.

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

I agree with the deletion but we need to add the SDK version note and I think we should keep the old fallback URL content around in another page for a while and call the fallback URL deprecated. It still works. And as Andrew pointed out we need to update the sample.

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.:x: jjustinms sign nowYou have signed the CLA already but the status is still pending? Let us recheck it.

Docs Build status updates of commit 27f6b54:
:warning: Validation status: warnings

File
Status
Preview URL
Details

msteams-platform/concepts/authentication/auth-flow-bot.md
:warning:Warning

Details

msteams-platform/concepts/authentication/auth-flow-bot.md

Line 49: [Warning] Invalid file link:(~/concepts/authentication/auth-bot-AAD).

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

Docs Build status updates of commit be3c116:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

msteams-platform/concepts/authentication/auth-flow-bot.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Research VLA to VL mapping

Research (both literature review and architectural scoping) around utilizing VLA models for Vision-Language or sole Language fine-tuning and inference.

Octo
Architecture
* Input tokenizers that convert language, observations, and goals to tokens
* Language inputs - T5 base
* Image observations and goals - Convnet stack
* Add position embeddings to task and obs tokens and arrange them sequentially
* Tokens are passed through a Transformer backbone
* Attention is block-wise masked
* Obs tokens can only attend to obs tokens from previous tokens, and task tokens
* Readout tokens - attends to prev obs and task tokens, but is not attended to by any other tokens. Analogous to CLS
   token in BERT
* Lightweight action head that implements diffusion process applied to readout embedding, and predicts chunk of
   consecutive actions
* Can flexibly add alternate output heads to the embeddings from Transformer backbone

Training objective
  * Conditional diffusion decoding head that predicts continuous multimodal actions
  * Only one forward pass of transformer backbone per action prediction. Multi-step denoising process carried out
    entirely in the diffusion head

OpenVLA
Architecture
* Prismatic-7B VLM backbone
* 2 part vision encoder - pre-trained SigLIP and Dinov2
   - input images are passed through both encoders and feature vectors from both are concatenated channel-wise
* Llama 2 for language
* Custom ActionTokenizer class that extends regular PreTrainedTokenizerBase to handle continuous robot actions. It
   discretizes continuous actions into bins, maps them to tokens, and provides methods to convert between actions
   and tokens. This allows the integration of continuous robot actions into the discrete token space of language
   models, enabling the model to work with both language and robot control in a unified manner.

Training objective
 * Fine-tune Prismatic for robot action prediction:
 * Formulate action prediction as a vision-language task
 * Use VLM backbone to predict actions by mapping continuous actions to discrete tokens used by LLM tokenizer
 * Llama tokenizer in the VLM backbone only reserves 100 special tokens for newly introduced tokens during fine-
    tuning, but OpenVLA uses 256 tokens for action space discretization
 * To overcome this, OpenVLA overwrites the 256 least used tokens to use as action tokens
 * Once actions are processed into seq of tokens, they are trained using regular next token prediction with cross-
   entropy loss

Architectural modification requirements:

Adapt OpenVLA for Vision-Language and just Language
Adapt OpenVLA for discrete action space - ActionTokenizer only works for continuous actions as of now


fix: Add a nudge in home for people still using org.openfoodfact.app fixes #2979

What
Old deprecated app fix

added the code to check if a user is using an older version of the app and a button to update the app

Screenshot

Fixes bug(s)
https://github.com/orgs/openfoodfacts/projects/61/views/1

Fixes: People who were still using the org.openfoodfact.app

Part of

#2979

it seems the original photo example is not what is wanted so it needs to be updated for me to make the correct changes in this PR

In germany we have a saying which would translate to "That was a rough birth" thats applicable here. Still thanks for your effort @juliethangeldeveloper 🥳

@M123-dev  Just to add on to make sure I clarify most of the code was provided for me when I join the open source talks. In this code, the keyboard as was in there and I just added to display on the screen. That's why all the comments did not make sense. Thanks for closing it and resolving the issues


Added new keys to flow

Closes #149
Summary
Added keys needed for of_multi_table
Local Tests
Installing pipelines
End-to-End Tests
Checking if needed

I am going to add the script here.


Replace provisional S3 C3 product to final

Replace provisional S3 C3 products to final S3 C3 products.
ga_s2am_provisional_ard_3 -> ga_s2am_ard_3
ga_s2bm_provisional_ard_3 -> ga_s2bm_ard_3
I have verified by eye that the outputs do not change for the changed notebooks. The automated 'GitHub Actions' check was  successful when it ran on the draft PR.

Awesome thanks @geoscience-aman ! I've asked a question about when the main S2 datasets get updated with NRT data on teams here; depending on that I might jump into this branch and make some small changes to the Exporting_satellite_images.ipynb notebook.


missing

license
readme structure
contributing guidelines

labels bhi

completed all above parts as you explained in the meet.


Changed to use CallKit instead of deprecated API.

Changed to use CallKit instead of deprecated API.

Hi @dowatugkins,
I was looking at your changes. Will this break support for iOS 9 and before, or is there something I'm not seeing?.
Thanks.

@soledaddiez It would break iOS 9 support. This could be modified to check for an iOS version and then implement either call kit or call telephony depending. As for me, I don't really have time to make that change.


command line  /usr/lib/zabbix/alertscripts/zbxtg.py  on zabbix 3.4

hello script not work after send this command
/usr/lib/zabbix/alertscripts/zbxtg.py "test" "hello" "how are you?"
error:
Traceback (most recent call last):
File "/usr/lib/zabbix/alertscripts/zbxtg.py", line 9, in
import requests
ImportError: No module named requests
[Exit 1]
any solution?

Install the Python library named Requests

Ok now we have install on my centos 7 with command  "pip install requests"
but I have same problem


Fixes the lifecyclepolicy problem reported at Issue #953

These are the minor code changes I've had to make in order to fix the problem I reported on https://github.com/cloudtools/troposphere/issues/953

Thanks for the issue/PR.


Version 2.0.0 not installed when using @latest reference

Hi,
It is normal that the version 1.5.0 is still installed when using the install command from the documentation ?
go install github.com/ffuf/ffuf@latest

I tried this morning and the version 1.5.0 was installed instead of the 2.0.0:

Thank you very much in advance for your feedback 😃

Looking into it, I have no idea why go install doesn't see the new version tag.

Thanks a lot for the feedback.

@righettod I had the same issue, but I realiised it was building the new binary correctly, but just not moving it to ~/go/bin.
So until a proper fix, after running git clone https://github.com/ffuf/ffuf ; cd ffuf ; go get ; go build you can just do mv ffuf ~/go/bin/

@xnl-h4ck3r Thank you very much for the hint and infos 💯

This is now fixed in the README, the go install command to get the latest ffuf v2 version is:
go install github.com/ffuf/ffuf/v2@latest

Thank you


Support JDBC DatabaseMetaData getImportedKeys function

This is required for external library integration, for example Liquibase.
I ported and simplified from mysql-connector-j. I also had to port a number of StringUtils functions, which I copied verbatim. However I've gone over those functions and have 100% coverage on them in the new StringUtilsTest.java.
Much of the lines here are for the tests and test cases.
@ashudeep-sharma @harshit-gangal

As discussed on Slack, I'm going to merge this without review because I want to rename our Java packages: https://github.com/youtube/vitess/issues/2725
Please still review this and @bbeaudreault promised to fix forward any comments :)

@harshit-gangal we have hundreds of databases, most of them are unsharded, and many of them have foreign keys.

The problem with FK is that vttablet will incorrectly report affected rows if you turn on cascade deletes. But if you use RBR, you should be ok.
I'm actually thinking this may play well with shared vindexes also. Cascade deletes+RBR will fill that missing feature that Shared Vindexes could not fulfill.

@harshit-gangal let me know if you have any other review comments. Once you've given it a full pass through I'll submit a followup PR to address


Saving and loading models?

I noticed that the train() method has a parameter checkpoint_to_load, which is currently unused in the code. I assume this would be part of a solution for saving and loading the current state of the model.
Would you be able to give any pointers on how to go about implementing something like this? I would like to be able to run this code on a machine that is not running 24/7, so being able to save and resume the training would be a great feature.

Yes, check this out: https://github.com/gsurma/face_generator. It's based on the image_generator and is suited for saving/loading models.


ci(test): test pr sync

This is a test

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Limitless Smite Chaos?

Is there anyway we could get Limitless Smite Chaos as a Mythic Ability as well? Need some Hellknight love

The single limitless smite already applies to both evil and chaos. I admit it's not very clear from the description.


Bump carbon identity framework version

PR builder started
Link: https://github.com/wso2/product-is/actions/runs/2536575461


Invalid response from karate.read inside a js function

Java8 and karate 0.6.1 & 0.6.2
Given the test.json file content
{ "foo":"bar"}
When calling the following function from a feature file
function() {
    var test = karate.read('classpath:api/test.json');
    karate.log(test);
    karate.log(JSON.stringify(test));
}

then the value of test is undefined
but if called in a feature like:
* def test = call('classpath:api/test.json')
* print test

then the value of test is the json structure
I have tried many combinations but it never reads the json correctly

@jbadeau thanks for this, a good catch that went undetected for a while ! fixed in dev. I guess you will be able to build from source and verify

Thanx, was really a blocker for us. Thanx fir the fast response. I’ll berify it in the morning

I can confirm the fix works. :)

@ptrthomas Is it expected that trying to read a Karate feature file within JS doesn't work?
For context, I'm trying to make a more simple, readable way to call some helper functions, like * run_helper("create", "client"), but the value of result returned by the example below is just the contents of the string passed in.
* def run_helper =
    """
        function(tag, file) {
            const result = karate.read(`classpath:helpers/${file}.feature@${tag}`);
            return result;
        }
    """

@justindwyer6 can't tell from this. follow this process please: https://github.com/intuit/karate/wiki/How-to-Submit-an-Issue

@justindwyer6 and most likely you are looking for karate.call(). I am still not sure what you are trying and whether a "feature type" works in JS - so we may still need that way to replicate

Ah yes, it looks like karate.call() essentially does what I'm looking for. Thanks!


docs: make all examples easy to open in Stackblitz

Links can be previewed at https://deploy-preview-2691--unocss.netlify.app/guide/#examples
Examples with quirks:

astro-vue: has a warning: duplicate compile class name 'uno-06j1dt', please choose different class name from unocss:transformers:pre, src/components/Example.vue - looks like #2667 is tracking it so we can ignore it here.
remix: prints out some deprecation notices and information on needed upgrades - probably an effortless update for @veritem if that seems helpful.

Examples that don't work
I've not included these in the "Play Online" column for now, but the relevant contributors are welcome to update samples on their own as desired:

next: fails to load SWC binary for @next/swc-linux, this has been an issue for awhile. Stackblitz has a Next template from a few versions back that works in Stackblitz but if you remove their package-lock.json, set next version to most recent and re-install the error is back. I don't have the necessary Next experience to make decisions here. Any thoughts @Sickisonfire?
vite-svelte-postcss: throws a Vite error: end must be greater than start, I know @sibbng is in process with the postcss stuff so I'm assuming this will resolve itself in time.
qwik: throws error: Cannot find module 'undici' - any thoughts, @vasilGerginski?
vite-elm: Broken but https://stackblitz.com/edit/elm-daisyui is an Elm + Stackblitz example that works fine. Any thoughts, @userquin?

@antfu, I have a few remaining thoughts/questions:

Is /guide/#examples the right location in the docs?
The two vue-cli examples did not include a pnpm-workspace.yaml file. I "fixed" this but please undo if it was intentional.
The majority of examples don't include "type": "module" in their package.json but I find it hard to believe that all of these are using commonjs. Looking at the first, Astro, their basic template includes  "type": "module". Is this just an oversight that is a result of being placed within a monorepo that has "type": "module"? I think the examples should be updated to include "type": "module" as applicable so if people open in Stackblitz and then decide to download the project, things are set right for them to start their new project.

@jacob-8 linked stackblitz example is wrong, should be https://stackblitz.com/fork/github/unocss/unocss/tree/main/examples/vite-elm
I'll check it later, it seems some Elm dependencies broken, maybe Elm has changed:

Error
$ elm-tooling install
/home/projects/unocss-unocss-ua9umf/elm-tooling.json
  0% elm 0.19.1
  0% elm-format 0.8.4
  0% elm-json 0.2.8
  0% elm-test-rs 2.0.0
ERR! elm-format 0.8.4
ERR! elm-json 0.2.8
ERR! elm-test-rs 2.0.0
ERR! elm 0.19.1

4 errors

elm 0.19.1
    < https://github.com/elm/compiler/releases/download/0.19.1/binary-for-linux-64-bit.gz
    > /home/.elm/elm-tooling/elm/0.19.1/elm

    socket hang up

elm-format 0.8.4
    < https://github.com/avh4/elm-format/releases/download/0.8.4/elm-format-0.8.4-linux-x64.tgz
    > /home/.elm/elm-tooling/elm-format/0.8.4/elm-format

    kill EPERM

    Maximum call stack size exceeded

elm-json 0.2.8
    < https://github.com/zwilias/elm-json/releases/download/v0.2.8/elm-json-v0.2.8-x86_64-unknown-linux-musl.tar.gz
    > /home/.elm/elm-tooling/elm-json/0.2.8/elm-json

    kill EPERM

    Maximum call stack size exceeded

elm-test-rs 2.0.0
    < https://github.com/mpizenberg/elm-test-rs/releases/download/v2.0/elm-test-rs_linux.tar.gz
    > /home/.elm/elm-tooling/elm-test-rs/2.0.0/elm-test-rs

    kill EPERM

    Maximum call stack size exceeded

@jacob-8 I've created https://stackblitz.com/edit/userquin-vite-elm-test-latest-unocss-8dmwhp from https://github.com/userquin/vite-elm-test-latest-unocss
In local works but still not working in StackBlitz

In local works but still not working in StackBlitz

I don't know Elm so I'm not much help but I just wanted to ask in case it was an easy thing to make it work in Stackblitz. If it's not that's fine, people still have a good demo. If you figure it out down the road, just add the link. :)

vite-svelte-postcss: throws a Vite error: end must be greater than start, I know @sibbng is in process with the postcss stuff so I'm assuming this will resolve itself in time.

It's a svelte compiler issue: https://github.com/sveltejs/svelte/issues/7518. For now we can remove @screen md {} from style block.

@jacob-8

next: fails to load SWC binary for @next/swc-linux, this has been an https://github.com/stackblitz/core/issues/1904 for awhile. Stackblitz has a Next template from a few versions back that works in Stackblitz but if you remove their package-lock.json, set next version to most recent and re-install the error is back. I don't have the necessary Next experience to make decisions here.

I also gave it a try but it doesn't work because Stackblitz has issues with loading swr which is Next rust-based compiler. Another possible solution here would be to add an example with a link to CodeSandBox which has support for it.
I noticed that the Next example is also based on the pages directory. App directory has become stable. I can open a PR if you guys are open for it.

Note that the solution that Next posted in https://nextjs.org/docs/messages/failed-loading-swc isn't working with Stackblitz playgrounds based on https://github.com/vercel/next.js/discussions/30468
Therefore I would consider trying CodeSandbox in this case.

I can open a PR if you guys are open for it.

Yes, please do @iamhectorsosa, with a CodeSandbox link. I don't know Next, but I will assume that just having one example using the app directory sufficient for learning's sake, even for those still using pages directory. Correct me if I'm wrong.

I've drafted a PR -> https://github.com/unocss/unocss/pull/2724
I still have to update the Readme file but I'm shooting to replicate the Next + TailwindCSS starter to be Next + UnoCSS. Let me know what you think.


subnetIds[] missing from VPCEndpoint resource

What happened?
To create a VPCEndpoint resource, we used to specify the subnetIds[]  in the contrib crossplane provider but it is missing in the Upbound official provider which we migrated to.
Plus, this field is also present in the AWS terraform provider.
Crossplane provider AWS<EMAIL_ADDRESS>Upbound officiel provider AWS : https://marketplace.upbound.io/providers/upbound/provider-aws/v0.40.0/resources/ec2.aws.upbound.io/VPCEndpoint/v1beta1
Terraform provider AWS : https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/vpc_endpoint.html#subnet_ids
I am aware of the VPCEndpointSubnetAssociations resource but in our case, the subnets already exist (not created by crossplane) and their numbers can vary depending on the environment. Consequently, handling dynamically the number of associations in a composition is easier if subnets are defined as a list in 1 resource than if they are defined in multiple association resources...
I guess we could use composition functions for this but we try to keep things as simple as possible and in this particular case, the official provider adds complexity comparing to the previous provider or to terraform corresponding resource
Even if subnetIds and VPCEndpointSubnetAssociations are exclusive, is it not possible to keep them both and to let users the way they want to specify it ?
Plus, it would allow the VPCendpoint resource with crossplane to be as close as possible as the experience in the AWS console where the subnets can be chosen in a list, and also as the corresponding terraform provider, which would ease migration to crossplane for users.
What environment did it happen in ?
Crossplane version : 1.12
Provider version : 0.40
K8s version : 1.24

is this duplicate of  https://github.com/upbound/provider-aws/issues/119  ?

This is still quite the problem, I was porting a terraform module to Crossplane today and the missing field is quite important.

/fresh

/fresh


nginx 404

Info:

Docker version: 18.06.1-ce
Laradock commit: db02fe7570c50615e18dd7371cc5d8ed443bab4f
System info: Linux
System info disto/version: Ubuntu 18.04.1 LTS

Issue:
Getting 404 error

Expected behavior:
show website

Reproduce:
Root directory:
my-project
laradock

cp .env-example .env
laradock/nginx/sites/default.conf changes:
root /var/www/public; to root /var/www/my-project;
docker-compose up -d nginx

On local env it's working, problem is just on the production.
Thank you!

check server_name
or
docker-compose exec nginx bash
ls /var/www/my-project

I already made same on a different machine and it's working. Thank you!


Refactor default Source sublanguage selection feature

Refactor default Source sublanguage selection feature
Summary: Code quality improvements, bug fixes and minor UX changes.
Description
WIP
Changelog
WIP
Verification instructions
WIP
Last updated 1 Jul 2020, 11:45PM

Pull Request Test Coverage Report for Build 5874

11 of 31 (35.48%)  changed or added relevant lines in 5 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.3%) to 35.612%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

src/commons/sagas/BackendSaga.ts
6
7
85.71%

src/commons/sagas/RequestsSaga.ts
0
2
0.0%

src/pages/academy/subcomponents/AcademyDefaultChapter.tsx
3
20
15.0%

Totals

Change from base Build 5799:
0.3%

Covered Lines:
1861

Relevant Lines:
4833

💛  - Coveralls


Update all dependencies

:tada: This PR is included in version 5.0.2 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


[feature] Filter option - Trigger only when NEW content (lines) are detected ( compared to earlier text snapshots )

[ ] needs test
[ ] tidy up field titles and description
[ ] limit to last x? (as a main system setting?)

Super handy for pages that just shuffle the content around! or when new products appear in a listing.
Don't trigger a change unless there is no content available when compared to all history saved

Is it possible to extend the features to include the functionality (as well as the proposed UI as in #500). I would imagine that it would be as simple as adding 2 more check boxes and just doing the inverse of the current checks?

@bwees that is a separate issue, but i'm open to PRs with test-cases if you want to provide it


Consensus change series appears to be inconsistent - we are applying the wrong block.

BUG REPORT
Stack Trace or error message
PS C:\Users\cscha> C:\Sia-v1.3.2-windows-amd64\siad.exe -d G:\Blockchains\SIA
Sia Daemon v1.3.2
Git Revision 72938f5
Loading...
(0/6) Loading siad...
(1/6) Loading gateway...
(2/6) Loading consensus...
(3/6) Loading transaction pool...
(4/6) Loading wallet...
(5/6) Loading host...
(6/6) Loading renter...
Finished loading in 20.4331806 seconds
panic: Consensus change series appears to be inconsistent - we are applying the wrong block.
goroutine 62407 [running]:
github.com/NebulousLabs/Sia/modules/transactionpool.(*TransactionPool).ProcessConsensusChange(0xc0421a62c0, 0x83d1b026941fa14, 0
x3159f5cd34fceb04, 0x2fc68a3e204f17be, 0xf1dc6d03b3a37b7b, 0x0, 0x0, 0x0, 0xc0c921cd20, 0x1, ...)
/home/luke/go/src/github.com/NebulousLabs/Sia/modules/transactionpool/update.go:223 +0x28d3
github.com/NebulousLabs/Sia/modules/consensus.(*ConsensusSet).updateSubscribers(0xc0421ac500, 0x0, 0x0, 0x0, 0xc0cc33c8e0, 0x1,
0x1)
/home/luke/go/src/github.com/NebulousLabs/Sia/modules/consensus/subscribe.go:118 +0x184
github.com/NebulousLabs/Sia/modules/consensus.(*ConsensusSet).managedAcceptBlocks(0xc0421ac500, 0xc0aeb0bd48, 0x1, 0x1, 0x1e8400
, 0x0, 0x0)
/home/luke/go/src/github.com/NebulousLabs/Sia/modules/consensus/accept.go:311 +0x5e0
github.com/NebulousLabs/Sia/modules/consensus.(*ConsensusSet).managedReceiveBlock.func1(0xb3e840, 0xc0af06db40, 0x0, 0x0)
/home/luke/go/src/github.com/NebulousLabs/Sia/modules/consensus/synchronize.go:529 +0x1fc
github.com/NebulousLabs/Sia/modules/gateway.(*Gateway).managedRPC(0xc0421a6000, 0xc042247ec0, 0x12, 0xa986bf, 0x7, 0xc0c8b61770,
0x0, 0x0)
/home/luke/go/src/github.com/NebulousLabs/Sia/modules/gateway/rpc.go:66 +0x2b2
github.com/NebulousLabs/Sia/modules/gateway.(*Gateway).RPC(0xc0421a6000, 0xc042247ec0, 0x12, 0xa986bf, 0x7, 0xc0c8b61770, 0x0, 0
x0)
/home/luke/go/src/github.com/NebulousLabs/Sia/modules/gateway/rpc.go:76 +0xcb
github.com/NebulousLabs/Sia/modules/consensus.(*ConsensusSet).threadedRPCRelayHeader.func5(0xc043c12fb0, 0xc0421ac500, 0xb3e840,
0xc0c5f8c020, 0xc04319fe00, 0xc0afea1030)
/home/luke/go/src/github.com/NebulousLabs/Sia/modules/consensus/synchronize.go:459 +0x147
created by github.com/NebulousLabs/Sia/modules/consensus.(*ConsensusSet).threadedRPCRelayHeader
/home/luke/go/src/github.com/NebulousLabs/Sia/modules/consensus/synchronize.go:457 +0x40f
PS C:\Users\cscha> C:\Sia-v1.3.2-windows-amd64\siad.exe -d G:\Blockchains\SIA\
Expected Behavior

No exception
siad runs again after re-started (but siad repeats giving above error message)

How to reproduce it (as minimally and precisely as possible)
No special action, three files were uploading at this point.
Environment

Sia version: 1.3.2
OS: Win 10 64bit

Same.

I renamed my 'transactionpool' folder and started siad to create a new one, it took about 40 minutes, but it is back up again.

@DavidVorick just an FYI that this has been reported several times on Discord as well as here, always with 1.3.2.  We may want to look at what introduced this bug.

I've seen the same error 3x today.
Running on Windows 10 pro 64-bit.
I'll try @chargr route with the transactionpool folder and will report back if that resolved it.

@tbenz9 iirc this sanity check was introduced by @DavidVorick after we noticed that our boltdb databases corrupted over time for some users. It should only be triggered after updating to 1.3.2 because 1.3.1 silently ignored this issue. I would recommend everyone to do a clean install of Sia 1.3.2 to avoid that potential issue.

Update: I tried deleting the transactionpool folder and doing a clean install.
Both resulted in the same "things take a while to load after updating" display (Sorry I don't have the exact on-screen display message).
After deleting the transactionpool folder, the App was hung for over 4 hours.
Then I closed the app and did a clean install of 1.3.2
It's been hung on that same screen for about an hour.

Another report of this issue
https://www.reddit.com/r/siacoin/comments/85p3hr/anyone_know_why_i_cant_run_the_siaui_ive_been/

We're actively troubleshooting this error in Discord right now. Can anyone try following these instructions to delete your transactionpool.db file and report back if it worked.
https://gist.github.com/tbenz9/8066d74853ab28c23bd7bc1d3c7cdeda#delete-the-transactionpooldb-file

For me, it worked to remove the transactionpool folder completely (renamed it to .old), created an empty one and restarted Sia. This fixed it for me (see comment by @chargr above). Basically, this folder holds the .db and the .log file. So yes, helped! (@tbenz9  notice my improved punctuation ;-) )

I encountered this on 1.3.2 (not fresh install and load, it's an upgrade from 1.3.1).  The issue happened after a system crash after siad was restarted, the system is working normally with no symptoms of any problem.  Stack below.
Sia Daemon v1.3.2
Git Revision 72938f5
Loading...
(0/6) Loading siad...
(1/6) Loading gateway...
(2/6) Loading consensus...
(3/6) Loading transaction pool...
(4/6) Loading wallet...
(5/6) Loading host...
(6/6) Loading renter...
Finished loading in<PHONE_NUMBER> seconds
panic: Consensus change series appears to be inconsistent - we are applying the wrong block.

goroutine 10418 [running]:
github.com/NebulousLabs/Sia/modules/transactionpool.(*TransactionPool).ProcessConsensusChange(0xc04226c000, 0xeb5078dc0065d61d, 0x4764b5c5f215a736, 0x7710f9a5be90d379, 0xf165500076cdda93, 0x0, 0x0, 0x0, 0xc042aed560, 0x1, ...)
	/home/luke/go/src/github.com/NebulousLabs/Sia/modules/transactionpool/update.go:223 +0x2b68
github.com/NebulousLabs/Sia/modules/consensus.(*ConsensusSet).updateSubscribers(0xc0420d7b80, 0x0, 0x0, 0x0, 0xc042663ea0, 0x1, 0x1)
	/home/luke/go/src/github.com/NebulousLabs/Sia/modules/consensus/subscribe.go:118 +0x17f
github.com/NebulousLabs/Sia/modules/consensus.(*ConsensusSet).managedAcceptBlocks(0xc0420d7b80, 0xc044809d40, 0x1, 0x1, 0x1e8400, 0x0, 0x0)
	/home/luke/go/src/github.com/NebulousLabs/Sia/modules/consensus/accept.go:311 +0x61e
github.com/NebulousLabs/Sia/modules/consensus.(*ConsensusSet).managedReceiveBlock.func1(0xdf80e0, 0xc0426ac640, 0x0, 0x0)
	/home/luke/go/src/github.com/NebulousLabs/Sia/modules/consensus/synchronize.go:529 +0x1fc
github.com/NebulousLabs/Sia/modules/gateway.(*Gateway).managedRPC(0xc0420ca580, 0xc0421ba7e0, 0x14, 0xac0901, 0x7, 0xc042df5650, 0x0, 0x0)
	/home/luke/go/src/github.com/NebulousLabs/Sia/modules/gateway/rpc.go:66 +0x2c9
github.com/NebulousLabs/Sia/modules/gateway.(*Gateway).RPC(0xc0420ca580, 0xc0421ba7e0, 0x14, 0xac0901, 0x7, 0xc042df5650, 0x0, 0x0)
	/home/luke/go/src/github.com/NebulousLabs/Sia/modules/gateway/rpc.go:76 +0xd8
github.com/NebulousLabs/Sia/modules/consensus.(*ConsensusSet).threadedRPCRelayHeader.func5(0xc042439960, 0xc0421627e0, 0xc0420d7b80, 0xdf80e0, 0xc044539340, 0xc042d76780)
	/home/luke/go/src/github.com/NebulousLabs/Sia/modules/consensus/synchronize.go:459 +0x147
created by github.com/NebulousLabs/Sia/modules/consensus.(*ConsensusSet).threadedRPCRelayHeader
	/home/luke/go/src/github.com/NebulousLabs/Sia/modules/consensus/synchronize.go:457 +0x447


Books as Home

Is it possible to have the /books page as the homepage, I would like our staff to be taken straight to that URL once they login.

Hi @SamB-GB
This is already possible. Please go to the settings screen and under Application Homepage set "select the books page as your homepage" as true.
Bottom right of the screenshot below.

Not possible to select just /books instead have to select a books page.

@SamB-GB - Please select the checkbox shown in the screenshot below,


SpanButton continuously fires press event

Hello, I'm going to use Doorbell service with my ESP32-CAM setup. But button function is being called every second even I don't click button on my breadboard.
Here is DEV_Doorbell.h;
struct DEV_Doorbell : Service::Doorbell {
  int buttonPin;
  SpanCharacteristic *switchEvent;
  SimpleTimer timer;

  DEV_Doorbell(int buttonPin, int index) : Service::Doorbell(){
    this->buttonPin=buttonPin;
    switchEvent = new Characteristic::ProgrammableSwitchEvent();
    new SpanButton(buttonPin);
  } // end constructor

  void button(int pin, int pressType) override {

      Serial.print("PRESS TYPE:");
      Serial.print(pressType);
      Serial.print("\n");
      switchEvent->setVal(pressType);

  }

};

Output:
Starting MDNS...

HostName:      HomeSpan-02DB7EB82017.local:80
Display Name:  HomeSpan Bridge
Model Name:    HomeSpan-ESP32
Setup ID:      HSPN

Starting HAP Server on port 80 supporting 14 simultaneous HomeKit Controller Connections...

PRESS TYPE:2
PRESS TYPE:2
PRESS TYPE:2
PRESS TYPE:2
PRESS TYPE:2
PRESS TYPE:2
PRESS TYPE:2
PRESS TYPE:2
PRESS TYPE:2

Does this occur even if you disconnect the button and all other wires from the esp-32?

No, it stops to fire event when I disconnect the button from ESP32. It's on PIN15 by the way and I tested all setup (wires, button and ESP32) in different project it works fine which means there is no any problem on hardware.

try to change the constructor for "spanButton" and put this code ->
new SpanButton(buttonPin, SpanButton::TRIGGER_ON_HIGH);

try to change the constructor for "spanButton" and put this code -> new SpanButton(buttonPin, SpanButton::TRIGGER_ON_HIGH);

Yes it worked, thank you!

Is the button normally open or normal closed? If it stops firing when you disconnect the button, and starts to repeatedly fire when you reconnect the button, it sounds like the button is normally closed, which is the wrong type of button. Or perhaps you wired the button incorrectly so that the leads are always connected (instead of across the switch)? Try using a plain jumper wire and manually touch your pin to ground - that should trigger a short press or long press depending on how long you touch ground. Having a repeated Press Type = 2 (Long) would occur if the pin is connected to ground permanently.

By the way this is a digital pin button, I have tested this wired setup with a different project and it works fine. But by using the code @ronny-antoon shared it works fine.


Missing License File

The license file specification is not working and is redirecting to an unexisting page
https://github.com/sheikalthaf/ngu-carousel/blob/master/LICENSE

@royalcas License file is added. Sorry for not merging the pull request. Because I only accepts the carousel changes and improvements. Thanks


Faulty palette reconstruction

Fixes #19

Thanks for pointing this out!
I also added an edit so that it fixes #21 as a related colour modal checkbox bug.


zoom failed at docusaurus version 2.0.0-beta.18

When I click the image, I get a error message in Chrome console:

Then I check the image html code:

It's seems like refactor: extract MDX components #6989 modify image class name, then image zoom plugin failed.

docusaurus: 2.0.0-beta.18
medium-zoom: 1.0.6

Ran into the same problem with Docusaurus 2.0.1. I discovered that if I remove the following medium-zoom options, the error does not occur.
          container: '#zoom-container',
          template: '#zoom-template',

Ran into the same problem with Docusaurus 2.0.1. I discovered that if I remove the following medium-zoom options, the error does not occur.
          container: '#zoom-container',
          template: '#zoom-template',

Yes, I'm success too, thank you very much! @ntolia


add: orml-xtokens and its dependancies in the runtime

Added xtokens from open-runtime-module-library and all its dependancies so that we can add it to our runtime.
Dependencies added:

[X] orml/tokens
[X] orml/traits
[X] orml/utilities
[X] orml/xcm
[X] orml/xcm-support

I still need to add the 2 following types to the runtime to fully implement xtokens

[ ] CurrencyIdConvert
[x] AccountIdToMultiLocation

Better to create a new branch dev based on main and merge there. @zycon91

Closing this as the branch is really outdated and it was easier to upgrade parachain-template with no added orml stuff and add xtoken from scratch.


Add the ability to generate $apply results with unique IDs

Context

The current $apply results create resources with IDs that are not unique i.e. executing $apply on the same PlanDef multiple times will generate MedicationRequests with the same IDs.
This means the Resources will override each other if the implementing applications try to persist them to the DB.

Implementation Requirement

[ ] Update the CQL evaluation to generate resources with unique UUIDs

Sample resources of the current resource generated
{
  "resourceType": "MedicationRequest",
  "id": "30",
  "meta": {
    "lastUpdated": "2024-01-24T19:31:01.327+05:00",
    "profile": [
      "http://hl7.org/fhir/uv/cpg/StructureDefinition/cpg-immunizationrequest"
    ]
  },
  "status": "draft",
  "intent": "proposal",
  "doNotPerform": false,
  "medicationCodeableConcept": {
    "coding": [
      {
        "system": "http://hl7.org/fhir/sid/icd-11",
        "code": "XM28X5",
        "display": "Measles vaccines"
      }
    ]
  },
  "subject": {
    "reference": "Patient/9a6188b4-cd60-41af-8880-c3f8464eef4b"
  },
  "instantiatesCanonical": [
    "http://smart.who.int/ig/smart-immunizations/ActivityDefinition/IMMZD2DTMeaslesMR"
  ]
}

{
  "resourceType": "OperationOutcome",
  "id": "apply-outcome-IMMZD2DTMeasles",
  "meta": {
    "lastUpdated": "2024-01-24T19:31:01.404+05:00"
  },
  "issue": [
    {
      "severity": "error",
      "code": "exception",
      "diagnostics": "Error encountered extracting b5887df8-a702-4d97-b207-c00991477e4a: Unable to retrieve Questionnaire code map for Observation based extraction"
    }
  ]
}

The OperationOutcome ID is generated here https://github.com/cqframework/clinical-reasoning/blob/1d033f61140709d5fc1f3d4702ea9f1460647157/cqf-fhir-cr/src/main/java/org/opencds/cqf/fhir/cr/plandefinition/r4/PlanDefinitionProcessor.java#L174


The "carousel" hack

The carousel component contains a hack that would be beneficial to fix
Noted here https://github.com/bootstrap-vue-next/bootstrap-vue-next/pull/1306#issuecomment-1642925223
BCarousel does some weird magic to get things to work correctly. In particular, it actually flows the carousel "backwards". In that it's carousel-item-next/prev classes are actually backwards.
This is not great because it doesn't line up with the actual getbootstrap spec
In addition to this, "carousel-item" is declared in the parent. For some reason, when I removed this value originally, it caused everything to break down. This is a static class that doesn't need to be changed, but for some reason, making it purely static causes it to break down.
The hack should be fixed up to where the next/prev start/end classes are actually in the correct order, and that the carousel-item can be correctly back into BCarouselItem, and not declared on the parent.

(Keep in mind, I'm not quite sure how to fix this so any help is appreciated)


refactor(ConnectionService): update connections

Please describe the changes this PR makes and why it should be merged:
If applicable, please reference Discord API Docs PRs or commits that influenced this PR:

https://github.com/discord/discord-api-docs/pull/5265
https://github.com/discord/discord-api-docs/pull/5306
https://github.com/discord/discord-api-docs/pull/5355

Once again 😔


Alert user of failure

Until implementation becomes more clear this will handle failed requests by alerting the user.

Great! Merging this now.


Allow using a disease label as a "genotype"

Currently we can only use the disease as a "phenotype".

Fixed in #245


[master]Remove unused lszdev in unpackdiskimage

Signed-off-by: ylpan<EMAIL_ADDRESS>

don't know the history, is this lszdev introduced by CoreOS recently or stay here for long time?

Introduced by rhcos, and used to get  DEST_DEV_BUS, but DEST_DEV_BUS never used. So would like to remove.
lszdev is in the rpm s390utils-base-2.4.0-1.el7.s390x,  a higher version than s390utils-base-1.23.0-45.el7_7.1.s390x, which is current VM usually  used

ok, thanks for the info~


Float precision

We are using a table containing floats, and have tests that use random values. The builtin serializers use std::ostringstream as underlying implementation, which defaults to std::ios_base::precision() == 6. This implies that values are quite quickly truncated.
A quick hack of
diff --git a/include/sqlpp11/sqlite3/connection.h b/include/sqlpp11/sqlite3/connection.h
index 79b3ae6..bd1a8ea 100644
--- a/include/sqlpp11/sqlite3/connection.h
+++ b/include/sqlpp11/sqlite3/connection.h
@@ -159,4 +159,5 @@ namespace sqlpp
       serializer_t(const connection& db) : _db(db), _count(1)
       {
+          _os.precision (std::numeric_limits<long double>::digits10 + 1);
       }

shows that the underlying implementation supports higher precision values without issue.
Was this an explicit design choice or are float values just so rare nobody noticed?

Hi,
It was not a design choice. I believe that most folks avoid float values. I know that I do :-)
I am happy to accept a pull request to change the precision.
Best,
Roland


Projects can have Projects

User story
As a Collective admin, I want to nest Projects within Projects, so I can further break down budgets and activities accordingly.
Best solution for this problem
A way to create a Project within a Project.
I don't know if this is a crazy idea or if would actually be simple. If it's crazy, ignore it. But if it's easy to replicate the same architecture recursively, we should consider it, as it's been requested by users and I think would add awesome flexibility to the platform.
The one concern I have is about this becoming an end-run to the host application process, where people create Projects underexisting Collectives instead of new Collectives, and the host doesn't have as much oversight. But since in #4611 @znarf said even if Projects have their own admins the parent Collective admins will still be admins as well, it might not be a problem.

I've decided I don't think this is a good idea if we're implementing #4611 and I think that's more important.


Are You sure SQL Server Components are always needed during BC install?

>On the Customize the installation page, select the following components as a minimum:
>AL Development Environment (optional but recommended); Server; SQL Server Components; Web Server Components.
I just did Update from 15.0 to 15.1 in customer environment without installing SQL Server Components.
Customer has SQL database along with SQL Standard in another server.
If I selected SQL Server Components to install I got error:
"The SQL Server instance 'InstanceName' could not be found. A new instance will be created for you. If a compatible version of SQL Server is not found, Setup will install SQL Server Express."
So hence I installed BC without installing SQL Server Components. So far all seems to be working fine.
Or am I totally wrong about something?
Please add to instructions how to install update when SQL express is not needed (since most customers use SQL Standard and some SQL Enterprise and SQL Express is mainly used by NAV consultant PC-s).
Thank You.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: e71477e4-f2ea-1481-7597-29b324bd4f57
Version Independent ID: a2bb3a05-e5c1-d9f7-f47f-d2f34c4a91ea
Content: Install an update - Business Central
Content Source: dev-itpro/upgrade/upgrading-cumulative-update-v15.md
Service: dynamics365-business-central
GitHub Login: @jswymer
Microsoft Alias: jswymer

Hello,
You are correct. The SQL Server Components are not required in this case. This option is only required if there is not already a compatible version of SQL Server on the database machine, in which case SQL Server Express 2016 will be installed. However, if you do choose this option and SQL Server is already installed, then SQL Express is not installed. I will update the article to make thing clearer. Thanks for the feedback!

Hello,
You are correct. The SQL Server Components are not required in this case. This option is only required if there is not already a compatible version of SQL Server on the database machine, in which case SQL Server Express 2016 will be installed. However, if you do choose this option and SQL Server is already installed, then SQL Express is not installed. I will update the article to make things clearer. Thanks for the feedback!

Hello, the article has been an updated and will be published soon. Thanks for the feedback!


Extract block hardness in block extractor

I've searched existing issues and couldn't find a duplicate.

[X] I confirm this is not a duplicate.

Is your feature request related to a problem? Please describe.
Block hardness is required for calculating block breaking speed, which is needed to determine whether a block was broken instantly or the breaking was cancelled, as the client does not send a breaking finished packet when a block was broken instantly, even in survival mode.
Describe the solution you'd like
block hardness should be extracted and added to blocks.json

While you're at it, wheter a player can stand on a block should also be extracted. That's required for get_top_block for example

While you're at it, whether a player can stand on a block should also be extracted. That's required for getting the top block when spawning players for example, currently that just checks whether a block is air

Im not sure what do you mean by "whether a player can stand on a block"

While you're at it, whether a player can stand on a block should also be extracted. That's required for getting the top block when spawning players for example, currently that just checks whether a block is air

Im not sure what do you mean by "whether a player can stand on a block"

What I meant was whether the block has a collision shape, but now that I think about it that doesn't make much sense, as collisions are a lot more complex than that. I also didn't realize before that there are in fact already collision shapes in blocks.json, so sorry for not doing my due diligence and thanks for adding block hardness!


Keys will be stuck in not supported/ non text fields with quick accent

Microsoft PowerToys version
v0.71.0
Installation method
WinGet
Running as admin
Yes
Area(s) with issue?
Quick Accent
Steps to reproduce
turn on quick accent play game fps or any other
press a alphabet key + space (space is after slight delay)
can also test on any key test sites online i used
https://en.key-test.ru/
✔ Expected Behavior
keys should not get stuck
❌ Actual Behavior
alphabet key stuck until pressed again
Other Software
No response

I also have this bug too. It really annoys me when I play games. It needs to be fixed asap.

/dup #20349


Make extension logger contextual

Make the extension logger better by either doing something similar to VSCode's Telemetry or make some system that prefixes logs with the extension name or something along those lines.
Depends on #50

Maybe fixed in #295


[STORM-3588] add GenericResourceAwareSchedulingPriorityStrategy to accommodate generic resource in grading topologies

Add GenericResourceAwareSchedulingPriorityStrategy by extending DefaultSchedulingPriorityStrategy to accommodate generic resources when sorting topologies.
TestGenericResourceAwareSchedulingPriorityStrategy provides situations where DefaultSchedulingPriorityStrategy wrongfully pre-empted all other running topologies. This is prevented by GenericResourceAwareSchedulingPriorityStrategy.

Add evictedTopologies set in ResourceAwareScheduler currently only for test since we did not find a better way to check eviction history. We added here considering that eviction is a RAS specific method and this new field could probably be useful in the future.

@kishorvpatil Thanks for your review. As I addressed your comment, I noticed that we actually can cache all types of total cluster resources since the superviors snapshot won't change in cluster object.
So we don't need to add another getClusterGenericResourceTypes() function. We can just cache generic resources map and only fetch the keySet while we need it.
I also added cache for CPU and memory in latest commit, any further comments would be appreciated.

@RuiLi8080
I would like us to have behavior what we test and test that check production behavior. We should expose any evictions as standard result of scheduling available for both production and tests instead of forTest kind of special code path executed only during tests.

@kishorvpatil I think your point makes sense. But I was worried that maintaining the information of evicted topos will add unnecessary computation and also flood logs with evicted topoids since it is calculated at every round of scheduling. I made a small change so we only do this when testing or debugging, not sure if this can address your point.

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-checkstyle-plugin:3.0.0:check (validate) on project storm-server: You have 1 Checkstyle violation. -> [Help 1]

checkstyle violations.

Only one server test failed in JDK11, JDK8 passed.


Untested solution for fixing build issue

Tried doing a build, got error:
+ go get -v github.com/lgierth/dnslink-dnsimple
github.com/lgierth/dnslink-dnsimple (download)
github.com/dnsimple/dnsimple-go (download)
github.com/google/go-querystring (download)
github.com/google/go-querystring/query
github.com/dnsimple/dnsimple-go/dnsimple
github.com/lgierth/dnslink-dnsimple
# github.com/lgierth/dnslink-dnsimple
src/github.com/lgierth/dnslink-dnsimple/main.go:46:24: cannot use a.ID (type int64) as type int in argument to strconv.Itoa
The command '/bin/sh -c set -ex &&   npm install -g aegir &&   go get -v github.com/lgierth/dnslink-dnsimple &&   go get -v github.com/davecheney/godoc2md' returned a non-zero code: 2

Tried this. It compiled. I have not tested it though.

Yeah I think this should do - thanks


Allow account creation from only a given domain name

Hi, I wanted to know if there exists, (and if not, could this be a feature in future) that on my instance of gitea -- say git.example.com, only those accounts can be created which have @example.com emails?

I think this exists with:
[service]
EMAIL_DOMAIN_WHITELIST = example.com

Closing, please reopen if needed


Environment variable to disable git repo checks

Context
I'm using a submodule to use one of my unity projects inside of a flutter app and have both built at the same time in the same repo. I'm running into the following error when building the unity side of things even though all of the files required to build the project are there:
Failed to run "git rev-list --count 1ae98cf9e5d20e8bd637b9efe21d2e25c4de0424".
 In-command error caught: Error: The process '/usr/bin/git' failed with exit code 128

Suggested solution
Add an environment variable that disables checking the git repo in the job.
Considered alternatives
Forking the project and adding this myself for my own use.
Additional details
https://discord.com/channels/710946343828455455/710946344264794219/991749950780223598

I have same problem and this blocks to move forward with game-ci. Now investigating another solution... :/


Multi-GPU training

I am attempting to run comet-train with multiple GPUs.
Command (abbreviated):
CUDA_VISIBLE_DEVICES=0,1,2,3 comet-train ...

Config (abbreviated):
init_args:
  accelerator: gpu
  devices: 4
  auto_scale_batch_size: True
  auto_select_gpus: True

Output with error (abbreviated):
...
Initializing distributed: GLOBAL_RANK: 0, MEMBER: 1/4
...
Initializing distributed: GLOBAL_RANK: 1, MEMBER: 2/4
Added key: store_based_barrier_key:1 to store for rank: 1
...
Initializing distributed: GLOBAL_RANK: 3, MEMBER: 4/4
Added key: store_based_barrier_key:1 to store for rank: 3
Added key: store_based_barrier_key:1 to store for rank: 0
Rank 0: Completed store-based barrier for key:store_based_barrier_key:1 with 4 nodes.
----------------------------------------------------------------------------------------------------
distributed_backend=nccl
All distributed processes registered. Starting with 4 processes
----------------------------------------------------------------------------------------------------
Rank 3: Completed store-based barrier for key:store_based_barrier_key:1 with 4 nodes.
Rank 2: Completed store-based barrier for key:store_based_barrier_key:1 with 4 nodes.
Rank 1: Completed store-based barrier for key:store_based_barrier_key:1 with 4 nodes.
...
LOCAL_RANK: 1 - CUDA_VISIBLE_DEVICES: [0,1,2,3,4,5,6,7]
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0,1,2,3,4,5,6,7]
LOCAL_RANK: 2 - CUDA_VISIBLE_DEVICES: [0,1,2,3,4,5,6,7]
LOCAL_RANK: 3 - CUDA_VISIBLE_DEVICES: [0,1,2,3,4,5,6,7]
...
1,161.732 Total estimated model params size (MB)
Sanity Checking: 0it [00:00, ?it/s]/opt/conda/lib/python3.10/site-packages/pytorch_lightning/trainer/connectors/data_connector.py:208: UserWarning: num_workers>0, persistent_workers=False, and strategy=ddp_spawn may result in data loading bottlenecks. Consider setting persistent_workers=True (this is a limitation of Python .spawn() and PyTorch)
  rank_zero_warn(
Traceback (most recent call last):
  File "/opt/conda/bin/comet-train", line 8, in <module>
    sys.exit(train_command())
  File "/opt/conda/lib/python3.10/site-packages/comet/cli/train.py", line 209, in train_command
    trainer.fit(model)
  File "/opt/conda/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 608, in fit
    call._call_and_handle_interrupt(
  File "/opt/conda/lib/python3.10/site-packages/pytorch_lightning/trainer/call.py", line 36, in _call_and_handle_interrupt
    return trainer.strategy.launcher.launch(trainer_fn, *args, trainer=trainer, **kwargs)
  File "/opt/conda/lib/python3.10/site-packages/pytorch_lightning/strategies/launchers/multiprocessing.py", line 113, in launch
    mp.start_processes(
  File "/opt/conda/lib/python3.10/site-packages/torch/multiprocessing/spawn.py", line 197, in start_processes
    while not context.join():
  File "/opt/conda/lib/python3.10/site-packages/torch/multiprocessing/spawn.py", line 160, in join
    raise ProcessRaisedException(msg, error_index, failed_process.pid)
torch.multiprocessing.spawn.ProcessRaisedException:

-- Process 1 terminated with the following error:
Traceback (most recent call last):
  File "/opt/conda/lib/python3.10/site-packages/torch/multiprocessing/spawn.py", line 69, in _wrap
    fn(i, *args)
  File "/opt/conda/lib/python3.10/site-packages/pytorch_lightning/strategies/launchers/multiprocessing.py", line 139, in _wrapping_function
    results = function(*args, **kwargs)
  File "/opt/conda/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 650, in _fit_impl
    self._run(model, ckpt_path=self.ckpt_path)
  File "/opt/conda/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 1112, in _run
    results = self._run_stage()
  File "/opt/conda/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 1191, in _run_stage
    self._run_train()
  File "/opt/conda/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 1204, in _run_train
    self._run_sanity_check()
  File "/opt/conda/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 1276, in _run_sanity_check
    val_loop.run()
  File "/opt/conda/lib/python3.10/site-packages/pytorch_lightning/loops/loop.py", line 199, in run
    self.advance(*args, **kwargs)
  File "/opt/conda/lib/python3.10/site-packages/pytorch_lightning/loops/dataloader/evaluation_loop.py", line 152, in advance
    dl_outputs = self.epoch_loop.run(self._data_fetcher, dl_max_batches, kwargs)
  File "/opt/conda/lib/python3.10/site-packages/pytorch_lightning/loops/loop.py", line 194, in run
    self.on_run_start(*args, **kwargs)
  File "/opt/conda/lib/python3.10/site-packages/pytorch_lightning/loops/epoch/evaluation_epoch_loop.py", line 84, in on_run_start
    self._data_fetcher = iter(data_fetcher)
  File "/opt/conda/lib/python3.10/site-packages/pytorch_lightning/utilities/fetching.py", line 178, in __iter__
    self.dataloader_iter = iter(self.dataloader)
  File "/opt/conda/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 441, in __iter__
    return self._get_iterator()
  File "/opt/conda/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 388, in _get_iterator
    return _MultiProcessingDataLoaderIter(self)
  File "/opt/conda/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 1042, in __init__
    w.start()
  File "/opt/conda/lib/python3.10/multiprocessing/process.py", line 121, in start
    self._popen = self._Popen(self)
  File "/opt/conda/lib/python3.10/multiprocessing/context.py", line 224, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "/opt/conda/lib/python3.10/multiprocessing/context.py", line 288, in _Popen
    return Popen(process_obj)
  File "/opt/conda/lib/python3.10/multiprocessing/popen_spawn_posix.py", line 32, in __init__
    super().__init__(process_obj)
  File "/opt/conda/lib/python3.10/multiprocessing/popen_fork.py", line 19, in __init__
    self._launch(process_obj)
  File "/opt/conda/lib/python3.10/multiprocessing/popen_spawn_posix.py", line 47, in _launch
    reduction.dump(process_obj, fp)
  File "/opt/conda/lib/python3.10/multiprocessing/reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
AttributeError: Can't pickle local object 'CometModel.val_dataloader.<locals>.<lambda>'

I'm using NVIDIA A10G GPUs and the following software versions:

Python - 3.10.9
COMET - upstream
torch - 2.0.1
pytorch-lightning - 1.9.5
transformers - 4.29.0
numpy - 1.24.3

Hi all, I would like to confirm this, I have the same issue with the above technology stack.

Hey @zouharvi @maxiek0071. Can you try the linked PR and let me know if that works (if it does not, post the error trace)?
You can install it like this:
python -m pip install git+https://github.com/Unbabel/COMET.git@refs/pull/160/head

Hi @BramVanroy,
I install comet from the branch you specified, and now I'm getting a similar error for EvaluationLoop.

Initializing distributed: GLOBAL_RANK: 1, MEMBER: 2/4
Added key: store_based_barrier_key:1 to store for rank: 1
Initializing distributed: GLOBAL_RANK: 2, MEMBER: 3/4
Added key: store_based_barrier_key:1 to store for rank: 2
Initializing distributed: GLOBAL_RANK: 3, MEMBER: 4/4
Added key: store_based_barrier_key:1 to store for rank: 3
Added key: store_based_barrier_key:1 to store for rank: 0
Rank 0: Completed store-based barrier for key:store_based_barrier_key:1 with 4 nodes.
----------------------------------------------------------------------------------------------------
distributed_backend=nccl
All distributed processes registered. Starting with 4 processes
----------------------------------------------------------------------------------------------------

Rank 3: Completed store-based barrier for key:store_based_barrier_key:1 with 4 nodes.
Rank 2: Completed store-based barrier for key:store_based_barrier_key:1 with 4 nodes.
Rank 1: Completed store-based barrier for key:store_based_barrier_key:1 with 4 nodes.
LOCAL_RANK: 1 - CUDA_VISIBLE_DEVICES: [0,1,2,3]
LOCAL_RANK: 2 - CUDA_VISIBLE_DEVICES: [0,1,2,3]
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0,1,2,3]
LOCAL_RANK: 3 - CUDA_VISIBLE_DEVICES: [0,1,2,3]

  | Name                | Type               | Params
-----------------------------------------------------------
0 | encoder             | XLMREncoder        | 558 M
1 | layerwise_attention | LayerwiseAttention | 26
2 | train_metrics       | RegressionMetrics  | 0
3 | val_metrics         | ModuleList         | 0
4 | estimator           | FeedForward        | 10.5 M
-----------------------------------------------------------
10.5 M    Trainable params
558 M     Non-trainable params
569 M     Total params
1,138.661 Total estimated model params size (MB)
Sanity Checking: 0it [00:00, ?it/s]/home/ubuntu/venv-comet-3.10/lib/python3.10/site-packages/pytorch_lightning/trainer/connectors/data_connector.py:208: UserWarning: num_workers>0, persistent_workers=False, and strategy=ddp_spawn may result in data loading bottlenecks. Consider setting persistent_workers=True (this is a limitation of Python .spawn() and PyTorch)
  rank_zero_warn(
[W CudaIPCTypes.cpp:15] Producer process has been terminated before all shared CUDA tensors released. See Note [Sharing CUDA tensors]
[W CudaIPCTypes.cpp:15] Producer process has been terminated before all shared CUDA tensors released. See Note [Sharing CUDA tensors]
[W CudaIPCTypes.cpp:15] Producer process has been terminated before all shared CUDA tensors released. See Note [Sharing CUDA tensors]
[W CudaIPCTypes.cpp:15] Producer process has been terminated before all shared CUDA tensors released. See Note [Sharing CUDA tensors]
Traceback (most recent call last):
  File "/home/ubuntu/venv-comet-3.10/bin/comet-train", line 8, in <module>
    sys.exit(train_command())
  File "/home/ubuntu/venv-comet-3.10/lib/python3.10/site-packages/comet/cli/train.py", line 192, in train_command
    trainer.fit(model)
  File "/home/ubuntu/venv-comet-3.10/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 608, in fit
    call._call_and_handle_interrupt(
  File "/home/ubuntu/venv-comet-3.10/lib/python3.10/site-packages/pytorch_lightning/trainer/call.py", line 36, in _call_and_handle_interrupt
    return trainer.strategy.launcher.launch(trainer_fn, *args, trainer=trainer, **kwargs)
  File "/home/ubuntu/venv-comet-3.10/lib/python3.10/site-packages/pytorch_lightning/strategies/launchers/multiprocessing.py", line 113, in launch
    mp.start_processes(
  File "/home/ubuntu/venv-comet-3.10/lib/python3.10/site-packages/torch/multiprocessing/spawn.py", line 198, in start_processes
    while not context.join():
  File "/home/ubuntu/venv-comet-3.10/lib/python3.10/site-packages/torch/multiprocessing/spawn.py", line 160, in join
    raise ProcessRaisedException(msg, error_index, failed_process.pid)
torch.multiprocessing.spawn.ProcessRaisedException:

-- Process 1 terminated with the following error:
Traceback (most recent call last):
  File "/home/ubuntu/venv-comet-3.10/lib/python3.10/site-packages/torch/multiprocessing/spawn.py", line 69, in _wrap
    fn(i, *args)
  File "/home/ubuntu/venv-comet-3.10/lib/python3.10/site-packages/pytorch_lightning/strategies/launchers/multiprocessing.py", line 139, in _wrapping_function
    results = function(*args, **kwargs)
  File "/home/ubuntu/venv-comet-3.10/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 650, in _fit_impl
    self._run(model, ckpt_path=self.ckpt_path)
  File "/home/ubuntu/venv-comet-3.10/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 1112, in _run
    results = self._run_stage()
  File "/home/ubuntu/venv-comet-3.10/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 1191, in _run_stage
    self._run_train()
  File "/home/ubuntu/venv-comet-3.10/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 1204, in _run_train
    self._run_sanity_check()
  File "/home/ubuntu/venv-comet-3.10/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 1276, in _run_sanity_check
    val_loop.run()
  File "/home/ubuntu/venv-comet-3.10/lib/python3.10/site-packages/pytorch_lightning/loops/loop.py", line 199, in run
    self.advance(*args, **kwargs)
  File "/home/ubuntu/venv-comet-3.10/lib/python3.10/site-packages/pytorch_lightning/loops/dataloader/evaluation_loop.py", line 152, in advance
    dl_outputs = self.epoch_loop.run(self._data_fetcher, dl_max_batches, kwargs)
  File "/home/ubuntu/venv-comet-3.10/lib/python3.10/site-packages/pytorch_lightning/loops/loop.py", line 194, in run
    self.on_run_start(*args, **kwargs)
  File "/home/ubuntu/venv-comet-3.10/lib/python3.10/site-packages/pytorch_lightning/loops/epoch/evaluation_epoch_loop.py", line 84, in on_run_start
    self._data_fetcher = iter(data_fetcher)
  File "/home/ubuntu/venv-comet-3.10/lib/python3.10/site-packages/pytorch_lightning/utilities/fetching.py", line 178, in __iter__
    self.dataloader_iter = iter(self.dataloader)
  File "/home/ubuntu/venv-comet-3.10/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 435, in __iter__
    return self._get_iterator()
  File "/home/ubuntu/venv-comet-3.10/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 381, in _get_iterator
    return _MultiProcessingDataLoaderIter(self)
  File "/home/ubuntu/venv-comet-3.10/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 1034, in __init__
    w.start()
  File "/usr/local/lib/python3.10/multiprocessing/process.py", line 121, in start
    self._popen = self._Popen(self)
  File "/usr/local/lib/python3.10/multiprocessing/context.py", line 224, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "/usr/local/lib/python3.10/multiprocessing/context.py", line 288, in _Popen
    return Popen(process_obj)
  File "/usr/local/lib/python3.10/multiprocessing/popen_spawn_posix.py", line 32, in __init__
    super().__init__(process_obj)
  File "/usr/local/lib/python3.10/multiprocessing/popen_fork.py", line 19, in __init__
    self._launch(process_obj)
  File "/usr/local/lib/python3.10/multiprocessing/popen_spawn_posix.py", line 47, in _launch
    reduction.dump(process_obj, fp)
  File "/usr/local/lib/python3.10/multiprocessing/reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
AttributeError: Can't pickle local object 'EvaluationLoop.advance.<locals>.batch_to_device'

I suppose further adjustments are necessary.

I've been looking at this over lunch and I have made some progress, but not enough I believe. I do not have the time/patience currently to dig deeper into the idiosyncracies of PyTorch Lightning (where the issue lies) but I've written about what the issue is and what some one with more experience can do to fix it, in this PR: https://github.com/Unbabel/COMET/pull/160#issuecomment-1669470367
So perhaps if you share that PR with your network, other people may chime in and we can quickly solve it. But I cannot dig deeper into this for now, sorry! Maybe @ricardorei has some ideas.

Thanks @BramVanroy for your help, I appreciate it! I will first evaluate how not using encoder fine-tuning impacts the QE quality (#158). If the speed stays at 13.98it/s throughout training, it takes about 12-15h for 3-4 epochs for me.
Could @ricardorei confirm that they executed comet-train on multi-GPUs? What was their Python venv and CUDA version?

Hi all! I'll look into this today.
I had this fixed before but Pytorch-Lightning likes to changes things. Maybe its just a quick fix... Like Bram said in his PR I think the problem is with torchmetrics.

I updated lightning and metrics and I tested multGPU training and it was working. I used strategy: ddp and devices: 2 and everything went well.
Please give it a try

Use the latest version 2.1.0

Hi @ricardorei, I have just checked with this version, and I can execute training on multiple GPUs.
Thank you for your help!

Thanks, @ricardorei 🙂


Ergonomic tools for working with Commands from &mut World

What problem does this solve or what need does it fill?
Even when working with exclusive world access, work often needs to be deferred to satisfy soundness.
This occurs most commonly when iterating over a query or in other looping contexts.
What solution would you like?
Add World::commands(), which fetches a fresh instance of Commands.
Add World::apply_commands(), for a simple
What alternative(s) have you considered?
This is a partial, tightly-scoped alternative to some of the frustrations that inspired https://github.com/bevyengine/rfcs/pull/42.
Additional context
World::apply_commands can currently be achieved with this snippet:
use bevy::prelude::*;
use bevy::ecs::system::SystemState;

let world = World::new();

let mut system_state = SystemState::<Commands>::new(&mut world);

// Manually adding a command to the list to verify that this works
let mut commands = system_state.get_mut();
commands.spawn();

// Applies all accumulated commands to the world, causing them to take immediate effect
system_state.apply(&mut world);

Quick callout that Commands requires an &Entities reference, which is owned by World. This would block direct mutable Word accesses, which means by itself it isn't a suitable alternative to https://github.com/bevyengine/rfcs/pull/42

Quick callout that Commands requires an &Entities reference, which is owned by World. This would block direct mutable Word accesses, which means by itself it isn't a suitable alternative to bevyengine/rfcs#42

Hmm, right 🤔 I'm running into this with my naive impl.

Another note: this used to be possible in bevy 0.4, thanks to entity reservation being behind a mutex.

Another note: this used to be possible in bevy 0.4, thanks to entity reservation being behind a mutex.

Actually not quite. Entity reservation is thread safe due to how it uses atomics (both in 0.4 and now), but it was never behind a mutex. We just shared &Entities references using what can only be described as "disgusting disregard for everything Rust works so hard to prevent". (I'm using this uncharitable phrasing because I wrote the code).
https://github.com/bevyengine/bevy/blob/3b2c6ce49b3b9ea8bc5cb68f8d350a80ff928af6/crates/bevy_ecs/src/core/entities.rs#L322

The current implementation plays by the rules, but also makes it impossible to do what we did in 0.4.

Playing with a draft of this, there are two serious problems that must be overcome.

As discussed above, we need a &Entities to initialize a new Commands. This is required to verify that an entity exists before operating on it with EntityCommands, as well as to reserve entities for Commands.spawn.
We need to store a CommandQueue somewhere. Naively adding it as a field to World won't work, because then we can't mutate the world as that would be self-referential.

When Commands is used as a system parameter, the second problem is somehow (??) solved by storing the CommandQueue in the Fetch of the system parameter.
We could get around this by forcing users to create a new CommandQueue in their exclusive system, and then manually applying that. This is the solution currently taken by the test cases that use commands. I'm not a huge fan: it's verbose and exposes CommandQueue to the user, when that should largely be an implementation detail.

#3946 makes significant progress towards this; see the included example.

Closed in favor of #6184.


I can't release 1.2.0 (I'm not a gem owner)

@mkremer, @ahmeij  - I created a release draft here (no changes to actual code, just deps) : https://github.com/guard/guard-process/releases/tag/untagged-6b45f957c6f45d2bfb2f
You can add me if you want. (gem owner guard-process -a<EMAIL_ADDRESS>

Hi, & thanks for taking this on.
Owner added successfully.

<EMAIL_ADDRESS>

Released! Thanks @ahmeij, my pleasure!

Thanks for your help @e2 :)

Glad to help.


QUESTION: Why is the minimum value for connectionTimeout limited to 1000?

Hi, we're evaluating this connection pool and i have a question as to why the minimum value of connectionTimeout is limited to 1000 ms?

This property controls the maximum number of milliseconds that a client (that's you) will wait for a  connection from the pool. If this time is exceeded without a connection becoming available, a SQLException will be thrown. 1000ms is the minimum value. Default: 30000 (30 seconds)

If we have an issue with a single shard we will fill up all available threads on the server waiting for a connection rather than just drop the request straight away. Any thoughts on this?

After finding the forums I came across this answer:

The issue with setting the connectionTimeout too low (for example 10's of milliseconds) is that the connectionTimeout doubles as both the time that the user is willing to wait for a connection from the pool and the time that HikariCP itself is willing to wait when creating new connections to add to the pool.  A timeout in the 10's of milliseconds might be insufficient in many environment to establish and authenticate a new connection.

https://groups.google.com/forum/#!searchin/hikari-cp/"connectionTimeout"/hikari-cp/ohaKBZma3R0/RPxqIkJ3DRQJ

Which unfortunately makes HikariCP unusable for our  needs.

@bysse What minimum would be sufficient for your use case?

@bysse We can certainly consider a lower floor, but 125ms would be the absolute minimum.
Both Windows and Linux have a default scheduler quantum of 20ms.  If 16 threads are running on a 4-core CPU, a single thread may have to wait up to 80ms just to be allowed to run.  If the pool has a vacancy due to, for example, the retirement of a connection at maxLifetime, this leaves precious little time to establish a connection to fill the slot without returning a spurious failure to the client.
If careful consideration is not taken to ensure the CPU and scheduler are not oversaturated, running at a 125ms timeout, puts your application tier at risk of spurious failures even if the pool has available connections.

@brettwooldridge Thanks for the response. I would say that the problem as I see it is that the timeout for connecting, and the timeout waiting for a free connection from a full pool is the same. However, since we already have a thin layer that is limiting the number of simultaneous connections to the database we're not affected by the later. I think my initial thoughts was a bit skewed by our solution :)
We do not need to go as low as 125 ms but a smaller connectionTimeout of say 250 - 500 ms would be better.

@bysse I think we can agree to a 250ms floor for the connectionTimeout.

@brettwooldridge :+1:

@bysse HikariCP 2.4.4 RC1 is available for testing.

@brettwooldridge Great thanks! We'll increase the number of nodes running Hikari next week then i guess.

@bysse After implementing this fix, I'm one fence about what to do.  The connectionTimeout should be fine, but the validationTimeout has constraints imposed by JDBC.
Where possible HikariCP uses the Connection.isValid() method rather than executing a database query.  However, the isValid() interface, as defined says:

timeout - - The time in seconds to wait for the database operation used to validate the
connection to complete.

Similarly, the Statement.setQueryTimeout() accepts units only in seconds.
I'm going to have to fix the validation code.  As it stands a timeout less than one second is going to round down to 0 seconds, which is interpreted by the driver (per spec.) as no timeout.

@brettwooldridge I appreciate your work on this. For our use the 250 ms connectionTimeout is sufficient even though there will be different timeouts depending on the previous state on the connection.
However, if you feel that the discrepancies between connectionTimeout and validationTimeout introduce inconsistencies into the framework then I think that should weigh heavier than a specific feature request.

@bysse It creates more of a documentation issue than anything.
For example, if the driver supports setNetworkTimeout() which is denominated in milliseconds (MySQL, PostgreSQL), HikariCP will still "time-box" the isValid() or setQueryTimeout() calls down to 250ms resolution.
So, setting a validationTimeout of 250ms will be honored if the driver supports setNetworkTimeout(), even though we will call isValid() with a 1 second timeout.  However, the same 250ms timeout on a driver that doesn't not support setting the network timeout will result in a minimum 1 second resolution.  This will not be apparent to the user and will require documentation to explain.
Not a problem, just an issue.

Even if desirable it is not practical even 'today' is explained in this old thread https://community.oracle.com/thread/552257?start=3


[SPARK-50327][SQL] Factor out function resolution to be reused in the single-pass Analyzer

What changes were proposed in this pull request?
Factor out function resolution code to the FunctionResolution object.
Why are the changes needed?
Some Analyzer code will be used in both fixed-point and single-pass Analyzers. Also, Analyzer.scala is 4K+ lines long, so it makes sense to gradually split it.
Context: https://issues.apache.org/jira/browse/SPARK-49834
Does this PR introduce any user-facing change?
No. It's a pure refactoring.
How was this patch tested?
Existing tests.
Was this patch authored or co-authored using generative AI tooling?
No.

+1, LGTM. Merging to master.
Thank you, @vladimirg-db.


Create script to convert data to BIDS

Asked ChatGPT:
I have the MRI file structure below for one example patient:

├── S02_T2_SAG_SPC_ISO0_8
│   ├── stack_ICEBERG_LM_166_V3_M_S2_T2_SAG_SPC_ISO0.8.json
│   ├── v_ICEBERG_LM_166_V3_M_S2_T2_SAG_SPC_ISO0.8.json
│   └── v_ICEBERG_LM_166_V3_M_S2_T2_SAG_SPC_ISO0.8.nii
├── S03_DTI_64DIR_TRA_5mm_2pulses
│   ├── stack_ICEBERG_LM_166_V3_M_S3_DTI_64DIR_TRA_5mm_2pulses.json
│   ├── v_ICEBERG_LM_166_V3_M_S3_DTI_64DIR_TRA_5mm_2pulses.bval
│   ├── v_ICEBERG_LM_166_V3_M_S3_DTI_64DIR_TRA_5mm_2pulses.bvec
│   ├── v_ICEBERG_LM_166_V3_M_S3_DTI_64DIR_TRA_5mm_2pulses.json
│   └── v_ICEBERG_LM_166_V3_M_S3_DTI_64DIR_TRA_5mm_2pulses.nii
├── S04_DTI_64DIR_TRA_5mm_2pulses
│   ├── stack_ICEBERG_LM_166_V3_M_S4_DTI_64DIR_TRA_5mm_2pulses.json
│   ├── v_ICEBERG_LM_166_V3_M_S4_DTI_64DIR_TRA_5mm_2pulses.bval
│   ├── v_ICEBERG_LM_166_V3_M_S4_DTI_64DIR_TRA_5mm_2pulses.bvec
│   ├── v_ICEBERG_LM_166_V3_M_S4_DTI_64DIR_TRA_5mm_2pulses.json
│   └── v_ICEBERG_LM_166_V3_M_S4_DTI_64DIR_TRA_5mm_2pulses.nii
├── S05_DTI_64DIR_TRA_5mm_2pulses
│   ├── stack_ICEBERG_LM_166_V3_M_S5_DTI_64DIR_TRA_5mm_2pulses.json
│   ├── v_ICEBERG_LM_166_V3_M_S5_DTI_64DIR_TRA_5mm_2pulses.bval
│   ├── v_ICEBERG_LM_166_V3_M_S5_DTI_64DIR_TRA_5mm_2pulses.bvec
│   ├── v_ICEBERG_LM_166_V3_M_S5_DTI_64DIR_TRA_5mm_2pulses.json
│   └── v_ICEBERG_LM_166_V3_M_S5_DTI_64DIR_TRA_5mm_2pulses.nii
├── S06_T1_SAG_MT_FL3D_2mm_HF
│   ├── stack_ICEBERG_LM_166_V3_M_S6_T1_SAG_MT_FL3D_2mm_HF.json
│   ├── v_ICEBERG_LM_166_V3_M_S6_T1_SAG_MT_FL3D_2mm_HF.json
│   └── v_ICEBERG_LM_166_V3_M_S6_T1_SAG_MT_FL3D_2mm_HF.nii
├── S07_T1_SAG_FL3D_2mm_HF
│   ├── stack_ICEBERG_LM_166_V3_M_S7_T1_SAG_FL3D_2mm_HF.json
│   ├── v_ICEBERG_LM_166_V3_M_S7_T1_SAG_FL3D_2mm_HF.json
│   └── v_ICEBERG_LM_166_V3_M_S7_T1_SAG_FL3D_2mm_HF.nii
├── S08_t1_mp2rage_sag_p3_1mm_iso_INV1
│   ├── stack_ICEBERG_LM_166_V3_M_S8_t1_mp2rage_sag_p3_1mm_iso_INV1.json
│   ├── v_ICEBERG_LM_166_V3_M_S8_t1_mp2rage_sag_p3_1mm_iso_INV1.json
│   └── v_ICEBERG_LM_166_V3_M_S8_t1_mp2rage_sag_p3_1mm_iso_INV1.nii
├── S09_t1_mp2rage_sag_p3_1mm_iso_INV2
│   ├── stack_ICEBERG_LM_166_V3_M_S9_t1_mp2rage_sag_p3_1mm_iso_INV2.json
│   ├── v_ICEBERG_LM_166_V3_M_S9_t1_mp2rage_sag_p3_1mm_iso_INV2.json
│   └── v_ICEBERG_LM_166_V3_M_S9_t1_mp2rage_sag_p3_1mm_iso_INV2.nii
├── S10_t1_mp2rage_sag_p3_1mm_iso_T1_Images
│   ├── stack_ICEBERG_LM_166_V3_M_S10_t1_mp2rage_sag_p3_1mm_iso_T1_Images.json
│   ├── v_ICEBERG_LM_166_V3_M_S10_t1_mp2rage_sag_p3_1mm_iso_T1_Images.json
│   └── v_ICEBERG_LM_166_V3_M_S10_t1_mp2rage_sag_p3_1mm_iso_T1_Images.nii
└── S11_t1_mp2rage_sag_p3_1mm_iso_UNI_Images
    ├── stack_ICEBERG_LM_166_V3_M_S11_t1_mp2rage_sag_p3_1mm_iso_UNI_Images.json
    ├── v_ICEBERG_LM_166_V3_M_S11_t1_mp2rage_sag_p3_1mm_iso_UNI_Images.json
    └── v_ICEBERG_LM_166_V3_M_S11_t1_mp2rage_sag_p3_1mm_iso_UNI_Images.nii

Can you write a python script that converts this file structure into the "Brain Imaging Data Structure". The tricks would be to use the following strings for conversion, that should systematically be present across all patients:
"T2_SAG" --> sub-xx/anat/sub-xx_T2.nii.gz
"DTI_64DIR" --> sub-xx/dwi/sub-xx_chunk-yy_DWI.nii.gz
"T1_SAG_MT_FL3D" --> sub-xx/anat/sub-xx_mt-on_MTS.nii.gz
"T1_SAG_FL3D" --> sub-xx/anat/sub-xx_mt-off_MTS.nii.gz
"mp2rage_sag_p3_1mm_iso_T1" --> sub-xx/anat/sub-xx_T1map.nii.gz
"mp2rage_sag_p3_1mm_iso_UNI" --> sub-xx/anat/sub-xx_UNIT1.nii.gz

with:
- xx being the patient number, which on this tree above, corresponds to "LM_166". Here, xx should be "LM166"
- note that the input files are nii. Make sure to add a code in the script to zip them into nii.gz
- for the DWI scans, the files with extension ".bval" and ".bvec" should also be copied and renamed with the same file prefix as the newly named DWI nii.gz scan.
- there are usually 3 DWI scans. They each correspond to different fields of view, and thus should be differenciated with chunk-1, chunk-2, chunk-3.
- for each NIfTI file, the associated JSON file should also be copied and renamed. The one without the "stack_" prefix should be copied.
- in case there is more than one scan that matches the definition above (except for the DWI scans), always use the latest one.

The input of the script is the folder that includes all the patient with the same structure as the one shown above.

@Kaonashi22 for healthy subjects, eg: "2023_10_02_DEV2_206_01_ICEBERG_ME_Sujet10", what would be the name to give in the BIDS structure? Suggestions:

sub-HC206
sub-DEV206
sub-DEV206Sujet10

Hi Julien,
For 2023_10_02_DEV2_206_01_ICEBERG_ME_Sujet10", I would rename

sub-DEV206Sujet10

Thanks!

From: Julien Cohen-Adad @.>
Sent: February 2, 2024 16:16
To: sct-pipeline/spine-park @.>
Cc: Lydia Chougar, Dr @.>; Mention @.>
Subject: Re: [sct-pipeline/spine-park] Create script to convert data to BIDS (Issue #1)
@Kaonashi22https://github.com/Kaonashi22 for healthy subjects, eg: "2023_10_02_DEV2_206_01_ICEBERG_ME_Sujet10", what would be the name to give in the BIDS structure? Suggestions:

sub-HC206
sub-DEV206
sub-DEV206Sujet10

—
Reply to this email directly, view it on GitHubhttps://github.com/sct-pipeline/spine-park/issues/1#issuecomment-1924707641, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BFCFJYXFTFJFPR5BK4KDGS3YRVJSBAVCNFSM6AAAAABCFOYNT2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMRUG4YDONRUGE.
You are receiving this because you were mentioned.Message ID: @.***>


Expand error message for missing/extra arguments in fragment(...)

Emit error message saying how many extra arguments were given
and expected when number of extra arguments does not line up with
number of question marks in fragment(...) string.

:heart: :green_heart: :blue_heart: :yellow_heart: :purple_heart:


Adding `if_exists="override" Mode

Is your feature request related to a problem? Please describe.
is there mode to do truncate load for existing table if_exists="override", it could be useful to avoid dropping table
Describe the solution you'd like
being able to override data in existing table.

Can you share a bit more about your use case? You'd like to replace the data, but keep the table intact? I suppose this is related to your other request about preserving clustering/partitioning?

correct i like to replace the data, but keep the table intact. dropping table always has risk of losing table metadata, and could cause failing queries for the time between drop and create. override is safer operation.

What if we call it "truncate" to match what the API is actually doing? If/when we support non-load jobs, we can run a DML "truncate" operation first via SQL.

could cause failing queries for the time between drop and create

To avoid these problems, a single "truncate" load job would work, but does risk changing the schema accidentally. I think some combination of writing to a staging table, then a SQL merge statement to delete existing rows and replace with rows from the staging table would be required to do this atomically without changing the table metadata.

could cause failing queries for the time between drop and create

To avoid these problems, a single "truncate" load job would work, but does risk changing the schema accidentally. I think some combination of writing to a staging table, then a SQL merge statement to delete existing rows and replace with rows from the staging table would be required to do this atomically without changing the table metadata.

Could such an implementation also be considered for the replace mode? The time between "drop" and "create" can be significant for larger tables and results in the failure of downstream queries.
Could the replace mode be re-implemented with a more atomic solution, i.e.:

Write to a staging location
Drop the existing table
Rename the staging location to the target location

What would be the downsides?


Additional Tesseract API methods, including ResultRenderer

Return the partially processed result image when pixCombineMasked failed rather than throwing exception. If result image is null, return the original.
Additional Tesseract API methods, including ResultRenderer.

Is there any reason for this behaviour? As I'm not sure this is a good idea
for the following reasons (in no particular order):

it breaks the generally accepted standard of throwing a error or
alternatively aborting and returning false in the case of a failure.
the behaviour is not obvious from the signature
prevents the consumer from choosing their own behaviour in the case of a
failure

Agree, but this case is a bit different. In the original example code lineremoval.c, pix8 is returned regardless of the outcome of pixCombineMasked, whose returned int value is generally ignored in Leptonica code. I have changed it to throw exception if pix8 is null.

Closing as functionality is covered by work done in issue #193.


User profiles do not render properly with new pipeline

Caused by https://github.com/lobsters/lobsters/issues/933
Needs to be fixed before next stable release

https://github.com/lobsters/lobsters/pull/1177


Support protos as part of State API

Since protos are serializable, we should support them as part of the state API

I haven't heard of a real-world use case for this yet so closing for now.


Transactions Q

Jan, hey.
First of all thanks for this great lib.
I have a Q regarding transactions.
I'm trying to implement persistent job queue on top of MapDB blocking queue with multiple producers/consumers (threads).
I'd like to have a behaviour similar to any other MQ broker - when an item is polled from the queue in a transaction, it won't be available to any other consumer unless the transaction is rolled back.
I guess the transactions out-of-the-box (TxMaker) won't blend - the item consumed will be available to other threads until the transaction is committed.
Any suggestions/ideas?
Thanks in advance.
Daniel

MapDB has regular transaction. So the item will be available to all transactions, but only single tx will be able to commit. Remaining transaction will be rolled back with TxRollbackException


Mitmproxy CLI tools

Similar to #1521, I'd like to open a discussion about the mitmproxy command line interface tools:
At the moment we have three core applications:

mitmproxy - a curses-based terminal UI / "frontend" to inspect, manipulate HTTP flows, etc.
mitmdump - basically only the proxy part of mitmproxy, without any interaction or "frontend"
mitmweb - serves a similar use case as mitmproxy, but through a HTTP-JS "frontend"

Since the main project is called mitmproxy, as well as the GitHub organisation, and the main repository, it was always a bit unintuitive how the actual binaries are called.
Therefore I'd like to propose that we change the naming of the three main applications (i.e. the words we type into our terminal to start one of the three "frontends") to:

mitmproxy - same as before, starts the curses-based UI
mitmproxy dump behaves as the old mitmdump
mitmproxy web behaves as the old mitmweb

Now, in this case the first argument would determine the frontend to be started. By default (i.e. without a first argument) the curses-based frontend should be started. However, we could also change that, if we detect we are running without a tty (or the screen is too small) and simply fallback to dump mode.
Or maybe instead of introducing a new argument, we could use mitmproxy-dump and mitmproxy-web as binary names. This probably requires one or two more keystrokes to autocomplete though.
For backward compatibility we should still ship the old binaries as well -and add a notification about the rename, so users can adapt their aliases / workflows / scripts.
What do you think?
I'm open for other naming conventions or even completely different names. 😄
/cc @mitmproxy/devs

Let's think this through. Our options are:

Keep the status quo: mitmproxy (console), mitmweb and mitmdump. Awkward, but reflects the reality that right now mitmproxy console is by a wide margin what most people use.
Be logically consistent. Because console mode is not supported on some platforms, just having it as the default doesn't really work. This gives us "mitmproxy console", "mitmproxy web" and "mitmproxy dump". I don't like the guessing of looking at the tty to determine what to fire up, so I'd like to be explicit. So just running mitmproxy would list the tools and exit.
Rename the mitmproxy binary to better reflect the fact that the project is not just about the terminal. All the options here are a bit ugly: "mitmconsole", "mitmterm"?
mitmproxy-console, mitmproxy-web, mitmproxy-dump. I don't like this one.

There are a few other benefits I can think of for option 2:

Producing a single binary would cut distribution sizes by two thirds. Not certain how significant this is at the moment, but would be a nice side-effect.
This opens ops the possibility of adding more commands. For instance "mitmproxy convert" to convert from mitmdumps to other formats is something I've occasionally wanted. Very soon now, we're going to have the ability to upload mitmdumps for viewing online (I just need to clear time to work on this), so "mitmproxy share" might be another one.

Let's wait for other folks to chip in on this, but tentatively I think the proposal makes sense. We can't make this sort of change often, though, so let's proceed cautiously.

Another aspect we should consider here is that we ideally ship mitmweb with
an Electron/cefpython wrapper further down the track. At that point, the
whole "web" naming does not even make sense to the user anymore and we'd
have to introduce new changes again. It may make sense to wait until we
have a clear picture of that (which platforms, portable binaries or
installers, ...?) before we start renaming stuff.
What I would dislike about option 2 is that it bloats up every current
console command. This would also require a command line interface for
invoking any of the tools (unless we ship installers/do $x to have useful
shortcuts). As anecdotal evidence, I've seen OSX users that aren't too
accustomed to CLI apps and just double-click on the mitmproxy binary. Not
our core audience, but we should keep in mind that this is just the tip of
the iceberg if we support Windows properly with mitmweb.
I'd like to throw in option 1b for consideration: De-emphasize the mitmweb
binary name (I could imagine having an installer that creates a "mitmproxy
ui" shortcut linking to mitmweb) and continue living with mitmproxy and
mitmdump.
Right now, I would lean towards figuring out mitmweb's distribution target
audience and user share first, even though that may still take a bit.
On Sun, 4 Sep 2016, 15:52 Aldo Cortesi<EMAIL_ADDRESS>wrote:

Let's think this through. Our options are:

Keep the status quo: mitmproxy (console), mitmweb and mitmdump.
Awkward, but reflects the reality that right now mitmproxy console is by a
wide margin what most people use.
Be logically consistent. Because console mode is not supported on
some platforms, just having it as the default doesn't really work. This
gives us "mitmproxy console", "mitmproxy web" and "mitmproxy dump". I don't
like the guessing of looking at the tty to determine what to fire up, so
I'd like to be explicit. So just running mitmproxy would list the tools and
exit.
Rename the mitmproxy binary to better reflect the fact that the
project is not just about the terminal. All the options here are a bit
ugly: "mitmconsole", "mitmterm"?
mitmproxy-console, mitmproxy-web, mitmproxy-dump. I don't like this
one.

There are a few other benefits I can think of for option 2:

Producing a single binary would cut distribution sizes by two
thirds. Not certain how significant this is at the moment, but would be a
nice side-effect.
This opens ops the possibility of adding more commands. For instance
"mitmproxy convert" to convert from mitmdumps to other formats is something
I've occasionally wanted. Very soon now, we're going to have the ability to
upload mitmdumps for viewing online (I just need to clear time to work on
this), so "mitmproxy share" might be another one.

Let's wait for other folks to chip in on this, but tentatively I think the
proposal makes sense. We can't make this sort of change often, though, so
let's proceed cautiously.
—
You are receiving this because you are on a team that was mentioned.
Reply to this email directly, view it on GitHub
https://github.com/mitmproxy/mitmproxy/issues/1526#issuecomment-244633254,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AA-NPvZr_lulxz0ys8M9w2GOmdlVHVqJks5qm0urgaJpZM4J0XhK
.

I'm pushing this one out beyond 1.0 - we don't need to make a decision now. I agree that making "normal" invocations harder is something to be concerned about - on Unixen we can go some way towards getting the benefits of multiple binaries with symlinks and then looking at the invocation binary name. I also feel that whatever the future of mitmweb, it should remain an option to run it outside of Electron. This is useful when, for instance, running on a headless device and debugging remotely.


Support latest diffuser version

Hi @guoyww,
if this project can support the latest diffusers like diffusers v0.17?
I would like to use this great project with diffusers v0.17, but I have run into several issues with mismatched functions and classes. I was wondering if you have any advice on how to resolve them.

Hi @LaLaLailalai . We will update the repo to newer diffusers in the future. Different diffusers versions define some layers/functions at different files, and you may pay attention to these changes. Thanks.

@guoyww  3 months have passed, when will support latest diffuser version??

+1
would also like to see animatediff with lateset diffusers. Many api and functions have changed since diffusers 0.11, making it extremely difficult to develop new applications based on animatediff


remove the requirement for window_relative_pos for ui_button_at

Describe the feature
ui_button_at currently requires a window_relative_pos or topLeftCorner in C#, is this absolutely necessary? It would be simpler to not have to use this and it doesn't seem like its necessary
An example use-case
ui_button_at("click_me", [0.0, 1.0]);

misread the documention, the vec 2 specifies the size, not the position


[Admin] Publish

Learn Build status updates of commit 5117528:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

README.md
:white_check_mark:Succeeded

SECURITY.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the learn.microsoft.com contributor guidesPost your question in the Learn support channel