When running the feature extraction pipeline with an array of strings as input,  the embedding result is different from running the pipeline for one string at a time.

System Info
transformer version: 2.17.2
Nodejs version: 18.19.0
Environment/Platform

[ ] Website/web-app
[ ] Browser extension
[X] Server-side (e.g., Node.js, Deno, Bun)
[ ] Desktop app (e.g., Electron)
[ ] Other (e.g., VSCode extension)

Description
When running the feature extraction pipeline with an array of strings as input,  the embedding result is different from running the pipeline for one string at a time. e.g.
import { pipeline } from "@xenova/transformers";
import * as assert from "node:assert";

const extractor = await pipeline(
  "feature-extraction",
  "Alibaba-NLP/gte-base-en-v1.5",
  {
    quantized: false
  }
);

const sentences = [
  "milk chocolate",
  "Title: a good milk chocolate receipt. Description: Milk chocolate is a type of chocolate that is made from a blend of cocoa solids, cocoa butter, sugar, and milk or milk powder. Unlike dark chocolate, which has a higher percentage of cocoa solids and little to no milk content, milk chocolate contains a significant amount of milk, giving it a creamier texture and a sweeter, milder flavor. Milk chocolate typically has a lower cocoa content (often between 10% to 50%) compared to dark chocolate. It is commonly used in candy bars, confections, and various desserts.",
  "Title: a good chocolate milk receipt. Description: Chocolate milk is a sweetened, flavored milk beverage that is made by mixing milk (usually cow's milk) with cocoa powder or chocolate syrup and often sugar. The drink can be served cold or hot and is popular as both a refreshing treat and a comfort beverage. Chocolate milk can be made at home or purchased pre-mixed in bottles or cartons. Some versions also include additional ingredients like vanilla extract or stabilizers to improve texture and flavor.",
  "sydney water revenue report",
];

//Encode all 4 strings in one go
const fullResult = (await extractor(sentences, { normalize: true, pooling: "cls" })).tolist();

//Run pipeline with one string at a time
const result0 = (await extractor(sentences[0], { normalize: true, pooling: "cls" })).tolist();
const result1 = (await extractor(sentences[1], { normalize: true, pooling: "cls" })).tolist();
const result2 = (await extractor(sentences[2], { normalize: true, pooling: "cls" })).tolist();
const result3 = (await extractor(sentences[3], { normalize: true, pooling: "cls" })).tolist();

//The first one matches, ok
assert.deepEqual(result0[0], fullResult[0]);
// second one matches, ok
assert.deepEqual(result1[0], fullResult[1]);
// third one doesn't match.  will fail.
// fullResult[2][0] == -0.030997734516859055 but result2[0][0] == -0.030997756868600845
assert.deepEqual(result2[0], fullResult[2]);
assert.deepEqual(result3[0], fullResult[3]);

Reproduction
Run the following code snippets with node 18 or higher to reproduce the problem:
import { pipeline } from "@xenova/transformers";
import * as assert from "node:assert";

const extractor = await pipeline(
  "feature-extraction",
  "Alibaba-NLP/gte-base-en-v1.5",
  {
    quantized: false
  }
);

const sentences = [
  "milk chocolate",
  "Title: a good milk chocolate receipt. Description: Milk chocolate is a type of chocolate that is made from a blend of cocoa solids, cocoa butter, sugar, and milk or milk powder. Unlike dark chocolate, which has a higher percentage of cocoa solids and little to no milk content, milk chocolate contains a significant amount of milk, giving it a creamier texture and a sweeter, milder flavor. Milk chocolate typically has a lower cocoa content (often between 10% to 50%) compared to dark chocolate. It is commonly used in candy bars, confections, and various desserts.",
  "Title: a good chocolate milk receipt. Description: Chocolate milk is a sweetened, flavored milk beverage that is made by mixing milk (usually cow's milk) with cocoa powder or chocolate syrup and often sugar. The drink can be served cold or hot and is popular as both a refreshing treat and a comfort beverage. Chocolate milk can be made at home or purchased pre-mixed in bottles or cartons. Some versions also include additional ingredients like vanilla extract or stabilizers to improve texture and flavor.",
  "sydney water revenue report",
];

//Encode all 4 strings in one go
const fullResult = (await extractor(sentences, { normalize: true, pooling: "cls" })).tolist();

//Run pipeline with one string at a time
const result0 = (await extractor(sentences[0], { normalize: true, pooling: "cls" })).tolist();
const result1 = (await extractor(sentences[1], { normalize: true, pooling: "cls" })).tolist();
const result2 = (await extractor(sentences[2], { normalize: true, pooling: "cls" })).tolist();
const result3 = (await extractor(sentences[3], { normalize: true, pooling: "cls" })).tolist();

//The first one matches, ok
assert.deepEqual(result0[0], fullResult[0]);
// second one matches, ok
assert.deepEqual(result1[0], fullResult[1]);
// third one doesn't match.  will fail.
// fullResult[2][0] == -0.030997734516859055 but result2[0][0] == -0.030997756868600845
assert.deepEqual(result2[0], fullResult[2]);
assert.deepEqual(result3[0], fullResult[3]);

// fullResult[2][0] == -0.030997734516859055 but result2[0][0] == -0.030997756868600845

Minor (insignificant) floating point precision differences like this are expected and will not impact your application. When comparing floating point values, you should set a tolerance value (like 1e-5)

@xenova Thanks a lot for your reply
Can I confirm if it is the expected behaviour? Also curious about how could it happen as:

it only happens to the result of the third string. The result of the first string (fullResult[0]) & the second string (fullResult[1]) 100% match result0[0] & result0[1] for all 18 decimal places.
the behaviour is reproducible. Always the result of the first & second string matches but third one doesn't.
if it's the floating point precision issue introduced during the extraction, I am wondering why it doesn't impact the first & second string

Thanks so much for your detailed reply. It helps a lot 👍

I have also encountered this problem before, and it seems that the larger the batch size, the less accurate the data becomes
Because after importing the vector database, when batch size=50, the search results are completely distorted


company portal not updating form Installamator

Hello,
Through Installamator company portal app its not installing the latest version need help on this,
@Armin Briegel - @scriptingosx

See: https://github.com/Installomator/Installomator/issues/1857
As a workaround If you are using Jamf with a parameter for the label, just add another parameter:
"downloadURL=https://go.microsoft.com/fwlink/?linkid=853070"
This will override the downloadURL for the specified label that is built into the script.

Problem is the appNewVersion variable no longer works either since they've removed the version number from the filename. You'll need to eliminate that variable as well. I haven't been able to find any pages on Microsoft's website that mention the release notes or version number of the latest Company Portal release which is what would be needed to use appNewVersion.

FYI I created pull request #1997 to fix this label.


Login screen

I am merging the login screen branch with the master code base. I have completed the login/sign up screens you to view the app, download or clone it into your local machine and run npm install and then run npm start to start up dev server. This new version includes some major bug fixes and new feature implementation I also want to make a feature to make sure a user's password must be more than 4 letters long to signup as well as a system to confirm that there is. not duplicated accounts by using the Chill&chat api's find user endpoint as outlined in the documentation.
Thanks and kind regards, Alvin :);

Ok, I will merge this branch now, this branch has no conflicts or errors nice👍 !!!

This branch will not be deleted there will be new features commin' soon!


Document initial database class diagram

The database for the Django backend of auto-grader can be modeled with a relatively simple class diagram.
There's a tool called Mermaid JS that allows you to create graphs from code pretty easily. I think we can use this to brainstorm an intitial schema for the backend.
The auto-grader architecture document already uses this format for showing off some graphs, but for this one we'll use a class diagram instead.
Here's an example of what the schema might look like. I think we can start with this, and create a PR to see what it looks like, then work on it a bit more.
Student
    - many-to-many with course
    - many-to-many with assignment
    - name
    - student id
Assignment
    - many-to-many with course
    - name
    - instructions
Course
    - name
    - code
Instructor
    - many-to-many with course
    - name
    - instructor id

 classDiagram
      Course <|-- Assignment
      Assignment <|-- Question
      Professor <|-- Course
      Course <|-- Student

      class Assignment{
          string description
          string name
          string dueDate
          int grade
          Question[] questions
      }

      class Course {
          string course_id
          string name
          string description
          string section
          string professor
      }

      class Student {
         int student_id
         string name
         Course[] courses_registered_in
         Assignment[] assignments_due
         Assignment[] assignments_finished
      }

      class Professor {
          string name
          string email
          course course_teaching
          Student[] students_in_course
          Assignment[] assignments_for_class
      }

      class Question {
          string description
          boolean complete
          int grade
      }

Ooh this is looking great! I do want to take a longer look at the relationships between tables, say how Assignment has Question[] (mostly because I don't know which syntax is best myself :P ). But I think this should be good to move into a markdown file!
Can you make a file SCHEMA.md and add this to it? Then you should be able to make a new branch and create a pull request for it 💯

SCHEMA.md
not sure where to make a new branch at


Stun prod

About the PR

adds a stun type weapon that the crew can access via crafting this is not a good sec alternative as it does less stun drain per hit and cannot be carried easily.
Media

[ ] I have added screenshots/videos to this PR showcasing its changes ingame, or this PR does not require an ingame showcase

Changelog

:cl:

add: Added craft able stun weapon.

time to figure out how do add to the crafting table
(:

time to figure out how do add to the crafting table

to the what?

time to figure out how do add to the crafting table

to the what?

CRAFTIN TABLE i need to make le stun prod a craft able weapon

add a crafting recipe prototype

add a crafting recipe prototype

working on this now i believe its working just testing with a copy of spears prototype then im going to alter it to something suitable.

yeah crafting is confusing

yeah crafting is confusing

Make it not confusing, shrimple as that

this will be left for a moment

@noudoit you coming back to this?


list index out of range on non-XML fileGrp

When I try to open a fileGrp which contains only images, but no PAGE files, and if it does not contain the original image (like OCR-D-IMG) but only derived images (referenced by other fileGrps via AlternativeImage, e.g. OCR-D-IMG-BIN), then browse-ocrd collapses with the following error:
Traceback (most recent call last):
  File "ocrd_browser/view/base.py", line 66, in <lambda>
    configurator.connect('changed', lambda _source, *value: self.config_changed(name, value))
  File "ocrd_browser/view/images.py", line 40, in config_changed
    self.reload()
  File "ocrd_browser/view/images.py", line 69, in reload
    self.pages.append(self.document.page_for_id(display_id, self.use_file_group))
  File "ocrd_browser/model/document.py", line 233, in page_for_id
    pcgts = self.page_for_file(page_files[0])
IndexError: list index out of range

(Otherwise display of derived page images works fine.)
Oh, and could you please set the ocr-d topic for this repo to increase its visibility with OCR-D users?

309b83c

thanks!
Hmm, it does work for image-only fileGrps now, but still happens if some fileGrp misses a pageId (as soon as I step to that page). Re-open or new issue?

Ah, ok. So the fileGrp has no PAGE-XML entry for a particular page_id, right?

So the fileGrp has no PAGE-XML entry for a particular page_id, right?

Exactly! (Happens when using -g selectively during processing, or adding pages to a workspace later-on.)

Can you check with the latest commit https://github.com/hnesk/browse-ocrd/commit/fd02515712775e159a54dae245ac602cc6f3f9b2 ?
I found it difficult to reproduce a crash, I tried it here
https://github.com/hnesk/browse-ocrd/blob/fd02515712775e159a54dae245ac602cc6f3f9b2/tests/model/test_document.py#L92
so I'm not sure if this is the fix.
Browsing to a fileGrp without PAGE-XML should now give you a warning:
11:04:37.785 WARNING ocrd_browser.model.document - No PAGE-XML and no image for page 'PHYS_0020' in fileGrp 'OCR-D-GT-PAGE'
and that should be it

Can you check with the latest commit fd02515 ?

Yes, this works perfectly – thanks a lot!

I found it difficult to reproduce a crash, I tried it here
https://github.com/hnesk/browse-ocrd/blob/fd02515712775e159a54dae245ac602cc6f3f9b2/tests/model/test_document.py#L92

Indeed, this is a correct way to simulate missing pages. (You also could have created that workspace dynamically, by removing the page ad-hoc in the test, but never mind.)

Browsing to a fileGrp without PAGE-XML should now give you a warning:

It does exactly that.


No more works on github gists

I think this extension worked on GitHub gists also earlier.
suddenly I am seeing this is not working on GitHub gists.

Fixed by #9


Error running app RVCStudio at streamlit

Error - "Oh no.
Error running app. If this keeps happening, please contact support."
The message above displays whenever I try to access the "https://rvc-studio.streamlit.app/" url.

unfortunately they've decided to close it due to the youtube music downloader.


Multiple APIs in the same project

Very nice lib, thank you for spending time on this.
I'm using the lib in an Expo project, where I call a Gateway application which routes requests to multiple backend services. Is it possible to create separate clients for those backends?
Atm. I have "codegen": "openapi-rq -i http://localhost:8082/v3/api-docs --base http://localhost:8080/svc/a -c axios --format prettier ", which is generating the code for the service behind port 8082, with a base URL pointing to service a. How could I add another service here? My main problem is that creating a second service will create second static OpenAPI object which I have to configure identically to the one from service a.

@HJK181
Is it possible to generate code for each service using the --output option?
a:
openapi-rq -i http://localhost:8082/v3/api-docs --base http://localhost:8080/svc/a -c axios --format prettier  --output openapi-a

b:
openapi-rq -i http://localhost:8082/v3/api-docs --base http://localhost:8080/svc/b -c axios --format prettier  --output openapi-b

@HJK181
Is it possible to generate code for each service using the --output option?
a:
openapi-rq -i http://localhost:8082/v3/api-docs --base http://localhost:8080/svc/a -c axios --format prettier  --output openapi-a

b:
openapi-rq -i http://localhost:8082/v3/api-docs --base http://localhost:8080/svc/b -c axios --format prettier  --output openapi-b

But this gives me two static OpenAPI objects in two folder’s right? Than, I have to set the same bearer token and interceptors on two independent objects instead of one?

@HJK181
Perhaps the exportCore option from @hey-api/openapi-ts could be useful. Try generating OpenAPI.ts with exportCore=true once, and then run it with exportCore=false afterwards. OpenAPI.ts won't be overwritten. By exporting the same object, you should be able to use it across multiple services.
In this library, the exportCore option is fixed to true, so I need to fix it.

Will this be fixed in the next major version?


Add support for schema and table filtering in redshift-usage ingestor.

Is your feature request related to a problem? Please describe.
I was giving a try to the new redshift-usage ingestor and I realised that all schemas and tables were processed. I tried to apply the same filtering settings than my regular redshift ingestor and these seem to have no effect.
I tried the following attributes:

schema_pattern
table_pattern
database_alias

Describe the solution you'd like
I would like to be able to use these settings in the same way they are used in the redshift ingestor.

Hi @cbuffevant, I've migrated this to our new Feature Request portal - please upvote & subscribe to the post for automated updates as we move through review & development! We'll close out this issue in the meantime.
https://feature-requests.datahubproject.io/b/feedback/p/redshift-usage-support-table-and-schema-filtering


Fix/deps

fixed: #196

@tushar5526 I've checked locally, the update version of deps does not break anything.

Have you tested the docker build as well? Which Python version are you using?

@tushar5526 I've tested it for python [3.9, 3.10, 3.11]

@Mr-Sunglasses let's not use >= as it might install a dependency with a change in the major version. Can you replace them with ~= to fix the major version?

@tushar5526 I've done the necessary changes, I think now we're ready to merge 🧜🏻♀


zarf tools kubectl version returns error code

Environment
Device and OS: OSX
App version: v0.25.2
Kubernetes distro being used: v1.23.16
Other:
When creating a pre-flight check for the kubernetes version, the zarf tools kubectl command was returning an error. It looks like the clientVersion is not getting set in a way the command likes, potentially based on how its being imported and used by Zarf tools.
Steps to reproduce
Run the following:
$ zarf tools kubectl version --short
Flag --short has been deprecated, and will be removed in the future. The --short output will become the default.
Client Version: v0.0.0-master+$Format:%H$
Kustomize Version: v4.5.7
Server Version: v1.23.16-eks-48e63af
error: could not parse pre-release/metadata (-master+$Format:%H$) in version "v0.0.0-master+$Format:%H$"
$ echo $?
1

Expected result
For the return code for the command zarf tools kubectl version --short  to be zero.
Actual Result
Visual Proof (screenshots, videos, text, etc)
Severity/Priority
Simple work around by:
  - name: one
    required: true
    actions:
      onDeploy:
        after:
        - cmd: "./zarf tools kubectl version --short || true"
          setVariable: OUTPUT

Additional Context

https://github.com/defenseunicorns/zarf/pull/1624 - this will actually be resolved in 1624


SendGrid plugin init script

SendGrid recently updated their Wordpress plugin and they dropped support for Username & Password authentication. It now requires an API key. Setup can't be done programmatically, it requires a user to create an API key on SendGrid dashboard manually.
An init script for SendGrid has been added. It provides instructions for user to create an API key, and sets Heroku env variable.

@xyu why not try this? I can confirm SendGrid is not working.

@xyu Can this be merged?
I made these changes manually to get SendGrid working.

@Tailzip this looks great I'm so sorry it's taken so long to merge because I had left this repo in a semi abandoned state. Thank you so much!!


Fallout: Vaults & Deathclaws PNP -Vehicle Update

Changes / Comments
Fixed a few bugs regarding some of the skills.
Removed Vehicle Condition Marks and Successful Hits.
Fixed an issue with Storage Space.
Roll20 Requests
Comments are very helpful for reviewing the code changes. Please answer the relevant questions below in your comment.

[x] Does the pull request title have the sheet name(s)? Include each sheet name.
[x] Is this a bug fix?
[ ] Does this add functional enhancements (new features or extending existing features) ?
[ ] Does this add or change functional aesthetics (such as layout or color scheme) ?
[ ] If changing or removing attributes, what steps have you taken, if any, to preserve player data ?
[ ] If this is a new sheet, did you follow Building Character Sheets standards ?

If you do not know English. Please leave a comment in your native language.

And next pull will be on January 7th, so there is no hurry really.


Refactor gas limit

It refactors the gas estimation to make it more readable (easier to maintain)

Needs to rebranch from master. I'll do that in another PR


Establish confirmed relationships with co-designers

https://github.com/orgs/informalsystems/teams/quint-co-designers


Test UI does not send HTTP GET body

My web service expect a get request with a body. The generated swagger look correct (see below), the ui shows it with parameter type "body" and even the generated curl statement is correct (and works), but when the request is send from UI, it has no body.
Method definition in controller:
[HttpGet("testbody/{param}")]
[SwaggerResponse(System.Net.HttpStatusCode.OK, Type = typeof(IEnumerable))]
public IActionResult GetTestBody(String param, [FromBody]String properties = null) {
curl: curl -X GET --header 'Content-Type: application/json' --header 'Accept: application/json' -d 'property' 'http://localhost:61240/api/Ldap/testbody/param'
Excerpt from generated swagger.json:
"/api/Ldap/testbody/{param}": {
"get": {
    "tags": [
        "Ldap"
    ],
    "operationId": "ApiLdapTestbodyByParamGet",
    "consumes": [
        "application/json",
        "text/json",
        "application/json-patch+json"
    ],
    "produces": [ ],
    "parameters": [
        {
            "name": "param",
            "in": "path",
            "required": true,
            "type": "string"
        },
        {
            "name": "properties",
            "in": "body",
            "required": false,
            "schema": {
                "type": "array",
                "items": {
                    "type": "string"
                }
            }
        }
    ],
    "responses": {
        "200": {
            "description": "OK",
            "schema": {
                "type": "array",
                "items": {
                    "type": "string"
                }
            }
        },
        "204": {
            "description": "No Content"
        }
    },
    "deprecated": false
}

},

I am seeing the same thing with 6.0.0-beta901 for posts. The CURL statement works, but the only way to get parameters to be passed from the UI is to mark the parameters as [FromQuery]. Parameters without an attribute, or using [FromBody], the UI does not pass. Verified using Fiddler.

If the generated Swagger is correct but the UI is not behaving as expected, then you would be better off logging this issue with the swagger-ui folks.


fix: chat responsiveness &  Auth box Scrolling

Related Issue
The support chat was not responsive , covering the entire screen of the device

Closes: #[issue number]
Description of Changes

Made changes to make the support chat responsive by adding a close button to it separately.
this button is only for sm screens only
doesn't cover the entier screen
adjusted the chat to be scrollable in the small screen
adjusted the margins to fit the sm sizes .

Checklist:

[ ] My code adheres to the established style guidelines of this project.
[ ] I have conducted a self-review of my code.
[ ] I have included comments in areas that may be difficult to understand.
[ ] I have made corresponding updates to the project documentation.
[ ] My changes have not introduced any new warnings.

Screenshots

Original
Updated

Please ensure all test should be passed.. see what causing error for prettier to fail.
npx prettier --write .

https://github.com/user-attachments/assets/db02d508-2de4-4a20-b57a-c3986d81d5ff
Just a small fix: this crosss bar should be uniform.. why we replaced it with purple cross with alignment in down

Screen.Recording.2024-07-23.at.9.32.24.PM.mp4
Just a small fix: this crosss bar should be uniform.. why we replaced it with purple cross with alignment in down

i did it coz u see that when user logs in we display their name in the header with that the cross in the header becomes very small and i thought i was not a good ux , it'll be hard to use that's why i added the cross btn separately below to enhance the ux

Screen.Recording.2024-07-23.at.9.32.24.PM.mp4
Just a small fix: this crosss bar should be uniform.. why we replaced it with purple cross with alignment in down

i did it coz u see that when user logs in we display their name in the header with that , the cross in the header becomes very small and i thought it was not a good ux , it'll be hard to use that's why i added the cross btn separately below to enhance the ux

Agreed! Fine, then could you do this will all screen sizes just to have continuity of design?

I think the chat box.. our message should be on right..? ok let's plan to re-create the chatbox, integrating AI to it..  let's plan to take this change a step further.

I'll create a requirement Doc on the support app. Like how we can enhance it functionality-wise and UI-wise. by today EOD

I'll create a requirement Doc on the support chat. Like how we can enhance it functionality-wise and UI-wise. by today EOD

sure Anmol

Some last UI fix:

The alignment of the message that the user sends should be on the right.
Provide some dark color or may be a shadow effect to the header and footer (where we write the message) so it will be distinguishable with the chat window.

Some last UI fix:

The alignment of the message that the user sends should be on the right.
Provide some dark color or may be a shadow effect to the header and footer (where we write the message) so it will be distinguishable with the chat window.

@anmode  just pushed the commit , take a look

i removed the comments anmol

Congrats @code-par-learn on your first PR 🚀


[BUG] ValueError: malformed node or string when fine tuning example

Bug report checklist
Hello! I was attempting to follow the example  fine tuning exercise. However, when I run the training.py script, I quickly get a "malformed node or string" error.
Is a CUDA compatible processor required for training? Or is this something else in my setup?
Describe the bug
This is the error stack:
File "/my-path/chronos-forecasting/scripts/training/train.py", line 692, in <module>
    app()
  File "/opt/homebrew/lib/python3.12/site-packages/typer/main.py", line 326, in __call__
    raise e
  File "/opt/homebrew/lib/python3.12/site-packages/typer/main.py", line 309, in __call__
    return get_command(self)(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.12/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.12/site-packages/typer/core.py", line 661, in main
    return _main(
           ^^^^^^
  File "/opt/homebrew/lib/python3.12/site-packages/typer/core.py", line 193, in _main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.12/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.12/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.12/site-packages/typer/main.py", line 692, in wrapper
    return callback(**use_params)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.12/site-packages/typer_config/decorators.py", line 92, in wrapped
    return cmd(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^
  File "/Users/williama/Library/Mobile Documents/com~apple~CloudDocs/William's Cloud/Oxford/Dissertation/chronos-forecasting/scripts/training/train.py", line 563, in main
    training_data_paths = ast.literal_eval(training_data_paths)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File<EMAIL_ADDRESS>line 112, in literal_eval
    return _convert(node_or_string)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File<EMAIL_ADDRESS>line 104, in _convert
    left = _convert_signed_num(node.left)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File<EMAIL_ADDRESS>line 85, in _convert_signed_num
    return _convert_num(node)
           ^^^^^^^^^^^^^^^^^^
  File<EMAIL_ADDRESS>line 76, in _convert_num
    _raise_malformed_node(node)
  File<EMAIL_ADDRESS>line 73, in _raise_malformed_node
    raise ValueError(msg + f': {node!r}')
ValueError: malformed node or string on line 1: <ast.BinOp object at 0x30a1455d0>

And this my config training/configs/chronos-t5-small.yaml:
training_data_paths:
- "./data_for_ft.arrow"
probability:
- 1.0
context_length: 512
prediction_length: 64
min_past: 60
max_steps: 200_000
save_steps: 100_000
log_steps: 500
per_device_train_batch_size: 32
learning_rate: 0.001
optim: adamw_torch_fused
num_samples: 20
shuffle_buffer_length: 100_000
gradient_accumulation_steps: 1
model_id: google/t5-efficient-small
model_type: seq2seq
random_init: true
tie_embeddings: true
output_dir: ./output/
tf32: true
torch_compile: true
tokenizer_class: "MeanScaleUniformBins"
tokenizer_kwargs:
  low_limit: -15.0
  high_limit: 15.0
n_tokens: 4096
lr_scheduler_type: linear
warmup_ratio: 0.0
dataloader_num_workers: 1
max_missing_prop: 0.9
use_eos_token: true

Expected behavior
I expected training to fulfill successfully.
To reproduce

Install the chronos package with the training extra.
Generate sample time series and convert it to an arrow format per the example (https://github.com/amazon-science/chronos-forecasting/tree/main/scripts)
Modify training/configs/chronos-t5-small.yaml to add the data_for_ft.arrow file path with 1.0 probability
Run the training command pointing to the config:

python training/train.py training/configs/chronos-t5-small.yaml

Environment description
Operating system: macOS 14.5
Python version: 3.12
CUDA version: None, running on Mac M1 Pro CPU
PyTorch version: 2.3.0
HuggingFace transformers version: 4.40.2
HuggingFace accelerate version: 0.30.1

Yes, you'd typically need a a CUDA compatible GPU for training and fine-tuning. You may be able modify the scripts to support CPU fine-tuning but it may be quite slow.

@williamrodz Closing this due to inactivity. Please feel free to re-open if you have questions.


coqPackages.category-theory: Set highest compatibility to coq 8.14

This is a simple bump, because category-theory builds fine with Coq 8.14, but not yet with 8.15.

Does this build? I get:

File "./Lib/Nomega.v", line 7, characters 2-17:
Error: Unable to locate library Coq.omega.Omega.

It builds with 8.14 on my machine, but not with 8.15. Perhaps something in the build environment you're in is different?

Can also confirm it works with just a clean nix-build here.

@GrahamcOfBorg build coqPackages.category-theory

I'm curious as to why it regressed the version date for all Coq versions 8.10 and up? It moved us back from the November release to the October release.

What do you mean? I cannot see any October or November release in this file, only a July and a December release. BTW, what about creating proper tags for these releases, instead of storing the relationship from version number to commit in nixpkgs? This would help anyone willing to package the same versions in the opam-coq-archive.

This test of an update nixpkgs-unstable shows that the selected version of category-theory for Coq 8.12 is the December release: https://github.com/coq-community/coq-nix-toolbox/runs/4617699649?check_suite_focus=true#step:6:11
Same thing for Coq 8.11: https://github.com/coq-community/coq-nix-toolbox/runs/4617732931?check_suite_focus=true#step:6:11

Never mind, I read it wrong. Thank you!


Valgrind reported memory access bugs

Fix one serious and two less serious uninitialized memory access bugs in openthread. All of these were found using valgrind.

zero signal handler memory access
zero otSecurityPolicy structure during SecurityPolicy class constructor
do not access netlink message error TLVs if the extended ack flag is not set in the message

this can happen when the kernel is too old to support the netlink socket option.
this bug has led to segmentation faults and in one case a livelock.

Looks good. Thanks. Couple of suggested changes below:

updated

make pretty still fails - not sure why exactly

@abtink @jwhui i can take over this for @gabekassel. rebased and nits should be fixed. PTAL.


MSVC support

Rebase it, don't "reverse merge"

Rebase correctly in order to remove e4d85732379fc6b5efead54879f20cc76ab4607e


Clarify support on CF, not on Neo

Clarify Mendix apps can be deployed only to CF, not Neo

@feddovanede
Is this a recent change? I note that we mention using Neo in a Marketplace release note. https://docs.mendix.com/releasenotes/app-store/index#october-1st-2018

No, as far as I know it's not. We have never supported running Mendix apps on SAP BTP Neo environments, as far as I know. I thought this was already in the documentation, but I can't find it (anymore).


Add OpenID Connect Discovery Endpoint

An authorization server should support the /.well-known/openid-configuration endpoint as specified by https://openid.net/specs/openid-connect-discovery-1_0.html.
This enables automatic client configuration (as already implemented in spring security 5) and we can also integrate JWKS endpoint as of gh-2.
The Nimbus SDK may be used for generating the endpoint response.

This is also required to support #30 as it calls this endpoint.

Hi @rwinch @jgrandja Can I work on this issue?
I'm a developer at VMware in Workspace ONE authorization server team and I have experience in developing applications on top of Spring Security.

@anoopgarlapati Thank you for the offer! Let's find another task for you to contribute as this one is lower on the priority. We need to focus on implementing core OAuth 2.0 before we get into OpenID Connect 1.0.
I'm in the process of putting together a roadmap that will prioritize the features to be developed. I will log an issue that provides a link to the roadmap along with the first set of epics and associated issues.
Please take a look at the next set of issues (coming this week) and let me know which ones interest you and we can assign to you.
Sounds good?

@jgrandja Sounds good and I understand. Looking forward to contributing. Thanks!

@andifalk I couldn't find a way to convert this issue to an epic so I'll close this in favour of #55.


Liquid Trades do not preserve price and amount precision

Exchange
Liquid
Subscription type
Trades
Describe the bug
When creating the trade object from the Liquid trade websocket response the price and amount are truncated to the default 2 decimal places using toFixed() because a precision is not specified
https://github.com/altangent/ccxws/blob/master/src/exchanges/liquid-client.js#L284
To Reproduce
Steps to reproduce the behavior:

Start a trade subscription for any product on liquid

const ccxws = require("ccxws");
const binance = new ccxws.liquid();

const market = {
  id: "celbtc", // liquid IDs from ccxt do not work, websocket stream uses [base][quote] format
  base: "CEL", // standardized base symbol for Bitcoin
  quote: "BTC", // standardized quote symbol for Tether
};

// handle trade events
binance.on("trade", trade => console.log(trade));

// subscribe to trades
binance.subscribeTrades(market);

Compare the websocket response with the ccxws Trade object

Example output:
ccxws:
Trade {
  exchange: 'Liquid',
  quote: 'BTC',
  base: 'CEL',
  tradeId: '333106471',
  unix:<PHONE_NUMBER>000,
  side: 'buy',
  price: '0',
  amount: '482',
  buyOrderId: undefined,
  sellOrderId: undefined
}

JSON response from liquid:
{
	"channel": "executions_cash_celbtc",
	"data": "{\"created_at\":1593603406,\"id\":333106471,\"price\":0.00005167,\"quantity\":482,\"taker_side\":\"buy\"}",
	"event": "created"
}

Expected behavior
Precision for price and amount is preserved from the exchange, which is 8 digits of precision for both.

Thanks for the issue and PR!  Will release shortly.

Released in https://github.com/altangent/ccxws/releases/tag/v0.35.2.  Thanks!


Fix release pipeline with component detection

Fix the release pipeline building the Google's implementation library libwebrtc to build on on-prem agents.
Add a Component Governance task before the build to detect components in use by the project.
Ensure the security report for those components is clean by deleting unused code before detection and building to make sure potentially vulnerable code present in Google's build tools etc. is not used in the build and not shipped.
Update cgmanifest.json with the correct commits for all used git repositories.

For context the Google checkout/build tools are pulling a lot more code than necessary to just build libwerbrtc (we do not build tests/examples/tools), and some of those use old dependencies with known security vulnerabilities. This has no impact on the actual security of MixedReality-WebRTC since those tests/examples/tools are not even built, but the Component Governance task which monitors security is conservative and will report vulnerabilities for any component detected on the Azure Agent disk. So we delete those unused components with known vulnerabilities before scanning and before building, which ensures that they cannot possibly make it into MixedReality-WebRTC and makes security checks pass.


Rails  capstone project : budget app

Rails  capstone project : budget app
During this capstone project, I successfully achieved the following milestones:

Ruby and Ruby on Rails (RoR) were used to design a RESTful application.
The application includes several models based on an Entity-Relationship Diagram (ERD) in PostgresSQL. The models include
User, Group, Expense, and a bridge table called GroupExpense.
Authentication is implemented using Devise, while authorization is handled by cancancan.
An unauthorized user is presented with a splash screen upon accessing the application.
The category page serves as the home page for the application.
A transaction page has been created.
Unit and integration tests have been written using RSpec.
Authenticated users have the ability to create groups and expenses.
I have recorded a video presentation showcasing the project's details. please check below
The web service has been deployed on the Render site, as shown below
The design of the application adhered to a mobile-first approach

Project presentation video ⏺ . To watch the video please click the link below 🔗
video presentation
Live demo Link 🚀 🔥 🔥 Please be aware that if you register using a live link, your email confirmation may end up in your email address's spam or trash bin.  🔥 🔥
Live demo Link

Thank you, @AmaduKamara , for providing valuable feedback on my previous comment. I have carefully considered your input and made improvements to the sign-out option.


Neobundle needs manual :helpt for itself.

I just did a fresh installation of system, and switched to use neobundle.
But it seems we should exciptly do a :helpt for neobundle itself.

:NeoBundleDocs is executed when you updated plugins.
I think you don't update by neobundle.

Sorry? Maybe I am not clear enough.
I was saying the doc for neobundle itself is not generated automatically, not the docs for the plugins managed by neobundle.
Regards.

Please upload minimal .vimrc and reproduce ways from Vim starting.
I will check it.

No response.


Player Extension

Steps to reproduce
Go to setting and enable player Extension and select player you prefer
Play any anime
App crashes
Expected behavior
Aniume should play on Extention player
Actual behavior
App crashes
Crash logs
No response
Aniyomi version
<IP_ADDRESS>
Android version
Android 11
Device
Poco x3 nfc
Other details
No response
Acknowledgements

[X] I have searched the existing issues and this is a new ticket, NOT a duplicate or related to another open or closed issue.
[X] I have written a short but informative title.
[X] If this is an issue with an extension, I should be opening an issue in the extensions repository.
[X] I have tried the troubleshooting guide.
[X] I have updated the app to version <IP_ADDRESS>.
[X] I have updated all installed extensions.
[X] I will fill out all of the requested information in this form.

are you on stable or the preview?


[#CHK-2548]  add logos to paymethods list

This PR add logos to the paymethods list
List of Changes

manage asset from BE service

Motivation and Context

How Has This Been Tested?

Screenshots (if appropriate):
Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklist:

[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.

DRAFT until the resolution of a comment on the relative Jira task


Updated style guide according to discussions on GitHub

New PR because I accidentally deleted the branch on which PR #278 was based (oops), before it got merged.
Minor typos and rephrasing
Updated style guides according to the discussions in #271 and #272
Added prompts before shell command examples
Added tip to describe Copy button behavior for shell examples

Closes #271
Closes #272

I think the rebase was messed up, but I merged, seems in the end the merge commit is fine.
When properly merged/updated with main it shouldn't bring in any old commit as it was the case here.
https://docs.egi.eu/about/contributing/git/#update-local-feature-branch-with-changes-made-on-the-main-branch


Add Dockerfile

Creates a ~77 MB container that makes it easy to access this tool. E.g.:
alias vodafone-station-cli="podman run --rm --secret VODAFONE_ROUTER_PASSWORD,type=env vodafone-station-cli:latest"

Thanks for the contribution @salevdns!
Is this something useful? There is already a published package on npm which one can execute directly via npx.

For me it is useful.
Containerizing the tool provides a consistent, isolated, and easily deployable solution, regardless of the host environment's configuration. In this case, it also reduces disk space usage a bit.
My personal motivation behind this was to enable execution on an immutable host.


pyright complains about type hints in azure speech SDK

Describe the bug
pyright, the static type checker from Microsoft, complains about the default values in speech SDK methods, consider this as an example:
class AudioOutputConfig():
    """
    ...
    """

    def __init__(self, use_default_speaker: bool = False, filename: str = None,
                 stream: AudioOutputStream = None, device_name: str = None):

The type hints are incorrect because they assume all objects are implicitly nullable which is where pyright disagrees:
error: Argument of type "None" cannot be assigned to parameter "stream" of type "AudioOutputStream" in function "__init__"
    "None" is incompatible with "AudioOutputStream" (reportArgumentType)

Speech SDK should use correct type hints with explicit nullability like this:
def __init__(self, use_default_speaker: bool = False, filename: str | None = None,
                 stream: AudioOutputStream | None = None, device_name: str | None = None):

Fixed in the Speech SDK 1.41.1 release.


Agregator logs zero until first battery comes online.

Please include system details:
Three JBD serial batteries
Hardware: Cerbo/RPi???
I have run it on both Cerbo and RPi with same results
Setup: DVCC?
NA
Please provide output from dbus-spy.
NA
Please attach any relevant logs. Log location is /var/log/BatteryAggregator/current.
NA
The zero volts is annoying, and just plain wrong.  Agregator should not report until there is something to report. This wacks the scale of my graphs which must now go from zero to 14.2 instead of about 12 to 14.2.

Wow, "just plain wrong", harsh. The only thing that is plain wrong is that statement. The aggregator is a battery, a battery not reporting a voltage can cause issues. What I can do is probably delay dbus registration until a valid voltage is available.

You are right, I was unnecessarily harsh.  Especially directed at someone providing something for free,  Please accept my apologies.

On Aug 25, 2024, at 09:18, pulquero @.***> wrote:
Wow, "just plain wrong", harsh. The only thing that is plain wrong is that statement. The aggregator is a battery, a battery not reporting a voltage can cause issues. What I can do is probably delay dbus registration until a valid voltage is available.
—
Reply to this email directly, view it on GitHub https://github.com/pulquero/BatteryAggregator/issues/67#issuecomment-2308834755, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACBAH46NENZJGPFPPD22D5DZTHKTPAVCNFSM6AAAAABNCBQFG2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMBYHAZTINZVGU.
You are receiving this because you authored the thread.


Scrollbar component: fixed low speed scrolling on Firefox

Description
Fixed low scrolling speed on scrollbar component on Firefox.  The detect_browser package wasn't correctly called so it was unable to detect Firefox and apply the correct scroll delta.
const { detect } = require('detect-browser')
const browser = detect()

Types of changes

[ x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] Improvement/refactoring (non-breaking change that doesn't add any feature but make things better)

Thanks! I merged manually.
This change will be released as part of v2.1.


Add pipelines import to fix scrapy bug

The scrapy command is currently failing because the settings are failing to import the logging pipeline from a nested file. I've corrected that problem by introducing an import to the file's parent module that will line up with the path provided in the settings.
Here's the bug I saw that sent me down this route
$ scrapy crawl accc
no pupa_settings on path, using defaults
2017-10-14 13:47:18 [scrapy.utils.log] INFO: Scrapy 1.4.0 started (bot: documenters_aggregator)
2017-10-14 13:47:18 [scrapy.utils.log] INFO: Overridden settings: {'NEWSPIDER_MODULE': 'documenters_aggregator.spiders', 'CLOSESPIDER_ERRORCOUNT': 5, 'COOKIES_ENABLED': False, 'COMMANDS_MODULE': 'documenters_aggregator.commands', 'USER_AGENT': 'Documenters Aggregator [development mode]. Learn more and say hello at https://city-bureau.gitbooks.io/documenters-event-aggregator/', 'BOT_NAME': 'documenters_aggregator', 'SPIDER_MODULES': ['documenters_aggregator.spiders']}
2017-10-14 13:47:18 [scrapy.middleware] INFO: Enabled extensions:
['scrapy.extensions.corestats.CoreStats',
 'scrapy.extensions.telnet.TelnetConsole',
 'scrapy.extensions.memusage.MemoryUsage',
 'scrapy.extensions.logstats.LogStats']
2017-10-14 13:47:18 [scrapy.middleware] INFO: Enabled downloader middlewares:
['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
 'documenters_aggregator.middlewares.DocumentersAggregatorRobotsTxtMiddleware',
 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
 'scrapy.downloadermiddlewares.stats.DownloaderStats']
2017-10-14 13:47:18 [scrapy.middleware] INFO: Enabled spider middlewares:
['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
 'scrapy.spidermiddlewares.referer.RefererMiddleware',
 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
 'scrapy.spidermiddlewares.depth.DepthMiddleware']
Unhandled error in Deferred:
2017-10-14 13:47:18 [twisted] CRITICAL: Unhandled error in Deferred:

2017-10-14 13:47:18 [twisted] CRITICAL:
Traceback (most recent call last):
  File "/home/palewire/.virtualenvs/documenters-aggregator/lib/python3.5/site-packages/twisted/internet/defer.py", line 1386, in _inlineCallbacks
    result = g.send(result)
  File "/home/palewire/.virtualenvs/documenters-aggregator/lib/python3.5/site-packages/scrapy/crawler.py", line 77, in crawl
    self.engine = self._create_engine()
  File "/home/palewire/.virtualenvs/documenters-aggregator/lib/python3.5/site-packages/scrapy/crawler.py", line 102, in _create_engine
    return ExecutionEngine(self, lambda _: self.stop())
  File "/home/palewire/.virtualenvs/documenters-aggregator/lib/python3.5/site-packages/scrapy/core/engine.py", line 70, in __init__
    self.scraper = Scraper(crawler)
  File "/home/palewire/.virtualenvs/documenters-aggregator/lib/python3.5/site-packages/scrapy/core/scraper.py", line 71, in __init__
    self.itemproc = itemproc_cls.from_crawler(crawler)
  File "/home/palewire/.virtualenvs/documenters-aggregator/lib/python3.5/site-packages/scrapy/middleware.py", line 58, in from_crawler
    return cls.from_settings(crawler.settings, crawler)
  File "/home/palewire/.virtualenvs/documenters-aggregator/lib/python3.5/site-packages/scrapy/middleware.py", line 34, in from_settings
    mwcls = load_object(clspath)
  File "/home/palewire/.virtualenvs/documenters-aggregator/lib/python3.5/site-packages/scrapy/utils/misc.py", line 44, in load_object
    mod = import_module(module)
  File "/home/palewire/.virtualenvs/documenters-aggregator/lib/python3.5/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 986, in _gcd_import
  File "<frozen importlib._bootstrap>", line 969, in _find_and_load
  File "<frozen importlib._bootstrap>", line 958, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 673, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 665, in exec_module
  File "<frozen importlib._bootstrap>", line 222, in _call_with_frames_removed
  File "/home/palewire/Code/documenters-aggregator/documenters_aggregator/pipelines/__init__.py", line 1, in <module>
    from logging import DocumentersAggregatorLoggingPipeline
ImportError: cannot import name 'DocumentersAggregatorLoggingPipeline'
Traceback (most recent call last):
  File "/home/palewire/.virtualenvs/documenters-aggregator/bin/scrapy", line 11, in <module>
    sys.exit(execute())
  File "/home/palewire/.virtualenvs/documenters-aggregator/lib/python3.5/site-packages/scrapy/cmdline.py", line 149, in execute
    _run_print_help(parser, _run_command, cmd, args, opts)
  File "/home/palewire/.virtualenvs/documenters-aggregator/lib/python3.5/site-packages/scrapy/cmdline.py", line 89, in _run_print_help
    func(*a, **kw)
  File "/home/palewire/.virtualenvs/documenters-aggregator/lib/python3.5/site-packages/scrapy/cmdline.py", line 156, in _run_command
    cmd.run(args, opts)
  File "/home/palewire/Code/documenters-aggregator/documenters_aggregator/commands/crawl.py", line 14, in run
    crawler = list(self.crawler_process.crawlers)[0]
IndexError: list index out of range

I am getting this error as well in testing scrapy.

Good catch, sorry about that! I added a commit to this PR to surface all of the pipelines this way.

Dig it.


Add Zeni Software to Hiring Network

I'm working on Echnobind today so I can add this one also

Done


Suggestion for package structure

Just an idea
└── emission
    ├── non_thermal
    │   ├── bremsstrahlung.py
    │   └── other.py
    └── thermal
        ├── continuum.py
        ├── lines.py
        └── other.py

Emm, I personally think the first "child" may be better to be categorized in terms of emission mechanisms: bremsstrahlung (free-free), line emission (free-bound, bound-bound), and others (e.g., pion decay). The difference between thermal and nonthermal is only related to the source electron/ion distribution. For example, bremsstrahlung can also take a thermal distribution, and a Kappa distribution bridges both thermal and nonthermal.

@binchensun Are you suggesting that we have one function/model that can produce both thermal/non-thermal bremsstrahlung spectra?  If so, what would the API look like?

@binchensun Are you suggesting that we have one function/model that can produce both thermal/non-thermal bremsstrahlung spectra? If so, what would the API look like?
I have not yet thought about the detailed implementation. But a non-thermal thin-target bremsstrahlung takes in, e.g., a power-law, which normalizes upon n_iVn_e_nonthermal (if nonthermal electron distribution is described in density, not in flux), and thermal bremsstrahlung takes in T and EM (the latter has the same unit as n_iVn_e_nonthermal). The outputs are both in photons/(cm2 s keV). I guess it should be straightforward to put both the thermal and nonthermal thin-target modules into, e.g., bremsstrahlung.py? Looking at the SSWIDL implementation of thermal bremsstrahlung, it should be pretty straightforward: https://hesperia.gsfc.nasa.gov/ssw/packages/xray/idl/brem_49.pro

So here's an idea for the structure based on the thread here.  (It needs further discussion though.)  Modules/models would be based on emission mechanisms, rather than thermal/non-thermal.  (Although we could also have thermal/non-thermal models that are just wrappers around the relevant mechanisms.)
└── emission
    ├── bremsstrahlung.py
    ├── line.py
    ├── free-bound.py
    ├── other.py
    ├── thermal.py  # Wraps around relevant mechanisms and takes thermal distributions as inputs.
    ├── nonthermal.py  # Wraps around relevant mechanisms and takes nonthermal distributions as inputs.
└── particles
    ├── electrons  # Electron distributions
        ├── thermal.py
        └── nonthermal.py

We have a module for the particle distributions which are taken as inputs to the emission modules.
Just an idea.

So a direction has been chosen in #151  so going to close this issue for now.


Updates

@CAnorbo can you please make the section about surveillance a bit prettier? :D Font is way too big and I'm not sure if pic is even needed for that part.


Collision logging, Transformers for JSON and Standard Files

This is the PR #773, but on a new, rebased branch (the original PR was made 2 years ago).
Hi, I fitted the collsion_logging merge request from chapmajs to your current source code (https://github.com/johnrengelman/shadow/pull/126).
Also I followed your wish to use typed parameters (which was really needed, because after adding them, some problems became obvious).
After that I used the modified plugin and with the warnings I could fix my problems instantly. Great!
The only problem was, that I was flooded with warnings about colliding META-INF/NOTICE, META-INF/license.txt, readme.txt and similar files. So I added a transformer, StandardFilesMergeTransformer, which takes these standard files, which exist in all big projects in big numbers, and merges them.
Since these are all pretty primitive files regarding their structure, no special YAML or JSON files, I just concatenate their contents the following way:

Standard files are "META-INF/license", "META-INF/notice",  "META-INF/readme" or "readme". Which may have the following extensions: none, "txt", "md", "htm" or "html".
If a standard file is found (case insensitive), its content is read, stripped of leading and trailing newlines, and then searched in the list of already existing contents of that file. The search is case insensitive.

If no existing entry was found a new entry is created and its content and origin is recorded.
If an existing entry is found, its list of origins is extended with the origin of the content.

Later, the transformed entries are written like that:

    Origins: attoparser-2.0.5.RELEASE.jar, commons-codec-1.15.jar

                                 Apache License
                           Version 2.0, January 2004
    ...
    ================================================================================
    Origins: flyway-core-8.5.4.jar, flyway-mysql-8.5.4.jar

    Copyright (C) Red Gate Software Ltd 2010-2022

    Licensed under the Apache License, Version 2.0 (the "License");
    ...

The StandardFilesMergeTransformer transformer is added as default. It can be removed with "removeDefaultTransformers".
I added it as default because otherwise:

In projects with many dependencies the user gets flooded with information about duplicate entries
like "META-INF/notice.txt", "META-INF/license.txt"...
Important licensing information written in META-INF/license.txt and other files may be lost.
Helpful information written in readme files may be lost.
The merging of this plain text files is safe, there is no important logic to follow. Merged HTML may not look that good, but it works.

Added JsonTransformer for JSON files, based on JsonTransformer from @LogicFan. Their code is also under Apache License 2. Extended it, so it can be applied to multiple files.

Lowered the logging level for colliding META-INF/MANIFEST.MF files, from warning to debug, so the user doesn't get flooded with unimportant messages.

In discussion #710 it came to light that some people need the original module-info.class not to be removed, which is also a possible source for file collision problems. While it's hard to find a nice, quick solution for that, I at least added an option to include the module-info files and list their contents (listing appears only if module-info.class's aren't excluded).
The option is: allowModuleInfos(), and used in action in def 'Tests the info about the module-info.class, if removed from standard excludes'().

With that changes, any user should be capable to build a fat JAR with more than one Spring (Boot) dependency, right out of the box. Otherwise it's a real pain to find out which files collide, and how to merge colliding JSON files, since  that is the only colliding file type, which misses a merging transformer from this plugin.
And JSON is very common!

Sorry for the delay. Can you rebase again? I changed a lot on the main...

Sorry for the delay. Can you rebase again? I changed a lot on the main...

Did the rebase.


Consider updating tasko (now called CircuitPython_async)

Here's the repo: https://github.com/kvc0/CircuitPython_async
Pros:

More recent, hopefully better tested version
Better docstrings and code style
Cons:
Might break something

Possibly switch to micropython and use native async lib


final VT tests

TODO
mapnik_vector_tile.cpp

[x] catch statement in addImageBufferSync: https://github.com/mapnik/node-mapnik/blob/v2_spec/src/mapnik_vector_tile.cpp#L3321
[x] not_simple_feature usage: https://coveralls.io/builds/5013302/source?filename=src%2Fmapnik_vector_tile.cpp#L4814
[x] json_value_visitor used in toJSON: https://github.com/mapnik/node-mapnik/blob/v2_spec/src/mapnik_vector_tile.cpp#L2182
[x] reportGeometrySimplicitySync try/catch statement: https://coveralls.io/builds/5013302/source?filename=src%2Fmapnik_vector_tile.cpp#L5012 - should the last return be within the catch statement?
[x] reportGeometryValiditySync try/catch statement: https://coveralls.io/builds/5013302/source?filename=src%2Fmapnik_vector_tile.cpp#L5042 - should the last return be within the catch statement?

mapnik_image.cpp

[x] synchronous version of image.resizeSync with offset_x and offset_y set.

mapnik_map.cpp

[x] call buffer_size in map.render so we can set it via options

Everything complete here expect for the not_simple_feature, which is a bugger of a testcase. Tried throwing a "bowtie" feature at it with properties set like this:
it('successfully creates a not_simple_feature with a non-simple geometry', function(done) {
    var vtile = new mapnik.VectorTile(0,0,0);
    var gj_str = fs.readFileSync('./test/data/bowtie.geojson').toString('utf8');
    vtile.addGeoJSON(gj_str, 'not-simple', {strictly_simple: false, fill_type: mapnik.polygonFillType.evenOdd});
    var simple = vtile.reportGeometrySimplicity();
    // some sort of assert
    done();
});

but we still come out simplified. Can't seem to hit this if statement and am starting to think it isn't possible to bypass our simplification/fill_type steps in node-mapnik with the v2 specification at play. Some guidance from @flippmoke would be great - are we able to get a not_simple_feature successfully from javascript?
cc/ @springmeyer

:100: :tada:


build_image.sh:remove unrelated if condition

during e19ae1250374db1541133a839ccda791593a6e6c i got some leftover if condition from my debuging session, not sure how did i missed it
Fixing it

Since the fix is still in next, I will fix it directly


FieldArray fields.getAll does not reflect redux state

This bug is related to #2355.
I want to create a dynamic list of inputs where there is always empty input on the bottom:
<CustomField
  key={index}
  name={name}
  component={this.renderDynamicIngredientInput}
  onUpdate={(event: React.ChangeEvent<HTMLInputElement>) => {
    //console.log(fields.getAll());
    const lastField = fields.getAll()[fields.length - 1];
    //console.log(event.target.value.length, lastField.length, lastField, fields.length);
    if (event.target.value.length > 0 && lastField.length > 0) {
      fields.push("");
    }
  }}
/>

<input
  {...input}
  {...props}
  onChange={event => {
    input.onChange(event);
    onUpdate(event);
  }}
/>

The view:

The store state:

The console.log with getAll:

I'm using "redux-form": "^7.2.0", 😉
Basically it's get updated only on first type after focus in input.

I don't know why it haven't want to work but it didn't as you can see on screenshots.
Now I'm using fields.length and it works 😃
onType={(event: React.ChangeEvent<HTMLInputElement>) => {
  if (
    index === fields.length - 1 &&
    event.target.value.length > 0
  ) {
    fields.push("");
  } else if (event.target.value.length === 0) {
    fields.remove(index);
  }
}}

So I'm closing it for now. If it happens again, I will create a sandbox for it 😉

I ran through this problem and fields.getAll() was always one step behind the store values, here is a solution that worked for us:
onCopyFieldValues(fields, options) {
    let documents = [];
    fields.forEach((item, idx) => documents.push(fields.get(idx)));
    const newFields = copyFieldValues(documents, options); // running custom changes to the values
    this.props.change('documents', newFields);
  }

TLTR: fields.getAll() didn't work but calling fields.get(<index>) with fields.forEach() did the trick.

I ran through this problem and fields.getAll() was always one step behind the store values, here is a solution that worked for us:
onCopyFieldValues(fields, options) {
    let documents = [];
    fields.forEach((item, idx) => documents.push(fields.get(idx)));
    const newFields = copyFieldValues(documents, options); // running custom changes to the values
    this.props.change('documents', newFields);
  }

TLTR: fields.getAll() didn't work but calling fields.get(<index>) with fields.forEach() did the trick.

@giocodes, I tried using fields.forEach((item, idx) => documents.push(fields.get(idx))); but it is still 1 behind when I console logged the documents. pls help

I ran through this problem and fields.getAll() was always one step behind the store values, here is a solution that worked for us:
onCopyFieldValues(fields, options) {
    let documents = [];
    fields.forEach((item, idx) => documents.push(fields.get(idx)));
    const newFields = copyFieldValues(documents, options); // running custom changes to the values
    this.props.change('documents', newFields);
  }

TLTR: fields.getAll() didn't work but calling fields.get(<index>) with fields.forEach() did the trick.

@giocodes, I tried using fields.forEach((item, idx) => documents.push(fields.get(idx))); but it is still 1 behind when I console logged the documents. pls help

It have been stuck on this for a couple of days now. Were you able to solve it?
Thanks in advance

@JesseTheNoob @jKh98  sorry I haven't used this in a loooong time, if you setup a codesandbox to reproduce the issue I'll be happy to take a look.


Possible Solution for Encryption on Exit

Apologies in advance if this has already been thought of and it did not work. I don't have enough experience with Obsidian yet and only just started using it on Android today. But I was wondering if perhaps there is a way to keep the encryption active by switching to another vault via the ribbon button. It wouldn't be a perfect solution, but it could allow for vaults to stay encrypted apart from one another perhaps. Sorry again if this was already attempted and thank you for the plugin regardless!

Sorry for the late response. I didn't actually get what you mean. Could you explain it more openly?

Hi!
I think @tdadvocate found a bug.
I mean, if I close obsidian without clicking the lock in the left bar, the notes will not be encrypted and I can read them until I open obsidian again.

Yes you're right. I did it because plugin is supported on phones. Obsidian doesn't let start a function after you close the app on phones (on pc it works well). So encryption should start after you "submit" the encryption.
Also i'm thinking about to make available encryption only for pc. As i noticed most of users who needs encryption use pc only. If i make it pc only my plugin will be way better.


Memory Leak

** Do **
Fix a memory leak in the App.

Coverage increased (+0.0009%) to 99.389% when pulling f783a6c3baacad6d0149c6d63a13483c96d05619 on next-memory-leak into d6f03f7715710f64af2e8f75f3aa7aa7c11da432 on next.

Could you kindly rebase your commits from latest next? The diff (commits list) should only show your commits :)
Very well done with RxJS usage! 💖

@fahad19 : rebased !

Coverage increased (+0.0009%) to 99.389% when pulling 6231a99dd93e834cdea6bf62c45c52bc9766faf8 on next-memory-leak into d6f03f7715710f64af2e8f75f3aa7aa7c11da432 on next.

The first two commits still need to go away:

examples: kitchensink
register widgets

As seen here: https://github.com/Travix-International/frint/pull/90/commits

I am closing this PR the rebase is messy.
New one here: https://github.com/Travix-International/frint/pull/93


Retrieve API definition/API synapse file when enabling Inbuilt artifact synchronization

Description:
Currently, there is no mechanism to retrieve the synapse artifact of the API with the Inbuilt artifact synchronization[1]. The recommended approach is to fetch the data from GW REST APIs[2] where we have no API for this purpose.
Affected Product Version:
APIM 3.2.0
[1] https://apim.docs.wso2.com/en/3.2.0/install-and-setup/setup/distributed-deployment/synchronizing-artifacts-in-a-gateway-cluster/#inbuilt-artifact-synchronization
[2] https://apim.docs.wso2.com/en/3.2.0/develop/product-apis/gateway-apis/gateway-v1/gateway-v1/#tag/Get-API-Artifacts

Pull requests:
PMT - WSO2-CARBON-PATCH-4.6.0-1114
https://github.com/wso2-support/carbon-apimgt/pull/3026
https://github.com/wso2-support/carbon-apimgt/pull/3450
https://github.com/wso2-support/carbon-apimgt/pull/3452
https://github.com/wso2-support/carbon-apimgt/pull/3453
https://github.com/wso2-support/carbon-apimgt/pull/3456 (reverted)
https://github.com/wso2-support/carbon-apimgt/pull/3457 (reverted)
https://github.com/wso2-support/carbon-apimgt/pull/3466
https://github.com/wso2-support/carbon-apimgt/pull/3467
https://github.com/wso2-support/carbon-apimgt/pull/3468
https://github.com/wso2-support/orbit/pull/186
https://github.com/wso2-support/update-artifacts/pull/1377
UMT - Update Number 1502
https://github.com/wso2-support/carbon-apimgt/pull/3470
https://github.com/wso2-support/carbon-apimgt/pull/3477
https://github.com/wso2-support/carbon-apimgt/pull/3448 (Additional PR)
https://github.com/wso2-support/carbon-apimgt/pull/3484 (Additional PR)
Integration tests
https://github.com/wso2-support/product-apim/pull/725 - PMT
https://github.com/wso2-support/product-apim/pull/738 - UMT
Docs
https://github.com/wso2/docs-apim/pull/4278


fix export of "125 days differentiation of embryonic stem cells" dataset - GSE93593

Description of the issue
The dataset Single-cell RNA-seq analysis throughout a 125-day differentiation protocol that converted H1 human embryonic stem cells to a variety of ventrally-derived cell types. was exported for Release 23 but had a couple of issues related to the import.
The issues were all related to schemas used in the entities being outdated. After a quick scan of the staging area, everything but the project entity had an outdated schema, that is incompatible with the exporter.
Acceptance criteria

[ ] All schema upgrades have been identified (e.g. cell_suspension 5.0.0 --> 12.0.0)
[ ] Upgrades are categorised based on the ability to do them automatically or not
[ ] All entities are updated to the latest schemas
[ ] Metadata is deleted from the staging area
[ ] Project is re-exported

Schemas in the submission:

https://schema.humancellatlas.org/type/biomaterial/13.1.0/cell_suspension
https://schema.humancellatlas.org/type/biomaterial/15.3.0/donor_organism
https://schema.humancellatlas.org/type/biomaterial/14.3.0/cell_line
https://schema.humancellatlas.org/type/biomaterial/10.2.0/specimen_from_organism
https://schema.dev.data.humancellatlas.org/type/biomaterial/13.1.0/cell_suspension
https://schema.humancellatlas.org/type/process/9.0.0/process
https://schema.humancellatlas.org/type/protocol/biomaterial_collection/2.1.0/differentiation_protocol
https://schema.humancellatlas.org/type/protocol/biomaterial_collection/6.1.0/dissociation_protocol
https://schema.dev.data.humancellatlas.org/type/protocol/biomaterial_collection/3.0.0/enrichment_protocol
https://schema.dev.data.humancellatlas.org/type/protocol/sequencing/6.1.0/library_preparation_protocol
https://schema.dev.data.humancellatlas.org/type/protocol/sequencing/10.0.0/sequencing_protocol

I have divided the types of changes into 2:

Minor changes: Can be automatically updated. Do not need to be looked into (Minor changes do not break backwards compatibility)
Major changes: Will need to be manually looked.

The list goes as follows:
Minor changes

https://schema.humancellatlas.org/type/biomaterial/13.1.0/cell_suspension: https://schema.humancellatlas.org/type/biomaterial/13.3.0/cell_suspension
https://schema.humancellatlas.org/type/biomaterial/15.3.0/donor_organism: https://schema.humancellatlas.org/type/biomaterial/15.6.1/donor_organism
https://schema.humancellatlas.org/type/biomaterial/10.2.0/specimen_from_organism: https://schema.humancellatlas.org/type/biomaterial/10.5.0/specimen_from_organism
https://schema.dev.data.humancellatlas.org/type/biomaterial/13.1.0/cell_suspension: https://schema.humancellatlas.org/type/biomaterial/13.3.0/cell_suspension
https://schema.humancellatlas.org/type/protocol/biomaterial_collection/2.1.0/differentiation_protocol: https://schema.humancellatlas.org/type/protocol/biomaterial_collection/2.2.0/differentiation_protocol
https://schema.humancellatlas.org/type/protocol/biomaterial_collection/6.1.0/dissociation_protocol: https://schema.humancellatlas.org/type/protocol/biomaterial_collection/6.2.0/dissociation_protocol
https://schema.dev.data.humancellatlas.org/type/protocol/biomaterial_collection/3.0.0/enrichment_protocol: https://schema.humancellatlas.org/type/protocol/biomaterial_collection/3.1.0/enrichment_protocol
https://schema.dev.data.humancellatlas.org/type/protocol/sequencing/6.1.0/library_preparation_protocol: https://schema.humancellatlas.org/type/protocol/sequencing/6.3.1/library_preparation_protocol
https://schema.dev.data.humancellatlas.org/type/protocol/sequencing/10.0.0/sequencing_protocol: https://schema.humancellatlas.org/type/protocol/sequencing/10.1.0/sequencing_protocol
https://schema.humancellatlas.org/type/process/9.0.0/process: https://schema.humancellatlas.org/type/process/9.2.0/process

Major changes:

https://schema.humancellatlas.org/type/biomaterial/14.3.0/cell_line: https://schema.humancellatlas.org/type/biomaterial/15.1.0/cell_line

AI:

[ ] Retrieve all the entities related to the submission
[ ] With the map of updates, automatically update the entities with their correct schema
[ ] Export the project

Retrieve all the entities related to the submission
Files were skipped, since we can't import them anyways (Also, skipping set-up of the IngestApi object)
submission = api.get_submission('https://api.ingest.archive.data.humancellatlas.org/submissionEnvelopes/5cdb0344d96dad000858fff4')
biomaterials = api.get_related_entities('biomaterials', submission, 'biomaterials')
biomaterials = list(api.get_related_entities('biomaterials', submission, 'biomaterials'))
protocols = list(api.get_related_entities('protocols', submission, 'protocols'))
processes = list(api.get_related_entities('processes', submission, 'processes'))

Map:
{'https://schema.dev.data.humancellatlas.org/type/biomaterial/13.1.0/cell_suspension': 'https://schema.humancellatlas.org/type/biomaterial/13.3.0/cell_suspension',
 'https://schema.dev.data.humancellatlas.org/type/protocol/biomaterial_collection/3.0.0/enrichment_protocol': 'https://schema.humancellatlas.org/type/protocol/biomaterial_collection/3.1.0/enrichment_protocol',
 'https://schema.dev.data.humancellatlas.org/type/protocol/sequencing/10.0.0/sequencing_protocol': 'https://schema.humancellatlas.org/type/protocol/sequencing/10.1.0/sequencing_protocol',
 'https://schema.dev.data.humancellatlas.org/type/protocol/sequencing/6.1.0/library_preparation_protocol': 'https://schema.humancellatlas.org/type/protocol/sequencing/6.3.1/library_preparation_protocol',
 'https://schema.humancellatlas.org/type/biomaterial/10.2.0/specimen_from_organism': 'https://schema.humancellatlas.org/type/biomaterial/10.5.0/specimen_from_organism',
 'https://schema.humancellatlas.org/type/biomaterial/13.1.0/cell_suspension': 'https://schema.humancellatlas.org/type/biomaterial/13.3.0/cell_suspension',
 'https://schema.humancellatlas.org/type/biomaterial/14.3.0/cell_line': 'https://schema.humancellatlas.org/type/biomaterial/15.1.0/cell_line',
 'https://schema.humancellatlas.org/type/biomaterial/15.3.0/donor_organism': 'https://schema.humancellatlas.org/type/biomaterial/15.6.1/donor_organism',
 'https://schema.humancellatlas.org/type/process/9.0.0/process': 'https://schema.humancellatlas.org/type/process/9.2.0/process',
 'https://schema.humancellatlas.org/type/protocol/biomaterial_collection/2.1.0/differentiation_protocol': 'https://schema.humancellatlas.org/type/protocol/biomaterial_collection/2.2.0/differentiation_protocol',
 'https://schema.humancellatlas.org/type/protocol/biomaterial_collection/6.1.0/dissociation_protocol': 'https://schema.humancellatlas.org/type/protocol/biomaterial_collection/6.2.0/dissociation_protocol'}

With the map of updates, update the entities
Biomaterials:
for b in biomaterials:
    b['content']['describedBy'] = mapped_schemas[b['content']['describedBy']]
    r = api.patch(b['_links']['self']['href'], json={'content': b['content']})

cell_lines need to be updated with publications.hca_publication = false

Export the dataset
The dataset needs to be put manually into graph valid state. It has 2 files errors (No idea why, there is no message, but we are not exporting files).
The dataset has been graph-validated before so I am going to use the restart-submission script in the hca-ebi-dev repo to restart the submission

The export is working - That means, system-wise, this is effectively a DCP2 dataset with the upload area cleaned.

tasks:

[ ] rename terra staging area
[ ] re-export
[ ] update DCP25 import form

Re-export did not work.
Stuck in metadata export and there are several messages acknowledging data transfer error.
{"log":"2023-02-20 15:57:20,665 - TerraSubmissionExporter - ERROR - submission_uuid:0948a727-228f-4cfc-857e-6243c6aed08d - export_job_id:63f3985969335b24a7094647 - project_uuid:2043c65a-1cf8-4828-a656-9e247d4e64f1 - Rejecting message: {\"exportJobId\":\"63f3985969335b24a7094647\",\"submissionUuid\":\"0948a727-228f-4cfc-857e-6243c6aed08d\",\"projectUuid\":\"2043c65a-1cf8-4828-a656-9e247d4e64f1\",\"callbackLink\":\"/exportJobs/63f3985969335b24a7094647\",\"context\":{}} due to error: FileTransferCouldNotStart\n","stream":"stderr","time":"2023-02-20T15:57:20.666247229Z"}

I have deleted the staging area and forced a re-export

Log shows exported, but project shows up as exporting:
"2023-02-24 11:21:24.455  INFO 6 --- [pool-5-thread-1] o.h.i.s.monitor.SubmissionStateListener  : \tEnvelope '0948a727-228f-4cfc-857e-6243c6aed08d' -> State EXPORTED\n"

Will wait for a bit

On hold until after the HCA GM in July. Just needs time from Enrique to complete the update together probably with the Immune Cell Atlas

it's actually been solved from what I can see in ingest/import forms/etc.
Will close it


Convert project imports to relative imports

Closes #59

Add conversion of project imports to relative imports for lib files.
Add test for convertToRelativeImports

Question:
The example app has the following import but no corresponding dependency
import 'package:flutter_gen/gen_l10n/translations.dart';

Hello @gupta-shrinath. Thank you so much for sending in this PR but I am sadly archiving import_sorter (see #63). This means that your work will not be merged into import_sorter. Thank you so much for the effort in trying to improve the project.

While I think this feature is useful, it is in my opinion out of the scope of import_sorter.
The sole responsibility of this package should be to sort imports, not update them.
@gupta-shrinath would you instead be interested in creating a separate package for this?
I see that someone else already tried to create a solution for this:

https://www.reddit.com/r/FlutterDev/comments/gvj6eu/cli_tool_to_change_package_imports_to_relative/
https://pub.dev/packages/drtimport

Thank you @gleich for the package and I understand your situation and I agree with making someone the maintainer as it is a good decision in spirit of open source.
Hey @vkammerer thank you for appreciating efforts. I think this feature could be added to this package as the code is heavily dependent on existing code. Regarding making a new package for this feature I think is not worth because as you said  there is already a package / tool available.


cuDNN sometimes does not work at all (but sometimes does)...

Dear all,
I tried running a rather simple ConvNet using cuDNN (the supervised demo [1] where I replaced the convolutions, relu, and pooling by the cuDNN equivalents) and it sometimes trains fine, but sometimes just seems do be doing nothing at all (train and test error remain at the initial 19% global acc. forever). There are no errors reported. Using the cunn modules instead of the cuDNN ones also always works fine, so this seems to be a problem specific to cuDNN.
Another interesting thing: when I print the biases for some layers (same for the weights) using e.g.
print(model:get(1).bias:float())

they are all reported as nan's (in a run where cuDNN "hangs". Otherwise they are small numbers). Doing the same with the cunn-convolutions always prints nice small numbers.
Anyone else seen this problem before? Is there a way to at least detect when cuDNN "hangs" (besides monitoring the biases for nan's)? It's kind of annoying when your net is not learning at all and you do not know whether this is due to bad hyperparameters, or just because cuDNN hangs.
I'm on OSX 10.8 with Cuda 6.5 and cuDNN R2. I also use Caffe and Mocha with cuDNN backends and have never seen this problem occurring with these libs before.
Best
Michael
[1] https://github.com/torch/tutorials/blob/master/2_supervised/2_model.lua

there's a CuDNN R3 available which fixes some bugs, try that.
Second, can you post your model spec?

i meant R2RC3

Ok after installing R2 RC3 and rebooting, it seems to work. Might be that it was indeed some strange bug in cuDNN. I'll close this issue for now and reopen should I experience this behavior again. Thank you very much for you help and for pointing me to cudnn R2RC3 !
PS: For completeness, this is the model spec:
nn.Sequential {
  [input -> (1) -> (2) -> (3) -> (4) -> (5) -> (6) -> (7) -> (8) -> (9) -> (10) -> (11) -> output]
  (1): cudnn.SpatialConvolution
  (2): cudnn.ReLU
  (3): cudnn.SpatialMaxPooling
  (4): cudnn.SpatialConvolution
  (5): cudnn.ReLU
  (6): cudnn.SpatialMaxPooling
  (7): nn.View
  (8): nn.Dropout
  (9): nn.Linear(1600 -> 128)
  (10): cudnn.ReLU
  (11): nn.Linear(128 -> 10)
}

Is there anything wrong with it?


[Broken] nebula extractor doesn't support cookies

Checklist

[ ] I'm reporting a broken site support
[x] I've verified that I'm running yt-dlp version 2021.07.07
[x] I've checked that all provided URLs are alive and playable in a browser
[x] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[x] I've searched the bugtracker for similar issues including closed ones

Verbose log

[debug] Command-line config: ['--cookies', 'cookies.txt', '-v', '-N8', 'https://nebula.app/videos/lindsay-ellis-could-blazing-saddles-be-made-today']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] yt-dlp version 2021.07.07
[debug] Python version 3.9.6 (CPython 64bit) - Linux-5.12.14.s0ix-8+-x86_64-with-glibc2.33
[debug] exe versions: ffmpeg 4.4, ffprobe 4.4
[debug] Proxy map: {}
[debug] [Nebula] Extracting URL: https://nebula.app/videos/lindsay-ellis-could-blazing-saddles-be-made-today
ERROR: This video is only available for registered users. Use --cookies, --username and --password or --netrc to provide account credentials
Traceback (most recent call last):
  File "/home/tpm/.local/lib/python3.9/site-packages/yt_dlp/YoutubeDL.py", line 1142, in wrapper
    return func(self, *args, **kwargs)
  File "/home/tpm/.local/lib/python3.9/site-packages/yt_dlp/YoutubeDL.py", line 1167, in __extract_info
    ie_result = ie.extract(url)
  File "/home/tpm/.local/lib/python3.9/site-packages/yt_dlp/extractor/common.py", line 569, in extract
    ie_result = self._real_extract(url)
  File "/home/tpm/.local/lib/python3.9/site-packages/yt_dlp/extractor/nebula.py", line 167, in _real_extract
    nebula_token = self._retrieve_nebula_auth(display_id)
  File "/home/tpm/.local/lib/python3.9/site-packages/yt_dlp/extractor/nebula.py", line 88, in _retrieve_nebula_auth
    self.raise_login_required()
  File "/home/tpm/.local/lib/python3.9/site-packages/yt_dlp/extractor/common.py", line 1043, in raise_login_required
    raise ExtractorError(msg, expected=True)
yt_dlp.utils.ExtractorError: This video is only available for registered users. Use --cookies, --username and --password or --netrc to provide account credentials

Description

When supplying a cookies.txt file, the extractor complains about having to be a registered user. Indeed, the cookie support was cut before merging, but I can't see why. The relevant commit just says Removed obsolete/irrelevant/misused authentication methods:   environment variable, cookie jar and videopassword., even though I can't find any complaints about the cookie method and even a brief improvement suggestion.
Does @hheimbuerger perhaps remember why cookie support was cut? I feel a little uncomfortable letting my credentials lie around in a file and cookies are at least somewhat ephemeral and can't be used to alter my account, afaict.

It was probably an oversight. Could you test whether adding back https://github.com/yt-dlp/yt-dlp/pull/122/commits/13154498ce0d94325e05346596a7d79b83bb6df5#diff-6fff743502081039f289ebbc5e851557d66676e18eeb1e20c6b62c7387604c7dL125-L133 makes authentication with cookies work correctly?

That's weird indeed.
I want to say that it couldn't possibly happen to me, that I would remove this by accident. Yet I cannot remember a specific conversation we had about removing it either. So I guess it did. 😉
I'll create a PR for it tomorrow.

Here's my branch: https://github.com/hheimbuerger/yt-dlp/tree/issue496-nebula-add-cookie-auth-support
Tested locally with my own credentials.
@TpmKranz Are you able to check out and test this branch? Note that it's only supporting the current nebula domain (nebula.app), so in case you're using a very old cookie jar file, you might need to re-login to Nebula once (or just edit the domain in your cookie jar).
@pukkandan What's the correct approach for the interaction between credential auth (--netrc or --password) and cookie jar authentication?
The approach I'm following here is that if you're provided login credentials, potential cookies will be ignored and it will always attempt to authenticate with those credentials. (If they're wrong, even if the cookie token would be fine, authentication will fail.) Only if there's no credentials, it will use cookies. Is that a valid approach? Or should cookies take preference?
I'll open a PR once you approve my general approach. 😀

What's the correct approach for the interaction between credential auth (--netrc or --password) and cookie jar authentication?

I am not sure either. There aren't many extractors that support both login and cookies explicitly (for most cookie based logins, the presence of the cookie is sufficent for auth)

The approach I'm following here is that if you're provided login credentials, potential cookies will be ignored and it will always attempt to authenticate with those credentials. (If they're wrong, even if the cookie token would be fine, authentication will fail.) Only if there's no credentials, it will use cookies. Is that a valid approach?

I think that is good enough and is in-line with how other extractors behave

@TpmKranz Are you able to check out and test this branch? Note that it's only supporting the current nebula domain (nebula.app), so in case you're using a very old cookie jar file, you might need to re-login to Nebula once (or just edit the domain in your cookie jar).

Works just as expected, thank you very much. 👍🏻

The approach I'm following here is that if you're provided login credentials, potential cookies will be ignored and it will always attempt to authenticate with those credentials. (If they're wrong, even if the cookie token would be fine, authentication will fail.) Only if there's no credentials, it will use cookies. Is that a valid approach? Or should cookies take preference?

If I could make a suggestion: Why not try cookie auth first and fall back to credential based auth on failure? In case of success, this would save an API call for an apiToken that's only used once, whereas currently, you request a new apiToken everytime and ignore the one given in the cookies file that wouldn't be there if one didn't want to use it multiple times in the first place.
Once that's in place, you could go a step further and save the fresh apiToken to the given cookies file so that the apiToken could be reused and future API calls avoided.
It's a bit more complicated but I could also not think of any other reason to provide both a cookie file and credentials at the same time.
This way, I wouldn't have to mess around with my browser everytime the cookie expires and the cookie file could even be empty before executing yt-dlp and afterwards, it would contain a fresh apiToken to be used again and again until it expires!
Just my 2c; the way it currently works is functional as well, just not as comfortable. Thanks again!

This way, I wouldn't have to mess around with my browser everytime the cookie expires and the cookie file could even be empty before executing yt-dlp and afterwards, it would contain a fresh apiToken to be used again and again until it expires!

This already works like this

Once that's in place, you could go a step further and save the fresh apiToken to the given cookies file so that the apiToken could be reused and future API calls avoided.

This is how extractors should normally work, but not Nebula. I can't remember if @hheimbuerger had any reason to fetch the auth token in _real_extract instead of _real_initialize

I'm sorry if I'm not following you correctly, but to me that seems like a contradiction:

This already works like this
…
This is how extractors should normally work, but not Nebula.

Calling yt-dl with a non-existent cookie jar and credentials, the cookie jar contains csrftoken and sessionid for api.watchnebula.com and uuid for player.zype.com. These are not enough to authenticate to nebula, apparently.
What's missing is a) the preference for cookie auth and b) something like:
self._set_cookie(
  'nebula.app',
  'nebula-auth',
  compat_urllib_parse_urlencode(
    f'{{"apiToken":"{response["key"]}","isLoggingIn":false,"isLoggingOut":false}}'
  )
)

Happy to hear the proof-of-concept works for you, @TpmKranz!
I'm following on your suggestion of changing the order of authentication methods. Give me a few days to review the details, because the login endpoint was a bit odd.
I'll create a PR then.

@pukkandan: I can't remember if @hheimbuerger had any reason to fetch the auth token in _real_extract instead of _real_initialize

Nah, I was just ignorant and had never heard of _real_initialize() before. 😉 I've switched to it now.

@TpmKranz: This way, I wouldn't have to mess around with my browser everytime the cookie expires and the cookie file could even be empty before executing yt-dlp and afterwards, it would contain a fresh apiToken to be used again and again until it expires!

I mean... yes, but that only applies if you're now willing to store your login credentials e.g. in .netrc. My understanding was that this is exactly what you don't want, and that's why we even started bringing back the cookie auth.
And if you're okay with storing that, then what are you gaining from this solution really? I would argue the only thing you gain is one less HTTP request (in most cases). Which, from a developer perspective sounds nice and clean and the right thing to do, but given that a video download anyway means about 5 HTTP requests for meta data retrieval and then another couple hundred to fetch all the DASH segments... meh!
Even though I'm kinda arguing against it, I implemented it nevertheless. It now tries to fetch the token from the cookie jar first, and only if no (non-expired) cookie is found, it runs the credential auth.
I've also implemented the obvious next step: checking for an invalid token response (HTTP 401) and then reauthenticating. This would be for the case that the server invalidates the token.
I'm hesitant whether to leave this in. This just feels like too much: I'm not sure whether trying to be overly clever is a good idea for something like yt_dlp, where you're always working against undocumented APIs you're not really supposed to use, and where long-term stability might be more important than smart functionality.


test: adds profiling script

The profiling script allows a user to inspect how time time and memory are spent across called functions when performing partitioning on a given document. There are a few different ways to profile and visualize the data and being able to quickly iterate any of these while debugging performance seems entirely reasonable. To account for this I've made this an interactive script, allowing the user to quickly iterate profiling on custom or fixed set test docs and quickly navigate various slices/views of the profiled data.
bonus: only mkdir for ci's cached nltk content if it doesn't already exist (I suspect this was necessary because there was a version of the ci cache floating out there before the changes made last week).
Install
Profiling has a few dependencies which can be installed with:
pip install -r scripts/performance/requirements.txt
Before running a test either populate the docs directory with test documents of interest or set environment variable SYNC_S3_DOCS=true
Export / assign desired environment variable settings:

SYNC_S3_DOCS: Set to true to sync test documents from S3 (default: false)
DOCKER_TEST: Set to true to run profiling inside a Docker container (default: false)

NOTE: the DOCKER_TEST option won't work on M1 Mac because there isn't an ARM-Linux build of memray, but this should work from an x86/AMD environment.
Usage
./scripts/performance/profile.sh

Run the script and choose the profiling mode: 'run' or 'view'.
In the 'run' mode, you can profile custom files or select existing test files.
In the 'view' mode, you can view previously generated profiling results.
The script supports time profiling with cProfile and memory profiling with memray.
Users can choose different visualization options such as flamegraphs, tables, trees, summaries, and statistics.
Test documents are synced from an S3 bucket to a local directory before running the profiles

This works wonderfully! 🙏 🙏 🙏

Awesome! Thanks! PS: snuck in another option to specify the strategy. Small change, but just wanted to give heads up.

This looks amazing.  Great job @ryannikolaidis

This looks amazing. Great job @ryannikolaidis

Thanks! 🙏


Bump base64id

closes https://github.com/SocketCluster/socketcluster/issues/467
npm run test is happy
https://github.com/faeldt/base64id/compare/1.0.0...2.0.0

:)


Write a test for GC that runs a real program for some steps

The infinite-looping two-space GC does not fail any tests at the moment, but it definitely should.

Brought up by #50


added 2 new machine learning books

What does this PR do?
Add resources
For resources
Description
Added 2 machine learning books.
Why is this valuable (or not)?
They are great books.
How do we know it's really free?
For book lists, is it a book? For course lists, is it a course? etc.
It's a book.
Checklist:

Read our contributing guidelines.
[ ] Search for duplicates.
Include author(s) and platform where appropriate.
[ ] Put lists in alphabetical order, correct spacing.
[ ] Add needed indications (PDF, access notes, under construction).
Used an informative name for this pull request.

Follow-up

Check the status of GitHub Actions and resolve any reported warnings!

This appears not to be an authorized distribution. Please see docs/CONTRIBUTING.md guidelines.


Large set of data download causes Broken Pipe

We successfully integrated your script how sometime if user stops downloading or if there are large set of file 400+ files to download we are getting write tcp <IP_ADDRESS>:8080->123.456.789.12:52399: write: broken pipe
Please advices, what could be the cause if it.

I haven't seen this myself but we do not have the need to bundle more than 100 files at a time.
Patches are welcome.


Fix automl_img_obj_vision_test and provide proper dataset

provide proper dataset file and update resources
delete redundant steps

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

The system test succeeded on staging

Closing to re-create on community


Fix minor typo in CONTRIBUTING.md

All N/A, single character typo fix.

[ ] Tests added for any new code
[ ] Ran make fmt-fix (or had formatting run automatically on all files edited)
[ ] Documentation added for any new functionality
[ ] Entry added to UNRELEASED.md for any new functionality

Thank you!

Please let us know if you have any questions about contributing, or if you hit any snags!

@shonfeder Thanks! Unfortunately I needed to merge in the latest changes from unstable and (since this is my first PR) it looks like it needs to be approved again before the workflow can re-run


.......\FirebasePlugin.java:349: error: cannot find symbol Visual Studio Cordova Project

Hi,
I am getting this error during compilation

Same here. Did you find solution for this problem?

closing for now, reopen if still an issue with the latest version of the plugin


Make "return to menu" button more visual

As you can see the return to menu button is not visible in the game view:

To address this:

Consider adding an appropriate background color to enhance visibility.
You can adjust its position for better visual prominence.

Hey @Ramzi-Abidi, I am so sorry I didn't add a comment to ask to be assigned this issue, but I hope my pr is still acceptable. I will ensure to ask you beforehand in the future.
PR to change styling for "return to home" button: https://github.com/Ramzi-Abidi/Pong/pull/79


Start and group after double click

Hey,
I need to pass datas from one function to another. So I need and action when I double click on some free cluster to open my modal form with filled start date and group where I clicked. Any ideas how to get this datas? Currently I've got modal form which is opening with addlistener.
One more thing - I'm using JSON array which is loaded from asp.net class but after saving new event I can't refresh datas. I mean I don't have problem with taking this Json again but redraw() function is not working that how I expected.

The Timeline of the successor of chap-links-library, vis.js, has eventlisteners like click and doubleClick which you can use.
About refreshing data: without a code example of what you're doing there isn't much I can say about that. When loading new data you have to apply it using Timeline.setData(), do you do that?

I have working action and modal form but I need to pass the datas from the clicked cluster - I mean if I have few groups and I click on 9 July in group "abcd" i need to pass the start date and name of group into this modal form.
Ohh, thanks. I didn't check with setData() - maybe it will help, let you know.

I mean if I have few groups and I click on 9 July in group "abcd" i need to pass the start date and name of group into this modal form.

There is no built-in way to get this information, you will have to hack something for this into the Timeline yourself. The event listeners of vis.js do provide this information, however, the Timeline of vis.js doesn't yet have support for clustering.

I did it this way:
First we need this function:
function getSelectedRow() {
var row = undefined;
var sel = timeline.getSelection();
if (sel.length) {
if (sel[0].row != undefined) {
row = sel[0].row;
}
}
return row;
}
Now everything what we have to do is add a variable into our listener function (it's running on double click):
var onadd = function () {
var test = timeline.getItem(getSelectedRow());
$('#form').modal('show');
};
links.events.addListener(timeline, 'add', onadd);

What do you mean? Do you mean you solved it like this?

Yep, It is working fine.
Variable test return an object like this:
content: "New"
group: "abcd"
start: Date {Thu Jul 16 2015 22:21:52 GMT+0200}
type: "box"

Ah now I get what you mean. Ok great you solved it, I will close the issue then.


Please add the grouping option in the download folder.

Additional comment
That is an option that currently exists in the file explorer and is useful, at least in downloads. I would be very grateful if you could add that option. Thank you very much for the application, it improves very fast and the consumption is significantly reduced.

Merging with#1074