Mixing multiple images

Hi this is a great project!
I notice that you had an image blender. It would be great if instead of proving a single Parent, you could provide an array of them. This would be much faster than the current solution of running the blender multiple times (or at least it seems like it would be). Perhaps this is already possible if so please let me know how.
This would be useful to merge B&W images into RGB pictures for instance.
Thanks,
Ghislain

This isn't possible right now, but it sounds like something that might not be too difficult to add. I'll have to look into it.


Feature to allow custom bootstrap classes for the OK/Cancel buttons

We sometimes use confirm dialogs in more of a "negative" context, where dismissing the dialog warrants a 'danger' class for the button.
I have created the ability to pass in the desired bootstrap button class to the directive for each of the OK and Cancel buttons.
I have pull requested this, but let me know if this isnt the process that you prefer, or if you want to ignore it entirely.
Thanks!

The issue with this is I don't want to add options that only work in the context of bootstrap css. Some users don't use bootstrap. So you would have to specify all of the classes: btn btn-warning instead of just warning. Just giving the ability to specify additional classes wouldn't work for your use case.
Perhaps the ability to create named templates would be better. The name would map to a templateUrl or template string defined ahead of time. Then you can completely customize it and easily reference it. Thoughts?
$confirm({templateName: 'danger'})

I agree with you about locking into bootstrap - I only made that assumption since this plugin has a dependency on angular bootstrap.
Why couldn't my change simply be adjusted to accept the full list of classes as you specified, wouldn't that actually work for my use case?
I am guessing with your suggestion of named templates, in the angular-confirm code you would define several different templates, and the client code would just supply which one to use? As long as I am understanding, wouldn't this be locked into bootstrap as well?  Or perhaps you were expecting that folks would end up customizing the templates to achieve what they want?
I am happy to work on this, once we have the best solution if that helps.

The full list of classes would work for your use case, however I don't want to add a ton of options. Having a singular feature that covers a broader use case is better.
So in the $confirmModalDefaults there would be an additionalTemplates that would be an object where the key is the name and the value would be an object with either a template key or a templateUrl key.
$confirmModalDefaults: {
    additionalTemplates: {
        "warning": {templateUrl: '/../../abc.html'},
        "danger": {template: 'some template string'}
    }
}

Then
$confirm({templateName: "danger"}) would supply the modal with the template for that key.
confirm-template="danger"
That way you can supply multiple templates for different situations and easily reference them. What do you think?

This is more flexible, it does leave a bit more up to the client than seems needed (at least for me), but would be easier to support.  I also understand wanting to control the number of options, but you are still gaining an option here. This also makes no assumptions about the UI elements needed or being used.
This setup would work for us, we are already defining our own custom template anyway simply to adjust the presentation aspects of the dialog.  We would just have to have 2 or 3 templates now - not a big deal.
I am also guessing you were NOT looking for angular-confirm to provide a set of default "additionalTemplates", which keeps you more agnostic as well.

OK.  If you are ok with it, I can proceed to make this work as you described.  Just wanted to know we are on the same page before I start that process.

@stguitar Yes that sounds good to me. The module should not provide additional templates, it should be initialized to an empty object.

Cool.  I will probably be able to start on that tonight, so hopefully will have something soon.

@Schlogen my pull request should be updated now with the proposed changes and test cases. let me know if i need to make any adjustments.

@Schlogen No prob, happy to help.


Move Playground to components

This is a draft PR for cleaning up the components. It has several goals:

[ ] Remove Playground and put it into components, so that we can embed it into our website
[ ] Remove the disgusting sliders for testing logarithmic scaling at the bottom of the charts, possibly having a few other options including CDF visualization, and togglable rather than slidable log scales
[ ] Attempt to fix some performance regressions
[ ] Fix some "undefined"s showing up in input fields
[ ] Allow the width of Squiggle editors to change, and also change all properties to em rather than px

It's also entirely possible that using emotion will break in observable, and I want to test whether that happens. If it doesn't, I would prefer using emotion to inline styles

I've started to review this. Please help it pass the automated tests.

| It's also entirely possible that using emotion will break in observable, and I want to test whether that happens
Note that this is going to staging, which might be hard to test in observable for a while. We could speed this up if that's important.

I'd suggest having including some images, for future PRs that make visual changes to the interfaces.
Also, I noticed that the Notebook now shows all of the variables presented?

It certainly does! And has for a while

Turns out emotion seems to not like storybook either, I'm giving styled components a go instead

It's also entirely possible that using emotion will break in observable, and I want to test whether that happens

Note that this is going to staging, which might be hard to test in observable for a while. We could speed this up if that's important.

I'm tempted to do a quick deploy from what I have here up to npm to simply test it. Also worth noting that master + current npm versions have a massive XSS vulnerability as it stands, so it would be a good opportunity to remove all previous versions from npm and then experiment with some different options for a replacement

Turns out emotion seems to not like storybook either, I'm giving styled components a go instead
Sounds good. I've had good success with styled components, with Foretold.io. One downside is that they didn't flesh super well with Tailwind, which I later wanted to use (and could see using for this later)

Btw, if we have the ``react-ace

Btw, if we have the react-ace editor now, I assume we should also use it for the SquiggleEditor (now using ReactCodeJar)? While you're at it, I'd probably change SquiggleEditor to a different name, to distinguish it more from the playground.

Absolutely! I had it doing that before, but I changed it back when I got some odd errors

Would be nice to look into #71, #55 and #40

Seems to be running faster for me now.
One issue is that for the one-line version, the height of the editor isn't quite big enough.

Btw; I think this uses Antd. Previously I had some trouble with Antd; it seemed pretty large, for what we actually wanted to do with it. My guess is that Tailwind will be nicer, though maybe overkill for these components.

I probably should have split this up into smaller parts, but I'd be happy to see this merged.
I'm now off to review #147 + possibly look to improve the Typescript interface

Seems to be running faster for me now.
One issue is that for the one-line version, the height of the editor isn't quite big enough.

I think I fixed this in newer versions

Btw; I think this uses Antd. Previously I had some trouble with Antd; it seemed pretty large, for what we actually wanted to do with it. My guess is that Tailwind will be nicer, though maybe overkill for these components.

I'm frankly concerned that most css frameworks won't work in observable, but I'm not really sure and I'd like to test it. I might do that first before asking for a merge.

I'm frankly concerned that most css frameworks won't work in observable, but I'm not really sure and I'd like to test it. I might do that first before asking for a merge.

Yea, this wouldn't surprise me. However, I'd expect inline styles would be respected still, if you can't load css.

I probably should have split this up into smaller parts, but I'd be happy to see this merged.

To be clear, my impression is that this isn't ready to be merged? This is still labeled as a draft and some of the tests are failing.

Oh yo! got a deploy:
https://deploy-preview-148--squiggle-components.netlify.app/
For those who are interested.

Ok, fixed all your comments hopefully.

You didn't respond to several of my comments? Maybe they weren't displayed or something? (We had that problem before)

For my sake, Todo:

[ ] change the onChange handler to return onChange rather than onBlur
[ ] Change the vega component to a checkbox
[ ] Create new IDs + check whether it works with multiple editors
[ ] Multiple line squiggle editors (noticed this was an issue)
[ ] Remove ace imports that are not used

Great. The yarn.lock file has a conflict, otherwise I'm happy for this to be merged in.


parsing ISO 8601 fails for first day of year e.g. "2018-01-01T00:00:00.000Z"

var book = XLSX.read('date,no date\n"2018-01-02T00:00:00.000Z","2018-01-01T00:00:00.000Z"\n',{type: 'string',cellDates:true})
book.Sheets.Sheet1 is
{
"A1": {
"t": "s",
"v": "date"
},
"B1": {
"t": "s",
"v": "no date"
},
"A2": {
"t": "d",
"v": "2018-01-02T00:00:00.000Z",
"w": "1/2/18"
},
"B2": {
"t": "s",
"v": "2018-01-01T00:00:00.000Z"
},
"!ref": "A1:B2"
}
sheetjs v 0.11.17
browser: chrome 63 (mac),

This happens because the fuzzydate function validates parsed dates that are parsed as the january first or any day of 2001 more strictly.
In these cases the unparsed string has to contain a 3 letter month substring or must only consist of characters 0-9-:,/.,
function fuzzydate(s/*:string*/)/*:Date*/ {
	var o = new Date(s), n = new Date(NaN);
	var y = o.getYear(), m = o.getMonth(), d = o.getDate();
	if(isNaN(d)) return n;
	if(y < 0 || y > 8099) return n;
	if((m > 0 || d > 1) && y != 101) return o;
	if(s.toLowerCase().match(/jan|feb|mar|apr|may|jun|jul|aug|sep|oct|nov|dec/)) return o;
	if(s.match(/[^-0-9:,\/\\]/)) return n;
	return o;
}

One possible fix for this without touching the general logic would be to allow T and Z in this regex: if(s.match(/[^-0-9:,\/\\]/)) return n;
Sidenote:
I think that trying to weed out false positives from the very lenient Date constructor like this is like playing a game of whack-a-mole.
A proper fix could be an option to supply column type information to the read call, include a set of default parsers for frequently used types and another option to define custom column-types/parsers.

Just overwrite the logic with function fuzzydate(s/*:string*/)/*:Date*/ { 	var o = new Date(s), n = new Date(NaN); 	var y = o.getYear(), m = o.getMonth(), d = o.getDate(); 	if(isNaN(d)) return n; 	if(y < 0 || y > 8099) return n; 	if((m > 0 || d > 1) && y != 101) return o; 	if(s.toLowerCase().match(/jan|feb|mar|apr|may|jun|jul|aug|sep|oct|nov|dec/)) return o; 	**if (d === 1 && m ===0) return o;** 	if(s.match(/[^-0-9:,\/\\]/)) return n; 	return o; }

I'm experiencing the same issue.

Still a problem in master branch.

wouldn't it be fair to always allow valid ISO8601 formats?
My dates which contain 2001 always fail, despite being valid ISO8601 dates
Besides, Date.prototype.getYear() is deprecated

Unfortunately Excel does not understand 2018-01-01T00:00:00.000Z to be a date, entered manually or in a CSV file. To test, try the following CSV
String,=TYPE(C1),="2018-01-01T00:00:00.000Z"
Date,=TYPE(C2),2018-01-01T00:00:00.000Z

Excel will show B1 = 2 and B2 = 2. The return value 2 means Excel thinks the cell is a text cell
You can manually enter an ISO8601 string in the formula bar and see the same result.

Actually I'm not facing the issue when seeing data in excel. The issue happens in XLSX.read('csv string', { cellDates: true })
If I debug the workbook produced from it, the "type" for any date which year is 2001 fails to reconize as date, and defaults to string
https://github.com/SheetJS/sheetjs/blob/0ddeba4fc358394438e3651910bce3b3ef29578a/xlsx.mjs#L2933
why does fuzzydate check for 'y != 101'?

Grouping into #1300
@FlavioRodr
new Date("This is not a Date 1")
// Mon Jan 01 2001 00:00:00 GMT-0500 (Eastern Standard Time)

Most of Chrome's bad date parsing is 2001, for which
new Date("This is not a Date 1").getYear()
// 101


DatePicker/DateRangePicker: On pressing left/right arrow keys while focus is on next/previous buttons is closing the picker

🐛 Bug Report
On DatePicker and RangePicker, if user presses left or right arrow keys while focus is on Previos/Next month buttons, the picker is closing.
🤔 Expected Behavior
Expected Behavior: This unexpected closing of the picker should not happen.
🔦 Context
Issue can be observed on Storybook or react-spectrum docs
🧢 Your Company/Team

🕷 Tracking Issue (optional)

Appears to be a chrome bug https://bugs.chromium.org/p/chromium/issues/detail?id=1473752


Bug: BulkAPI isCompleted does not interpret all Batch States

Reference: https://developer.salesforce.com/docs/atlas.en-us.api_asynch.meta/api_asynch/asynch_api_batches_interpret_status.htm
BulkAPI#isCompleted only considers the Complete and Failed state. It does not consider the other states (Queued, InProgress, Not Processed).
This PR adds logic to interpret all the batch states.

LGTM


MViT2 Support

🚀 Feature
Hi, thanks for the good package, I wonder if there is any plan to support MViT2 (new arch proposed by Improved Multiscale Vision Transformers for Classification and Detection)

Yes, we have the plan to add the improved MViT in PyTorchVideo, so please stay tuned!

I noticed that the query residual in MViT has been added to the attention.py file.  Is there also plans to add the relative position embedding?  I am curious whether that would take the same form as the relative positional embedding used in Bottleneck Transformer.

SlowFast already support MViT2. Hope pytorchvideo can follow it.

Hi, just want to summarize the current status:

MViTv2 is already released in PySlowFast
PyTorchVideo currently add the residual pooling from MViTv2 and we will add relative positional embedding later to fully support MViTv2.


Can flutter desktop engine support window 32bit?

I found flutter desktop engine support window 64bit only. Can it support 32bit?

The flutter engine doesn't support 32-bit Windows: https://github.com/flutter/flutter/wiki/Custom-Flutter-Engine-Embedders
No plans to support 32-bit Windows.


Cell is not passed inside events

For example here it is explicitly set to nil
func collectionView(_ collectionView: UICollectionView, didSelectItemAt indexPath: IndexPath) {
    let (model, adapter) = context(forItemAt: indexPath)
    adapter.dispatchEvent(.didSelect, model: model, cell: nil, path: indexPath, params: nil)
}

And in most of the cases it is set to be nil

Fixed, thanks


✨(back) manage administrator role like an instructor

Purpose
A pedagogic member has an administrator role and we don't take care of
this role. This role should have the same permission as an instructor.
He should be able to access the dashboard and then use every marsha's
features.
Proposal

[x] add new administrator and use it like an instructor.

Closes #447

This PR is integrated into #396 and can be closed.


pushing viewController leftMenu cannot work

Hi as title
I follow the example _ = panel!.center(myVC)
It turns out that the center viewController is pushed right, but the leftMenu is not working...
I cannot open leftMenu even called panel?.openLeft(animated: true)
I also print panel?.left, and I can get the right menuVC, just cannot use it
do I need to extension other transition stuff?
Here is my setUp inside appDelegate
  private func setUpControllers() {
    window = UIWindow(frame: UIScreen.main.bounds)
    guard let window = window else { fatalError("no windows") }

    let centerVC = UINavigationController(rootViewController: myCenterVC())
    let menuVC = MyMenuVC()

    let rootVC = FAPanelController()
    _ = rootVC.center(centerVC).left(menuVC)

    window.rootViewController = rootVC
  }

thanks

Hello @fidmor89 ,
i am facing this issue on my app if i can present controller from the MenuController tableview cell its work fine. But i am using push controller from menu that will be not work.
Thanks


Store receipts in stable directory on the native platforms

Explanation of Change

The PR improves the app to store captured receipts in a new stable directory - /Receipts-Upload.
Fixed Issues

$ https://github.com/Expensify/App/issues/51761
PROPOSAL: https://github.com/Expensify/App/issues/51761#issuecomment-2473936965
Tests

Go offline.
Submit an expense and scan a receipt with the in-app camera (submit it to anyone).
Kill the app (close).
Re-open the app.
Go back online
The receipt is uploaded and scanned correctly. If it fails for some reason, then the user can download the receipt.

[x] Verify that no errors appear in the JS console

Offline tests

Same as "Tests".
QA Steps

// TODO: These must be filled out, or the issue title must include "[No QA]."
Same as "Tests".

[x] Verify that no errors appear in the JS console

PR Author Checklist

[x] I linked the correct issue in the ### Fixed Issues section above
[x] I wrote clear testing steps that cover the changes made in this PR

[x] I added steps for local testing in the Tests section
[x] I added steps for the expected offline behavior in the Offline steps section
[x] I added steps for Staging and/or Production testing in the QA steps section
[x] I added steps to cover failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[x] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)
[x] I tested this PR with a High Traffic account against the staging or production API to ensure there are no regressions (e.g. long loading states that impact usability).

[x] I included screenshots or videos for tests on all platforms
[x] I ran the tests on all platforms & verified they passed on:

[x] Android: Native
[x] Android: mWeb Chrome
[x] iOS: Native
[x] iOS: mWeb Safari
[x] MacOS: Chrome / Safari
[x] MacOS: Desktop

[x] I verified there are no console errors (if there's a console error not related to the PR, report it or open an issue for it to be fixed)
[x] I followed proper code patterns (see Reviewing the code)

[x] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick)
[x] I verified that comments were added to code that is not self explanatory
[x] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[x] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method

[x] If any non-english text was added/modified, I verified the translation was requested/reviewed in #expensify-open-source and it was approved by an internal Expensify engineer. Link to Slack message:

[x] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[x] I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is either coming verbatim from figma or has been approved by marketing (in order to get marketing approval, ask the Bug Zero team member to add the Waiting for copy label to the issue)
[x] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[x] I verified the JSDocs style guidelines (in STYLE.md) were followed

[x] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[x] I followed the guidelines as stated in the Review Guidelines
[x] I tested other components that can be impacted by my changes (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar are working as expected)
[x] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[x] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[x] I verified that if a function's arguments changed that all usages have also been updated correctly
[x] If any new file was added I verified that:

[x] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[x] If a new CSS style is added I verified that:

[x] A similar style doesn't already exist
[x] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(theme.componentBG))

[x] If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
[x] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[x] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[x] If the PR modifies a component or page that can be accessed by a direct deeplink, I verified that the code functions as expected when the deeplink is used - from a logged in and logged out account.
[x] If the PR modifies the UI (e.g. new buttons, new UI components, changing the padding/spacing/sizing, moving components, etc) or modifies the form input styles:

[x] I verified that all the inputs inside a form are aligned with each other.
[x] I added Design label and/or tagged @Expensify/design so the design team can review the changes.

[x] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[x] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.

Screenshots/Videos

Android: Native

https://github.com/user-attachments/assets/a5fd7d92-c7ff-4b5b-aec1-19e0c50a9ca3
https://github.com/user-attachments/assets/120611bd-74da-42c9-813c-e12ca5f4a7ca

Android: mWeb Chrome

https://github.com/user-attachments/assets/67adc093-04bb-4a64-bdda-93284434a24f

iOS: Native

https://github.com/user-attachments/assets/92536863-dfb6-486c-a27a-4da285054d00

iOS: mWeb Safari

MacOS: Chrome / Safari

MacOS: Desktop

@allgandalf could you please attach a video because it works correctly on IOS on dev for me? Btw, is it possible to add my iPhone to the whitelist so I can test it as well?

I've investigated the network deeply and realized the server returns the 405 error. Also, the FormData contains the image and it exists in the fs obviously. Can we double-check the server logs via the requestID?

cc @allgandalf

All good, I just want to figure it out if the backend receives the file and why sends this error. Thank you!

Can we double-check the server logs via the requestID?

Yeah, thanks for providing that. Looking at the logs I can see that when the request is made from our api layer to our database layer, the receipt is empty, when it should be the filename from the S3 upload response of the receipt image. So I think the problem is somewhere on the frontend with the way the receipt is sent. Can we add some front end logging to verify that the receipt file is present, non-empty, and in the proper format?

Bedrock\Client - Starting a request ~~ command: 'CreateMoneyRequest' clusterName: 'auth' headers: '[authToken: '<REDACTED>' debtorEmail: '' amount: '0' currency: 'USD' comment: '' created: '2024-12-04' merchant: '(none)' iouReportID: '5259455211799816' chatReportID: '8301563627134920' transactionID: '3114714428657765752' reportActionID: '8074998914474577022' createdChatReportActionID: '0' createdIOUReportActionID: '0' reportPreviewReportActionID: '5351253755105051316' receipt: '' receiptState: 'SCANREADY' receiptID: '0' category: '' tag: '' billable: '' transactionThreadReportID: '5397098710674854' createdReportActionIDForThread: '3412775657730853189' commentNVP: '{"comment":""}' uploadingIP: '<IP_ADDRESS>' gps: '' personalPolicyValue: '[]' debtorAccountID: '0' reimbursable: '1' policyID: '' idempotent: '' logParam<EMAIL_ADDRESS>urlToNewDot: 'https://staging.new.expensify.com/' shouldReadUsingThreads: '' shouldHandleOnyxUpdates: '1' isNewDotRequest: '1' clientUpdateID: '3250954058' maxNumberOfUpdates: '500' requestID: '8ecce0ce9f2ff978-PRG' lastIP: '<IP_ADDRESS>' writeConsistency: 'ASYNC' priority: '500' timeout: '110000']'
--

<br class="Apple-interchange-newline">

To be clear, I do see the screen shot there showing the receipt object and it looks like there's a blob for the receipt image, so it's confusing why it doesn't upload. Is everything about the request sent to the backend exactly the same between the request online that succeeds and the one that fails? I wonder if maybe there's some other receipt meta-data that gets removed when the app is killed?

Working on it.

@allgandalf, it is my priority right now. I am testing it with Proxyman to understand why the file is missing. Yeah, I can confirm that the blob object disappears. Working on the fix.

It looks like there is a problem with Blob between app restarts. I am trying some solutions.

I've asked my Callstack colleagues for help with this blob issue. Tbh, I don't think we will merge the PR tomorrow. However, we could split the work up. We merge this PR because it stores receipts in the stable directory and this part of the work is done. I will work on a follow-up PR to solve the blob problem which has not occurred due to this PR. WDYT @allgandalf ?

Thanks @rezkiy37, that sounds good to me

:hand: This PR was not deployed to staging yet because QA is ongoing. It will be automatically deployed to staging after the next production release.

@rezkiy37 Found issue https://github.com/Expensify/App/issues/54345 on iOS
https://platform.applause.com/services/links/v1/external/f80e8be259b8fe00c0f0be8cd97c5f8be4e5300e895d30be7e27919b99edbd07

@rezkiy37 Found issue #54345 on iOS
https://platform.applause.com/services/links/v1/external/f80e8be259b8fe00c0f0be8cd97c5f8be4e5300e895d30be7e27919b99edbd07

@izarutskaya the issue you mentioned is not associated with this PR and the video you attached.

@izarutskaya, also I don't see a problem on the video. Could you please clarify?


Remove the child trie feature

I would suggest to remove child tries before they make it in production.
Conceptually this feature doesn't bring any advantage over a single trie.
It is overly complicated, with many different host functions, corner cases (https://github.com/w3f/polkadot-spec/issues/540), and a weird concept of a "default" child trie.
It also pollutes many different other aspects of the client (e.g. the JSON-RPC API needs to know about them, the chain specs format needs to know about them).
It seems that its only advantage is that it is easier to do it this way than implement retrieving the hash of trie node. However the trade-off in terms of complexity is IMO hilariously in favor of removing child trie.

Child tries are already in production.
Removing them is still somehow partially doable: migrate some data, personally I think it makes sense to have them iff we want to have different kind of child trie in the future. But the host function are already quite called and would need to remain.
Conceptually over a single unbalance trie it forces what would otherwise be a branch to be at a given location (but this could be enforced by setting null value where we want to ensure there is a branch to be able to use its root).
I agree that it is somehow a weak advantage (no smaller proof from it maybe a bit clearer if we want to split proof, but I wrote iff in my previous sentence).
"Default child trie" should be name differently as it seems confusing, this is the default kind of child trie the child trie where the state is of the same kind as the parent state.

We are going to eventually support amortized O(1) deletion of child tries at the database level.
@tomaka What do you mean by "retrieving the hash of trie node. " and how would that replace the child tree functionality?

What do you mean by "retrieving the hash of trie node. " and how would that replace the child tree functionality?

Correct me if I'm wrong, but writing key K of child trie C could be replaced with writing the key concat(C, K).
The only feature that child tries provide and that can't be replicated is the ability to determine their trie root hash, but this could be replicated by adding a host function that returns the node value of a certain node.
As far as I know, child tries are used only for contracts, so that contracts are sandboxed. I don't exactly know how the sandboxing works, but I assume that restrict the contract so that it can only read/write from/to a specific child trie. However I don't see how that couldn't be easily changed to restrict the contract to a certain trie prefix instead.

Child tries are already used by Polkadot for crowdloans. All contributions per crowdloan are stored in one child trie.

Also used by crowdloan in polkadot and kusama. I remember parachains team asking question about the api, so I would not be surprise if used by some parachain.
I also know that regularly people team are looking at using some specialized child trie for their projects, and I really support the idea of allowing different kind of child trie/state (which current CT implementation show the foundation).
But making it generic is quite challenging (I remember trying it a while back but not delivering).
IMO, out of the fact that CT are already used, CT should not be removed and quite the opposite: it should be improved to allow adding custom or additional states.
One could argue that what is needed is just for substrate to allow one customizable state, and let the users compose their state by themself only if strictly needed, not false, and pretty much well aligned with polkadot verification model. But having this top standard state ensure a common api for pallets (but yes you can always reduce the core code and move a lot for users to implement themselves, could indeed also make sense for child state).

The only feature that child tries provide and that can't be easily replicated right now is the ability to determine their trie root hash, but this could be replicated by adding a host function that returns the node value of a certain trie node.

and forcing a parent node to be at C - 1 nibble so the partial key at the start of node K is in the root of K. Or having a changing key C return with root. (just mentioning, not something that should be taken in consideration, the question is more api related I think)

it should be improved to allow adding custom or additional states.

Why should it? What's an example use case?
Personally I find it completely insane that we even consider that kind of insanely complex design.
Can we not declare the trie as "done", remove child tries as too weird and complex, and move on to something else? Do we really have to dig into the trie further and further just for the sake of justifying our jobs as engineers?

Personally I find it completely insane that we even consider that kind of insanely complex design.

On paper the design is just writing merkle proof of another state in a parent state and allow composing both state proof.
As people use to say api design is bad, maybe this is the issue. Or it is just the thing in place that are too complex and would
require an overhaul, or removal as you propose.
I really don't see composing two merkle state as 'insanely' complex, it is just put the merkle root of a second state in the parent one.

Can we not declare the trie as "done", remove child tries as too weird and complex, and move on to something else?

Sounds doable.

Do we really have to dig into the trie further and further just for the sake of justifying our jobs as engineers?

Sorry if I frighten you, I was just saying that improving child state could be good, I was not implying a massive task force was in place, for instance I think the last time someone look at this problematic was me and shortly to make a example branch for proposing an exercise for academy. Last change to child trie is really old I think (except the removal api change which is merely applying same constraint you have with remove prefix in trie).

What's an example use case?

Could attach some small in memory trie with different cost/weight model, could add mmr for indexed based api, could add state with zk proof (I know some are looking in this direction).
But as I mention you can always try to entirely switch the substrate state to something else, having child trie only allow to compose state in a easier way (could be fully under user responsability).

Afaik we should do O(1) deletion below any node, not merely at child tries.
We discussed trie write-locks for runtime multi-threading too, but they too should happen anywhere, not just at child trie boundaries.

I think it makes sense to have them iff we want to have different kind of child trie in the future.

We do want alternative storage proof types, like MMR, KZG, and zk proof friendly Merkle trees, especially for zkUTXOs ala ZCash sapling, ZEXE, etc., and non-membership proof friendly nullifier sets.  Ain't clear how these fit in practice though, like do we make them easily accessible from substrate code, or do we make people handle them manually.  In particular, we've had trouble using trie proofs into older past state, which maybe fixed now, but likely not..  I think older past state shows up way more alongside alternative proofs, so maybe they wind up manually anyways.  ¯_ (ツ)_/¯
Is the mental model that child tries work like mount points?  If so, one could even attach them dynamically during initialization or whatever.  At least conceptually, each parachain shares a small trie with the relay chain, into which we'll place things like XMP roots, and maybe simpler if you lean into the mount points model.  I suppose the mount points model also simplifies one parachain sharing a subtrie with another parachain, just by attaching merkle proofs to a message.

Conceptually over a single unbalance trie it forces what would otherwise be a branch to be at a given location (but this could be enforced by setting null value where we want to ensure there is a branch to be able to use its root).

At least to me, this sounds like "domain separation" in hashing, which simply means "users cannot touch this part of the hash input".  As as example, if you've a binary tree, then you could index via a slice of enum { Left, Right, Leaf, DomainSeparator } where user bits encode only to Left or Right, but only the application itself can encode a DomainSeparator (or a `Leaf).
You can always do domain separators manually by altering user inputs, but this brings mental costs elsewhere, like downstream standards efforts, or insecurity from broken domain separation.  It's likely child tries wind up more complex than what one wants from domain seperation, but the domain seperation functionality likely matters.
tl;dr It sounds like child tries maybe conflating the role of "mount points" and "domain separators".


Improve documentation

After we moved similar functions under same manual page, we lost some "hidden" parameters that were described under .... That is because one manual page can have only one description for each parameter, i.e., if we have documentation coming from multiple R files, descriptions of ... parameters are not merged; the documentation has only one description of ....
--> Check these problems. If this kind of issue is found, ensure that all the documentation can be found from manual page (merge descriptions manually to single R file).
For instance, agglomeration functions are under single "agglomeration" manual page. However, it seems that agg.na.rm parameter is missing from [here](https://microbiome.github.io/mia/reference/agglomerate-methods.html  even though it is listed here

I'd also probably split at least agglomerateByRank and agglomerateByPrevalence functions in separate pages, as for example, detection and prevalence parameters are not required for agglomerateByRank (?).

The idea was to harmonize the naming and documentation so that it is intuitive for user that these agglomerate* function do similar things. We cannot avoid having documentation pages which include parameters that are not shared with every function in the page; they are very common. But this is something that we can discuss what is the best way to organize the manual. Updating manuals was already in task list.

Given that other functions like estimate* and calculate* have separate manual pages, and if the intention is to harmonize the package documentation, separating the agglomerateBy* functions will provide several benefits such as simplicity, specific info about fxns and ease of maintenance.
However I also see a WIP where we index estimate* docs into addAlpha, so I am a bit unsure of the consequences on harmonization. Does this imply we index other functions of nature fxn* such as calculate*, agglormerate*, convert*, get*, import* , run*, split* etc. This is imperative if we need to ensure consistency in the code and perhaps the docs.

estimate* functions were merged into addAlpha. So there is no separate methods for estimateDiversity, estimateDominance etc anymore; all alpha diversity indices are calculated with addAlpha
The initial idea was to harmonize the documentation, but as Artur pointed out, there are drawbacks.  Since we already harmonized the naming, that should be already clear that functions named similarly do similar things.
So I would prefer simplicity and clarity. That is achieved probably by separating agglomerateByRank and agglomerateByPrevalence functions since they have too many different parameters. Also the same goes with import* functions. So the changes that we made previously were suboptimal. However, it should be quite simple to revert this since we only modified the targeted documentation name (@rdname tag).
@Daenarys8 Can you go through mia documentation and give suggestions how to proceed? https://microbiome.github.io/mia/
The documentation is currently a mess.

I am thinking we need to  restructure the documentation. Placing emphasis on providing a clear, logical, and comprehensive guide that users can easily follow. We can start with an Introduction and Overview section/menu that gives a high-level summary of the mia package, its purpose, and key features, along with installation instructions. Follow this with a Getting Started section/menu that introduces users to TreeSE objects, providing a quick start guide and basic usage examples to get users up and running quickly.
The next section/menu Workflows, could include Data Preparation, which should focus on importing and preprocessing data, managing metadata, and performing data normalization, agglomeration and transformation to ensure users can properly prepare their datasets for analysis. After data preparation, include an analysis subsection that covers core analysis functionalities such as alpha and beta diversity analysis, taxonomic analysis, etc.
The next section References detailing user accessible methods.
Optionally we could address common user questions and issues in a FAQs and Troubleshooting section, offering solutions to common problems and best practice tips. Finally, the Contributing and Community side navigation with guidelines for contributing to the package, a code of conduct, citations and links to community resources can remain the same.
However this is just a suggestion and very open to scrutiny and improvements.

Thanks for the suggestion! Hmmm, that sounds nice, but we should avoid doing overlapping work. The things that you described are already in OMA.
We should keep the mia documentation as simple as possible, but currently the problem is that there might be missing parameters, unclear explanations etc. So I suggest to

Check if we should separate documentation of agglomerateByPrevalence & agglomerateByRank
Check explanations if they are clear enough
Use @inheritparam

Yes, please. This also is related to things that we discussed in Mattermost privately (the description should be unified: the class of input, what the parameter does, details, default value.). We can discuss more if you feel so

@Daenarys8 the status of this one?

It is ready pending review. #603

Perfect. Please close this after the PR is merged.

Related to #372 - is there anything relevant to add into this?
Close all related issues once the PR is done and things are ready.


Read data for displacement per atom value computation

Read 'total displacements rate' and explicit 'time' of irradiation step from the FISPACT output file.

Fixed in gihub:dvp2015/pypact project, branch read_dpa


Remove docker from QGIS tests

Use local pytest instead.
Had to remove the two tests that really instantiate the GUI.
No more docker integration simplifies the environment.
Adds coverage reporting for ribasim_qgis

I guess this can be closed in favor of #939. Still would be nice to be able to drop Docker from our test stack, but preferably not at the cost of reduced testing capability.

I guess this can be closed in favor of #939. Still would be nice to be able to drop Docker from our test stack, given we have QGIS in our Pixi environment as well, but preferably not at the cost of reduced testing capability.

Since the latest attempt also didn't fix it, maybe we merge this and add a high priority issue to add something equivalent with pixi? Or does anyone want to give docker another try?

I guess this can be closed in favor of #939. Still would be nice to be able to drop Docker from our test stack, given we have QGIS in our Pixi environment as well, but preferably not at the cost of reduced testing capability.

Since the latest attempt also didn't fix it, maybe we merge this and add a high priority issue to add something equivalent with pixi? Or does anyone want to give docker another try?

Yeah, I have given up on the Docker path (although I don't expect it is the root cause), but I can try the pixi path in this branch.

Make sure to merge main, which has a more recent QGIS that is not broken.

I can't run the tests locally because of https://github.com/qgis/QGIS/issues/52987 on mac, but it seems to be setup ok. 👍🏻


Upgrade base-image to 1.10

Why isn't a more recent image used as the base image?
1.10.1 contains a small security-fix.
Also why don't you use the baseline (currently 1.11) image?

FYI: This requires also updating Forego and Docker gen. I've done it in my branch https://github.com/BenHall/nginx-proxy/blob/sticky/Dockerfile

Thank you but why don't you make a PR for it?

The update has been done by now, closing.


bugs-tab: Expected element not found: .Counter

Description
See issue description at: https://github.com/matthizou/github-show-avatars/issues/6#issuecomment-2544227501
ElementNotFoundError: Expected element not found: .Counter
    at expectElement (chrome-extension://hlepfoohegkhhmjieoechaddaejaokhf/assets/npm/select-dom.js:21:11)
    at addBugsTab (chrome-extension://hlepfoohegkhhmjieoechaddaejaokhf/assets/features/bugs-tab.js:92:22)
    at async init (chrome-extension://hlepfoohegkhhmjieoechaddaejaokhf/assets/features/bugs-tab.js:158:3)
    at async chrome-extension://hlepfoohegkhhmjieoechaddaejaokhf/assets/feature-manager.js:157:20
    at async Promise.all (index 0)
    at async asyncForEach (chrome-extension://hlepfoohegkhhmjieoechaddaejaokhf/assets/helpers/async-for-each.js:6:2)

How to replicate the issue + URL
https://github.com/facebook/react/pulls?page=5&q=sort%3Aupdated-desc+is%3Apr+is%3Aopen
Extension version
24.12.9
Browser(s) used
Edge
Token ⚠

[X] I set a token in the options page. I understand that some features require a token to work.

Thanks for the report! Does it happen all the time? The feature works correctly for me

It seems to be about 50% of the time working, 50% not working when the feature is enabled and the github-show-avatars extension is also enabled.


If a KameletBinding is invalid, the related status is not updated to report an error

Assuming a KameletBinding lacks sourece/sink, as example:
piVersion: camel.apache.org/v1alpha1
kind: KameletBinding
metadata:
  name: c82iernvdc4irh6evn20
  namespace: default
spec:
  integration:
    configuration:
    - type: secret
      value: c82iernvdc4irh6evn20
    profile: OpenShift
  steps: []

The operator will fail with something like:
{
  "level":"error",
  "ts":1644832192.1764352,
  "logger":"controller.kamelet-binding-controller",
  "msg":"Reconciler error",
  "name":"c82iernvdc4irh6evn20",
  "namespace":"cos",
  "error":"could not determine source URI: no ref or URI specified in endpoint",
  "errorVerbose":"no ref or URI specified in endpoint"
}

But the error is not reported in the KameletBinding's status

This is similar to #3389 . The problem is that we don't set an upper limit for retries in the reconciliation loop (we probably do it only for the Build CR), so, this is left pending until the fix is applied. I wonder if we should immediately at least report the error condition or stop reconciliation when these kind of situations happen.


Add kind (intel/mac) column on apple silicon, and remove min loading time?

So its two things, first is self explanatory, the second, when i opened the app i noticed it took a while to open, went to check why, and saw that there is a min loading time for the loading screen on the main +page.svelte, is there a reason for that? Or can that be removed? (And maybe remove the entire loading screen all together, or only show it after 0.5s for example)

@jose-manuel-silva min loading time is meant to be there, what do you mean by Add kind (intel/mac) column on apple silicon, and where?

Just curious, is there a reason for the loading time? Or its just a design decision?
The kind column is to show if a process is x86 (intel) or arm (mac)


feat: Impl controller logic v1

What this PR does / why we need it:
Add your description
Which issue(s) this PR is related to (optional, link to 3rd issue(s)):
Fixes #
Reference to #

Special notes for your reviewer:
/cc @your-reviewer

Release note:

NONE

/lgtm


Refactor: Add code format in detailed description

Modified files that reside in the UserGuide folder to use code format
in parameters of the detailed description.
Closes #697
Signed-off-by: Kapil Bansal<EMAIL_ADDRESS>

@claireyuw can you please verify if the changes done in this PR are correct?

Hi @devkapilbansal , thank you for your contribution. Please see what I have commented in the files changed and make necessary modifications.


vendor: fix import path of protobuf

A prior commit to add vendor accidentally used the wrong import path
for the protobuf package. We fix this path:
github.com/protobuf -> github.com/golang/protobuf
We also commit the protoc-gen-go package into protobuf since it is a
dependency for the unit tests. I left that out in the original commit
for vendoring to keep the dependency tree smaller.
We also add go1.7 to the list of Go versions to test for.

\cc @alexwoz

I'm removing tip from the list of versions we test on. It can be fairly noisy.


Implement correct mapping for contract creation code

fixes #446 , #473

@JoranHonig  any additional changes required here?

@JoranHonig I think this can be merged?


quick fix for cli build command

current master throws:
TypeError: builder.cleanup is not a function

@rwjblue make sense. I'm on it

Apparently, I don't understand the TDZ.

@rwjblue @stefanpenner those tests were not too easy to write and I'm not 100% sure if those are what we want from them. Please review.

@stefanpenner can you please replay appveyor. Wonder if it is random fail or something is different on node 9.

@stefanpenner can you please replay appveyor. Wonder if it is random fail or something is different on node 9.

Unfortunately I cannot, appveyor does not believe I am an owner of this repo.... for unknown reasons.

Same here. 🤔

I'll merge, and keep an eye on master.
@chrmod much appreciated!!


Create boruvkasAlgorithm.py

@MMVonnSeek Thanks for the PR. Make sure to leave a star on the repo and follow me on Github.

Ol, Thank you


JSON - unable to fetch jsconfig schema

I get this warning when opening a jsconfig.json file.

It's fixed in the latest


It would be great to be able to configure a max width of the console output

Hi,
I don't know, perhaps this is already possible, but if I Log a complex Object the line gets so long that it is broken to the next line and the output is not very readable.
Would be great if I could set a max width and that the the output is wrapped in a way that makes sense like:
2016-05-13 13:11:08 [Verbose] REQUEST "http://localhost:3579/sync/syncstate"
2016-05-13 13:11:08 [Verbose] REQUEST "http://localhost:3579/sync/day"
2016-05-13 13:11:08 [Verbose] SyncEvent: UserId: "78eb5882-07f5-43b1-8eea-d11f9beea428" -
Dayfrom Device:
QmdSyncDay
{ LastModified: 05/13/2016 11:11:08,
DateOfDay: 05/12/2016 22:00:00,
Quote: ""
} -
DayOnServer: null
2016-05-13 13:11:08 [Verbose] REQUEST "http://localhost:3579/sync/syncstate"
Or something like that.
Cheers
THomas

@escamoteur have you had a look at https://github.com/serilog/serilog-sinks-literate

I tried literate Console, but hat's not what I meant. I'm talking about wrapping and indentation. Sorry when I post above was not really good formated. I try it again:
2016-05-13 13:11:08 [Verbose] REQUEST "http://localhost:3579/sync/syncstate"
2016-05-13 13:11:08 [Verbose] REQUEST "http://localhost:3579/sync/day"
2016-05-13 13:11:08 [Verbose] SyncEvent: UserId: "78eb5882-07f5-43b1-8eea-d11f9beea428" -
   Dayfrom Device:
    QmdSyncDay
   {
        LastModified: 05/13/2016 11:11:08,
        DateOfDay: 05/12/2016 22:00:00,
        Quote: ""
    }
    DayOnServer: null
2016-05-13 13:11:08 [Verbose] REQUEST "http://localhost:3579/sync/syncstate"

Cheers

@escamoteur it's an interesting suggestion :-)
It'd be a (really quite) substantial amount of work to design and implement robustly, I'd guess. If you're keen to propose a design, please jump in, but I can't see myself tacking this in the immediate future.
Since this repo is being wound down in favor of https://github.com/serilog/serilog-sinks-console, let's close the issue here and open a new one if someone's keen to take on the work.
Cheers!


Improve the speed of preview (caching, threading?)

Currently, the markdown preview is always recomputed, even without changes to the source markdown file. There would be several ways of improving this very poor design:

[ ] store a hash of the source code files if a preview has been generated. When reasking to preview, check the hash, and skip to display if nothing changed
[ ] make an incremental modification. When adding a new note to an existing notebook, it could be beneficial to compute the markdown preview only for the tiny bit we are interested in (plus, this could be used in previewing the note before posting it).
[ ] thread the preview away. While launching the application, and finding notebooks in the home folder, always assume that the user will want to preview it, and start in a separate thread (if available) the previewing. This way, the UI is not blocked, and the response will be faster.

We have to think of a way to store these hashes of already previewed files:

in the same way we store the preferences of the code? (so registry in Windows)
in .noteorganiser?

First: I don't have a problem with the current speed. But it's allways a good thing to improve it.

storing hashes sounds good. I would store them in the tempfolders as they belong to the tempfiles and html-files. The only thing we have to think of are the config-entries that can change and need a recompution even when the hash adds up
being able to compute the single notes sounds good in general, as you said, for a preview. Maybe we can generate a template-file where we only have to put the notes in. This could lead to different templates where the user can choose from.
threading is a nice thing in general, but the last thing I read was that it's still a mess in python. If it's possible without too much hassle, sure..


Error runing docker image

Problem encountered on https://dotnet.microsoft.com/learn/aspnet/microservice-tutorial/run-docker
Operating System: macos
Sesion-de-ilia:Microservicio1 ilarius$ docker run -it --rm -p 3000:80 --name mymicroservicecontainer mymicroservice
It was not possible to find any installed .NET Core SDKs
Did you mean to run .NET Core SDK commands? Install a .NET Core SDK from:
https://aka.ms/dotnet-download
failed to resize tty, using default size

So did you download the SDK as instructed in the install step? This article might help you determine that:
https://docs.microsoft.com/en-us/dotnet/core/install/how-to-detect-installed-versions?pivots=os-macos


automerge label is not being added to PRs

dependabot.yml content
version: 2
updates:
  - package-ecosystem: 'npm'
    directory: '/'
    schedule:
      interval: 'weekly'
      day: 'saturday'
      time: '10:00'
      timezone: 'Australia/Sydney'
    labels:
      - 'dependencies'
      - 'automerge'

What you expected to see, versus what you actually saw
I expected dependabot to open PR with automerge label, as per config
Images of the diff or a link to the PR, issue or logs
https://github.com/Maxim-Mazurok/google-api-typings-generator/pull/551 (had to add automerge label manually)

I imagine it's because it's a kinda "reserved" label. In which case, dependabot probably should warn/throw when it sees it in config?

Thanks for raising this. Agree we should either warn during config file validation or we should remove automerge from the reserved label list since we intentionally don't currently support automerge. If we could do the latter, than we could allow folks to use the github actions workaround with a label like automerge. But I haven't poked at the code yet, so not sure what's possible...

unstale


Improve restore reliability and performance

The restore script now:

Uses smaller batch sizes to reduce the likelihood of excessively large batches causing timeouts
Uses retries to ensure that all data is inserted
Uses a process pool to decrease the overall restore time

Given our discussion about working towards a different approach for backup and restore operations, I don't think it makes sense to increase the complexity of this tool.


Feature: to_list missing from python async API

SDK
Python
Description
The synchronous API had a to_list method that was a sibling to to_arrow and to_pandas.  The method simply did to_arrow().to_pylist() but could be helpful for people that are not familiar with arrow and just want to get python data directly.
It would be pretty easy to add back into the async API.

Closing as duplicate of #1517


fix(linter/no-unused-vars): panic in variable declarator usage checks

#5160  👈
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @DonIsaac and the rest of your teammates on  Graphite


wav2vec 2.0 pretraining

This PR introduces pertaining for wav2vec 2.0 based on the HF interface. The example is given with CommonVoice but works with everything. It has been compared to fairseq -> Better results on the same setup.

Thank you @TParcollet  for this add. @aheba did you have the chance to try it at least for few epochs?

@TParcollet, one thing I noticed is that the code is put in recipes/self-supervised-learning. Normally, the current standard is "recipes/" (which is not ideal in some cases, but it is what we did so far)
Isn't it better to create it in a recipe using the official "librilight" dataset rather than using commonvoice?  We might not be able to train it now, but it could be useful for the users.
I think it could be a good idea to couple this recipe with the webdataset dataloader that @Gastron set up. Also the voxlingua107 recipe (https://github.com/speechbrain/speechbrain/blob/develop/recipes/VoxLingua107/lang_id/hparams/train_ecapa.yaml) that we recently merged is using it. This might speed up everything quite a bit.

Well for 1. I think it makes no sense to have a SSL recipe under a dataset name, simply because it can't be found as there isn't a dataset that is specific to SSL (which is not true for TTS for instance). 2. We could, but I can't train it as it would cost a LOT. The good thing with CommonVoice is that it already is correctly sized (2K hours, which is enough for a robust SSL model already) and easily downloadable. 3. I can try, yeah. It will take some time though.

As for point 1, isn't LibriLight the standard dataset used in most
cases for self-supervised learning? It looks like they have different
chunks like small, medium, large.
On Tue, 2 Nov 2021 at 13:40, Parcollet Titouan @.***>
wrote:

Well for 1. I think it makes no sense to have a SSL recipe under a dataset
name, simply because it can't be found as there isn't a dataset that is
specific to SSL (which is not true for TTS for instance). 2. We could, but
I can't train it as it would cost a LOT. The good thing with CommonVoice is
that it already is correctly sized (2K hours, which is enough for a robust
SSL model already) and easily downloadable. 3. I can try, yeah. It will
take some time though.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/speechbrain/speechbrain/pull/1062#issuecomment-957979995,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AEA2ZVWUDWCWJSAURXUTG43UKAPAJANCNFSM5GTXTWMA
.

Not really :-/ Peoples are using CV, LibriLight, LibriSpeech, VoxPopulli or even LeBenchmark now for French :p I guess we could add it under LibriSpeech ? This would be quick and relatively cheap to train (48H with 20 GPUs) ...

Hey @mravanelli , @TParcollet ,
I have 6 GPUs, let me try it for half epochs..

@aheba could you add the fix to DDP (local_rank) to this PR ?

let me add it as well

as discussed offline with @TParcollet , the DDP fix will be in separate PR.

@TParcollet , @yan-gao-GY , Thanks you for this PR,
This work will be very valuable for Spbrain community.
@mravanelli, LGTM.

Next work in other PR:

[ ] add optional arg in run_on_main/if_main_process to specify local and global run_on_main according to the user. etc (each server need to do data preparation, vs at the end of epoch, only one process do evaluation)...
[ ] support torchrun for next integration with pytorch version

@aheba looking forward for the first one as I need it to do multinode training properly ahah

@mravanelli it should be ready for a last try. The model is running on 32 GPUs right now and everything looks fine. Note: I will certainly make an announcement once this is merged because it may interest quite a lot of peoples. @Gastron If you have some spare time, I would love to get some help in building the Webdataset shard script for that ...

@mravanelli ready. I will just add a wav2vec 2.0 (CV EN) Base pretrained model to show that it works (to the YAML).

The only thing remaining (that we will maybe handle later with the engineer) is to have a way of saving our pretrained model in a HuggingFace compatible way ... For now it's impossible.

I started a downstream training to verify that everything is ok on that side as well. Please just try to run the pretraining if you wish.

@mravanelli now it works. It reverted back to transformers 4.11, you can try the PR again but from my side it worked :-) Things to try : train few epochs and then try to use it with a fine-tuning recipe. I will modify the tutorial to explain the steps as you must create a HF like folder (i.e. just add the config.json and feature_extractor.json with your wav2vec.ckpt)

Finally fixed. I have to say that this implementation sample negatives from the whole sentence and not only from the masked indices. The latter behaviour was a design choice made by Facebook that is clearly problematic for short sentences that might not have enough mask indices to sample negatives from.

I tried it on my side and everything is running. Of course, I didn't double checked the big run with 32 GPUs but we can definitely trust you and assume everything is working now. I think we can merge! Thank you @TParcollet


License detection from composer.lock

Type of change

New feature (non-breaking change which adds functionality)

Description
Checklist

+1


PCB standoff holes for securing Raspberry PI

I noticed that the image a314_with_rpi.jpg shows a Raspberry Pi (model unknown) dangling from the GPIO pins - modern Raspberry PI (later revision 1 plus models and later IIRC) have common hole placements for standoffs, would it be possible to incorporate at least one if not two of the standoff holes on the top of the board so that the Raspberry Pi can be secured properly to the board (as one would a normal Raspberry HAT).
Likewise on both the A500 and A600 boards having at least one standoff hole that conforms with a Raspberry Pi Zero would also be most helpful.

The RPi in question on that picture is the Model 3. Yes, you're right about it not having any stand-offs, but that is the result of numerous iterations and thorough tests of the PCB and it's mechanical integrity. You see, stand-offs are intentionally excluded for several reasons:

Excluding them allow for a a better and more relaxed routing of the PCB traces, avoiding us to use unnecessary thin traces or close track spacing.
The A314 system (meaning the A314 board together with the RPi attached on top, inside an Amiga) is in a physically secure location in space, not prone to vibration or chock. The RPi is secured by the 40-pin header as well as being fixed between the A314 PCB and the Amiga keyboard backplate. The connector isolator which you see in the picture you linked to (the white TPU strip on top of the RPi GPIO connector) makes contact with said backplate. There is just no room for lateral movement.
Further, the RPi lays with its video and audio connectors resting on top of the trapdoor side ledge, providing excellent support for the board. The only situation where this could be a possible issue, is if you were to use your Amiga upside-down. I'd say that is a an edge-case scenario...
I hope this eases your worries about the mechanical aspects of the A314 construction.

I just saw the reverse of the board on Ebay and noticed that one of the distance standoff holes would have in the same location as the CP2032 battery holder, so that one is definitely not practical :-D
I assume that whilst a Raspberry Pi Zero would not suffer from the same level mechanical stress as a full sized model, but it would also not benefit from the video and audio connectors positioning - looking at the current design it would also put a hole rather close to IC4 (SRAM) and that is impractical to move.
What are the expected height specifications for the GPIO socket and pin headers (I assume that the pins are assumed to add little to no separation, so something like this header would cause issues) ?

We have based the design upon the RPi 3 as it is supplied by the RPF. For that model (and compatible ones), you will find the appropriate header component listed in the BOM.


Added 0x

Hey MM team! Tom from 0x here. Check out our contract address on Etherscan

Hey @danfinlay -- sorry to ping, but any ETA on reviewing this PR? We're working on a 0x onboarding experience that uses Metamask, so would be great to get ZRX into this pkg so people can see their balances in the onboarding flow!

Sorry for the long wait, thanks for the PR!


Incorrect parameter displaying in tooltip.

Describe the bug
When comment of last parameter is an unordered list, the parameter is displayed incorrectly.
To Reproduce
---@param par1 string|'"text1"'|'"text2"' @event
---@param par2 function|' function(client, topic, message) end' @`function(client, topic, message)`.
--- - comment par2
---@return nil
function rr:on(par1, par2) end

Screenshots

Environment (please complete the following information):

OS: Windows
Is WSL remote? no
Client:  VSCode

I did some research and found out that it was because a code like this was generated in hover:
- xx
```lua
print(1)
```
---
```lua
print(1)
```

This code is explained in different places in different formats:

github and markdown extension of VSCode:

vscode and some other online play grounds:

The latter way of explanation will lead to some strange phenomena, and eventually lead to the problems you encountered.
I don't know what the correct specification is, so I can only suggest that you avoid this writing first (for example, insert a blank line under -comment par2 to explicitly indicate the end of the list syntax).

Thank you for your detailed answer. I have inserted blank line under unnumbered list, and this solved issue.


%0A in output

I encountered this in Java (using patch_toText(patch_make(...)), but I can see it on the JavaScript demo page, too, and I saw a past Google Code discussion touch on it for a Python usage, plus some Go code working around it.
Enter the following into the demo page:
Old version:

a
a

New version:

a
a
a

(I entered both with trailing newlines, but the behavior is similar without.)
Compute Patch gives:
@@ -1,4 +1,6 @@
 a%0Aa%0A
+a%0A

I would have expected something more like:
$ diff -u <(echo a; echo a) <(echo a; echo a; echo a) | tail -n +3
@@ -1,2 +1,3 @@
 a
 a
+a

(Hmm, I notice only now that diff-match-match disagrees with diff -u about the number of lines involved. (At least, I think that's what the ,4/,6 and ,2/,3 are?))
As a workaround, I can replace %0A sequences with the empty string, but as best I can tell, even that will leave various other characters escaped.
I'm a little confused as to why URL encoding comes into this at all. I would expect for Patch.toString() to avoid all that, instead looking like this:
        text.append(aDiff.text.replaceAll("\n", ""))
            .append("\n");

(It seems strange that I need to remove internal \n characters at all. I haven't dug into it.)

Just got back to this.
patch does understand the format produced by diff-match-patch. I am confused about this, since it's not clear how patch differentiates between actual sequences of %0A and those intended as newlines. Possibly it's based on the counts given by the line numbers, or it might just do whatever works. I couldn't find anything in the GNU description or Guido's description.
Also, I'm wrong about both the workaround and my proposed change to Patch.toString(). (I had written some tests, and I think I was looking at the expected values rather than the actual....) If I remove all \n characters, I end up with the lines run together. And if I replace them with newlines, I end up with lines without the appropriate leading characters, plus some extra newlines:
 a
a

-m
n
o
p
+q
r
s
t
u
v

z
z

Then, if I remove append("\n") from Patch.toString(), I can eliminate the blank lines -- but also the newline between p and +q. (And I'd still need to reinsert the + and - signs.)

Maybe I should look at it as "diff-match-patch does all kinds of fun stuff that is overkill for line-based diffs; for those, use java-diff-utils or something."

The root issue here is that DMP is character-based, whereas Unix diff/patch is line based.  There's documentation on this here:  https://github.com/google/diff-match-patch/wiki/Unidiff


Case sensitive search not supported on index fields

Is your feature request related to a problem? Please describe.
when we try to search the index field with some value, Vespa returns the result as case insensitive.
e.g if we are doing search for fieldA="Aabc", it also returns doc that has value "aabc" for fieldA.
Describe the solution you'd like
want the case sensitive result only. Docs with fieldsA=Aabc should be returned in the search
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.

Vespa already supports case-sensitive searches using attribute fields using match:cased doc.
It's not natural to have case sensitive search against index field with text matching and linguistic integrations IMHO.

There are use cases for it though, and what we have done recently is to make this completely the discretion of the linguistics module, but the linguistics module will have to support it and most don't. So, I think what's needed is to be able to select a linguistics module per field.

Hi  @bratseth,
We are looking to enable  case sensitive search on index fields only with exact match enabled. Is there a way we can achieve case sensitive search on these fields with out using a different lingiustic module since we dont need stemming and other parsing for exact field

To be clear you want both of a) cased exact match, no partial match, b) on an index field not an attribute?
And if so, why can't it be an attribute?

The only issue with using these fields as attribute is that they are stored in memory or we would need to enable paged similar to our current attributes. Since we already have a lot of attribute with paged enabled this would increase the page in out since these fields are heavily used.
We can definitely try the attribute approach but were wondering if there is another way.

Ok, got it. No other way right now, but it's not a lot of work to add when also doing "exact" - we can consider doing that.

Feature request: Allow option to make index fields with {match: word} case sensitive


calling .exit() has unexpected result: wllamaExit is not a function

I verify that the exit function exists, but calling it results in the error above.
try{
	if(window.llama_cpp_model_being_loaded){
		if(typeof window.llama_cpp_app.unloadModel === 'function'){
			await window.llama_cpp_app.unloadModel();
		}else{
			console.error("window.llama_cpp_app was not null, but had no unloadModel function?  window.llama_cpp_app: ", window.llama_cpp_app);
		}

	}
	else{
		if(typeof window.llama_cpp_app.exit === 'function'){
			await window.llama_cpp_app.exit();
		}else{
			console.error("window.llama_cpp_app was not null, but had no exit function?  window.llama_cpp_app: ", window.llama_cpp_app);
		}
	}
}
catch(err){
	console.error("caught error trying to stop/unload Wllama: ", err);
}

I just noticed something while doing mobile debugging. I couldn't figure out why the smallest model (Danube 3 500m, 320MB) wasn't working. According to my debug information the Wllama object was null (which it normally gets set to after calling exit() succesfully.
On mobile the memory debugging was working, so I noticed that Wllama's workers still seemed to exist?

Maybe I should never set the Wllama object back to zero.
Is there another sure-fireway to fully destroy wllama? The UI allows users to load all kinds of models, some of which are handled by Wllama, but others are handled by WebLLM or even Transformers.js. I try to juggle these 'runners' memory-wise, only allowing one of them to exist at a time.
At least, that was the theory...

Maybe I should never set the Wllama object back to null.

You can set it to a new wllama instance instead of setting to null

You can set it to a new wllama instance instead of setting to null

Thanks. Will that kill the workers and unload the model to release the memory properly?
I just noticed that resetWllamaInstance effectively does what you describe.
const resetWllamaInstance = () => {
  wllamaInstance = new Wllama(WLLAMA_CONFIG_PATHS, { logger: DebugLogger });
};

For now I've modified the code to no longer null the Wllama instance, and just unload the model when it's WebLLM's turn. Or could that result in memory not being fully recovered?

I'm running into a situation where await wllama.exit() is stuck. The code (similar to that in the first post) doesn't get beyond it.

I'm trying to unload the old model before loading a new one.
if(typeof window.llama_cpp_app.isModelLoaded != 'undefined'){
	let a_model_is_loaded = await window.llama_cpp_app.isModelLoaded();
	console.warn("WLLAMA: need to unload a model first?: ", a_model_is_loaded, window.llama_cpp_app);
	if(a_model_is_loaded && typeof window.llama_cpp_app.unloadModel != 'undefined'){
		console.log("wllama: unloading loaded model first.  window.llama_cpp_app: ", window.llama_cpp_app);
		await window.llama_cpp_app.unloadModel();
	}
	else if(a_model_is_loaded && typeof window.llama_cpp_app.exit != 'undefined'){
		console.error("wllama: unloading loaded model first by calling exit instead of unloadModel.  window.llama_cpp_app: ", window.llama_cpp_app);
		await window.llama_cpp_app.exit();
		console.log("wllama exited.  window.llama_cpp_app is now: ", window.llama_cpp_app);
	}
	else if(a_model_is_loaded){
		console.error("WLLAMA HAS A MODEL LOADED, BUT NO WAY TO UNLOAD IT?  window.llama_cpp_app: ", window.llama_cpp_app);
		return false;
	}
	create_wllama_object(); // TODO: potential memory leak if the old model isn't unloaded properly first
}
else{
	console.error("llama_cpp_add has no isModelLoaded: ", window.llama_cpp_app);
}

The reason I ask it because I've read that Mobile Safari doesn't clean up orphaned web workers properly.
I'm now attempting this:
if(typeof window.llama_cpp_app.proxy != 'undefined' && window.llama_cpp_app.proxy != null && typeof window.llama_cpp_app.proxy.worker != 'undefined'){
        console.warn("wllama.proxy still existed, attempting to terminate it manually");
        window.llama_cpp_app.proxy.worker.terminate();
}

Calling window.llama_cpp_app.proxy.worker.terminate(); has been working well for now.
I'll leave this issue open because I'm curious what the recommended route for unloading models is, and how memory can be optimally recovered while keeping an instance of Wllama alive for housekeeping tasks.


Simplify entity table construction from DataFrames

Introduces automatic construction of entity mappings by exploiting naming conventions on DataFrame columns.
Fixes #359

We should also document these conventions when we address https://github.com/opencypher/cypher-for-apache-spark/issues/379


Chat users API, list users detail

List only user detail with last chat date time

List only user details with necessary informations


Implement multi-variable environment in spn.Product

https://github.com/probcomp/sum-product-dsl/blob/c482341d04ef35febefc46f8821350e9236e8013/src/spn.py#L409-L413

The multivariable transform must be of type event.
If a condition or logprob query has multivariate symbol, then we cannot call logprob_factored on the children.  Instead, we need to reassemble the query post-substitution and call self.logprob on the new query, which will take care of converting it to dnf, solving it, and factoring it again.

Related https://github.com/probcomp/sppl/issues/92, since min/max can be of the form min(X1, X2, ..., Xn).


chore: update coverage for 2024-12-03-21-22

updates coverage metadata for 2024-12-03-21-22

/approve
/lgtm


[Bug]: json report is invalid if no findings written

What happened?
If prowler is executed with the --quiet flag it will only write failed checks to the report files. In case no finding was found the JSON report will end up with an invalid content o ]
How to reproduce it
Steps to reproduce the behavior:
prowler aws \
      --verbose --quiet \
      --output-modes json \
      --allowlist-file prowler-allowlist.yaml \
      --region eu-central-1 \
      --excluded-services trustedadvisor macie \
      --excluded-checks accessanalyzer_enabled_without_findings

Expected behavior
a JSON report file with [] content
From where are you running Prowler?
Please, complete the following information:

Resource: workstation and docker
OS: Ubuntu / Alpine
Prowler Version [./prowler --version]: 3.1.1
Python version [python --version]: Python 3.10.6
Pip version [pip --version]:  pip 22.0.2 from /usr/lib/python3/dist-packages/pip (python 3.10)
Installation method (Are you running it from pip package or cloning the github repo?): pip
Others:

Hi @jansepke we are going to review this and let you know.
Thanks for using Prowler !!


auth: fix remove subset when there are equal ranges

/cc @heyitsanthony @mitake
If we have two roles with same range permissions, the end result is an empty permission set due to a bug in subset removal function. This patch fixes it.

@xiang90 lgtm, thanks


[COLLECTIONS 706] Add set utils.as set method

PR adds the method described in https://issues.apache.org/jira/projects/COLLECTIONS/issues/COLLECTIONS-706 .
Could be discussed how to handle this case:
SetUtils.asSet(null)

The compile will warn (at least in Eclipse) that this actually means passing a null array as the var-args argument.

Right now, I implemented it such that in this case, the method will return null.
It might be arguable to better return an empty set in this case.
It could even be argued to return a set with the value null in this case.

Opinions?

Ola @reckart !
Out of curiosity, do you have already a practical use for that in another ASF project, like UIMA perhaps?
And regarding the change, is it different than SetUtils.unmodifiableSet ? If so, do you think it would be worth to add a comment in the documentation about that?
Cheers
Bruno

Closing; we have in 4.3:

org.apache.commons.collections4.SetUtils.hashSet(E...)
org.apache.commons.collections4.SetUtils.unmodifiableSet(E...)

olá @kinow :) The use case was not in an ASF project but in another OS project I am working on. I don't remember the exact case though.


[Enhancement] Add consistent hashing for hdfs backend selector

What type of PR is this：

[ ] bugfix
[ ] feature
[x] enhancement
[ ] refactor
[ ] others

Which issues of this PR fixes ：

Fixes #
Problem Summary(Required) ：

Checklist:

[ ] I have added test cases for my bug fix or my new feature
[ ] I have added user document for my new feature or new function

[FE PR Coverage Check]
:disappointed: fail : 0 / 161 (00.00%)
file detail

path
covered_line
new_line
coverage
not_covered_line_detail

:large_blue_circle:
com/starrocks/common/util/ConsistentHashRing.java
0
48
00.00%
[21, 30, 31, 32, 33, 36, 37, 38, 39, 45, 46, 47, 48, 49, 50, 51, 52, 53, 56, 57, 58, 59, 60, 63, 66, 67, 68, 69, 70, 73, 76, 77, 78, 79, 80, 82, 83, 84, 85, 87, 88, 91, 92, 93, 96, 97, 100, 102]

:large_blue_circle:
com/starrocks/planner/HudiScanNode.java
0
1
00.00%
[37]

:large_blue_circle:
com/starrocks/common/util/RendezvousHashRing.java
0
42
00.00%
[23, 24, 25, 26, 29, 32, 33, 34, 35, 36, 37, 38, 39, 42, 43, 44, 47, 48, 49, 50, 53, 56, 62, 63, 64, 65, 67, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 81, 83, 84, 85, 87]

:large_blue_circle:
com/starrocks/planner/HdfsScanNode.java
0
1
00.00%
[50]

:large_blue_circle:
com/starrocks/qe/HDFSBackendSelector.java
0
69
00.00%
[65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 80, 83, 84, 85, 87, 88, 89, 90, 91, 93, 94, 97, 98, 100, 116, 117, 120, 121, 124, 125, 126, 127, 128, 129, 130, 132, 133, 136, 139, 140, 141, 144, 147, 148, 168, 169, 170, 171, 173, 174, 175, 178, 179, 180, 181, 182, 184, 188, 196, 197, 198, 201, 202, 206, 215]

[FE PR Coverage Check]
:disappointed: fail : 88 / 166 (53.01%)
file detail

path
covered_line
new_line
coverage
not_covered_line_detail

:large_blue_circle:
com/starrocks/planner/HudiScanNode.java
0
1
00.00%
[37]

:large_blue_circle:
com/starrocks/planner/HdfsScanNode.java
0
1
00.00%
[50]

:large_blue_circle:
com/starrocks/qe/HDFSBackendSelector.java
0
72
00.00%
[66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 81, 84, 85, 86, 88, 89, 91, 92, 93, 94, 96, 97, 100, 101, 103, 119, 120, 123, 124, 127, 128, 129, 130, 131, 132, 133, 135, 136, 139, 142, 143, 144, 147, 150, 151, 155, 156, 157, 158, 178, 179, 180, 181, 183, 184, 185, 188, 189, 190, 191, 192, 194, 198, 206, 209, 210, 214, 223]

:large_blue_circle:
com/starrocks/common/util/ConsistentHashRing.java
47
49
95.92%
[87, 94]

:large_blue_circle:
com/starrocks/common/util/RendezvousHashRing.java
41
43
95.35%
[60, 75]

[FE PR Coverage Check]
:disappointed: fail : 110 / 188 (58.51%)
file detail

path
covered_line
new_line
coverage
not_covered_line_detail

:large_blue_circle:
com/starrocks/planner/HudiScanNode.java
0
1
00.00%
[37]

:large_blue_circle:
com/starrocks/planner/HdfsScanNode.java
0
1
00.00%
[50]

:large_blue_circle:
com/starrocks/qe/HDFSBackendSelector.java
0
72
00.00%
[66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 81, 84, 85, 86, 88, 89, 91, 92, 93, 94, 96, 97, 100, 101, 103, 119, 120, 123, 124, 127, 128, 129, 130, 131, 132, 133, 135, 136, 139, 142, 143, 144, 147, 150, 151, 155, 156, 157, 158, 178, 179, 180, 181, 183, 184, 185, 188, 189, 190, 191, 192, 194, 198, 206, 209, 210, 214, 223]

:large_blue_circle:
com/starrocks/common/util/ConsistentHashRing.java
47
49
95.92%
[87, 94]

:large_blue_circle:
com/starrocks/common/util/RendezvousHashRing.java
41
43
95.35%
[60, 75]

:large_blue_circle:
com/starrocks/common/util/PlainHashRing.java
22
22
100.00%
[]

[FE PR Coverage Check]
:disappointed: fail : 114 / 192 (59.38%)
file detail

path
covered_line
new_line
coverage
not_covered_line_detail

:large_blue_circle:
com/starrocks/planner/HudiScanNode.java
0
1
00.00%
[37]

:large_blue_circle:
com/starrocks/planner/HdfsScanNode.java
0
1
00.00%
[50]

:large_blue_circle:
com/starrocks/qe/HDFSBackendSelector.java
0
72
00.00%
[66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 81, 84, 85, 86, 88, 89, 91, 92, 93, 94, 96, 97, 100, 101, 103, 119, 120, 123, 124, 127, 128, 129, 130, 131, 132, 133, 135, 136, 139, 142, 143, 144, 147, 150, 151, 155, 156, 157, 158, 178, 179, 180, 181, 183, 184, 185, 188, 189, 190, 191, 192, 194, 198, 206, 209, 210, 214, 223]

:large_blue_circle:
com/starrocks/common/util/RendezvousHashRing.java
41
43
95.35%
[60, 75]

:large_blue_circle:
com/starrocks/common/util/ConsistentHashRing.java
51
53
96.23%
[97, 104]

:large_blue_circle:
com/starrocks/common/util/PlainHashRing.java
22
22
100.00%
[]

[FE PR Coverage Check]
:disappointed: fail : 114 / 193 (59.07%)
file detail

path
covered_line
new_line
coverage
not_covered_line_detail

:large_blue_circle:
com/starrocks/planner/HudiScanNode.java
0
1
00.00%
[37]

:large_blue_circle:
com/starrocks/planner/HdfsScanNode.java
0
1
00.00%
[50]

:large_blue_circle:
com/starrocks/qe/HDFSBackendSelector.java
0
73
00.00%
[60, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 81, 84, 85, 86, 88, 89, 91, 92, 93, 94, 96, 97, 100, 101, 103, 119, 120, 123, 124, 127, 128, 129, 130, 131, 132, 133, 135, 136, 139, 142, 143, 144, 147, 150, 151, 155, 156, 157, 158, 178, 179, 180, 181, 183, 184, 185, 188, 189, 190, 191, 192, 194, 198, 206, 209, 210, 214, 223]

:large_blue_circle:
com/starrocks/common/util/RendezvousHashRing.java
41
43
95.35%
[60, 75]

:large_blue_circle:
com/starrocks/common/util/ConsistentHashRing.java
51
53
96.23%
[97, 104]

:large_blue_circle:
com/starrocks/common/util/PlainHashRing.java
22
22
100.00%
[]

[FE PR Coverage Check]
:heart_eyes: pass : 173 / 193 (89.64%)
file detail

path
covered_line
new_line
coverage
not_covered_line_detail

:large_blue_circle:
com/starrocks/planner/HudiScanNode.java
0
1
00.00%
[37]

:large_blue_circle:
com/starrocks/planner/HdfsScanNode.java
0
1
00.00%
[50]

:large_blue_circle:
com/starrocks/qe/HDFSBackendSelector.java
59
73
80.82%
[72, 73, 74, 75, 76, 77, 78, 79, 86, 92, 93, 94, 124, 192]

:large_blue_circle:
com/starrocks/common/util/RendezvousHashRing.java
41
43
95.35%
[60, 75]

:large_blue_circle:
com/starrocks/common/util/ConsistentHashRing.java
51
53
96.23%
[97, 104]

:large_blue_circle:
com/starrocks/common/util/PlainHashRing.java
22
22
100.00%
[]

[FE PR Coverage Check]
:heart_eyes: pass : 225 / 253 (88.93%)
file detail

path
covered_line
new_line
coverage
not_covered_line_detail

:large_blue_circle:
com/starrocks/planner/HudiScanNode.java
0
1
00.00%
[37]

:large_blue_circle:
com/starrocks/planner/HdfsScanNode.java
0
1
00.00%
[50]

:large_blue_circle:
com/starrocks/qe/HDFSBackendSelector.java
59
73
80.82%
[72, 73, 74, 75, 76, 77, 78, 79, 86, 92, 93, 94, 124, 192]

:large_blue_circle:
com/starrocks/common/util/StaticPartitionedHashRing.java
50
57
87.72%
[80, 81, 82, 83, 84, 92, 99]

:large_blue_circle:
com/starrocks/common/util/RendezvousHashRing.java
43
46
93.48%
[60, 75, 85]

:large_blue_circle:
com/starrocks/common/util/ConsistentHashRing.java
51
53
96.23%
[97, 104]

:large_blue_circle:
com/starrocks/common/util/PlainHashRing.java
22
22
100.00%
[]

[FE PR Coverage Check]
:heart_eyes: pass : 175 / 196 (89.29%)
file detail

path
covered_line
new_line
coverage
not_covered_line_detail

:large_blue_circle:
com/starrocks/planner/HudiScanNode.java
0
1
00.00%
[37]

:large_blue_circle:
com/starrocks/planner/HdfsScanNode.java
0
1
00.00%
[50]

:large_blue_circle:
com/starrocks/qe/HDFSBackendSelector.java
59
73
80.82%
[72, 73, 74, 75, 76, 77, 78, 79, 86, 92, 93, 94, 124, 192]

:large_blue_circle:
com/starrocks/common/util/RendezvousHashRing.java
43
46
93.48%
[60, 75, 85]

:large_blue_circle:
com/starrocks/common/util/ConsistentHashRing.java
51
53
96.23%
[97, 104]

:large_blue_circle:
com/starrocks/common/util/PlainHashRing.java
22
22
100.00%
[]

[FE PR Coverage Check]
:heart_eyes: pass : 176 / 198 (88.89%)
file detail

path
covered_line
new_line
coverage
not_covered_line_detail

:large_blue_circle:
com/starrocks/planner/HudiScanNode.java
0
1
00.00%
[37]

:large_blue_circle:
com/starrocks/planner/HdfsScanNode.java
0
1
00.00%
[50]

:large_blue_circle:
com/starrocks/qe/HDFSBackendSelector.java
59
73
80.82%
[72, 73, 74, 75, 76, 77, 78, 79, 86, 92, 93, 94, 124, 192]

:large_blue_circle:
com/starrocks/common/util/RendezvousHashRing.java
43
46
93.48%
[60, 75, 85]

:large_blue_circle:
com/starrocks/common/util/PlainHashRing.java
23
24
95.83%
[48]

:large_blue_circle:
com/starrocks/common/util/ConsistentHashRing.java
51
53
96.23%
[97, 104]

[FE PR Coverage Check]
:heart_eyes: pass : 192 / 214 (89.72%)
file detail

path
covered_line
new_line
coverage
not_covered_line_detail

:large_blue_circle:
com/starrocks/planner/HudiScanNode.java
0
1
00.00%
[37]

:large_blue_circle:
com/starrocks/planner/HdfsScanNode.java
0
1
00.00%
[50]

:large_blue_circle:
com/starrocks/qe/HDFSBackendSelector.java
75
89
84.27%
[73, 74, 75, 76, 77, 78, 79, 80, 87, 93, 94, 95, 125, 215]

:large_blue_circle:
com/starrocks/common/util/RendezvousHashRing.java
43
46
93.48%
[60, 75, 85]

:large_blue_circle:
com/starrocks/common/util/PlainHashRing.java
23
24
95.83%
[48]

:large_blue_circle:
com/starrocks/common/util/ConsistentHashRing.java
51
53
96.23%
[97, 104]

[FE PR Coverage Check]
:heart_eyes: pass : 219 / 241 (90.87%)
file detail

path
covered_line
new_line
coverage
not_covered_line_detail

:large_blue_circle:
com/starrocks/planner/HudiScanNode.java
0
1
00.00%
[37]

:large_blue_circle:
com/starrocks/planner/HdfsScanNode.java
0
1
00.00%
[50]

:large_blue_circle:
com/starrocks/qe/HDFSBackendSelector.java
102
116
87.93%
[74, 75, 76, 77, 78, 79, 80, 81, 88, 94, 95, 96, 126, 257]

:large_blue_circle:
com/starrocks/common/util/RendezvousHashRing.java
43
46
93.48%
[60, 75, 85]

:large_blue_circle:
com/starrocks/common/util/PlainHashRing.java
23
24
95.83%
[48]

:large_blue_circle:
com/starrocks/common/util/ConsistentHashRing.java
51
53
96.23%
[97, 104]

run starrocks_admit_test

run starrocks_admit_test


Update grpc_tools_node_protoc_ts version to 5.3.2

Hello folks,
Requesting an update to the typescript generation tools to the latest version. A bug was recently fixed https://github.com/agreatfool/grpc_tools_node_protoc_ts/issues/113 in the current version, this issue causes typescript to be broken if some oneOf types are used.

Some additional context: My team uses the built images from namely for protoc generation, hence need to update the defaults.
I've opened a PR with the new version updated, please lmk if you have any further questions.

resolved in #301


Generics shouldn't escape their scope

I ran into this issue when I was working on scoped type variables, but the underlying problem is actually independent.
The following snippet should not cause an error.
function identity<U>(x: U) {
  return x;
}
var test: number = identity(0)

generics.js:2:10,10: U
 This type is incompatible with
generics.js:4:11,16: number

This is expected behavior and a result of the way we verify polymorphic functions. The function body is type checked twice, once with T = empty and once with T = mixed (or the bound, if T has a bound).
Because the return type is not explicitly annotated, it is monomorphic. That is, the implicit return type is an inferred type variable and both instantiations of T flow into it. Thus when we invoke indentity(0) the return type is effectively mixed | empty. While empty is a subtype of all types, including number, mixed is not. When Flow hits a constraint where mixed runs into number, we see an error.
The solution is to annotate the return type, like so:
function identity<U>(x: U): U {
  return x;
}
var test: number = identity(0)


DSS Sync Failure Alert

Build health alert for dss-sync-sfn failures
Push alerts for any failures into #data-store-eng
Resolves: https://github.com/HumanCellAtlas/data-store/issues/2102

there does not appear to be anything under Makfile:test perhaps we can place a manual test in the DSS that results in a ExecutionFailed status, however I'm not sure if we can force the failure.


add skia license

We now have the full Skia repository as submodule, or do you want it at top level directory? (I wonder if symbolic link from the submodule would work?).

well, we will still need a license file of our own.

i think it's enough that the submodule contains the whole tree including its own license file.
for the license of the python wrapper, I don't know. You choose

Lets just use the same license for simplicity (and if Skia is ever re-licensed we should remember to update ours 🙂).

sgtm


how i get chat id and user id?

how i get chat id and user id to use it and send message?

Hi!
First thing is to decide whether to enable a webhook or call getUpdates.
In case of the webhook, follow this documentation: https://github.com/unreal4u/telegram-api/wiki/Getting-updates-via-Webhook
Otherwise, please refer to our example: https://github.com/unreal4u/telegram-api/blob/master/examples/get-updates.php

hello,
i not know anything on telegram BOT and telegram API..
haven't a full example from start creating bot to save messages between bot and human in DB?
thanks..

I have another repo that implements a few bots, but you'll have to begin reading some stuff so that you know what you're doing: https://github.com/unreal4u/telegram-bots
Have a look at the following doc which explains the very basics: http://telegra.ph/Introduction-to-bot-programming-02-21

Closing this due to inactivity. Should you have the need to reopen it, please do!
Greetings.


Develop tool bug with https://giscus.app/client.js on Chromium brower

I find something strange when I use Google Chrome and open develop tool to select html element. Some element can't select，like this (on https://laymonage.com/posts/giscus):

Same bug on other chromium browser (e.g. Brave). Normal on Safari and Firefox. When I change script scr to local server's js file, everything normal.
So why happened this?

This is a Chromium bug: https://bugs.chromium.org/p/chromium/issues/detail?id=1219647


Accessing Other Dependent Resources for a Managed Dependent Resource

The purpose of the issue to prepare and discuss a design and implementation, regarding how Dependent Resource might access other dependent resources in managed mode.
Problem statement
Dependent Resources are not independent of each other, it's a common usecase to use output or generally speaking data from other dependent resources. In standalone mode, when dependent resources created manually reference to an other dependent resource can be passed when dependent resource is created. For managed mode this is more problematic, since those are instantiated and managed by the system. But should have access to other dependent resources.
Proposed Solution
A way would be to inject the other managed dependent resources. (Eventually the plan is to allow to describe most of the workflows with dependent resources), making this available via an interface:

public interface ManagedDependentResource {

void setDependentResourceContext(DependentResourceContext context)

}

Where the DependentResourceContext will contain a list of managed dependent resources.
Notes:

With depends_on relation, should be the dependent resources accessible somehow more explicitly? see: https://github.com/java-operator-sdk/java-operator-sdk/issues/850

Note that you can already pass information around using the Context object but it might be interesting to make things a little bit more structured, indeed.

yes, would like to remove context  eventually if possible.

yes, would like to remove generic map from context eventually if possible.

Out of curiosity: why? This provides a way for users to pass information around without the SDK having to know about it.

I meant that, it might be eventually not needed or necessary to have. But we will see.


A command to populate/update gh-pages for a dataset

Might actually be the one creating or updating GitHub action which would update gh-pages for that dataset when pushed

Could you give an example to illustrate what you mean, since I'm having trouble understanding exactly what is suggested here.
Would this be e.g. if there is a datalad dataset that has a corresponding record in an existing catalog, and you want an updated metadata record to be generated and then rendered in the catalog if the datalad dataset itself is updated?

target use-case: I have a dataset for which I extracted metadata (or may be even not?! that would be cool if action automatically extracted it), I push dataset to github, and GitHub action does everything necessary so that gh-pages for the dataset serves a datalad-catalog website for that dataset.  create-sibling-github could gain (somehow) an option --with-gh-pages=datalad-catalog (not unlike we have create-sibling --with-ui which creates http://datasets.datalad.org/ like website) to deploy such an action upon creation of the github repo.

Thanks for clarifying, this makes a lot of sense. I have done something like this before here: https://github.com/jsheunis/automaticat-demo/blob/main/.github/workflows/generate_cat_entry.yml. This workflow checks the diff for every push to main, and if there's a new subdataset added or if any files changed, then it extracts metadata for those datasets/files, translates the records to the catalog schema, and adds it to the catalog which is contained in the gh-pages branch.
Your use case is slightly different in that it operates from a single dataset which is not assumed to be part of a super-sub-dataset nested structure. But this actually makes the process a bit simpler.
I like the idea of adding the option to create-sibling-github.


Caesar Cipher -Cryptography (Python)

Have you read the Contributing Guidelines on Pull Requests?
Yes
Description
I have implemented Caesar Cipher in PYTHON
To shed some light on cryptographic algorithms, we begin with coding the basic Caesar Cipher which encrypts and decrypts messages with a specific shift value
Checklist

[x] I've read the contribution guidelines.
[x] I've checked the issue list before deciding what to submit.
[x] I've edited the README.md and link to my code.

Related Issues or Pull Requests
#422

@HarshCasper  @Kajol-Kumari please do review .. 😄

I have made the changes to the code and readme. Please review @SKAUL05 @siddharth25pandey @Kajol-Kumari @atarax665


Data Enrichment Feature

Merge Data Enrichment Feature

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.2 out of 3 committers have signed the CLA.:white_check_mark: novking:white_check_mark: LukeMainwaring:x: Eytan HanigEytan Hanig seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.


Added doc/VolumeNames.md

LGTM

@kerneltime Looks like CI is failing with "Out of resources"

LGTM.
In last paragraph, did you intend to put docker commands as code?
docker run -v volume:/path ... , docker volume create, docker volume rm, docker volume inspect


Adding Extensions Documentation to the Website

Description
As a part of MVP Integration of Extensions Catalogue we need to add extensions catalog document on the website.
For this I have raised two PRs:

https://github.com/asyncapi/website/pull/2226
https://github.com/asyncapi/extensions-catalog/pull/132

Raising this issue for central point of communication.
cc" @derberg

closing this issue


Javascript Error

Describe the bug
Whenever I start the app or do a full reinstall it starts but then shows a java script error. The accual widgets are seen behind the error box but dissapear if i go to desktop or click Ok. Sometimes they stay there but there is no updates happening so it just stays the same. Really love how it looks though, love your work.
Specificly, the error states "
Error
XA JavaScript error occurred in the main process
Uncaught Exception:
Error: listen EADDRINUSE: address already in use :::23564
at Server.setupListenHandle [as_listen2] (node:net:1740:16)
at listeninCluster (node:net:1788:12)
at Server.listen (node:net:1876:7) at Function.listen
X
(C:\Users------\AppData\Local\Programs\galaxy-widgets\resources\app.asar\node_m...:24)
at Object.
(C:\Users------\AppData\Local\Programs\galaxy-widgets\resources\app.asar\src\js\bat...5)
at Module._compile (node:internal/modules/cjs/loader:1271:14)
at Object..js (node:internal/modules/cjs/loader: 1326:10)
at Module.load (node:internal/modules/cjs/loader:1126:32)
at node:internal/modules/cjs/loader:967:12
at Function._load (node:electron/js2c/asar_bundle:2:13327)
OK"
To Reproduce
For me i just opened it idk
Screenshots

Desktop:
Windows 11 Home 23H2
Additional context
Add any other context about the problem here.

This appears if the application is already started and one starts it aain.


[ACL] Update Compute Library to v23.05.1

Update Compute Library to follow latest developments in 23.05.1.
Changelog: https://arm-software.github.io/ComputeLibrary/v23.05.1
cc @ashutosh-arm @Liam-Sturge @neildhickey for reviews

Thanks @leandron