Refactor Dropdown

We need to re-visit Dropdown to add missing features and update its API so it matches Salt's API goals.
### Tasks
- [ ] #2359
- [ ] #2460
- [ ] #2363
- [ ] #2361
- [ ] #2357
- [ ] Update Dropdown to use declarative children

Copied from #2607
20 Nov - snapshot release last week, waiting for MTK feedback, build needs reviewing

22 Nov - Adding some examples to the WIP Combobox and Dropdown and currently writing tests

24 Nov - Added type="button" but haven't worked on Dropdown much focus has been on ComboBox

1 Dec - Still haven't been able to work on this

18 Dec - Call tomorrow with stakeholders, @bhoppers2008 to take a quick look today and re-sync tomorrow

09/01 - Working on finishing addressing the feedback, have been raising separate PRs to fix issues and had to update floating-ui to solve some issues.

Closed by #3131


[Suggest] English or Chinese support

First of all, I will add English. Chinese is probably not. I don't think online translators can accurately translate some words.

Thank you very much, because I can't read Russian, I can read English.

The first time you can use https://www.deepl.com/translator . I will add support for at least English within 2-3 weeks. At the moment I am improving DDOS.

English is support!


azure devops plugin behind proxy never ends

Describe the bug
I am using the dependency check plugin of azure devops on a self hosted server. The connection to internet is using a proxy server.
The command executed is:
C:\Windows\system32\cmd.exe /D /S /C "D:\azure\ag1_work_tool_owasp\dependency-check\bin\dependency-check.bat --project MyProject --out D:\azure\ag1_work\42\a\dependency-check --scan D:\azure\ag1_work\42\a\myFolder --format ALL --proxyserver myproxy.local.net --proxyport 8080 --nvdApiKey ***"
Version of dependency-check used
The problem occurs using version 9.0.4
The Azure devops plugin version is 6.1.3
Log file
What I see in the Azure Devops console is:
C:\Windows\system32\cmd.exe /D /S /C "D:\azure\ag1_work_tool_owasp\dependency-check\bin\dependency-check.bat --version"
Dependency-Check Core version 9.0.4
C:\Windows\system32\cmd.exe /D /S /C "D:\azure\ag1_work_tool_owasp\dependency-check\bin\dependency-check.bat --project MyProject --out D:\azure\ag1_work\42\a\dependency-check --scan D:\azure\ag1_work\42\a\myFolder --format ALL --proxyserver myproxy.local.net --proxyport 8080 --nvdApiKey ***"
[WARN] NVD API request failures are occurring; retrying request for the 6 time
[INFO] Recoverable I/O exception (org.apache.hc.client5.http.HttpHostConnectException) caught when processing request to {s}->https://services.nvd.nist.gov:443
[WARN] NVD API request failures are occurring; retrying request for the 7 time
[INFO] Recoverable I/O exception (org.apache.hc.client5.http.HttpHostConnectException) caught when processing request to {s}->https://services.nvd.nist.gov:443
[WARN] NVD API request failures are occurring; retrying request for the 8 time
[INFO] Recoverable I/O exception (org.apache.hc.client5.http.HttpHostConnectException) caught when processing request to {s}->https://services.nvd.nist.gov:443
[WARN] NVD API request failures are occurring; retrying request for the 9 time
[INFO] Recoverable I/O exception (org.apache.hc.client5.http.HttpHostConnectException) caught when processing request to {s}->https://services.nvd.nist.gov:443
[WARN] NVD API request failures are occurring; retrying request for the 10 time
[INFO] Recoverable I/O exception (org.apache.hc.client5.http.HttpHostConnectException) caught when processing request to {s}->https://services.nvd.nist.gov:443
[WARN] NVD API request failures are occurring; retrying request for the 11 time
[INFO] Recoverable I/O exception (org.apache.hc.client5.http.HttpHostConnectException) caught when processing request to {s}->https://services.nvd.nist.gov:443
[INFO] Recoverable I/O exception (org.apache.hc.client5.http.HttpHostConnectException) caught when processing request to {s}->https://services.nvd.nist.gov:443
[INFO] Recoverable I/O exception (org.apache.hc.client5.http.HttpHostConnectException) caught when processing request to {s}->https://services.nvd.nist.gov:443
[INFO] Recoverable I/O exception (org.apache.hc.client5.http.HttpHostConnectException) caught when processing request to {s}->https://services.nvd.nist.gov:443
[WARN] NVD API request failures are occurring; retrying request for the 5 time
[INFO] Recoverable I/O exception (org.apache.hc.client5.http.HttpHostConnectException) caught when processing request to {s}->https://services.nvd.nist.gov:443
[WARN] NVD API request failures are occurring; retrying request for the 6 time
[INFO] Recoverable I/O exception (org.apache.hc.client5.http.HttpHostConnectException) caught when processing request to {s}->https://services.nvd.nist.gov:443
[WARN] NVD API request failures are occurring; retrying request for the 7 time
[INFO] Recoverable I/O exception (org.apache.hc.client5.http.HttpHostConnectException) caught when processing request to {s}->https://services.nvd.nist.gov:443
[WARN] NVD API request failures are occurring; retrying request for the 8 time
[INFO] Recoverable I/O exception (org.apache.hc.client5.http.HttpHostConnectException) caught when processing request to {s}->https://services.nvd.nist.gov:443
[WARN] NVD API request failures are occurring; retrying request for the 9 time
[INFO] Recoverable I/O exception (org.apache.hc.client5.http.HttpHostConnectException) caught when processing request to {s}->https://services.nvd.nist.gov:443
[WARN] NVD API request failures are occurring; retrying request for the 10 time

See the proxy configuration page: https://jeremylong.github.io/DependencyCheck/data/proxy.html
Unfortunately, we are in transition right now on how the proxy is configured.

I'm shocked - did you read the documentation I linked to that stated how you need to set the proxy? It is possible that setting the proxy just via the properties will work - but just the two args you've specified will fail. Please take a look at https://jeremylong.github.io/DependencyCheck/data/proxy.html

understood. I configured the plugin in this way:
inputs;
.....
.....
additionalArguments: '--proxyserver "myproxy.net" --proxyport "8080" --nvdApiKey "$(nvdApiKey)"'
env:
JAVA_TOOL_OPTIONS: "-Dhttps.proxyHost=myproxy.net -Dhttps.proxyPort=8080"
Thankyou

I've tried the suggested configuration, but still the Pipeline igmoring the Proxy setting,
I'm getting:
[ERROR] Could not connect to Central search. Analysis failed. java.io.IOException: Finally failed connecting to Central search. Giving up after 7 tries. 	at org.owasp.dependencycheck.analyzer.CentralAnalyzer.fetchMavenArtifacts(CentralAnalyzer.java:364) 	at org.owasp.dependencycheck.analyzer.CentralAnalyzer.analyzeDependency(CentralAnalyzer.java:229) 	at org.owasp.dependencycheck.analyzer.AbstractAnalyzer.analyze(AbstractAnalyzer.java:131) 	at org.owasp.dependencycheck.AnalysisTask.call(AnalysisTask.java:88) 	at org.owasp.dependencycheck.AnalysisTask.call(AnalysisTask.java:37) 	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) 	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) 	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) 	at java.base/java.lang.Thread.run(Thread.java:829) Caused by: java.net.SocketTimeoutException: connect timed out
am I missing something here?
- task: Bash@3
  inputs:
    env:
    JAVA_TOOL_OPTIONS: "-Dhttps.proxyHost=my-proxy -Dhttps.proxyPort=8080"
    targetType: 'inline'
    script: '/Owasp/dependency-check-9.0.5/bin/dependency-check.sh ...

Hi @RobSHK
not sure but I see two difference respect my task:

that definition of env: should be at the same level of inputs:
I addede even the arguments --proxyserver and --proxyport as arguments for the script

I am using the dependency check plugin in the azure marketplace (https://marketplace.visualstudio.com/items?itemName=dependency-check.dependencycheck&targetId=28c2a7af-7473-4d9f-8a07-00377c831903&utm_source=vstsproduct&utm_medium=ExtHubManageList)


TypeError: globalScope.addEventListener is not a function

Expected Behavior
Start the dev server without error
Current Behavior
After installing I tried running npm run dev and got the following error:
TypeError: globalScope.addEventListener is not a function
at Object. (/.../node_modules/@amplitude/plugin-page-view-tracking-browser/src/page-view-tracking.ts:78:21)
at step (/.../node_modules/@amplitude/plugin-page-view-tracking-browser/node_modules/tslib/tslib.js:195:27)
at Object.next (/.../node_modules/@amplitude/plugin-page-view-tracking-browser/node_modules/tslib/tslib.js:176:57)
at /.../node_modules/@amplitude/plugin-page-view-tracking-browser/node_modules/tslib/tslib.js:169:75
at new Promise ()
at Object.__awaiter (/.../node_modules/@amplitude/plugin-page-view-tracking-browser/node_modules/tslib/tslib.js:165:16)
at Object.setup (/.../node_modules/@amplitude/plugin-page-view-tracking-browser/src/page-view-tracking.ts:71:63)
at Timeline. (/.../node_modules/@amplitude/analytics-core/src/timeline.ts:28:23)
at step (/.../node_modules/@amplitude/analytics-core/node_modules/tslib/tslib.js:195:27)
at Object.next (/.../node_modules/@amplitude/analytics-core/node_modules/tslib/tslib.js:176:57)
Steps to Reproduce

npm install @amplitude/analytics-browser
npm run dev

Environment
My dependencies in package.json:
"@amplitude/analytics-browser": "^2.1.2",
"next": "12.1.6",
"nextjs-cors": "^2.1.2",
"ngrok": "^4.3.1",
"react": "^18.1.0",
"react-dom": "18.1.0",
"react-markdown": "^8.0.7",
"sharp": "^0.30.7",
"swr": "^1.3.0"

Hi @nabusman, for NextJS we usually recommend using the Amplitude Node SDK. This is because NextJS code may be run on both on the client (browser) and the server (SSR).
Hope this helps and let me know if it doesn't!

@Mercy811 I am also facing this issue. And I have some questions too.

Ideally, in the server-side case, tracking should be triggered on the client side. wdyt?
The plugin @amplitude/plugin-page-view-tracking-browser/ is the trigger for the issue. Would this plugin work with the Node SDK? And in general, does the NodeSDK work on the browser?

@Mercy811 what do you think of the following approach?

use the browser-sdk and send all analytics from the browser when the component mound (React.useEffect(,track []))
don't load ampli on the server-side (if (typeof window === "undefined") return;)

I don't think that the NodeSDK is a good fit for a NextJS app

Hi @glung, yes if you want to use @amplitude/plugin-page-view-tracking-browser plugin, you have to use the browser SDK because the Node SDK doesn't support that. But have to be carefully and make sure that the the browser SDK only runs on client side as it include platform specific code which can not run on server side. Here is an example Next.js app using browser SDK. Hope this is helpful.

@Mercy811 I followed your recommendation yesterday and switched from Browser SDK to Node SDK. The errors disappeared but Amplitude is no longer capturing events.
But now you have crossed it out.
What is the correct recommendation here? How do we install the SDK in a Next.js 13 project which has both server and client? It's attempting to load Browser SDK in server, which fails obviously, but it looks like Node SDK doesn't work as expected on the client.

I am also running into this issue. Could an option be added to completely disable this plugin (e.g. here)? The other default plugins are all wrapped in a check to see if they are needed before being loaded.

@Mercy811 Checking in on this, are there any plans to fix this issue on the Amplitude side? NextJS renders components on the server side and this crash happens during import, so even if we don't init Amplitude on the server side it still crashes. It seems like it totally breaks recent versions of Amplitude in NextJS without the workaround @ADTC shared. 😔

Hi @georgewitteman, thanks for your suggestion and choosing Amplitude. We've re-prioritize this and I will keep you updated.

Hi @georgewitteman<EMAIL_ADDRESS>supports to not install the page view plugin by either config.defaultTracking = false or config.defaultTracking.pageViews = false.


Tick boxes when specimen or label obscured

Some discussion has been going on with the collection managers and digitisers, but it looks like we should make a note of instances in the Digi App where the specimens and/or label will not be able to be imaged as they are obscured (has implications for reports to collection managers, status and for transcription). In Specify, I was thinking that if we add them as separate lines in the remarks table using controlled terminology, we should be able to run reports on them to send to collection managers to review. However, in the App it might be easiest to have tick boxes for these.

[ ] @chelseagraham and Matilde to confirm with Jens S regarding the use of "Specimen obscured" and "Label Obscured" and provide a link here to that ticket with examples of how we use it and why.
[ ] Decide where in UI boxes will go and what will look like
[ ] Hand over to DigiApp team to implement
[ ] Fedor to review the below steps and add additional/edit as needed
[ ] Create tick box in UI for "Specimen Obscured"
[ ] Create tick box in UI for "Label Obscured"
[ ] Create columns for each of the tick boxes in the specimen table in the db
[ ] These should then map to separate lines in the Remarks table in Specify and so need to update GREL script
[ ] Fully test
[ ] Demonstrate to Chelsea
[ ] Update user documentation
[ ] Update GREL documentation
[ ] Update App documentation

Sent email on 26/10/2023 to Fedor, Bhupjit and Jan asking whether it is theoretically possible to map notes from DigiApp to different lines in Remarks table in Specify

just linking a ticket on the Herbarium board from which I can draw information relevant to this ticket https://github.com/NHMDenmark/Herbarium-Sheets-workstation/issues/55

Current procedures (from @Gomismis document https://github.com/NHMDenmark/Herbarium-Sheets-workstation/issues/55) include

Flagging a specimen with visible labels and obscured or missing specimens as an MSO and scanning a data matrix with the controlled vocabulary "Specimen obscured" into the notes field. The number of barcode labels assigned corresponds to the number of visible labels. A single image is taken. (1.2, 1.6, 1.8, 4.1, 4.4)
Flagging a specimen with obscured labels and obscured specimens as an MSO and scanning a data matrix with the controlled vocabulary "Specimen obscured; Label obscured" into the notes field. A single barcode is assigned. A single image  is taken. (1.3)
Entering a specimen with layered specimens and all information on the bottom sheet as a single specimen (no radio buttons selected) and scanning a data matrix with the controlled vocabulary "Specimen obscured; Label obscured" into the notes field. A single barcode is assigned. A single image  is taken. (1.4, 1.7)
Entering a specimen with layered specimens and the same label information as a single specimen (no radio buttons selected) and scanning a data matrix with the controlled vocabulary "Specimen obscured" into the notes field. A single barcode is assigned. A single image  is taken. (1.5)
Entering a specimen with label obscured as a single specimen (no radio buttons selected) and scanning a data matrix with the controlled vocabulary "Label obscured" into the notes field. A single barcode is assigned. A single image  is taken. (1.7)
Entering photos and drawings with associated data and scanning a data matrix with the controlled vocabulary "Non-botanical object" into the notes field. The number of barcode labels assigned corresponds to the number of visible labels. A single image is taken. (4.2)

@bhsi-snm  @PipBrewer Are the requirements of this ticket ready for me to start implementing, or who has the ball on this issue?

@chelseagraham What is the status?

@chelseagraham Where on the UI are these two (?) tick boxes to be placed?

Placement specified here:
https://github.com/NHMDenmark/Mass-Digitizer/issues/439#issuecomment-1934131095

I just talked with @chelseagraham and I believed that these tickboxes are to be mapped to flag fields. However, looking through the ticket specification I can see these are to be mapped to collection object remarks (properties), so will do that instead.
Teased out the tasks that are relevant to me:

[ ] Create tick box in UI for "Specimen Obscured"
[ ] Create tick box in UI for "Label Obscured"
[ ] Create columns for each of the tick boxes in the specimen table in the db
[ ] These should then map to separate lines in the Remarks table in Specify

@chelseagraham Should I group these together as proposed and use the caption "Specimen Flags" as per below?

I think this works well. As Rebekka mentioned, they are not mutually exclusive, so the check boxes makes the most sense. :)

This ticket was closed prematurely. Steps were missed, there still need to be made changes to the GREL script and mapping to accommodate the mapping of "Specimen obscured" and "Label obscured" to the DaSSCo remarks table in Specify.

The GREL script has now been updated and pushed to github. It has been set up in a similar fashion to the notes field with a remark, source and date. The remark is "Specimen obscured" and "Label obscured" respectively, the source is "DaSSCo digitisation", and the date is the same as the cataloged date.
The mapping to the remarks table will be added with the next import.


feat!: Remove isSequenceOf and instead add Segment to DSP-API (DEV-3326)

Pull Request Checklist
Task Description/Number
Issue Number: DEV-3326
This PR:

Breaking: Removes isSequenceOf and hasSequenceBounds from the knora-base ontology
Adds Segment etc. as a replacement of sequences

The following will be done in follow-up PRs:

Add documentation on the new feature
Add to the persisted test data (so we have examples in make init-db-test)
write non-E2E tests

PR Type

[ ] build/chore: maintenance tasks (no production code change)
[ ] docs: documentation changes (no production code change)
[x] feat: represents new features
[ ] fix: represents bug fixes
[ ] perf: performance improvements
[ ] refactor: represents production code refactoring
[ ] test: adding or refactoring tests (no production code change)

Basic requirements for bug fixes and features

[ ] Tests for the changes have been added
[ ] Docs have been added / updated

Does this PR introduce a breaking change?

[x] Yes

Does this PR change client-test-data?

[ ] Yes

As discussed:
I understand that the removed properties of the ontology are not used in any of our projects and it is therefore safe to delete them from the ontology.
Nonetheless, I would appreciate if we are handling breaking changes in a more not-so-surprising way for our users. Until we have a process for handling this I am indifferent on whether or not we should remove them in this PR. Now I will shutup and write a proposal on a proper depreciation procedure for implementing breaking changes.

I didn't find anything suspicious. Should this PR bump the major api version?

Yes, that was the idea (because removing the sequence thing is a breaking change), I guess the exclamation mark should do the trick, right?

I didn't find anything suspicious. Should this PR bump the major api version?

Yes, that was the idea (because removing the sequence thing is a breaking change), I guess the exclamation mark should do the trick, right?

Indeed. I think there were some braking changes in between last major version update and now which didn't bump the number, therefor I needed to ask. Actually we should maybe analyze this maybe on the BE meeting.


Distance rates - Distance rate order in Rate page is random after clear cache and restart app

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Version Number: 9.0.45-2
Reproducible in staging?: Y
Reproducible in production?: Y
Email or phone of affected tester (no customers<EMAIL_ADDRESS>Issue reported by: Applause Internal Team
Action Performed:

Go to staging.new.expensify.com
Go to workspace settings > Distance rates
Add a few distance rates
Go to workspace chat.
Click + > Submit expense > Distance.
Enter waypoints > Next.
Click Rate on confirmation page.
Note that order of distance rates is A, B, C, D, E etc
Go to Troubleshoot > Clear cache and restart
Select Reset and refresh
Repeat Step 4 to 7.

Expected Result:
The order of distance rates in Rate page should remain A, B, C, D, E etc.
Actual Result:
The order of distance rates in Rate page is random after clear cache and restart app.
Workaround:
Unknown
Platforms:

[x] Android: Native
[x] Android: mWeb Chrome
[x] iOS: Native
[x] iOS: mWeb Safari
[x] MacOS: Chrome / Safari
[x] MacOS: Desktop

Screenshots/Videos
https://github.com/user-attachments/assets/3d377d29-d644-4d0a-beed-4f52aa0ef400

View all open jobs on GitHub

We think that this bug might be related to #wave-collect - Release 1

@CortneyOfstad  FYI I haven't added the External label as I wasn't 100% sure about this issue. Please take a look and add the label if you agree it's a bug and can be handled by external contributors

Proposal
Please re-state the problem that we are trying to solve in this issue.
Distance rate order in Rate page is random after clear cache and restart app
What is the root cause of that problem?
We are not sorting distanceRates here.
https://github.com/Expensify/App/blob/99f280b0edae75ff59614a1c5e98c47b39800aa5/src/pages/iou/request/step/IOURequestStepDistanceRate.tsx#L67
What changes do you think we should make in order to solve the problem?
Implement sorting after reading rates from ONYX here.
https://github.com/Expensify/App/blob/99f280b0edae75ff59614a1c5e98c47b39800aa5/src/pages/iou/request/step/IOURequestStepDistanceRate.tsx#L67
What alternative solutions did you explore? (Optional)
Reminder: Please use plain English, be brief and avoid jargon. Feel free to use images, charts or pseudo-code if necessary. Do not post large multi-line diffs or write walls of text. Do not create PRs unless you have been hired for this job.

Updated Proposal
Updated Proposal added alternate solution.

@cretadn22 thanks for your insights,
But I believe Proposals are about sharing ideas how a problem could be solved isn't it?
And I believe sorting is not any sort of complex problem where it's must to share pseudo code to make reviewer believe in your solution.
But anyways I will respect whatever thought  reviewer might have.

Proposal
Please re-state the problem that we are trying to solve in this issue.
The order of distance rates in Rate page is random after clear cache and restart app.
What is the root cause of that problem?

I notice that other proposals aren't clear why the display rate is different when we reset cache and restart

When we add new distances, these objects are added to Onyx in turn then this displays the current order that we added is A, B, C, D, E
https://github.com/Expensify/App/blob/6d8e739f7b1c286ae65565a6689b27b5983fe48a/src/pages/iou/request/step/IOURequestStepDistanceRate.tsx#L73
After we reset the cache and restarted, BE returned another order of rates object then it displayed in a different order
https://github.com/Expensify/App/blob/6d8e739f7b1c286ae65565a6689b27b5983fe48a/src/pages/iou/request/step/IOURequestStepDistanceRate.tsx#L73
What changes do you think we should make in order to solve the problem?
Sort the distance rates by rate value as we do here
const sections = Object.values(rates)
        .sort((rateA, rateB) => (rateA.rate ?? 0) - (rateB.rate ?? 0))
        .map((rate) => {

https://github.com/Expensify/App/blob/6d8e739f7b1c286ae65565a6689b27b5983fe48a/src/pages/iou/request/step/IOURequestStepDistanceRate.tsx#L73-L75
What alternative solutions did you explore? (Optional)
NA

@paultsimura we have a few proposals above for you to review when you have a chance — thanks!

I agree with @paultsimura, I don't how we're sorting the data is the most important part of the proposal.

Hey @trjExpensify! I am heading OoO this afternoon (10/15 to 10/23) so reassigning to keep this moving! The PR here has not been launched to staging just yet 👍
Thanks!

Sounds good!

I'm back from OoO — thanks for holding down the fort @trjExpensify!
Agree with your proposal @paultsimura!

Payment Summary
@paultsimura — paid $250 via Upwork
@ChavdaSachin — paid $250 via Upwork
Regression Test
https://github.com/Expensify/Expensify/issues/439059


Cannot get space before function paren in JS/TS "class"es

Operating System (+Version): Windows 10 build 1809
VS Code Version: 1.31.1 (user setup)
beautify Version: 1.4.9

Provide the settings you use:
.jsbeautifyrc
{
    "end_with_newline": false,
    "indent_size": 4,
    "indent_char": " ",
    "eol": "\n",
    "brace_style": "collapse,preserve-inline",
    "space_after_anon_function":true,
    "space_after_named_function": true,
    "comma_first": false,
    "break_chained_methods": false,
    "keep_array_indentation": true,
    "keep_function_indentation": true,
    "space_before_conditional": true,
    "max_preserve_newlines": 2
}

Action performed
Format typescript file with HookyQR.beautifyFile command - TypeScript configured to be handled by the JavaScript parser
import { Argument, ArgumentType, CommandoClient, CommandoMessage } from 'awesome-commando';

export default class StringArrayType extends ArgumentType {
    constructor (client: CommandoClient) {
        super(client, 'stringarray');
    }

    public validate (value: string, msg: CommandoMessage, arg: Argument) {
        if (/([\S ]*,[\S ]*)*/i.test(value) && value.split(',').filter(Boolean).length >= 1) {
            return true;
        }

        return 'This command requires a command any word or set of words comma separated. For example `one` or `one,two,three`';
    }

    public parse (value: string) {
        return value.split(',').filter(Boolean);
    }
}

Expected results
Expected spaces before the parens in constructor, validate and parse to remain
expected result
    constructor (client: CommandoClient) {
        super(client, 'stringarray');
    }

Actual results
Spaces before aforementioned functions are removed causing a stylistic (and TSLint) issue as I prefer there to be spaces.
actual output
    constructor(client: CommandoClient) {
        super(client, 'stringarray');
    }

Nope, it's not something js-beautify is capable of.

So... This cannot even be considered a future request?

You can log it over at https://github.com/beautify-web/js-beautify which is the code that does the actual work. We just wrap that functionality for VS Code.


gofabric8 clean

I am trying to remove all Fabric8 stuff -(apps, pods, services, etc etc).
I tried gofabric8 clean cluster and got:
* cluster.
* environ.
....```

I then `oc project fabric8` and get ```Now using project "fabric8" on server "https://apps.mydomain.com:8443".

Then I try gofabric8 clean system and get:
gofabric8 clean system
WARNING this is destructive and will remove ALL fabric8 apps, environments and configuration from cluster fabric8/apps-teammachine-us:8443/anton.  Continue? [y/N] y
Removing...
Cannot find parent namespace for: fabric8
Deleting pvc with label provider=fabric8
Error deleting PVC auth-db-postgresql-data
Error deleting PVC init-tenant-postgresql-data
Error deleting PVC keycloak-db-postgresql-data
Error deleting PVC wit-postgresql-data
Deleting projects with label provider=fabric8
Successfully cleaned up

Then I try:
gofabric8 clean apps
Cannot find parent namespace for: fabric8
WARNING this is destructive and will remove all of your apps!
for your tenant:  running in namespaces: -stage & -run

Continue [y/N]: y
Cleaning apps running in namespace: -stage
No resources found
No resources found
No resources found
No resources found

the clean system is deleting everything already, i don't believe you need to delete apps after!


Update sainlogic stable to 0.10.4

1.10.4 created 23.7.2023
8 user
1.10.5 created 23.7.2023
77 user

@phifogg
Are you sure that an update to 1.10.4 is desired?
1.10.5 has been created at same date and hence 1.10.5 has been tested latest sinde 23.7.

@phifogg
Are you sure that an update to 1.10.4 is desired?
1.10.5 has been created at same date and hence 1.10.5 has been tested at latest since 23.7.

AH.. good point. I'll update to 10.5 for stable. I didn't expect people to update that quickly :D


Crash at startup

I downloaded the latest Sponge cored file and put it in a fresh forge server install of the compatible version, and I get this
http://pastebin.com/5ZtXsGgF

This was a bug in build 625, update Sponge to latest


Magellan class does not initialize

I have been unable to initialize the Magellan class from pycromanager with recent versions of micromanager. Not certain on when it changed, but some time between 9-29-2023 and 10-30-2023. In the screen shots below, I have pycromanager version 0.29.9 and am running various versions of micromanager in demo configuration.
9-29-2023 micromanager version

10-30-2023 micromanager version

2-06-2024 micromanager version

Any update on this?  We're getting the same issue.

@nicost it appears you may have changed the API in https://github.com/micro-manager/micro-manager/pull/1832, which has broken Magellan access through pycro-manager
Could you look into this? Or if you don't have time and aren't using the changes you made, perhaps revert that PR?

Do you have the studio object available in Pycromanager?  If so, can you call
magellan - Magellan(studio)
?

@nicost, I appreciate your quick response. I attempted the following approach:
from pycromanager import Magellan, Studio

s = Studio()
magellan = Magellan(s)

but unfortunately, it didn't resolve the issue. I don't understand why--it may be related to the inheritance structure of the Studio and limitations of the Java python bridge.
While I understand the changes in micro-manager/micro-manager#1832 were necessary, they've resulted in a breaking API change that currently leaves Magellan unusable from Python. Considering your previous focus on avoiding such disruptions, I'm hopeful you see the need for a resolution. I lack both a comprehensive understanding of the alterations made in that PR and the equipment to conduct proper testing. Could you please investigate a solution?

I'll dive into this, but it may take some time.  First thought is to take
studio out of the constructor, create a function that adds it in, then test
everywhere it is used to make sure that it is not null.
On Mon, Apr 1, 2024 at 3:56 PM Henry Pinkard @.***>
wrote:

@nicost https://github.com/nicost, I appreciate your quick response. I
attempted the following approach:
from pycromanager import Magellan, Studio
s = Studio()magellan = Magellan(s)
but unfortunately, it didn't resolve the issue. I don't understand why--it
may be related to the inheritance structure of the Studio and limitations
of the Java python bridge.
While I understand the changes in micro-manager/micro-manager#1832
https://github.com/micro-manager/micro-manager/pull/1832 were
necessary, they've resulted in a breaking API change that currently leaves
Magellan unusable from Python. Considering your previous focus on avoiding
such disruptions, I'm hopeful you see the need for a resolution. I lack
both a comprehensive understanding of the alterations made in that PR and
the equipment to conduct proper testing. Could you please investigate a
solution?
—
Reply to this email directly, view it on GitHub
https://github.com/micro-manager/pycro-manager/issues/739#issuecomment-2030724947,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAQN2K257E7PSHAMBSKQD2LY3HQ2ZAVCNFSM6AAAAABC72NQJWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMZQG4ZDIOJUG4
.
You are receiving this because you were mentioned.Message ID:
@.***>

and sorry about the disruption, I clearly had forgotten about the use of
Micro-Magellan outside of the MMStudio eco-system.
On Tue, Apr 2, 2024 at 11:37 AM Nico Stuurman @.***>
wrote:

I'll dive into this, but it may take some time.  First thought is to take
studio out of the constructor, create a function that adds it in, then test
everywhere it is used to make sure that it is not null.
On Mon, Apr 1, 2024 at 3:56 PM Henry Pinkard @.***>
wrote:

@nicost https://github.com/nicost, I appreciate your quick response. I
attempted the following approach:
from pycromanager import Magellan, Studio
s = Studio()magellan = Magellan(s)
but unfortunately, it didn't resolve the issue. I don't understand
why--it may be related to the inheritance structure of the Studio and
limitations of the Java python bridge.
While I understand the changes in micro-manager/micro-manager#1832
https://github.com/micro-manager/micro-manager/pull/1832 were
necessary, they've resulted in a breaking API change that currently leaves
Magellan unusable from Python. Considering your previous focus on avoiding
such disruptions, I'm hopeful you see the need for a resolution. I lack
both a comprehensive understanding of the alterations made in that PR and
the equipment to conduct proper testing. Could you please investigate a
solution?
—
Reply to this email directly, view it on GitHub
https://github.com/micro-manager/pycro-manager/issues/739#issuecomment-2030724947,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAQN2K257E7PSHAMBSKQD2LY3HQ2ZAVCNFSM6AAAAABC72NQJWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMZQG4ZDIOJUG4
.
You are receiving this because you were mentioned.Message ID:
@.***>

No worries, thanks!

should be fixed in https://github.com/micro-manager/micro-manager/pull/1914

I'm running the 20240426 nightly build downloaded from the website and I am getting this error. did the issue return since fixing it?

This would be surprising. I suspect you may have an old version of a library hanging around. Can you try deleting your MM installation and performing a fresh new install of latest version?


Brackets in Macro Name Prevent It From Being Referenced By A Rule

Describe the bug
Falco allows a macro to be saved with a '[' character, but there does not appear to be a way to have the macro referenced by a rule even with escaping.
How to reproduce it
Save the below content to a file strange_characters.yaml:
---
- macro: "[][]]----Macro"
  condition: "container.id != host"
  append: false
- rule: "test"
  desc: "test"
  condition: '"[][]]----Macro"'
  output: "test"
  priority: "WARNING"
  tags: []
  source: "syscall"
  append: false

Then run "falco -V strange_characters.yaml"
Output is
Fri Jul 31 19:38:02 2020: Validating rules file(s):
Fri Jul 31 19:38:02 2020:    strange_characters.yaml
Compilation error when compiling ""[][]]----Macro"": 1: syntax error, unexpected '"[][]]----Macro"', expecting '(', 'Macro', 'FieldName', 'not'
---
- rule: "test"
  desc: "test"
  condition: '"[][]]----Macro"'
  output: "test"
  priority: "WARNING"
  tags: []
  source: "syscall"
  append: false
---
Fri Jul 31 19:38:02 2020: Runtime error: Compilation error when compiling ""[][]]----Macro"": 1: syntax error, unexpected '"[][]]----Macro"', expecting '(', 'Macro', 'FieldName', 'not'
---
- rule: "test"
  desc: "test"
  condition: '"[][]]----Macro"'
  output: "test"
  priority: "WARNING"
  tags: []
  source: "syscall"
  append: false
---. Exiting.

Expected behaviour
I would expect this to validate successfully as the macro has single quotes escaping the name. Alternatively if it is not possible to escape the special characters I would expect the macro to fail validation.
Environment

Falco version:
Falco version: 0.22.1
Driver version: a259b4bf49c3330d9ad6c3eed9eb1a31954259a6

System info:
system_info":{"machine":"x86_64","nodename":"vagrant-ubuntu-trusty-64","release":"3.13.0-163-generic","sysname":"Linux","version":"#213-Ubuntu SMP Thu Nov 15 02:19:07 UTC 2018"}

NAME="Ubuntu"
VERSION="14.04.6 LTS, Trusty Tahr"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 14.04.6 LTS"
VERSION_ID="14.04"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"

Linux vagrant-ubuntu-trusty-64 3.13.0-163-generic #213-Ubuntu SMP Thu Nov 15 02:19:07 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

/help

@leogr:
This request has been marked as needing help from a contributor.
Please ensure the request meets the requirements listed here.
If this request no longer meets these requirements, the label can be removed
by commenting with the /remove-help command.

In response to this:

/help

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
Provide feedback via https://github.com/falcosecurity/community.
/lifecycle stale

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
If this issue is safe to close now please do so with /close.
Provide feedback via https://github.com/falcosecurity/community.
/lifecycle rotten

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
If this issue is safe to close now please do so with /close.
Provide feedback via https://github.com/falcosecurity/community.
/lifecycle rotten

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.
Provide feedback via https://github.com/falcosecurity/community.
/close

@poiana: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.
Provide feedback via https://github.com/falcosecurity/community.
/close

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.


Blocking the Space key While Gaming

Microsoft PowerToys version
0.67.1
Installation method
Microsoft Store
Running as admin
Yes
Area(s) with issue?
General
Steps to reproduce
Whenever I am playing I have to quit PowerToys because otherwise it does not allow me to use the space bar at the same time as other keys.
-Start the game
-I enter my game
-The problem appears
✔ Expected Behavior
Normally I can be using the arrow keys or the WASD keys at the same time I use the Space key to jump, without releasing the previous keys.
❌ Actual Behavior
I have to release the other keys to be able to use the space key.
Other Software
Just Cause 2
Minecraft V. 1.19

I have same issue. In my case the game is Genshin Impact

Are you still experiencing this issue? /needinfo


Avoid exception checking nullability

Do not throw if we cannot determine the nullability of a dictionary. It's better to potentially be wrong, rather than be unable to generate a document at all.
Clean-up some code analysis suggestions.

Resolves #3070.
Resolves #2793 (as it's the last linked issue).

:warning: Please install the  to ensure uploads and comments are reliably processed by Codecov.
Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 91.60%. Comparing base (da07a53) to head (2a14a3e).

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #3119      +/-   ##
==========================================
+ Coverage   91.43%   91.60%   +0.16%
==========================================
  Files          76       76
  Lines        3119     3121       +2
  Branches      519      519
==========================================
+ Hits         2852     2859       +7
+ Misses        267      262       -5

Flag
Coverage Δ

Linux
?

Windows
?

macOS
91.60% <100.00%> (+0.16%)
:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Update using-the-windows-namespace-with-unity-apps-for-hololens.md

Adding a section to this page to describe usage of the new Microsoft.Windows.MixedReality.DotNetWinRT package which enables use of Windows APIs in Unity scripts without preprocessor conditionals, including in Play Mode.

@thetuvix please review these changes.

Docs Build status updates of commit 945c31d:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

mixed-reality-docs/using-the-windows-namespace-with-unity-apps-for-hololens.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

Docs Build status updates of commit d339644:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

mixed-reality-docs/using-the-windows-namespace-with-unity-apps-for-hololens.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

@jessemcculloch I think this is ready to merge; @thetuvix has signed off. Something seems wedged with the "OpenPublishing.Build (2 of 3) Preparing" check, but everything else has passed...

Docs Build status updates of commit 4ec0705:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

mixed-reality-docs/using-the-windows-namespace-with-unity-apps-for-hololens.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


how to use prompt-free feature? Is there any demo.py about prompt-free?

Hello, we are still refining our API now, thanks a lot for providing this issue, we will release this feature recently.

so now, prompt-free feature is not working?

@mengyao2022 @xyl3902596 We've already update our code to dds-cloudapi-sdk==0.3.3 and supported the prompt-free anything detection demo.


unable to send vite-pwa-plugin:dev-ready message to register service worker in dev mode!

Describe the bug
Hello!
Trying to work out a minimal PWA (Vue 3) in dev mode with this plugin and getting the following errors on my console:

I've also noticed that no service workers were generated. I'm using "vite-plugin-pwa": "^0.13.1" and "vite": "^3.1.0".
Also a production build fails with the following error:

Reproduction
https://github.com/alchemist-s/vite-pwa-repro
System Info
System:
    OS: macOS 10.15.7
    CPU: (6) x64 Intel(R) Core(TM) i5-8500 CPU @ 3.00GHz
    Memory: 114.02 MB / 8.00 GB
    Shell: 5.7.1 - /bin/zsh
  Binaries:
    Node: 14.17.2 - /usr/local/bin/node
    npm: 8.5.4 - /usr/local/bin/npm
    Watchman: 2<IP_ADDRESS> - /usr/local/bin/watchman

Used Package Manager
npm
Validations

[X] Follow our Code of Conduct
[X] Read the Contributing Guide.
[X] Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
[X] Check that this is a concrete bug. For Q&A, please open a GitHub Discussion instead.
[X] The provided reproduction is a minimal reproducible of the bug.

@alchemist-s rename vite.config.ts to vite.config.mts:

upps, maybe node version? I need to install a Node 14 version, can you try with a Node 16?

@userquin I did try with node 16 and got the same error. Let me try out your suggestion and I'll get back to you. Thanks for the quick turnaround!

@alchemist-s in my local working with Node 16.17.0 with ts and mts

@alchemist-s try removing the node_modules and reinstalling deps, it seems vite not being updated: we need Vite 3.1.0 that comes with the new sequential object hook, just check the last error!

also Rollup 2.79.0+, it seems something preventing the update

@alchemist-s try switching to pnpm instead using npm...

@userquin Okay so I had a global version of vite installed which was at 2.8.6...my bad. Its working perfect with Vite 3.1.0.
Sorry about that.


Enhancements - Remember last migrations folder & use migration list

While adding new migration, migrations folder is being selected. It would be nice if it remembers the last selection.

While updating the database, target migration input might be an option box. INstead of writing the exact name of the migration user may select from a list.

Hi, regarding the first point - it's a well-known feature request and we are working on it :)
Regarding the second one - I was thinking about it and it's a good point. Will create a task for that, thanks!

Please track the issues below according to your request:

#54
#130


userDataDir and options merging

Here the ability to choose a custom userDataDir for the chrome instance instead of a temporary one.
Also added a library 'deepmerge' to enable deep merging of the default options in browser.js
In my opinion this should be the default way to merge default objects, so i can override a single field of an object, instead of the whole object.
If you have any question, we can discuss without any problem!
:+1:

Thanks @mlessio !
I just merged it, and published (v3.1.0).
I've got one question about this anyway. What information can you get in the tmp directory of the browser? I always ignored it.
Cheers,
Lucho

Hi @LucianoGanga !
If you are talking about the userDataDir that CDP stores by default on /tmp/someRandomDir, it contains the chrome user profile. So inside you will find caches, stored preferences, cookies, localstorage, history, managed passwords and so on. If you want to simulate via headless or normal chrome a user session, i think you will have to use the same userDataDir across navigation sessions.
The actual behavior is just more likely an incognito window.
M

Thanks for the explanation @mlessio !!
You made it really clear :)


How To?

Essentially a "How To" - guide is missing.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 3604fe7e-46a3-6f5b-41e0-d3e69e01c8ae
Version Independent ID: 9415c8a5-177e-fe11-7198-3d97ed438dc8
Content: Speaker Recognition API
Content Source: articles/cognitive-services/Speaker-recognition/Home.md
Service: cognitive-services
GitHub Login: @dwlin
Microsoft Alias: dwlin

@rajivramanjani Thanks for the feedback! We are currently investigating and will update you shortly.

@rajivramanjani Thanks for the feedback! I have assigned the issue to the content author to investigate further and update the document as appropriate.
@cjgronlund Hi, this is a issue about document enhancement, and the user wants more details about Speaker Recognition API, especially for How to do that. Can you please check it and update the document as necessary? Thanks a lot!

#in-progress

@rajivramanjani Thank you for your feedback. I'm forwarding your recommendations to the Cognitive Services documentation team. In the meantime, please use the links at the bottom of the article pages to post questions to the forum.

Hi, I tried to create Speaker Verification Enrollment, I spoke "you can get in without your password" by microphone, but I got a error like this: "error": { "code": "BadRequest", "message": "InvalidPhrase" } Could you help me to fix it?
my code language: javaScript
API: rest api

There is no documentation, please prepare an example for the speaker identification

Can you please tell me that how to change the voice file to binary data?

Please reassign to @erhopf

#please-close
@rajivramanjani - Thanks for reach out. We have added your request for a new how to article to our backlog.


[Bash] Declare and assign separately

As proposed by static analysis tool:
https://github.com/koalaman/shellcheck/wiki/SC2155

Can one of the admins verify this patch?

Jenkins, test this please

Jenkins, test this please.

Jenkins, test this please.

Jenkins, test this please.


Question: (possible?) Option to boot into Xploder/Xplorer/other Cheat Device menu

Just out of curiosity
Do you see a way (when an Xploder/Xplorer is attached), to unlock the cdrom drive and then boot into the xploder/xplorer menu to set some cheats (just the possbility, neglect the fact that you would need to detect the attached xploder/xplorer or would need to select this option "somehow" (controller or something else).
I checked Martins documentation about the xploder/xplorer boot process, but I really don't understand it.
Just ask, because how else should people enjoy "blonde Lara" with tonyhax (https://patrickvogt.net/blond_lara_blond.jpg) :-).
In the end there are other ways to use cheats with cdrs (without a modchip e.g. Unirom), I really just ask out of curiosity.

I think if your hardware supports a physical cheat device you'd be much better using that with Unirom to unlock the drive and just skip the lengthy process of using tonyhax.


Update ClickBench benchmarks with DataFusion 43.0.0

Is your feature request related to a problem or challenge?
Like https://github.com/apache/datafusion/issues/11567
Requires

[x] https://github.com/apache/datafusion/issues/12470

Once DataFusion 43.0.0 is released, It would be great to update ClickBench https://benchmark.clickhouse.com/ with runs from the latest version. It looks like we are still reporting numbers for DataFusion 40 and there have been significant improvements since then. See for more details:

https://github.com/apache/datafusion/issues/12821

Describe the solution you'd like
Perhaps we can follow the model of  https://github.com/ClickHouse/ClickBench/pull/210 (thanks @pmcgleenon  )
We will also need to update DataFusion to apply the new binary_as_string option added by @goldmedal  in https://github.com/apache/datafusion/pull/12816. TLDR is that we need to update the create table statements to have the OPTIONS ('binary_as_string' 'true') clause
https://github.com/ClickHouse/ClickBench/blob/main/datafusion/create_partitioned.sql
CREATE EXTERNAL TABLE hits
STORED AS PARQUET
LOCATION 'partitioned'
OPTIONS ('binary_as_string' 'true');

Note this is the same as the DuckDB runner, as explained in https://github.com/apache/datafusion/issues/12788
Describe alternatives you've considered
No response
Additional context
No response

Hi @pmcgleenon
I just double checked and this should be good to go now -- the clickbench runner uses datafusion-cli rather than the python bindings (which haven't yet released a version 43.0.0)
Thank you so much

take

Hi @alamb  here are the initial results.
I've created a composite html file with results for DataFusion releases 33, 34, 36, 40 and 43 so we can compare across the previous releases.
clickbench.html.zip
Single results

Partitioned results

Before running the tests I've updated the SQL to include OPTIONS ('binary_as_string' 'true')
https://github.com/ClickHouse/ClickBench/commit/30fa096fbf47cbf48938848ff670dfabc7c11968
It looks like we've seen some significant gains across a number of queries.    Are these results in line with your expectations?
If so I can update the Clickbench to update it with the latest results.

It looks like we've seen some significant gains across a number of queries. Are these results in line with your expectations?

Yes, indeed, @korowa @Rachelint @jayzhan211 @XiangpengHao   and many others have been hard at work improving the speed. For an idea of what was done: https://github.com/apache/datafusion/issues/12821#issuecomment-2400801350
It is sad that we are not yet the fastest engine but interestingly it looks like many places where we are the farthest from the top are the really fast queries (that take 100s of ms)
Thank you again @pmcgleenon

YES! you are totally right @Dandandan
With some more finagling, I filtered for only c6a.metal 500gb gp2:

WOOHOO -- once this gets published it will be time to write a blog post !

After profiling, I believe we still can get obvious improvement after finishing #11943 and its related epic!

It is sad that we are not yet the fastest engine but interestingly it looks like many places where we are the farthest from the top are the really fast queries (that take 100s of ms)

That's an interesting observation.  Maybe we can look at benchmarking on c6a.metal at some point - it might highlight some more areas which could be improved and would show how DataFusion scales out to 192 vCPUs

I've created a PR on ClickBench to update the results https://github.com/ClickHouse/ClickBench/pull/251
Congratulations everyone on another amazing project milestone!

After profiling, I believe we still can get obvious improvement after finishing #11943 and its related epic!

Indeed. I also wanted to point out that item did not make it into 43.0.0 (it was merged shortly after we made the release candodate) so I expect the 44.0.0 release to be even better

https://github.com/apache/datafusion/pull/12996

The ClickBench PR has been merged, so the v43 results are available at https://benchmark.clickhouse.com/

Thanks again @pmcgleenon ❤

How is it compared to polars in performance in this benchmark?

I don't think Polars can run SQL so it can't run this benchmark

Polars is included in the benchmark (via in memory DataFrame API) and runs on the biggest machine.
It seems DataFusion is faster on quite some queries even though it reads parquet from disk and runs on a smaller machine.

See results

See results

This is quite interesting. It seems like Polars may also compute min/max/count/sum during load as some queries take 0 time (but there is 275s of load time).
Still 🎣 this would be an excellent optimization for anyone else working on

#13448

The polars test is in python so I don't think that is a fear comparison

The polars test is in python so I don't think that is a fair comparison

I don't think that's very relevant, as the execution is done natively (Polars is written in Rust). Likewise, DataFusion also would perform about the same in Python.
Running on the biggest machine / and / or reading everything in memory actually gives Polars an advantage, so I think it’s even more impressive to see the difference here.
I think it mainly shows the different optimizations done last ~2 years to make (tricky/expensive) aggregations fast.

I'm very pleased to see DataFusion achieving such results. However, I encountered some anomalies while trying to reproduce the benchmark, so I'd like to ask for some guidance.
Following the scripts in the ClickBench repository, I ran ClickBench on partitioned parquet files.
During the cold run phase, DataFusion was about 20% faster than ClickHouse, but in the hot run phase, DataFusion was about 20% slower than ClickHouse.
We used a machine with specifications similar to c6a.4xlarge, featuring 16 vCPUs, 32GB of memory, and an SSD with 2GB/s bandwidth. Additionally, we ran ClickBench on a machine with similar specifications but using HDD, and the results were consistent - DataFusion was slower than ClickHouse in the hot run phase.
This was quite unexpected, and I'd like to know if there might be some configuration/compilation parameters that could be causing this issue.
@alamb I would really appreciate any advice you could give when you have a moment.

@alamb I would really appreciate any advice you could give when you have a moment.

I think we would have to get some detailed profiling to really know for sure, but I suspect that ClickBench has non trivial caches (buffer caching, page caches, etc)
DataFusion, as a serverless engine, does not have any such caching (the only difference between cold/hot run is that on the hot run, data from disk will be in the Linux page cache (so may not do any actual IO)
It might also help to break down which queries showed the biggest discrepancy -- were they queries that already ran in 100ms (in which case caching , avoiding re-reading metadata might be a bigger part of processing)

@alamb I would really appreciate any advice you could give when you have a moment.

I think we would have to get some detailed profiling to really know for sure, but I suspect that ClickBench has non trivial caches (buffer caching, page caches, etc)
DataFusion, as a serverless engine, does not have any such caching (the only difference between cold/hot run is that on the hot run, data from disk will be in the Linux page cache (so may not do any actual IO)
It might also help to break down which queries showed the biggest discrepancy -- were they queries that already ran in 100ms (in which case caching , avoiding re-reading metadata might be a bigger part of processing)

For parquet files, ClickHouse uses local mode. In my understanding, in local mode, ClickHouse, like DataFusion, is a stateless query engine with only Linux page cache available. So I'm very surprised by these results. I will conduct more experiments to try to find out the reason.

@alamb I would really appreciate any advice you could give when you have a moment.

I think we would have to get some detailed profiling to really know for sure, but I suspect that ClickBench has non trivial caches (buffer caching, page caches, etc)
DataFusion, as a serverless engine, does not have any such caching (the only difference between cold/hot run is that on the hot run, data from disk will be in the Linux page cache (so may not do any actual IO)
It might also help to break down which queries showed the biggest discrepancy -- were they queries that already ran in 100ms (in which case caching , avoiding re-reading metadata might be a bigger part of processing)

After conducting more experiments, I made some unexpected discoveries:
In the public clickbench results, Clickhouse was using a version newer than 24.11, while our server had 24.1/24.3 installed. Therefore, I re-ran the benchmark using the latest version 24.12, and this time, the results were similar to those on the clickbench website - Datafusion was faster than Clickhouse in both cold run and hot run phases, and these results were consistently reproducible. This means that recent updates to Clickhouse have led to a decline in its query performance for parquet files. In the earlier versions, Clickhouse still had better performance during the hot run phase.
@alamb FYI

Thank you for the update @waruto210

@alamb I would really appreciate any advice you could give when you have a moment.

I think we would have to get some detailed profiling to really know for sure, but I suspect that ClickBench has non trivial caches (buffer caching, page caches, etc)
DataFusion, as a serverless engine, does not have any such caching (the only difference between cold/hot run is that on the hot run, data from disk will be in the Linux page cache (so may not do any actual IO)
It might also help to break down which queries showed the biggest discrepancy -- were they queries that already ran in 100ms (in which case caching , avoiding re-reading metadata might be a bigger part of processing)

After conducting more experiments, I made some unexpected discoveries:
In the public clickbench results, Clickhouse was using a version newer than 24.11, while our server had 24.1/24.3 installed. Therefore, I re-ran the benchmark using the latest version 24.12, and this time, the results were similar to those on the clickbench website - Datafusion was faster than Clickhouse in both cold run and hot run phases, and these results were consistently reproducible. This means that recent updates to Clickhouse have led to a decline in its query performance for parquet files. In the earlier versions, Clickhouse still had better performance during the hot run phase.
@alamb FYI

Do you know which queries are we still lag behind in the old version of clickhouse?

Do you know which queries are we still lag behind in the old version of clickhouse?
On our test machines, roughly the following queries are slower:

Q4: SELECT count(DISTINCT UserID) FROM hits LIMIT 20000
Q8: SELECT RegionID, count(DISTINCT UserID) AS u FROM hits GROUP BY RegionID ORDER BY u DESC LIMIT 10
Q13：SELECT SearchPhrase, count(DISTINCT UserID) AS u FROM hits WHERE SearchPhrase <> '' GROUP BY SearchPhrase ORDER BY u DESC LIMIT 10
Q29: SELECT sum(ResolutionWidth), sum(ResolutionWidth + 1), sum(ResolutionWidth + 2), sum(ResolutionWidth + 3), sum(ResolutionWidth + 4), sum(ResolutionWidth + 5), sum(ResolutionWidth + 6), sum(ResolutionWidth + 7), sum(ResolutionWidth + 8), sum(ResolutionWidth + 9), sum(ResolutionWidth + 10), sum(ResolutionWidth + 11), sum(ResolutionWidth + 12), sum(ResolutionWidth + 13), sum(ResolutionWidth + 14), sum(ResolutionWidth + 15), sum(ResolutionWidth + 16), sum(ResolutionWidth + 17), sum(ResolutionWidth + 18), sum(ResolutionWidth + 19), sum(ResolutionWidth + 20), sum(ResolutionWidth + 21), sum(ResolutionWidth + 22), sum(ResolutionWidth + 23), sum(ResolutionWidth + 24), sum(ResolutionWidth + 25), sum(ResolutionWidth + 26), sum(ResolutionWidth + 27), sum(ResolutionWidth + 28), sum(ResolutionWidth + 29), sum(ResolutionWidth + 30), sum(ResolutionWidth + 31), sum(ResolutionWidth + 32), sum(ResolutionWidth + 33), sum(ResolutionWidth + 34), sum(ResolutionWidth + 35), sum(ResolutionWidth + 36), sum(ResolutionWidth + 37), sum(ResolutionWidth + 38), sum(ResolutionWidth + 39), sum(ResolutionWidth + 40), sum(ResolutionWidth + 41), sum(ResolutionWidth + 42), sum(ResolutionWidth + 43), sum(ResolutionWidth + 44), sum(ResolutionWidth + 45), sum(ResolutionWidth + 46), sum(ResolutionWidth + 47), sum(ResolutionWidth + 48), sum(ResolutionWidth + 49), sum(ResolutionWidth + 50), sum(ResolutionWidth + 51), sum(ResolutionWidth + 52), sum(ResolutionWidth + 53), sum(ResolutionWidth + 54), sum(ResolutionWidth + 55), sum(ResolutionWidth + 56), sum(ResolutionWidth + 57), sum(ResolutionWidth + 58), sum(ResolutionWidth + 59), sum(ResolutionWidth + 60), sum(ResolutionWidth + 61), sum(ResolutionWidth + 62), sum(ResolutionWidth + 63), sum(ResolutionWidth + 64), sum(ResolutionWidth + 65), sum(ResolutionWidth + 66), sum(ResolutionWidth + 67), sum(ResolutionWidth + 68), sum(ResolutionWidth + 69), sum(ResolutionWidth + 70), sum(ResolutionWidth + 71), sum(ResolutionWidth + 72), sum(ResolutionWidth + 73), sum(ResolutionWidth + 74), sum(ResolutionWidth + 75), sum(ResolutionWidth + 76), sum(ResolutionWidth + 77), sum(ResolutionWidth + 78), sum(ResolutionWidth + 79), sum(ResolutionWidth + 80), sum(ResolutionWidth + 81), sum(ResolutionWidth + 82), sum(ResolutionWidth + 83), sum(ResolutionWidth + 84), sum(ResolutionWidth + 85), sum(ResolutionWidth + 86), sum(ResolutionWidth + 87), sum(ResolutionWidth + 88), sum(ResolutionWidth + 89) FROM hits LIMIT 20000
Q35：SELECT ClientIP, ClientIP - 1, ClientIP - 2, ClientIP - 3, count(*) AS c FROM hits GROUP BY ClientIP, ClientIP - 1, ClientIP - 2, ClientIP - 3 ORDER BY c DESC LIMIT 10


Show Full Text of Talents on character sheets

Per a suggestion by MadCow on the Modiphius forums:

Is there a way to make the generator kick out the talent’s full text (similar to an NPC/Pre-Gen charatcer)? This would be great, especially for new players that don’t know the talents well.

A version of this has been implemented. I'm still looking to do some refinement, though.

I now have two character sheets that include the full text: the "2 page" character sheet and the "landscape" character sheet.


More tweaks and fixes (about/ads/forgottenpassword/feed sections)

Another bunch of grammar and typo fixes, some new translations, more “frenchy”-looking tweaks.

Nah, in French typography you have unbreakable spaces before 'high' punctuation signs (so ;, :, !, ?, but not , or .). ;)

@lucsurr  , thanks again for the help in French. Really improves it.

@yianni3000 my pleasure! :)


serve status page using sub-domain

⚠ Please verify that this bug has NOT been raised before.

[X] I checked and didn't find similar issue

🛡 Security Policy

[X] I agree to have read this project Security Policy

📝 Describe your problem
How can i make the status page accesible through a sub-domain like: status.mydomain.com (no paths, just a direct sub-domain)
📝 Error Message(s) or Log
No response
🐻 Uptime-Kuma Version
this doesn't really matter
💻 Operating System and Arch
Raspian aka Debian
🌐 Browser
this doesn't really matter
🐋 Docker Version
No response
🟩 NodeJS Version
No response

See: https://github.com/louislam/uptime-kuma/wiki/Status-Page


Craft 3 support?

We are currently working on migrating a large project from craft 2 > 3. As the project uses the linkIt plugin I was wondering whether you were planning an update.
Thanks in advance and thank you for this great plugin.

Currently migrating, hoping to have a version ready by the end of the week.

This week? :P


Adding a License

I plan on adding the MIT license, given that all contributors agree (Just comment on this issue whether you agree or not)
@djmaze
@chukysoria
@plietar
@diogocp
@sashahilton00
@Alex131089
@JozoVilcek

:+1:

👍

:+1:

👍

Agreed

👌
El dom., 17 de julio de 2016 13:10, Sasha Hilton<EMAIL_ADDRESS>escribió:

Agreed
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/Fornoth/spotify-connect-web/issues/63#issuecomment-233176827,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABf4iFqphvBMGRLEpRTeN8JLp6aX2PvFks5qWg2jgaJpZM4JOIwe
.

Yep fine


(maint) Externalize strings in config.clj

Test PASSed

CLA signed by all contributors.

Test PASSed


[community] Eta Contributor Github organization

I think it would be pretty cool if there was an organization called "Eta Contributor" or something like that so contributors could have a "badge" on their Github profile. 😄
In my opinion this would be a nice incentive to attract contributors and also to advertise Eta, as quite a lot of people browse the Github profiles of people found on the internet.
What do you think?

@NickSeagull That sounds like a great plan! We already have eta-lang org - https://github.com/eta-lang. We can definitely start making use of it.
Feel free to create any repositories and let me know if you want access to it.


qt: Finetune RPCConsole

See commit messages.
Preview

Before
With this PR

pls drop a5b264d

pls drop a5b264d

Damn it happened again 😆 We really need to backport some stuff now to make building on catalina happy haha

utACK, images look good, will try to build and actually test later today

Assuming utACK above was meant to be an approval and not a regular comment, merging


added support for ordered hashtables

This fix allows the cmdlet New-MDTable to support ordered hashtables.
I've found out, that the cmdlet New-MDTable does not support ordered hashtables or [pscustomobject].
If i declare a [pscustomobject][ordered] with a defined order of properties, this order is ignored by the cmdlet and the column order of the table is randomly:
C:\> Import-Module MarkdownPS
C:\> $Object = [pscustomobject][ordered]@{
  Name = "ThisIsMyNameProperty"
  AnotherProperty = "This is another property"
  LastProperty = "This property should be the last column in the Markdown table"
}
C:\> New-MDTable -Object $Object
| AnotherProperty                                               | Name                                                          | LastProperty                                                  |
| ------------------------------------------------------------- | ------------------------------------------------------------- | ------------------------------------------------------------- |
| This is another property                                      | ThisIsMyNameProperty                                          | This property should be the last column in the Markdown table |

The expected result of the cmdlet should be:
| Name                                                          | AnotherProperty                                               | LastProperty                                                  |
| ------------------------------------------------------------- | ------------------------------------------------------------- | ------------------------------------------------------------- |
| ThisIsMyNameProperty                                          | This is another property                                      | This property should be the last column in the Markdown table |

I've always had an issue with this part. I find your fix surprizingly easy and small :) but I would like to run my use cases with you to better understand.
I always did something like this adjusted to your example to achieve the same effect
    $columns=[ordered]@{
        Name =$null
        AnotherProperty =$null
        Scheme=$null
        LastProperty =$null
    }

There is an example about this in the main readme file
#region Tables
$markdown+=New-MDHeader "Tables"
$markdown+=New-MDParagraph -Lines "Without aligned columns"
$markdown+=Get-Command -Module MarkdownPS |Select-Object Name,CommandType,Version | New-MDTable
$markdown+=New-MDParagraph -Lines "With aligned columns"
$markdown+=Get-Command -Module MarkdownPS | New-MDTable -Columns ([ordered]@{Name="left";CommandType="center";Version="right"})
#endregion

The drivers for the -Columns property were:

Override the order of columns upon rendering
Implicitly filter out columns which could be also be done by a Select-Object

Notice that the -Columns is expected to be an ordered hash but it is not defined in the parameter and I don't remember why. When I chose the -Columns parameter as a solution, I had noticed that there was no connection between how objects would render in the output directly or through Format-Table with the properties. This is because PowerShell offers this concept of formatters for types that are not directly related to the type definitions.
So my question is which part of the problem does your solution address. My understanding is that this is about rendering object produced by the New-Object cmdlet like in your example. When developing did you try this with other inputs, like the examples included in the readme and mentioned above?
Can we add this pattern also in the tests

To be honest, i've overlooked your example in the main readme file.
My solution is just a easy fix to not use the parameter -Columns explicitly. Additionally my expection is, if i provide a ordered hashtable to the cmdlet, that the cmdlet doesn't destroy the order of the ordered hashtable.

I've now added new tests to verify if New-MDTable does not distord the order of the properties.
Additionally i've found a bug in the test -Object is null.
This test will always fail on non-english systems, because the string is different.
The test now validates against the FullyQualifiedErrorId of the error message. This should be the same on all systems.

Looks ok. But it is very busy so I'll take some time before publishing. Sorry. Btw, if you have some ideas, please do share or create issues.

So I modified a bit the testing because it wasn't actually checking for the sequence. I added two randomized tests one for following the objects propery sequence and one to overwrite with specified columns. (In essence you can also render less column but a Select-Object can do that as well although I think this would mess up with the sequence.
Sorry for the delay I was away,.


Use official tei gaudi image and update tgi gaudi version

Description
Use official tei gaudi image and update tgi gaudi version
Issues
n/a.
Type of change
List the type of change like below. Please delete options that are not relevant.

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds new functionality)
[ ] Breaking change (fix or feature that would break existing design and interface)
[x] Others (enhancement, documentation, validation, etc.)

Dependencies
None
Tests
Local test

Please fix the code structure.


visualising embedding labels in Tensorboard

When I visualise the encoder source_embedding parameters in TensorBoard and enable the labels mode, I see the numbers of the embedding running from 0 to the source vocabulary size. I would like to see the vocabulary entry for each number instead, as in https://www.tensorflow.org/get_started/embedding_viz. How can I set this up?

I am also looking for that feature. It would be very helpful

This is possible by clicking the "Load Data" button in the upper left. Then you need to provide your vocab file for either the source or target vocab.
You will need to append your vocab file with a line for UNK, SEQUENCE_START, and SEQUENCE_END so that the length of your embeddings matches the size of your vocabulary


[5.7] Add the ability to add arbitrary attributes to a NavigatorIndex

Cherry-pick of https://github.com/apple/swift-docc/pull/258
This stores a handle to the attributes in the index, the tree, and the
individual nodes. Attributes are stored in a [String: Any] and the client is
expected to provided extensions with computed properties as needed.
Additionaly this commits refactors the API for reading a NavigatorIndex from an
on-disk LMDB database. The prveious API used a designated initializer to do
this. However, this initializer was complex as it opened database connections
etc...  Using a static function enables clients to be more resilient to API
changes, by letting them define protocols that specify the new API and providing
a no-op default implementation that delegates to old API. This is not currently
possible with the initializer base API as the initializer would need to be
marked required to ensure subclasses of NavigatorIndex implement this. This will
be made possible in the future as clients transition to the static function
based API.
rdar://93503303

[X] Added tests
[X] Ran the ./bin/test script and it succeeded
[X] Updated documentation if necessary

@swift-ci please test


Reject invalid collab close offer

Proposal: implement a simple if check which does not automatically accept a collab close offer. In a later phase we will then implement the reject flow.

This is unfortunately not reasonably feasible without changing rust-dlc. And if we need to change rust-dlc already we should do it directly with this ticket https://github.com/get10101/10101/issues/2019


[FEATURE] Navigate to Respective Organization Repositories on Click

Is your feature request related to a problem? Please describe.
I often find it frustrating when I browse the organizations listed on OSS.gg. Currently, clicking on an organization doesn't direct me to its respective repository, which requires extra steps to find what I’m looking for. This can be time-consuming and leads to a less efficient experience.
Describe the solution you'd like
I would like to implement a feature where clicking on a listed organization directly navigates the user to the respective organization's repository page. This would involve adding click event listeners to the organization items, ensuring that each click triggers a redirect to the appropriate URL. This change would streamline navigation and improve user experience on the platform.
Describe alternatives you've considered
No response
Additional context
No response

/assign

/assign

/assign

/assign


Enhancement try to download commit specific source code when git submodule failed

Proposed changes
Issue Number: close #xxx

Further comments
If this is a relatively large or complex change, kick off the discussion at<EMAIL_ADDRESS>by explaining why you chose the solution you did and what alternatives you considered, etc...

run buildall

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.8 seconds
stream load tsv:          534 seconds loaded<PHONE_NUMBER>9 Bytes, about 133 MB/s
stream load json:         20 seconds loaded<PHONE_NUMBER> Bytes, about 112 MB/s
stream load orc:          65 seconds loaded<PHONE_NUMBER> Bytes, about 16 MB/s
stream load parquet:          30 seconds loaded 861443392 Bytes, about 27 MB/s
insert into select:          28.8 seconds inserted 10000000 Rows, about 347K ops/s
storage size:<PHONE_NUMBER>0 Bytes


All strapi nodes data remain the same on /__refresh

I'm new into Gatsby and Strapi ecosystem. First of all, thank you for creating this plugin, since Strapi v4 has been out for a while, and there are compatibility issue for plugins including gatsby-source-strapi, your plugin indeed help a lot and the data is coming great, the only issue is when Strapi triggering a webhook on localhost:8000/__refresh, all the Strapi nodes data in Graphql aren't getting updated.

Above is a webhook settings for refreshing Gatsby without doing yarn develop

Meanwhile when using<EMAIL_ADDRESS>after triggering a webhook, it does updated all the Strapi nodes data.

Is there any specific config to make that happen when using this plugin?
Thanks

Ah I see, I can repro locally, looking into it!

Ok I have fixed it in v4.1.0, forgot that apollo client by default caches all queries. Have also added support for using the api token if you're into that ;)

Ah thank you! It's working as expected.

Ok I have fixed it in v4.1.0, forgot that apollo client by default caches all queries. Have also added support for using the api token if you're into that ;)


Making a voice call

so yeah :), i hope there is call feature in Rhymen/go-whatsapp <3

https://en.softonic.com/articles/whatsapp-web-calls hope, this will be released

Not implemented in web.whatsapp.com yet, so it is not possible at the moment


Don't work for Windows 10 (require_once problem?)

Hello !
First of all, sorry for my english but I'm french..
I have a problem with Beautify for Windows, there is an error while executing in command line.
Description
The results of beautification are not what I expect.
Steps to Reproduce

Add code to Atom editor
Run command Atom Beautify: Beautify Editor
This beautified code does not look right!

Debug
Here is a link to the debug.md Gist: https://gist.github.com/anonymous/8ff73f92e825ec1e46968972785375e1
And the error from the CMD
Warning: Uncaught ErrorException: require_once(C:\Users\Antoine\Documents\PHP-CS-Fixer-1.12/vendor/autoload.php): failed to open stream: No such file or directory in C:\Users\Antoine\Documents\PHP-CS-Fixer-1.12\php-cs-fixer:38
Stack trace:
#0 C:\Users\Antoine\Documents\PHP-CS-Fixer-1.12\php-cs-fixer(38): {closure}(2, 'require_once(C:...', 'C:\\Users\\Antoin...', 38, Array)
#1 C:\Users\Antoine\Documents\PHP-CS-Fixer-1.12\php-cs-fixer(38): require_once()
#2 {main}
  thrown in C:\Users\Antoine\Documents\PHP-CS-Fixer-1.12\php-cs-fixer on line 38

Fatal error: main(): Failed opening required 'C:\Users\Antoine\Documents\PHP-CS-Fixer-1.12/vendor/autoload.php' (include_path='.;C:\php\pear') in C:\Users\Antoine\Documents\PHP-CS-Fixer-1.12\php-cs-fixer on line 38

2016-10-05T13:39:26.249Z - verbose: [beautifiers\beautifier.coffee] spawn result 255
Warning: Uncaught ErrorException: require_once(C:\Users\Antoine\Documents\PHP-CS-Fixer-1.12/vendor/autoload.php): failed to open stream: No such file or directory in C:\Users\Antoine\Documents\PHP-CS-Fixer-1.12\php-cs-fixer:38
Stack trace:
#0 C:\Users\Antoine\Documents\PHP-CS-Fixer-1.12\php-cs-fixer(38): {closure}(2, 'require_once(C:...', 'C:\\Users\\Antoin...', 38, Array)
#1 C:\Users\Antoine\Documents\PHP-CS-Fixer-1.12\php-cs-fixer(38): require_once()
#2 {main}
  thrown in C:\Users\Antoine\Documents\PHP-CS-Fixer-1.12\php-cs-fixer on line 38

Fatal error: main(): Failed opening required 'C:\Users\Antoine\Documents\PHP-CS-Fixer-1.12/vendor/autoload.php' (include_path='.;C:\php\pear') in C:\Users\Antoine\Documents\PHP-CS-Fixer-1.12\php-cs-fixer on line 38

Checklist
I have:

[X] Tried uninstalling and reinstalling Atom Beautify to ensure it installed properly
[X] Reloaded (or restarted) Atom to ensure it is not a caching issue
[X] Searched through existing Atom Beautify Issues at https://github.com/Glavin001/atom-beautify/issues
so I know this is not a duplicate issue
[X] Filled out the Input, Expected, and Actual sections above or have edited/removed them in a way that fully describes the issue.
[X] Generated debugging information and added link for debug.md Gist to this issue

Would you be able to test your PHP-CS-Fixer installation manually from your command-line? See https://github.com/FriendsOfPHP/PHP-CS-Fixer#usage
For example, Atom-Beautify used the following command generated from https://github.com/Glavin001/atom-beautify/blob/master/src/beautifiers/php-cs-fixer.coffee#L37-L44
C:\wamp64\bin\php\php7.0.4\php.EXE C:\Users\Antoine\Documents\PHP-CS-Fixer-1.12\php-cs-fixer fix FILEPATH

It may be an issue with php.EXE PATH/TO/php-cs-fixer part.


action to post a message to Kafka

The Kafka/Message Hub package should include an action that allows the caller to post a message to some topic in some Kafka or Message Hub instance.

@glikson Had an interesting suggestion to try to reuse an existing/initialized producer if possible. This may be as simple as declaring the producer in the global namespace and then first checking if this producer exists and has the correct options set. In the case that the action is run in a warm container, this could reduce the overhead of creating/initializing a new producer.

readme update?

@csantanapr, README update was in the PR.

I see action documented in the README but I didn't see the file in this PR https://github.com/openwhisk/openwhisk-package-kafka/pull/127/files
maybe it was in different PR

@csantanapr, https://github.com/openwhisk/openwhisk-package-kafka/pull/131/files


Possible support for Java UI in the future?

Hello budy, how are you?
Do you think Java UI support would be possible in the future?
If not, do you know of any alternatives to deal with this problem?
I need to fill a textbox in a java application, but LibreAutomate cannot capture the element, only the window.
Thanks!!!

Need to enable Java Access Bridge in Options -> OS -> Java. Then restart LibreAutomate and that Java app.

lol
Dude, I didn't know about this feature.
This software is perfect!!!
One more thing... is it possible to use this function in C# (Visual Studio)?
I use LibreAutomate through Nuget.

https://www.libreautomate.com/api/Au.elm.html (click link "Known problematic applications").
See also
https://github.com/qgindi/LibreAutomate/blob/3eee17d7a587fcb3be9ba4ceb322d89e74fc26fb/Au.Editor/Tools/Delm.cs#L1659C2-L1659C2


run: expose prometheus metrics

Allow containerized gadgets to expose Prometheus metrics.
This PR makes the feature work and enable the configuration file to select the gadget. Hopefully the user interface will be improved later on.
How to use
Prepare my-prometheus-config.yaml as follows:
metrics_name: guide
metrics:
  - name: dns_traffic
    type: counter
    category: ""
    gadget: run
    gadgetArgs:
      - albantest.azurecr.io/trace-dns:v3
    labels:
      - namespace
      - pod
      - container

$ sudo -E ig prometheus --config @my-prometheus-config.yaml
INFO[0000] Experimental features enabled
INFO[0000] Running. Press Ctrl + C to finish
INFO[0000] Publishing metrics...

Testing done
Tested in this branch:
https://github.com/alban/inspektor-gadget/tree/alban_asg2023
TODO
I am not sure if this should be done in this PR or later on:

[ ] Exposing the gadget name via category="", gadget=run & gadgetArgs is awkward. The configuration file should be reworked.

I'd like us to take a moment to think about all the eventual export types. We are using the OTel library for the Prometheus export and we should assume that we'll support the other formats/destinations that the OTel library provides. My current understanding is that we will support the following...

Prometheus
OTel traces
Logs
gRPC API

Probably more but I'd think those cover a large part of what would be useful.
So, for me it's important that we make sure the metadata supports a way to define the various exports. It could be as easy as.
...
export:
  -type: prometheus
     [details here]
  -type: logs
     [details here]
...

I agree with Chris, I think we should hold this PR a bit until we handle https://github.com/inspektor-gadget/inspektor-gadget/issues/1926

I propose to close this one as we need to think better the design of this feature.


Hash Probe performance issue

We collect some performance data from Gluten + Velox and it shows hash probe is the hotspot in some SHJ or BHJ operations and may have some place to enhance the performance.
Below are some details from the individual queries running with Gluten with Velox backend.

22 Queries perf report(sort by Symbol)
Top 10 hot functions
7.02 percent for snappy::RawUncompress
6.84 percent for LZ4 compress fast extState fastReset
4.08 percent for facebook::velox::exec::HashTable::joinNormalizedKeyProbe
3.97 percent for facebook::velox::exec::HashTable::insertForJoin
3.62 percent for LZ4_compress_fast_extState_fastReset
3.32 percent for facebook::velox::exec::HashTable::groupNormalizedKeyProbe
2.45 percent for LZ4_decompress_safe
1.63 percent for facebook::velox::exec::HashTable::listJoinResults
1.61 percent for facebook::velox::dwio::common::fixedWidthScan<long, false, false, facebook::velox::common::IsNotNull, …>
1.51 percent for LZ4_decompress_safe

query5 stage86 perf report (sort by Symbol):
Stage86 is a hotspot spend 22 second with one SHJ InnerJoin, compared to 3rd party vendor's P-engine 13 second
Hash Build Time in Gluten is 11.13 second compared to 3rd party vendor's P-engine 13.23 second
Hash Probe Time in Gluten is 128.4 second compared to 3rd party vendor's P-engine 49.25 second
Top 10 hot functions
15.94 percent for facebook::velox::exec::HashTable::joinNormalizedKeyProbe
10.50 percent for LZ4_decompress_safe
5.32 percent for __cpuidle_text_start
4.64 percent for LZ4_compress_fast_extState_fastReset
1.70 percent for facebook::velox::exec::VectorHasher::lookupValueId
1.58 percent for facebook::velox::exec::HashTable::insertForJoin
1.27 percent for facebook::velox::exec::HashTable::listJoinResultsNoDuplicates
1.26 percent for copy_user_enhanced_fast_string
1.21 percent for 0x000000000018b643
1.02 percent for __lock_tet_start

query10 stage277 perf report (sort by Symbol):
Stage277 is a hotspot spend 13 sec with one SHJ InnerJoin and one BHJ, but most of the time spend is in SHJ, compared to 3rd party vendor's P-engine 6sec
Hash Build in Gluten is 12.51second in totaltime compared to 3rd party vendor's P-engine 10.61second in totaltime.
Hash Probe in Gluten is 51.9second in totaltime compared to 3rd party vendor's P-engine 29.58second in totaltime.
Top10 hot functions
5.98 percent for snappy::RawUncompress
5.06 percent for LZ4_compress_fast_extState_fastReset
4.49 percent for facebook::velox::exec::HashTable::joinNormalizedKeyProbe
2.93 percent for LZ4_decompress_safe
2.74 percent for LZ4_compress_fast_continue
2.51 percent for folly::hash::SpookyHashV2::ShortEnd
2.19 percent for facebook::velox::exec::RowContainer::extractString
2.07 percent for folly::hash::SpookyHashV2::ShortMix
2.00 percent for __cpuidle_text_start
1.99 percent for folly::hash::SpookyHashV2::Rot64

query17 stage416 perf report (sort by Symbol):
Stage416 is a hotspot spend 35 sec with one BHJ and one SHJ innerJoin, most of the time are spend in one Agg operator before BHJ and one Hash Probe in BHJ, and compared to 3rd party vendor's P-engine 31sec.
HashAgg in Gluten is 366second in totaltime compared to 3rd party vendor's P-engine 347sec in totaltime.
BHJ in Gluten spend 53.35second in totaltime for hashprobe and 101.75sec in totaltime for stream input compared to 3rd party vendor's P-engine 22.78sec in totaltime for BHJ.
top 10 hot functions
19.41 percent for facebook::velox::exec::HashTable::groupNormalizedKeyProbe
12.18 percent for facebook::velox::aggregate::AverageAggregate::addIntermediateResults(…)
4.39 percent for LZ4_decompress_safe
4.07 percent for facebook::velox::exec::Aggregate::clearNull
3.15 percent for facebook::velox::exec::VectorHasher::lookupValueId
2.10 percent for __cpuidle_text_start
1.92 percent for LZ4_compress_fast_extState_fastReset
1.62 percent for 0x0000000000863c9f
1.55 percent for facebook::velox::exec::HashTable::storeRowPointer
1.38 percent for 0x0000000000863ca3

CC: @frankobe @majetideepak @oerling

We
So, we'd have to double probe speed. This can be done in a couple of ways, depending on the case.

If the join has no payload and a single key, we can use the IN filter hash table instead. This could match some of the use cases.

Does this match the joins in question?
Could we have a microbenchmark repro? We'd have to have the same data distributions and table sizes. This may also depend on load factor.  Would have to try maybe different things if the first trick does not apply.
We'll match the competition in any case.

Add more information for 3 queries highlight in the above,
query5 stage86 detail information listed in below:
SHJ with o_orderkey, l_orderkey|Inner|BuildLeft
LeftTable with
total 45,515,350 rows
Key: o_orderkey in LongType
Payload: c_nationkey in LongType
RightTable with
total 1,200,018,434 rows
Key: l_orderkey in LongType
Payload: l_suppkey in LongType, l_extendedprice in DoubleType, l_discount in DoubleType
query10 stage277 detail information listed in below:
SHJ with o_orderkey, l_orderkey|Inner|BuildLeft
LeftTable with total 11,464,157 rows
Key: o_orderkey in LongType
Payload: c_custkey in LongType, c_name in StringType, c_address in StringType, c_nationkey in LongType, c_phone in StringType, c_acctbal in DoubleType, c_comment in StringType
RightTable with total 296,117,231 rows
Key: l_orderkey in LongType
Payload: l_extendedprice in DoubleType, l_discount in DoubleType
query17 stage416 detail information listed in below:
BHJ with l_partkey, p_partkey|Inner|BuildRight
LeftTable with total 1,200,018,434 rows
Key: l_partkey in LongType
Payload: l_quantity in DoubleType, l_extendedprice in DoubleType
RightTable with total 39,899 rows
Key: p_partkey in LongType
Payload: n/a

Update the latest hotspot in TPC-H SF200 testing with Gluten + Velox running on Azure cloud,
There are three queries q7, q18, and q21 listed in below, which is still slower than P-engine:
query7:
The hotspot are
stage 10 with one SHJ run in 16sec.
stage 15 with one SHJ run in 25sec.
Below perf report covers both stages,
11.83%  LZ4_compress_fast_extState_fastReset
8.58%  facebook::velox::exec::HashTable::joinNormalizedKeyProbe
6.27%  facebook::velox::exec::RowContainer::extractColumn(char const* const*, int, facebook::velox::exec::RowColumn, int, std::s
5.56%  LZ4_decompress_safe
5.43%  facebook::velox::exec::HashTable::insertForJoin
2.98%  facebook::velox::exec::RowContainer::extractValuesWithNulls<false, long>
2.93%  LZ4_compress_fast_continue
2.50%  gluten::shuffle::Splitter::SplitBinaryType
2.01%  facebook::velox::FlatVector::copyValuesAndNulls(facebook::velox::BaseVector const*, facebook::velox::SelectivityV
1.95%  facebook::velox::exec::HashTable::joinProbe
1.83%  gluten::shuffle::Splitter::SplitFixedWidthValueBuffer
q7 stage 10 and 15 perf report screenshot:

query18:
The hotspot are
stage 55 with one HashAgg + one SHJ run in 23sec.
stage 67 with one HashAgg + one SHJ + one BHJ + two HashAgg run in 26sec.
Below perf report covers both stages,
34.38%  facebook::velox::exec::HashTable::groupProbe
11.18%  facebook::velox::exec::ProbeState::fullProbe<(facebook::velox::exec::ProbeState::Operation)1, facebook::velox::exec::Hash
8.61%  LZ4_decompress_safe
5.99%  facebook::velox::exec::HashTable::insertForGroupBy
3.03%  facebook::velox::exec::VectorHasher::lookupValueId
2.85%  facebook::velox::exec::Aggregate::clearNull
2.40%  facebook::velox::exec::RowContainer::newRow
1.24%  facebook::velox::exec::HashTable::storeRowPointer
q18 stage 55 and 67 perf report screenshot:

query21:
The hotspot is
stage77 with two SHJ run in 110sec.
Below perf report covers this stage only,
21.19%  facebook::velox::exec::HashTable::insertForJoin
18.74%  facebook::velox::exec::HashTable::listJoinResults
15.59%  facebook::velox::exec::HashTable::joinNormalizedKeyProbe
5.64%  facebook::velox::exec::RowContainer::extractValuesWithNulls<false, long>
4.47%  facebook::velox::exec::HashTable::pushNext
3.91%  facebook::velox::exec::SimpleFunctionAdapter<facebook::velox::core::UDFHolder<facebook::velox::functions::EqFunction<face
2.98%  facebook::velox::exec::HashProbe::evalFilter
2.41%  LZ4_decompress_safe
1.50%  facebook::velox::exec::RowContainer::store
1.06%  facebook::velox::exec::RowContainer::setProbedFlag
q21 stage 77 perf report screenshot:

Hello, may I know what "P-engine" represents that you mentioned? Is it Photon Engine?

any progress for this task ?

If the join has no payload and a single key, we can use the IN filter hash table instead. This could match some of the use cases.

Does this match the joins in question? Could we have a microbenchmark repro? We'd have to have the same data distributions and table sizes. This may also depend on load factor. Would have to try maybe different things if the first trick does not apply.

@weiting-chen Would you reply to these questions from @oerling?

@weiting-chen  you mentioned this problem has been optimized by partners,  Is there any progress

Please check: PR6406.


[asio] update to 1.19.2

Library name: asio
New version number: 1.19.2
Other information that may be useful (release notes, etc...)
FYI:
https://github.com/chriskohlhoff/asio/issues/873
https://github.com/chriskohlhoff/asio/commit/2d3a4c9fcdeeea48dc6974389b4c5167da9e34f7 (included in 1.19.2)
See also PR https://github.com/chriskohlhoff/asio/pull/892 (Open)

Thats not the latest stable release. I would prefer to not use the latest development version