feature support struct_element function

Proposed changes
This commit support a function allows return a field column in struct column.
Since the function can return any type, this commit also supports ANY_STRUCT_TYPE
and ANY_ELEMENT_TYPE.
example:
mysql> select struct_element(named_struct('f1', 1, 'f2', 'a'), 'f2');
+--------------------------------------------------------+
| struct_element(named_struct('f1', 1, 'f2', 'a'), 'f2') |
+--------------------------------------------------------+
| a                                                      |
+--------------------------------------------------------+
1 row in set (0.03 sec)

mysql> select struct_element(named_struct('f1', 1, 'f2', 'a'), 1);
+-----------------------------------------------------+
| struct_element(named_struct('f1', 1, 'f2', 'a'), 1) |
+-----------------------------------------------------+
|                                                   1 |
+-----------------------------------------------------+
1 row in set (0.02 sec)

mysql> select struct_col, struct_element(struct_col, 'f1') from test_struct;
+-------------------------------------------------+-------------------------------------+
| struct_col                                      | struct_element(`struct_col `, 'f1') |
+-------------------------------------------------+-------------------------------------+
| {1, 2, 3, 4, 5}                                 |                                   1 |
| {1, 1000, 10000000,<PHONE_NUMBER>00,<PHONE_NUMBER>00} |                                   1 |
| {5, 4, 3, 2, 1}                                 |                                   5 |
| NULL                                            |                                NULL |
| {1, NULL, 3, NULL, 5}                           |                                   1 |
+-------------------------------------------------+-------------------------------------+
9 rows in set (0.01 sec)

Problem summary
Describe your changes.
Checklist(Required)

[ ] Does it affect the original behavior
[x] Has unit tests been added
[x] Has document been added or modified
[ ] Does it need to update dependencies
[ ] Is this PR support rollback (If NO, please explain WHY)

Further comments
If this is a relatively large or complex change, kick off the discussion at<EMAIL_ADDRESS>by explaining why you chose the solution you did and what alternatives you considered, etc...

run buildall

run buildall

run buildall

run buildall

run buildall

run buildall

run buildall

run buildall

run buildall

run buildall

run buildall

run buildall

run buildall

run beut

It seems that struct_element will read the whole struct and discard other none related columns，will we implement a function that only read the target column.This could save lots of scan io.

It seems that struct_element will read the whole struct and discard other none related columns，will we implement a function that only read the target column.This could save lots of scan io.

That's a very good idea! How to scan target field is important. But I think what we need is not only a function.
What we should do is that we prune sub fields in struct/map type while generating query plan in Fe.
And then pass the pruned fields into column_reader.new_iterator() function in be, so that we read only the target field in Block, which will be passed to function.
This is very important when we support nested complex type.


feat : Refactored partasplanned and  SingleLevelBoMAsPlanned maven module to support multi versions

Description

Refactored/Updated PartAsPlanned and SingleLevelBoMAsPlanned maven module to support multi version.

Pre-review checks
Please ensure to do as many of the following checks as possible, before asking for committer review:

[x] DEPENDENCIES are up-to-date. Dash license tool. Committers can open IP issues for restricted libs.
[x] Copyright and license header are present on all affected files

@almadigabor  Please review and approve the PR.


Add some communication between client and server

We must discuss the scope of multiplayer before merging this in. Lets call tomorrow.

Get a discord lol, my phone is dead

I am working on a basic model for multiplayer, i will push it to a new branch, you can then add to that once it is made final by the end of this week. After i upload a basic skeleton for networking you too will fin it easier to understand. I don't fully understand your code so i will not merge it into master right now as it will not integrate with the backend rewrite i'm doing.

@farhansolodev I have added to the Start-Backend branch, You can begin work on multiplayer using my code as a starting point. I have made sure to comment every line.


Is it possible to use multiple CPU cores?

Since the official docs recommend placing Vibora without a proxy or load balancer, I'm curious how Vibora is meant to be used in a multi-core machine to best utilize available resources?

I was wondering this as well. I have been playing with Nginx Unit + a Python WSGI script, but using Vibora would be so much easier ...

Vibora is multi-processed by default which means it'll make use of all your CPU cores. You can configure how many workers do you want in the "app.run(workers=10)" method but beware that the default is a pretty good one.

Out of curiosity, how does the main process communicate with its workers? Sockets? Pipes?

how can I run in production a vibora Script then?

Just use the app.run()

For example, I have my script.py then in my ubuntu y Type
python3.6 script.py &
then I can close my ssh session and my script will live forever?

Maybe you have to use nohup or just something that runs it in background.
My suggestion is to use docker.

Do you have an example how can I run with docker :)

there are so many examples in the web...
https://runnable.com/docker/python/dockerize-your-python-application


generate ".flow" files in lib

Closes #290 LiveURL: https://smartfaq-290-generate-flow-files.surge.sh

does it work with frontend? that was why we removed it previously

aaah okok I'll check


Add "sync now" and options for sync table when "never" selected as a timeframe

It would be helpful to not have to select a timeframe or "never" for auto-syncing tables , and then be able to manually sync when needed as is the case with sync tables that have a timeframe selected:

There has been no activity on this issue for more several months. We are closing it. If you think this still needs to be addressed please open a new issue.


Update Chrome Extension Sample Link

The sample link has been updated to reference the
new url path that the chrome web store enforces.
Note that thhe actual extension is the same leveraging
the ID of: ehflheedljfngcklcnigfonmllgkadbb

Hi guys

Yikes!  @oshalygin I have no idea how this got dropped for so long.  Thank you for taking the time to contribute and really sorry you had a bad experience.


Changed installer operator checks to cluster operator checks

Which issue this PR addresses:
Fixes https://dev.azure.com/msazure/AzureRedHatOpenShift/_workitems/edit/7519638/
What this PR does / why we need it:
This PR changes the way we check if operators are ready after cluster creation. With 4.4 there's a unified way to check if a cluster operator is available.
Test plan for issue:
E2E tests will verify if the installation progresses. The logic of checking if operators are available hasn't changed, only the method.
Is there any documentation that needs to be updated for this PR?
No docs changes, code got a bit simpler in conditions.go.

/azp run e2e

/azp run e2e


Dispose KieSession after rules execution when using ruleFlowGroup

Closes https://github.com/apache/incubator-kie-issues/issues/1491

PR job #2 was: UNSTABLE
Possible explanation: This should be test failures

Reproducer
build-chain build full_downstream  -f 'https://raw.githubusercontent.com/${AUTHOR:apache}/incubator-kie-kogito-pipelines/${BRANCH:main}/.ci/buildchain-config-pr-cdb.yaml' -o 'bc' -p apache/incubator-kie-kogito-runtimes -u https://github.com/apache/incubator-kie-kogito-runtimes/pull/3678 --skipParallelCheckout
NOTE: To install the build-chain tool, please refer to https://github.com/kiegroup/github-action-build-chain#local-execution

Please look here: https://ci-builds.apache.org/job/KIE/job/kogito/job/main/job/pullrequest_jobs/job/kogito-runtimes-pr/job/PR-3678/2/display/redirect
Test results:

PASSED: 3319
FAILED: 5

Those are the test failures:

org.jbpm.bpmn2.IntermediateEventTest.testEventBasedSplit2
expected: 2 but was: 1

org.kie.kogito.integrationtests.quarkus.reactive.ProcessSvgAddonIT.testGetSvgProcessInstances
1 expectation failed.Response body doesn't match expectation.Expected: Expected namespace uri 'http://www.w3.org/2000/svg' but was 'null' - comparing  at /svg[1]/defs[1] to  at /svg[1]/defs[1]:  Actual:   EndStartProcessSecond Line    Approval    First Line Approval

org.kie.kogito.integrationtests.quarkus.ProcessSvgAddonIT.testGetSvgProcessInstances
1 expectation failed.Response body doesn't match expectation.Expected: Expected namespace uri 'http://www.w3.org/2000/svg' but was 'null' - comparing  at /svg[1]/defs[1] to  at /svg[1]/defs[1]:  Actual:   EndStartProcessSecond Line    Approval    First Line Approval

org.kie.kogito.svg.service.SpringBootProcessSvgServiceTest.annotateExecutedPathTest
Expecting value to be false but was true

org.kie.kogito.integrationtests.springboot.ProcessSvgAddonTest.testGetSvgProcessInstances
1 expectation failed.Response body doesn't match expectation.Expected: Expected namespace uri 'http://www.w3.org/2000/svg' but was 'null' - comparing  at /svg[1]/defs[1] to  at /svg[1]/defs[1]:  Actual:   EndStartProcessSecond Line    Approval    First Line Approval


Ensure defaultChecked is implemented correctly everywhere

This should be the case at the moment. Radio Buttons and Checkboxes are built on top of the default React API.
If you encounter a component which doesn't behave like this feel free to open an issue.


Added coverage for BooleanPrimaryExpression

Codecov Report

Merging #28 (215e0c4) into master (3546df4) will increase coverage by 0.16%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master      #28      +/-   ##
==========================================
+ Coverage   91.31%   91.48%   +0.16%
==========================================
  Files         551      551
  Lines       14275    14275
  Branches     1904     1904
==========================================
+ Hits        13035    13059      +24
+ Misses        994      971      -23
+ Partials      246      245       -1

Impacted Files
Coverage Δ

src/mysql-grammar-2/compiled-grammar.generated.ts
100.00% <ø> (ø)

...grammar-2/expression/boolean-primary-expression.ts
100.00% <ø> (ø)

src/mysql-grammar-2/expression/predicate.ts
100.00% <ø> (ø)

src/scanner-2/scan-all.ts
81.99% <0.00%> (+11.37%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3546df4...215e0c4. Read the comment docs.


Document how to set up sandbox orgs

/cc https://github.com/18F/cg-product/issues/82

Is there anything involved here? Doesn't the sandbox bot take care of it?

Not much. It's probably worth documenting how the sandbox bot is configured at least.


Permission error on google drive archive with pretrained models

Hi,
I get a permission error when I try to download the pretrained models from the linked google drive archive.
Since I'm from Italy, I cannot use the Baidu service.
Thanks!

yeahme too, i tried to download the dataset and i couldn't.

Sorry for inconvenience. My google drive storage is full and I will try migrate it to other net drive such as onedrive.


[Feature]: when will support torch2.4 use it out of box?

🚀 The feature, motivation and pitch
[Feature]: when will support torch2.4 use it out of box?
Alternatives
[Feature]: when will support torch2.4 use it out of box?
Additional context
No response

We are working on update soon. Currently we require torch 2.4 wheels to be released for our xformers, flashinfer, and flash-attn dependencies. I believe flashinfer is the only hold-up at the moment.

We are working on update soon. Currently we require torch 2.4 wheels to be released for our xformers, flashinfer, and flash-attn dependencies. I believe flashinfer is the only hold-up at the moment.

Flashinfer already supports torch 2.4 check the latest 0.1.3 release with torch 2.4.
https://github.com/flashinfer-ai/flashinfer/releases

thank u for the catching up


Dark theme not being saved. Reverts to Light on page reload.

Select dark theme in Settings
Reload page and theme reverts to light.

This is likely the same issue I had where settings weren't being stored in localStorage, due to me setting chrome to block 3rd party cookies, which also affects extensions.
I resolved this by adding vue-devtools to the allow list on the chrome cookies setting page (chrome://settings/content/cookies).
chrome-extension://{vue-devtools extension id}
Hopefully that helps anyone else in the same boat.

I'm not sure I'm entering this correctly. This is what I have which isn't working.

I'm not sure I'm entering this correctly. This is what I have which isn't working.

you need to use the extension id (found on extensions settings page - chrome://extensions/). {vue-devtools extension id} is just a placeholder for example.
this is what mine looks like - chrome-extension://nhdogjmejiglipccpnnnanhbledajbpd/

Thanks, that sorted it.


+# to Level of all Intelligence Skill Gems not calculated

New influence mod +1 to Level of all Intelligence Skill Gems is not calculated in POB
(needed to change it to all spell skill gems) probably All Dex and all STR are also incorrect.
Hunter mods
82 | +1 to Level of all Chaos Skill Gems | Amulets
82 | +1 to Level of all Strength Skill Gems | Amulets
82 | +1 to Level of all Dexterity Skill Gems | Amulets
82 | +1 to Level of all Intelligence Skill Gems | Amulets

Can confirm and test in ver 1.4.157
82 | Prefix | +1 to Level of all Strength Skill Gems | amulet_basiliskdefault
82 | Prefix | +1 to Level of all Dexterity Skill Gems | amulet_basiliskdefault
82 | Prefix | +1 to Level of all Intelligence Skill Gems | amulet_basiliskdefault
82 | Prefix | +1 to Level of all Lightning Skill Gems | amulet_crusaderdefault
82 | Prefix | +1 to Level of all Fire Skill Gems | amulet_adjudicatordefault
82 | Prefix | +1 to Level of all Physical Skill Gems | amulet_adjudicatordefault
82 | Prefix | +1 to Level of all Cold Skill Gems | amulet_eyriedefault
82 | Prefix | +1 to Level of all Chaos Skill Gems | amulet_basiliskdefault


Make vxlan port configurable (#341)

networkservicemesh/sdk-vpp commit message:
Make vxlan port configurable
networkservicemesh/sdk commit message:
Make vxlan port configurable
Signed-off-by: Zoltan Lugossy<EMAIL_ADDRESS>

https://github.com/networkservicemesh/sdk-vpp/pull/394/files is merged.
@zolug Could you please update sdk-vpp via go get to main version?


Jodit uploader

HI. I have been using Jodit for a while and I like it.  Thank you for sharing!
I have been trying to get the uploader to work following the tutorial on https://xdsoft.net/jodit/v.2/doc/tutorial-uploader-settings.html but with no result.
The first error I get is "Uncaught TypeError: this.o.filesVariableName is not a function".  How can I fix that?
Also I am not clear on which path to give to upload.php on laravel, as well as where the files should be uploaded.
Has someone managed to get this work, including file manager to browse through files and folders on server?
I would greatly appreciate some help, ideally a working sample showing all these features.
Anyone?

I have been playing with Alexusmai filemanager and tinyMCE 5... quite easy to set up and it is working fine.  However I still prefer Jodit editor for its layout and functionality.
How can I integrate AFM in the same way in Jodit as I did in tinyMCE?
window.onload = function () {
tinymce.init({
file_picker_callback (callback, value, meta) {
let x = window.innerWidth || document.documentElement.clientWidth || document.getElementsByTagName('body')[0].clientWidth
let y = window.innerHeight|| document.documentElement.clientHeight|| document.getElementsByTagName('body')[0].clientHeight
tinymce.activeEditor.windowManager.openUrl({
url : '/file-manager/tinymce5',
title : 'Laravel File manager',
width : x * 0.8,
height : y * 0.8,
onMessage: (api, message) => {
callback(message.content, { text: message.text })
}
})
}
...


[DO NOT MERGE] Update go_GBK_Flink_Batch_10b.txt

Please add a meaningful description for your change here

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[ ] Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
[ ] Update CHANGES.md with noteworthy changes.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.
To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md
GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.

when the load is small, it seems working fine: https://github.com/apache/beam/actions/runs/11983626872/job/33413411736


scipy.fft.fht - documentation issue

Describe your issue.
In your published documentation for scipy.fft.fht (https://docs.scipy.org/doc/scipy/reference/generated/scipy.fft.fht.html) and scipy.fft.ifht, the formulas you report for the forward and inverse Hankel transforms appear incorrect. For the forward transform, I believe the integral kernel should be written as "a(r) J_u(kr) r dr". For the inverse transform, I believe the integral kernel should be written as "A(k) J_u(kr) k dk". See, for example, the link here:
https://mathworld.wolfram.com/HankelTransform.html
I believe this is just a typo in the documentation but wanted to alert you of it. I also wanted to confirm that the algorithm you developed was based around the correct formula.
Reproducing Code Example
issue does not concern code

Error message
issue does not concern code

SciPy/NumPy/Python version and system information
issue does not concern code

@ntessore mind taking a look?

I get this question so often, it might be best to put a note in the documentation:
The definition as written is not a typo; the DHT and IDHT are commonly defined using $k \, dr$ and $r \, dk$, respectively (see for example Hamilton's excellent reference). The DHT makes clever use of the convolution theorem for a function of argument $r$, and a kernel of argument $kr = \exp(\ln k + \ln r)$. Defining the integrals using $k \, dr$ means that, after the logarithmic change of variables, $k \, dr = kr \, d{\ln r}$ and we obtain a factor of $kr$ that is absorbed into the kernel. Conversely, using $r \, dr = r^2 \, d{\log r}$ would yield a factor of $r^2$ that is absorbed into the function, not the kernel.
Using the definition as it is, one only needs to multiply inputs by $r$ and divide outputs by $k$ to recover the textbook definition of the Hankel transform.

thanks for the explanation @ntessore ! much appreciated
@smcguire88 would you like to suggest a note to add to the documentation which would have helped you understand this?

Thank you for your quick reply. I have no suggestion for a note but, should you wish to add something, I think what you wrote here is quite clear and could be included to avoid any misunderstanding.

Hi I am interested in contributing to the documentation,
I am trying to set up the project and build the doc using dev.py
but i am getting this error

Do i need a 32 bit version of python to build the documentation part or did i do anything wrong ?
I need help

Hi @srinivaspavan9, could you let us know what steps you have taken so far? In particular, what version of Python do you have? Here are our instructions for building SciPy from source.
Also, rather than replying on here, it would probably be better to work through this on our Slack space, which you can find at https://scipy.org/community/ (feel free to ping me on there). If that doesn't work for you, please open a new issue here so that anyone else subscribed to this issue doesn't get the noise.


Error when bundling the npm distro with Vite.

When trying to import the @parse/react library using the npm distro and bundling with vite, the import fails with the following error:
inherits.js:9 Uncaught TypeError: Super expression must either be null or a function
    at _inherits (inherits.js:9:11)
    at LiveQueryClient.js:274:18
    at node_modules/parse/lib/browser/LiveQueryClient.js (LiveQueryClient.js:759:1)
    at __require2 (chunk-W7STUX7H.js?v=aa49bf2b:18:50)
    at node_modules/parse/lib/browser/ParseLiveQuery.js (ParseLiveQuery.js:25:47)
    at __require2 (chunk-W7STUX7H.js?v=aa49bf2b:18:50)
    at node_modules/parse/lib/browser/Parse.js (Parse.js:339:19)
    at __require2 (chunk-W7STUX7H.js?v=aa49bf2b:18:50)
    at node_modules/parse/index.js (index.js:1:18)
    at __require2 (chunk-W7STUX7H.js?v=aa49bf2b:18:50)

This is another example of this issue: https://github.com/parse-community/Parse-SDK-JS/issues/1362 from the Parse-sdk-js repo, and the error stems from the same line of code. The following .jsx file will reproduce the issue.
import * as React from 'react';

import { initializeParse } from '@parse/react';

function App() {
  return (
      <h1> Hello</h1>
  );
}

export default App;

Same for me


feat(aya-metrics + aya-metrics-ebpf): Add metrics crates to emit metrics from eBPF

This is just an idea I had which I am wondering if it would bring any value to this project and its users.
I am using aya in an XDP program which doesn't really interact with user space - I am mangling packets in XDP driver mode and sending them straight back out using the XDP_TX action. This does however mean that it is difficult to know what the XDP program is doing.
The aya-log and aya-log-ebpf are great for logging - but it could be really cool to do the same thing with metrics.
The proposal is quite simple - just like how aya-log-ebpf provides the logging macros and aya-log connects it into the commonly used log crate - the idea is that aya-metrics-ebpf provides the metric macros and aya-metrics connects it to the commonly used metrics crate.

I really like the idea, I would love to see such macros!
That said, I'm not sure if that should be a project under Aya umbrella (at least for now). Logging is an essential thing for programming, therefore we keep aya-log even in the monorepo. But for metrics, there is no hard dependency on Aya during development - metrics macros can be easily implemented in completely separate crates.
I would just implement the idea as a new project. That would also show a strong advantage of Aya - ability to use external crates even in eBPF programs.

Great! getting aya-bpf into crates.io would really help with that :)
https://github.com/aya-rs/aya/issues/464
If I can successfully qualify aya for my use case I'll try and get aya-metrics + aya-metrics-ebpf out over the next couple of months or so.

Since we agreed that metrics could start in a separate crate, I'm closing this issue and will keep #464 open for the aya-bpf release.


spicetify-cli: 2.25.1 -> 2.25.2

Description of changes
https://github.com/spicetify/spicetify-cli/releases/tag/v2.25.2
Diff: https://github.com/spicetify/spicetify-cli/compare/v2.25.1...v2.25.2
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[x] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[x] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
23.11 Release Notes (or backporting 23.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

@ofborg eval


Add github like checks tab to the pull requests page

Feature Description
now that the actions are implemented and checks are triggered  on pull requests.
Is it possible to add checks tab to the pull request tab like this:

Screenshots
No response

You can assign to me.


chore(main): release 2.4.1

:robot: I have created a release beep boop
2.4.1 (2024-01-02)
Bug Fixes

EMS-2492: Quote - Policy type form validation (#1602) (80a785a)
manifest: updated manifest version (d5644ae)
release-please: specified default target branch (#1625) (89a7aac)
test: date issue (b2ee757)
tests: Mock months generated as 0 (#1623) (978a58b)

This PR was generated with Release Please. See documentation.

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

:exclamation: No coverage uploaded for pull request base (main@89a7aac). Click here to learn what that means.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1630   +/-   ##
=======================================
  Coverage        ?   97.38%
=======================================
  Files           ?      534
  Lines           ?     8294
  Branches        ?      908
=======================================
  Hits            ?     8077
  Misses          ?      215
  Partials        ?        2

Flag
Coverage Δ

api-test
91.57% <0.00%> (?)

unit-test
98.48% <0.00%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

:robot: Release is at https://github.com/UK-Export-Finance/exip/releases/tag/v2.4.1 :sunflower:


Has all the code been updated to python3 ?

Yes, code itself and required dependencies have also been modified.

Yes, code itself and required dependencies have also been modified.

good job, thanks bro.


testing: cft test run without --verbose

The --verbose flag is causing sensitive terraform outputs to be displayed in plaintext as part of test runs. This is an initial experiment to see if simply disabling verbose hides sensitive information without hindering troubleshooting.

This is hiding all terraform stdout, severely limiting troubleshooting insight into Cloud Build logs. Not a workable solution.


How can extractVueStyles output multiple CSS files？

It can't at the moment. You can inline it into the HTML or extract to a single file.

@JeffreyWay ok, thanks


DomainNet Experimental Settings

Hi John,
Fantastic work you have done!
I see you add an additional dataset experiment on this repo, that's the DomainNet which is not included in the paper.
May I have the Experimental Settings for the DomainNet? For example the training epoch, batch size, pseudo-labeling threshold and etc.

We use batch size 32, learning rate 3e-3 and train the model for 15 epoch. You can try other hyper-parameters.


How can I help?

Hi,
I'm Mark. I'm following the welcome wiki which told me to create an issue asking how I can help. I've done very little JavaScript coding, but I've done extensive coding in Java, C++, and Android, so I could probably pick up small- to medium-sized tasks and muddle through.
How can I help?
Thanks,
Mark

Hi Mark, just look through the issues and pick one youd like to work on. Fork the code-for-nashville/inclucivics repository and make your changes to your local fork. Then when you're ready, make a pull request by merging your fork to the master branch of this project.

Hi @mdelders!
Thanks so much for doing the welcome wiki and finding this project!
Our highest priority with this project is actually data analysis now that we have severals years of data, so we're not doing a lot of work on the current UI, but here are a subset of the tickets that are good to get started: https://github.com/code-for-nashville/inclucivics/labels/good first issue.
Please let me know if there's more you need or are interested in! You can find me @combinatorist on the C4N slack as well.


Simplify cli arguments for benchmark definition

There are too many cli arguments to offer different functionalities, deploy docker standard, deploy rkt standard container, deploy default shell unit, use a benchmark definition to deploy custom docker/rkt or unit file.
We have to make easy to play with this tool.

ping @htr @denderello. I know I removed --raw-instructions option which I liked. But I don't find how to keep it without making more complex the benchmarking definition.

I only used the raw-instructions. I like the simplicity of scp-ing a binary once, run it with different instructions, get useful output to stderr/out and use it again

but you could achieve the same with different configuration files, wouldn't you? I favour to avoid a possibility of 10 flags that are maybe conflicting with each other.

sure, but that's much less practical then having a script doing things like
for intervals in 10 100 500 100
do
  ssh core@remote ./fleemmer -addr=<IP_ADDRESS>:54541 -v=12 -dump-html-tar -instructions="(start 1000 ${interval}ms) (sleep 10s)  (stop-all) (sleep 10s)" > results-${interval}.tar
done

easy scriptability was one of the biggest reasons I designed it to work via args and stdout, no files needed.

@htr I agree with you, but we now have more options. Before we have 5 flags to run by default beacon units. If you want to deploy custom docker apps you need to specify additional info. What am I trying to find is a balance ? So maybe having two different files one for the app definition if required and another to define the benchmark instructions.

I'll update this PR today with similar changes to those proposed above.

ping @denderello @htr

Looks like a nice compromise to me. @htr ?

Yup. Lgtm

@puja108 Could you have a quick look to the doc changes ? Feel free to make any change.

minor edits, LGTM now


Pull docker images from private and insecure registry

Is there any way to pull docker images from a private and insecure registry instead of docker.io ??
This is my testing code:
import anyio
import dagger

async def main():
    config = dagger.Config(log_output=sys.stdout)

    async with dagger.Connection(config) as client:
        # use a node:16-slim container
        # mount the source code directory on the host
        # at /src in the container

        source = (
            client.container()
            .from_("<private-registry-url>/node:20.5.0-slim")
            .with_directory(
                "/src",
                client.host().directory("."),
                exclude=["node_modules/", "ci/"],
            )
        )

        # set the working directory in the container
        # install application dependencies
        runner = source.with_workdir("/src").with_exec(["npm", "install"])

        # run application
        await runner.with_exec(["npm", "test"]).stderr()

anyio.run(main)

I add the /etc/dagger/engine.toml to enable insecure registry as follow:
debug = true
insecure-entitlements = ["security.insecure"]

[registry."private-registry-url"]
        http = true
        insecure = true

Also, I update the Python script to set the config file and enable insecure_root_capabilities flag to .with_exec. This is the new script:
import anyio
import dagger

async def main():
    config = dagger.Config(config_path='/etc/dagger/engine.toml', log_output=sys.stdout)

    async with dagger.Connection(config) as client:
        # use a node:16-slim container
        # mount the source code directory on the host
        # at /src in the container

        source = (
            client.container()

            .from_("<private-registry-url>/node:20.5.0-slim")

            .with_directory(
                "/src",
                client.host().directory("."),
                exclude=["node_modules/", "ci/"],
            )
        )

        # set the working directory in the container
        # install application dependencies
        runner = source.with_workdir("/src").with_exec(["npm", "install"], insecure_root_capabilities=True)

anyio.run(main)

When I run the script I get this error:
/root/.cache/dagger/dagger-0.8.1 session --label dagger.io/sdk.name:python --label dagger.io/sdk.version:0.8.1 --project /etc/dagger/engine.toml

According to dagger command help, there is no --project flag.
I'm using Dagger 0.8.1.

Did you try with our latest version? 0.8.8?
It might fix your issue, I know that Dagger supports mirror so it should work with your insecure registry

I tried the latest version but the error persist.
There is no --project flag yet in the latest version:
# ~/.cache/dagger/dagger-0.8.8 --help
Usage:
  dagger [command]

Available Commands:
  completion  Generate the autocompletion script for the specified shell
  help        Help about any command
  login       Log in to Dagger Cloud
  logout      Log out from Dagger Cloud
  query       Send API queries to a dagger engine
  run         Runs a command in a Dagger session
  version     Print dagger version

Flags:
      --cpuprofile string   collect CPU profile to path, and trace at path.trace
      --debug               Show more information for debugging
  -h, --help                help for dagger
      --pprof string        serve HTTP pprof at this address
      --progress string     progress output format (auto, plain, tty) (default "auto")
  -s, --silent              disable terminal UI and progress output
      --workdir string      The host workdir loaded into dagger (default ".")

Use "dagger [command] --help" for more information about a command.


Can't locate path: </srv/www/peeps.gwaz.dev/vendor/rachidlaasri/laravel-installer/src/Providers/../views>

the publishFiles() method in the Service Provider is trying to publish DIR.'/../views' with a lowercase v, rather than Views with a capital V.

Thanks for reporting this issue.
It is fixed now : https://github.com/RachidLaasri/LaravelInstaller/commit/154d5141d8aa603346e131a3b57b1614a5d6b210


Message parser should be able to support arbitrary whitespace such as '\n', '\t', '\r', and ' ' within and between messages

How do I configure the stream parser to be able to discard whitespace between JSON messages?
I am facing an issue during implementation of a RPC system, which requires usage of JSONParser to parse binary to JSON for transmitting thru RPC. The issue I am facing is that input streams could be separated by a variety of different whitespaces, and the current implementation of seperator in the lib only supports a single separator.
Assuming as such, we have to keep out input streams in the following manner :
{...message}{...message}.
However, to improve readability, we wish to be able to add whitespaces in between messages, in a similar manner as demonstrated below.
// Normal separation
{...message}{...message}{...message}

// Spaces
{...message} {...message}    {...message}

// New lines
{...message}
{...message}
{...message}

// Any combination
{...message}                                  {...message}
                       {...message}
{...message}

Hi @addievo,
There is a separator option that you can use to set whatever you want to be the separation between the JSON objects.
You could set the separator to /n, /n/n or whatever you want.
Would that work for you?

Hi @juanjoDiaz,
Thank you for your reply. While the separator option does exist, it appears limited to recognizing only one type of separator between JSON messages. In our use-case, we need the parser to accommodate a range of whitespace characters such as '\n', '\t', '\r', and ' ' as valid separators between messages. Is it possible to extend the functionality to support multiple types of separators?

@juanjoDiaz is it possible to make separator option into a regex string? That way we could easily capture any kind of characters we expect to be whitespace?

I think that regex is too flexible.
It will have a big performance impact for such a concrete use case.
I'm thinking of allowing an array of separators.
Since you can chain as many separators as you want, it will solve your case.

I implemented this but the performance impact is too big for me to accept it.
So I need to think of a way to keep the current performance for people that is fine working with a single separator while allowing multiple separators for those that can accept the performance impact.

Wouldn't the performance only be impacted if there were actual whitespace between the json strings? And also if I give an array of 1 character wouldn't that be the same as just specifying a single character?
And when you say we can chain whitespace you mean that if there is more than 1 whitespace character in the stream, it will auto-drop them all?

Hey @juanjoDiaz,
What about starting off with a quick check to see how many separators are in the array? If it's just one, we could stick to the fast route we already have. If there's more, then we switch to the slower, but more flexible, multi-separator logic. This way, we get the best of both worlds without a big hit on performance. What do you think?

Just to clarify the input stream may have 0 or more whitespace characters. The idea is for the stream parser to completely remove them all regardless of how many there is. So we cannot fix the number of expected whitespace separators like '\n\n'.

Wouldn't the performance only be impacted if there were actual whitespace between the json strings? And also if I give an array of 1 character wouldn't that be the same as just specifying a single character?

During tokenization, we need to check for every character to see if it is a separator o no.
Checking a single character is very fast.
Checking a sequence of characters is a bit slower but still fast.
Checking against a list of possible characters or character sequences is also slower.
Considering that you might be streaming a million JSON object separated by a simple character, the overhead of checkign against the list piles up and becomes very noticeable.

And when you say we can chain whitespace, do you mean that if there is more than 1 whitespace character in the stream, it will auto-drop them all?

Yes. If your separator is "\n", any number of new lines will be skipped. (e.g.: \n\n\n\n).
There is a unit test specific for this.
The problem is with matching "\n" or "\s".

What about starting off with a quick check to see how many types separators are in the array? If it's just one, we could stick to the fast route we already have. If there's more, then we switch to the slower, but more flexible, multi-separator logic. This way, we get the best of both worlds without a big hit on performance. What do you think?

Yes. This is what I want to try next.

Hey @juanjoDiaz,
Thanks a bunch for your reply and assistance.
Looking forward to having the array based separator.

Hi @juanjoDiaz ,
I think I might be running into something similar,
What about a 'Transform' callback where I could parse the chunk coming through in the read stream and 'transform' any whitespace or other characters as in chunk.replace(x,y) (or whatever would be appropriate for what's actually made available)
This places the performance hit on my code (with plenty of caveats in the documentation for this capability).
Of course, I would want access to the whole chunk or buffer about to be parsed not each character at a time.

Hi @KWLandry-acoustic ,
You issue is not related.

I do remember reading this somewhere, in the docs perhaps, but it did not translate, or actually I remember I ran into a problem with it and had to specify a separator,
Regardless, in any case, I'll set and test that,
Thanks,
KWL


Fix theme.js in custom themes

I believe this (along with the original PR#252) fully corrects the original issue #227 as well as the regression #382.

Can you explain how this fixes the issue? In the linked PR I did, the code doing this sort of behaviour was already commented out and this change is not consistent with that commented out behaviour.

Sure, I don't have much historical knowledge of the project but I believe this part of your commit:
+        this.addDirectory('default', Renderer.getDefaultTheme());
+        this.addDirectory('theme', theme.basePath);

Fixed the 'fallback' part which may not have ever been working properly.  I essentially took the other part of your commit, re-instituted the code and fixed it so that it would work properly.

Shoot I did not mean for this to come into this same PR.  LMK what you'd like me to do here...
Close this one and submit them separately?  I'm not sure there is a way to pull this commit out of the PR, if you know of a way, let me know and I will.

You can use rebase on the branch and fix it up.
I'd also prefer that you use const over let for the PR, so feel free to also fix that up. I'd also say that class names should always use PascalCase, so that change might be useful. I'm not 100% on using default always - maybe we should check default before using it or use the old behaviour of eval?

I ran the tests on my local machine and they all passed, I think Travis was running slow or something.

@blakeembrey I think that may be why it was commented out as it didn't work with eval when tested against the typedoc-theme-markdown.  I based the fix on export default which is what the markdown theme had.
If you didn't want to use the default, we could make it the same as for plugins.  The require() should pass back a function which accepts the app.  Thoughts?
I'll fix them up and re-submit.

Sure, but we can make them both consistent by including a check like typeof x=== 'function' ? x : x.default. I think that'd be the best compromise to enable both use-cases for now.
Yeah, Travis sometimes has issues with the timing. Might need to increase it sometime, usually I just restart it.


tvOS branch with Carthage does not build scheme

When using Carthage to build the tvOS branch:
git "https://github.com/Hearst-DD/ObjectMapper" "tvOS"
carthage update
*** Fetching ObjectMapper
*** Checking out ObjectMapper at "61c91a8c71f32fc8e3adc4df296d798507bcc1e1"
*** xcodebuild output can be found in /var/folders/19/xzwjb7553p74ghltct8tb6gw0000gq/T/carthage-xcodebuild.AqkOOG.log
*** Building scheme "ObjectMapper-Mac" in ObjectMapper.xcworkspace
*** Building scheme "ObjectMapper-watchOS" in ObjectMapper.xcworkspace
*** Building scheme "ObjectMapper-iOS" in ObjectMapper.xcworkspace
The problem is that tvOS is not being built.
Maybe there is something you need to configure to get it working with Carthage? Is there a tvOS scheme?
Thanks

It seems that it needs the build scheme...

The tvOS branch should now build correctly. Make sure xcode 7.1 is selected on the command line and you are using the latest version of Carthage.


[WIP][SPARK-33527][SQL] Extend the function of decode so as consistent with mainstream databases

What changes were proposed in this pull request?
In Spark, decode(bin, charset) - Decodes the first argument using the second argument character set.
Unfortunately this is NOT what any other SQL vendor understands DECODE to do.
DECODE generally is a short hand for a simple case expression:
SELECT DECODE(c1, 1, 'Hello', 2, 'World', '!') FROM (VALUES (1), (2), (3)) AS T(c1)
=>
(Hello),
(World)
(!)

There are some mainstream database support the syntax.
Oracle
https://docs.oracle.com/en/database/oracle/oracle-database/19/sqlrf/DECODE.html#GUID-39341D91-3442-4730-BD34-D3CF5D4701CE
Vertica
https://www.vertica.com/docs/9.2.x/HTML/Content/Authoring/SQLReferenceManual/Functions/String/DECODE.htm?tocpath=SQL Reference Manual|SQL Functions|String Functions|_____10
DB2
https://www.ibm.com/support/knowledgecenter/SSGU8G_14.1.0/com.ibm.sqls.doc/ids_sqs_1447.htm
Redshift
https://docs.aws.amazon.com/redshift/latest/dg/r_DECODE_expression.html
Pig
https://pig.apache.org/docs/latest/api/org/apache/pig/piggybank/evaluation/decode/Decode.html
Teradata
https://docs.teradata.com/reader/756LNiPSFdY~4JcCCcR5Cw/jtCpCycpEaXESG4d63kMjg
Snowflake
https://docs.snowflake.com/en/sql-reference/functions/decode.html
Why are the changes needed?
It is very useful.
Does this PR introduce any user-facing change?
Yes
How was this patch tested?
Jenkins test.

Test build #131628 has started for PR 30479 at commit cd81faf.

Test build #131628 has finished for PR 30479 at commit cd81faf.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
case class Decode(fakeChild: Expression, params: Seq[Expression]) extends RuntimeReplaceable
case class StringDecode(bin: Expression, charset: Expression)

Build finished.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/131628/

Test build #131645 has started for PR 30479 at commit 338d28b.

Test build #131645 has finished for PR 30479 at commit 338d28b.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Build finished.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/131645/

Build finished.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/36230/

Build finished.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/36246/

Test build #131747 has started for PR 30479 at commit 1cf326b.

Test build #131747 has finished for PR 30479 at commit 1cf326b.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
case class Decode(params: Seq[Expression], fakeChild: Expression) extends RuntimeReplaceable

Build finished.

Build finished.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/36345/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/131747/

cc @cloud-fan

Test build #132426 has started for PR 30479 at commit 80c56a1.

Test build #132427 has started for PR 30479 at commit 6a1b919.

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37026/

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37027/

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37026/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37026/

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37027/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37027/

Test build #132427 has finished for PR 30479 at commit 6a1b919.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/132427/

Test build #132426 has finished for PR 30479 at commit 80c56a1.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
case class Decode(params: Seq[Expression], child: Expression) extends RuntimeReplaceable

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/132426/

Test build #132460 has started for PR 30479 at commit 1614e62.

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37062/

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37062/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37062/

Just a question; the other systems don't have a function like Spark decode/encode?

Just a question; the other systems don't have a function like Spark decode/encode?

Only PostgreSQL have the function.

Test build #132460 has finished for PR 30479 at commit 1614e62.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/132460/

Test build #132484 has started for PR 30479 at commit eea967a.

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37086/

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37086/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37086/

Test build #132484 has finished for PR 30479 at commit eea967a.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/132484/

thanks, merging to master!

@cloud-fan Thanks for your work
@maropu @HyukjinKwon Thanks for your review too.


Stacked chart doesn't contain any producing technologies

I think we're still missing Solar PV production in the stacked load chart of a LES.

#1063


chore: set artworks feed view as ready for release

This PR resolves []
Description
Set artworks feed view as ready for release

PR Checklist

[x] I have tested my changes on iOS and Android.
[x] I hid my changes behind a feature flag, or they don't need one.
[x] I have included screenshots or videos, or I have not changed the UI.
[x] I have added tests, or my changes don't require any.
[x] I added an app state migration, or my changes do not require one.
[x] I have documented any follow-up work that this PR will require, or it does not require any.
[x] I have added a changelog entry below, or my changes do not require one.

To the reviewers 👀

[ ] I would like at least one of the reviewers to run this PR on the simulator or device.

Changelog updates
Changelog updates

Cross-platform user-facing changes

set artworks feed view as ready for release - mounir

iOS user-facing changes

Android user-facing changes

Dev changes

Need help with something? Have a look at our docs, or get in touch with us.

This PR contains the following changes:

Cross-platform user-facing changes (set artworks feed view as ready for release - mounir)

  Generated by :no_entry_sign: dangerJS against d31b406048c0ee78bdb7e6f6195c9efb6f0c9b7b


server: fix update diff on single value sets

The server was notifying single value set updates using XOR but for
single value sets are notified with the final value.
fixes #324
Signed-off-by: Jaime Caamaño Ruiz<EMAIL_ADDRESS>

/lgtm
works now, thanks @jcaamano!

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

10 of 12 (83.33%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.1%) to 74.83%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

server/transact.go
10
12
83.33%

Totals

Change from base Build<PHONE_NUMBER>:
0.1%

Covered Lines:
4944

Relevant Lines:
6607

💛  - Coveralls


Highlight related items when hovering

To understand better how all the pieces fall together, highlighting related items would be very nice. The color coding already helps with that but for starters it's much more intuitive if related items change their color on hover.

$20 from me for a fix. thumbs up if you want to bump the bounty with an equal amount.


Add save-the-date symposium info

👷 Deploy Preview for ucsc-ospo processing.

Name
Link

🔨 Latest commit
c7908d985b0632a17ddc921956c0d4c5efe4f9aa

🔍 Latest deploy log
https://app.netlify.com/sites/ucsc-ospo/deploys/62e1a2a6424cc80009341b48

Note that this is relative to main and doesn’t include all the other changes that are more long-term. The cool thing I just discovered is that netlify gives you a preview of the site! See “Deploy Preview”. I didn’t try to force the same formatting but just went with what easiest in the interest of time. I did spend some time on a different banner that includes some of things I was thinking. If you like the “open doors” picture better, I’ve included the powerpoint in the /content/post/20220727 directory which you can use to remix the banner. I’ve also edited the that this is a UC Santa Cruz event, not a UC event. The latter jumps the gun a bit.


omnitrace user API

I've been trying to use omnitrace_user_push_region  (and equivalent *pop* function) as an equivalent of roctxRangePush to mark regions of code within omnitrace output. The code compilers fine, but when run with this functionality it crashes during the run with errors like this:
[omnitrace][89177][0] Signal 11 caught : Segmentation fault (Address not mapped to object [0x100])
### ERROR ### [omnitrace][PID=89180][TID=0] signal=11 (SIGSEGV) segmentation violation. code: 1 (SEGV_MAPERR :: Address not mapped), address of faulting memory reference: 0x100
Backtrace:
[PID=89174][TID=0][0/6] __restore_rt
[PID=89174][TID=0][1/6] hipMemcpy3D_spt +0x892
[PID=89174][TID=0][2/6] hipMemcpy3D_spt +0x1043
[PID=89174][TID=0][3/6] hipMemcpy +0x11e
[PID=89174][TID=0][4/6] _ZN7dolfinx3acc14MatrixOperatorIdEC2ESt10shared_ptrINS_3fem4FormIddEEERKSt6vectorIS3_IKNS4_11DirichletBCIddEEESaISC_EE +0x499
[PID=89174][TID=0][5/6] main_dyninst +0x30c6

These errors don't occur when profiling with omnitrace without the user api calls, or when running normally, or when running with the rocm_prof api. I know I can just use the roctxRangePush( functions to enable some level of region marking with omnitrace, but should the user API work as I'm currently using it?

Could you provide me the code in question? Are you passing string literals, a malloced char*, or the .c_str() from a locally allocated string?

In the user-api example, I used only allocated strings or string literals so I may have overlooked the scenario where the string allocation is temporary.
That being said though, that bactrace doesn't make a whole lot of sense for a user API bug but that is very likely bc the binary rewrite invalidated the line info mapping. Do you see this issue on a non-instrumented binary where you are just sampling?

Code in question is https://github.com/IgorBaratta/pmg-dolfinx. The file implementing the trace is https://github.com/IgorBaratta/pmg-dolfinx/blob/main/src/amd_gpu.hpp . Don't see any other failures when doing omnitrace or rocprof stuff without these calls (although the system/configuration can be quite fragile).

Examples of using the function calls are in https://github.com/IgorBaratta/pmg-dolfinx/blob/main/examples/cg/main.cpp

Examples of using the function calls are in https://github.com/IgorBaratta/pmg-dolfinx/blob/main/examples/cg/main.cpp

So based on the example I see here, you are compiling with -DROCM_TRACING? And therefore, every *_profiling_annotation(...) is pushing/popping both a rocTX marker and the omnitrace user API marker (since you have surrounded all the *_profiling_annotation calls with #ifdef ROCM_TRACING)?
Please try two things:

If you have OMNITRACE_USE_ROCTX=ON, please set that to OFF and see if the issue remains

The potential issue is the ordering of roctxRangePop and omnitrace_user_pop_region in remove_profiling_annotation... these are out of order: you should always pop in the inverse order that you pushed

Run omnitrace in sampling mode on the original (un-instrumented) executable and see if the issue remains

i.e., mpirun -n <ranks> omnitrace-sample -- ./cg <args...>
Even if the issue remains, this should give a more accurate backtrace (even moreso if you add -g3 to the CMAKE_CXX_FLAGS here)

Side Note
You may want to implement the {add,remove}_profiling_annotations like this and remove all the #ifdef guards around calls to {add,remove}_profiling_annotations in the examples/cg/main.cpp:
inline void add_profiling_annotation(const char * const tag)
{
#ifdef ROCM_TRACING
  roctxRangePush(tag);
#elif defined(OMNITRACE)
  omnitrace_user_push_region(tag);
#else
  (void) tag;
#endif
}

In addition to making the code far more readable, if neither ROCM_TRACING nor OMNITRACE are defined, at any compiler optimization level greater than zero (i.e. even at -O1), these function calls will be optimized away by the compiler (i.e. an "production" build will effectively compile to the same assembly as when you guard the add/remove calls with preprocessor ifdefs):

Note how the compiled assembly at -O0 has associated assembly for the {add,remove}_profiling_annotation lines (i.e. debug build)
Note how the compiled assembly at -O1 has no associated assembly for the {add,remove}_profiling_annotation lines (i.e. non-debug build)

even in the case of RUN_LABEL at lines 106 and 111 where the label is a from a std::string

Thanks for the suggestions. Unfortunately, even with the modified code and the roctxRange... function calls turned off, I'm still seeing the crashing, with errors similar to before, i.e.:
MPI rank 11 can see 8 AMD GPUs
[omnitrace][85176][0] Signal 11 caught : Segmentation fault (Address not mapped to object [0x100])

[omnitrace][85175][0] Signal 11 caught : Segmentation fault (Address not mapped to object [0x100])
MPI rank 10 can see 8 AMD GPUs

MPI rank 15 can see 8 AMD GPUs
[omnitrace][85180][0] Signal 11 caught : Segmentation fault (Address not mapped to object [0x100])

MPI rank 9 can see 8 AMD GPUs
[omnitrace][85174][0] Signal 11 caught : Segmentation fault (Address not mapped to object [0x100])

### ERROR ### [omnitrace][PID=85174][TID=0] signal=11 (SIGSEGV) segmentation violation. code: 1 (SEGV_MAPERR :: Address not mapped), address of faulting memory reference: 0x100
Backtrace:
### ERROR ### [omnitrace][PID=85175][TID=0] signal=11 (SIGSEGV) segmentation violation. code: 1 (SEGV_MAPERR :: Address not mapped), address of faulting memory reference: 0x100
Backtrace:
### ERROR ### [omnitrace][PID=85176][TID=0] signal=11 (SIGSEGV) segmentation violation. code: 1 (SEGV_MAPERR :: Address not mapped), address of faulting memory reference: 0x100
Backtrace:
### ERROR ### [omnitrace][PID=85180][TID=0] signal=11 (SIGSEGV) segmentation violation. code: 1 (SEGV_MAPERR :: Address not mapped), address of faulting memory reference: 0x100
Backtrace:

MPI rank 13 can see 8 AMD GPUs
[omnitrace][85178][0] Signal 11 caught : Segmentation fault (Address not mapped to object [0x100])

### ERROR ### [omnitrace][PID=85178][TID=0] signal=11 (SIGSEGV) segmentation violation. code: 1 (SEGV_MAPERR :: Address not mapped), address of faulting memory reference: 0x100
Backtrace:

MPI rank 8 can see 8 AMD GPUs
[omnitrace][85173][0] Signal 11 caught : Segmentation fault (Address not mapped to object [0x100])

### ERROR ### [omnitrace][PID=85173][TID=0] signal=11 (SIGSEGV) segmentation violation. code: 1 (SEGV_MAPERR :: Address not mapped), address of faulting memory reference: 0x100
Backtrace:

MPI rank 12 can see 8 AMD GPUs
[omnitrace][85177][0] Signal 11 caught : Segmentation fault (Address not mapped to object [0x100])

### ERROR ### [omnitrace][PID=85177][TID=0] signal=11 (SIGSEGV) segmentation violation. code: 1 (SEGV_MAPERR :: Address not mapped), address of faulting memory reference: 0x100
Backtrace:

[omnitrace][85179][0] Signal 11 caught : Segmentation fault (Address not mapped to object [0x100])
MPI rank 14 can see 8 AMD GPUs

### ERROR ### [omnitrace][PID=85179][TID=0] signal=11 (SIGSEGV) segmentation violation. code: 1 (SEGV_MAPERR :: Address not mapped), address of faulting memory reference: 0x100
Backtrace:
[PID=85173][TID=0][0/6] __restore_rt
[PID=85173][TID=0][1/6] hipMemcpy3D_spt +0x892
[PID=85173][TID=0][2/6] hipMemcpy3D_spt +0x1043
[PID=85173][TID=0][3/6] hipMemcpy +0x11e
[PID=85173][TID=0][4/6] _ZN7dolfinx3acc14MatrixOperatorIdEC2ESt10shared_ptrINS_3fem4FormIddEEERKSt6vectorIS3_IKNS4_11DirichletBCIddEEESaISC_EE +0x4d0
[PID=85173][TID=0][5/6] main_dyninst +0x2ffb

I will try the tracing without the functions.

is this the full extent of the output? I'd like to see the /proc/<pid>/maps file. Are you definitely using a version of omnitrace that is build against the same major and minor version of rocm as the application? I'll try to reproduce with the code from the repo you provided. Any special build instructions?

Wait a minute... I see this in the output:
MPI rank 11 can see 8 AMD GPUs

from this line which is only printed when ROCM_SMI is defined. I assumed you were getting that from hipDeviceCount. So it's likely one of two problems:

Omnitrace is built against a different rocm version than the app so the first call to hipMemcpy tries to load omnitrace as the HSA tools lib and it segfaults bc of the rocm incompatibilities across different minor versions
I've never tested a code using rocm-smi when omnitrace is also trying to use rocm-smi. You definitely need to try running this in a build without ROCM_SMI defined.

With regard to 2, omnitrace makes most of the rocm-smi calls in a background thread but it does make a few calls on the main thread... mainly initialize/finalize/get-num-gpus. And that segfault appears to be happening on the main thread. if your app is holding a lock or something on rocm-smi when omnitrace tries to make some rocm-smi calls but earlier ones succeeded, that could definitely cause some problems.

Closing bc I believe this is a bug originating from your use of ROCm-smi


Moving windows on El Capitan

Moving the window in Radiant has become a bit of a chore. Often when I drag the top, I begin selecting text within the website.

I'm seeing a similar issue. Hardly noticeable if you move the window slowly, but as soon as you try to drag it at any sort of speed, it starts to lag and I end up with the search box highlighted

This is also an issue while starting Radiant, as it is always positioned in a different place than I'd like to have it, I have to wait until Radiant is fully up and running to be able to drag the window at all.

I have in on El Capitan that I cannot move the window at all (using 1.5.0)

Same here. I'm on El Capitan and 1.5.0 and sometimes I can't move the window at all.  It seems that this is only the case when I either don't have Chrome, or I have it open but am using a different account than the one I use for my Google Music subscription.  Once I switch accounts or open another Chrome window with my personal Google account (which is what's subscribed to Google Music) then I can move the window with ease.

Same here. Can't move the window at all. Tried with Chrome open and closed, but nothing changes.
Version 1.5.0 running on 10.11.1

The app itself uses an embedded webview, which is closer to Safari than Chrome - if I get some time later today I'll take a look and see if I can find anything.

i find that i can move the window if i've clicked in another window (say a open chrome window) and then click-and-drag the radiant window. (i.e., so the radiant window gets focus and immediately starts being dragged in one action)
If I do it as separate actions (click to focus radiant window + click-drag to try and move the window) it doesn't work.

So, Google updated their element names again.  #413 should enable dragging as it was previously implemented.
IMO we should look at alternate implementations - two ideas for further research:

implement something like -webkit-app-region: drag from Electron
implement the app bar in native code

I have found that if Radiant player is not in focus (like if I click on my desktop first) I can move the window without problem, but as soon as it has focus, it just selects the text.
OS X: 10.11.2
Player: 1.5.0

This is fixed in master and should go out today or tomorrow as 1.6

Hm, same problem here. Newest Version. Can't move the window.


ValueError: If eos_token_id is defined, make sure that pad_token_id is defined.

Describe the bug

A clear and concise description of what the bug is.

使用官方给的脚本https://github.com/zjunlp/DeepKE/blob/main/example/llm/OneKE.md，运行后报错
Traceback (most recent call last):
File "/home/llen/DeepKE-main/example/llm/run.py", line 42, in
generation_output = model.generate(input_ids=input_ids, generation_config=GenerationConfig(max_length=1024, max_new_tokens=512, return_dict_in_generate=True))
File "/root/miniconda3/envs/deepke-llm/lib/python3.9/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "/root/miniconda3/envs/deepke-llm/lib/python3.9/site-packages/transformers/generation/utils.py", line 1602, in generate
return self.greedy_search(
File "/root/miniconda3/envs/deepke-llm/lib/python3.9/site-packages/transformers/generation/utils.py", line 2489, in greedy_search
raise ValueError("If eos_token_id is defined, make sure that pad_token_id is defined.")
ValueError: If eos_token_id is defined, make sure that pad_token_id is defined.
虚拟环境与版本都是按照requirements.txt安装的
Environment (please complete the following information):

OS: [e.g. mac / window]
Python Version [e.g. 3.6]

transformers==4.33.0
torch==2.0.0
Screenshots

If applicable, add screenshots to help explain your problem.

Additional context

Add any other context about the problem here.
code.zip

将代码改为
generation_output = model.generate(input_ids=input_ids, generation_config=GenerationConfig(max_length=1024, max_new_tokens=512, return_dict_in_generate=True),pad_token_id=tokenizer.eos_token_id)
已解决

感谢您关注本项目和OneKE，请问您的问题解决了吗？


Safari extension?

I love using Fontanello and was wondering if a Safari version is in the works or even technically viable. Safari is my primary browser and I’d love to use it there. I’d be happy to contribute towards the development in any way I can but am not sure where to begin as I’m not familiar with developing Safari extensions. So, if there are any resources you could point me to that could help me understand the necessary Safari frameworks, that’d be awesome as well.
Either way, thanks a lot for creating Fontanello. It’s one of my most used browser extensions :)

Hi @yashghelani and thanks for the kind words!
It's definitely viable to make a version for Safari on MacOS, but guess it would need to be re-worked for iOS. Apple has made it fairly easy to convert extensions built for Chrome/Firefox/Edge to also work in Safari. Made some quick tests and got what seems to be a working version for MacOS. Will need to do some more testing and also think about how it should be distributed.

@frederfred Let me know if I can be of help in any way - testing, bug fixing, anything :)

I really would like to use Fontanello on Safari.

Same here. Love Fontanello and it's one of few reasons I still use Chromium-based browsers. Hope we can get a Safari version.

Hi @frederfred,
Is there any plan for Safari?


Multiple errors on app start after upgrading to 0.4.0

Getting multiple errors at app start after upgrading an existing app from 0.3.11 to 0.4.0.
Using 0.3.11 everything is fine.
First, when running cmd + R from Xcode, this error is appearing in the packager window:
 ERROR  Watcher took too long to load
Try running `watchman` from your terminal
https://facebook.github.io/watchman/docs/troubleshooting.html
Error: Watcher took too long to load
Try running `watchman` from your terminal
https://facebook.github.io/watchman/docs/troubleshooting.html
    at null._onTimeout (/Users/mikewilcox/code/randUM/node_modules/react-native/packager/react-packager/src/FileWatcher/index.js:74:16)
    at Timer.listOnTimeout [as ontimeout] (timers.js:112:15)

so then followed and tried to npm start from the shell and got this:
Error: watch EMFILE
    at errnoException (fs.js:1030:11)
    at FSWatcher.start (fs.js:1062:11)
    at Object.fs.watch (fs.js:1087:11)
    at NodeWatcher.watchdir (/Users/mikewilcox/code/randUM/node_modules/react-native/node_modules/sane/src/node_watcher.js:144:20)
    at Walker.<anonymous> (/Users/mikewilcox/code/randUM/node_modules/react-native/node_modules/sane/src/node_watcher.js:351:12)
    at Walker2015-04-17 19:42 node[25755] (FSEvents.framework) .emit (events.js:98:17)
    at /Users/mikewilcox/code/randUM/node_modules/react-native/node_modules/sane/node_modules/walker/lib/walker.js:69:16
    at Object.oncomplete (fs.js:107:15)
FSEventStreamStart: register_with_server: ERROR: f2d_register_rpc() => (null) (-21)

See http://facebook.github.io/react-native/docs/troubleshooting.html
for common problems and solutions.
2015-04-17 19:42 node[25755] (FSEvents.framework) FSEventStreamStart: register_with_server: ERROR: f2d_register_rpc() => (null) (-21)

Just to update how I solved this:
brew uninstall watchman
brew link automake
brew install --HEAD watchman

Cheers

This fixed it for me :
rm -rf /usr/local/var/run/watchman/ && brew uninstall watchman && brew install watchman

Confirmed. Reinstall watchman fixed this issue. I got this issue when upgrading from 0.44 to 0.46.4.

Did you salute it?

I did not have watchman installed (new pc). So the solution for me was just:
brew install watchman

reinstalling watchman is marvelous

Life rule - Just install watchman!!!


relay: returning an strawberry object with node: strawberry.relay.Node = strawberry.relay.node() breaks

After the latest strawberry / strawberry django updates, the code

@strawberry.type
class SecretgraphObject:
    node: strawberry.relay.Node = strawberry.relay.node()

@strawberry.type
class Query:
    @strawberry_django.field
    @staticmethod
    def secretgraph(
        info: Info, authorization: Optional[AuthList] = None
    ) -> SecretgraphObject:
        return SecretgraphObject

doesn't work anymore.

Describe the Bug

System Information

Operating system: linux
Strawberry version (if applicable): 193.1

Additional Context
GraphQL request:2:3
1 | query serverSecretgraphConfigQuery {
2 |   secretgraph {
  |   ^
3 |     config {
Traceback (most recent call last):
  File "/home/alex/git/secretgraph/.venv/lib/python3.11/site-packages/graphql/execution/execute.py", line 528, in await_result
    return_type, field_nodes, info, path, await result
                                          ^^^^^^^^^^^^
  File "/home/alex/git/secretgraph/.venv/lib/python3.11/site-packages/asgiref/sync.py", line 479, in __call__
    ret: _R = await loop.run_in_executor(
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/alex/git/secretgraph/.venv/lib/python3.11/site-packages/asgiref/sync.py", line 538, in thread_handler
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/alex/git/secretgraph/.venv/lib/python3.11/site-packages/strawberry_django/resolvers.py", line 91, in async_resolver
    return sync_resolver(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/alex/git/secretgraph/.venv/lib/python3.11/site-packages/strawberry_django/resolvers.py", line 77, in sync_resolver
    retval = retval()
             ^^^^^^^^
TypeError: SecretgraphObject.__init__() missing 1 required keyword-only argument: 'node'

@devkral do you still get this?
can you do a reproduction here? https://play.strawberry.rocks/

I think it is fixed, I tried to provoke the error in my code without success


Replace backslash with slash to avoid json errors on Windows

Amazing tool by the way, thanks a lot! Our build times are already cut in half with help from this tool 😊 If there is a definitive success I’ll write a blog post explaining my workflow, because it’s quite interesting and works from Visual Studio.

Our build times are already cut in half with help from this tool 😊 If there is a definitive success I’ll write a blog post explaining my workflow, because it’s quite interesting and works from Visual Studio

@mrexodia that sounds great! Would be very interested in a blog post indeed.


Add Prefix Tuning to BLOOM

This PR adds prefix tuning to BLOOM.
let me know if you cannot follow what I've changed, but I've confirmed that when not enabling prefix tuning, this implementation gets the same initial loss exactly on the first logging steps.
There is no longer the slowdown issue that I mentioned on Slack!!!!
(Turns out just creating the alibi tensor using hidden states and only keeping the preprocessing of alibi to happen after the prefix tuning did the trick)
I am testing the loss curves now.

still TODO: test if prefix tuning + task adapters works.

Loss curve comparison, with equal settings + what should be equivalent reduction factors + param budget. Prefix tuning is a lot more unstable. But this could just mean that prefix tuning is less expressive/powerful, I am not sure yet.

@haileyschoelkopf  Do you want to work on this and get this PR merged? LoRA has already been integrated 😄


[FEATURE] Improve Code Coverage

Feature Description
Improve the code coverage of the qbraid_qir package to > 97%
Implementation (Optional)
No response

@TheGupta2012 The vast majority of the missing coverage is in qasm3/maps.py, which we may be better if incorporated into to the pyqasm.rebase() function after the qBraid-SDK migration? https://github.com/qBraid/pyqasm/issues/12


Fountain of Youth is a Fountain of Adulthood

var partnerAge = props.partner.ageTracker.AgeBiologicalTicks;
props.partner.ageTracker.AgeBiologicalTicks = Math.Max(MINIMUM_AGE, partnerAge - AGE_REDUCTION);

Should be something like
var partnerAge = props.partner.ageTracker.AgeBiologicalTicks;
If (partnerAge - AGE_REDUCTION > MINIMUM_AGE)
    props.partner.ageTracker.AgeBiologicalTicks = partnerAge - AGE_REDUCTION;

Hi,
my code is fine.

Get some help.

If I decreased minimum age the code would just never trip and age down further.
The minimum age is 20 in your mod, so 18 yo skip straight to 20.

I will give you the benefit of the doubt and put the Min_Age to 18 and add activation-age also at 18.
But I will not account for stuff like Races that "are adult with 10" etc.


feat/node-metadata

This PR adds the possibility of injecting pod metadata into Envoy's configuration so individual pods can be identified from the discovery service using that metadata (which is sent in the discovery requests of the xDS protocol). So far the discovery service only saw anonymous subscribers requesting resources. With this the logic for the self-healing functionality can be improved and prometheus metrics can be exposed with pod/host/namespace identifying labels.
As metadata can only be injected in Envoy's bootstrap configuration, we have moved from having a single ConfigMap for all the pods to having an init container create the bootstrap configuration with the specific values for pod name, host name and namespace included in the metadata.
This PR effectively deprecates the EnvoyBootstrap controller, which will be removed from the code in a post v0.8 release.
/kind feature
/priority important-soon

/ok-to-test

Awesome change making it simple with the initContainer with better info for the SD, instead of the rigid configmap!
/lgtm


Set azure devops default status to ByDesign

User description
Using a default status of Active = 1 causes issues because a common setup is to enable comment check for Azure PRs. Setting Active=1 means that reviewers need to manually mark each comment as resolved.
Use a better ByDesign status (https://learn.microsoft.com/en-us/javascript/api/azure-devops-extension-api/commentthreadstatus), which better fits what we see in github and gitlab

PR Type
Enhancement

Description

Changed the default status of Azure DevOps comment threads from Active (1) to ByDesign (5)
This change addresses issues with common PR review setups that use comment checks
The new ByDesign status better aligns with GitHub and GitLab behavior
Eliminates the need for reviewers to manually mark each comment as resolved

Changes walkthrough 📝
Relevant filesEnhancement

azuredevops_provider.pyUpdate Azure DevOps comment thread default status

pr_agent/git_providers/azuredevops_provider.py
Changed the default status of CommentThread from 1 (Active) to 5 (ByDesign)

+1/-1

💡 PR-Agent usage:
Comment /help on the PR to get a list of all available PR-Agent tools and their descriptions

seems reasonable


Update index.html CSAF community days

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


PS4 8 minute timout

While using on a PS4 console, with a DS4 controller connected via pasthrough or with uploaded files the Raspberry PI Pico times out after 8 minutes and becomes unresponsive until unplugged from the USB socked and then is plugged in again.
All features work as expected on the PS4 during the 8 miuntes the Pico is working for.
I am using the latest GP2040-CE_0.7.10_Pico.uf2 file and did a clean install on a brand new pico.
I was expecting with the uploaded security files that this would work, but it didn't so I cleaned the flash using flash nuke and the reuploaded the GP2040-CE_0.7.10_Pico.uf2 file and set up again. This time I plugged a DS4 controller in and set up to use passthrough but even then after 8 minutes the PI stopped working until unplugged and plugged in again.

@mrdude2478 the DS4 does not work for passthrough authentication.
We offer no support about file upload.


Daemon to monitor changes in real-time and run restic

Having used CrashPlan and Obnam for a while, I'm very impressed by Restic.  When it gains compression (and optionally disabled encryption), I think it will be a great alternative (for the software, not the backend service).
One of the nice things about CrashPlan is its real-time service that watches for changes to backup sets (using e.g. inotify on Linux) and runs the actual backup every so often, as configured.
And one of the reasons I'm so impressed with Restic is how fast its actual backup phase is.  When backing up a large set of files with few of them having changed, most of the time is spent in the scan phase (e.g. #1160).
So, since Restic is so flexible about how it receives the list of paths to backup, it would be nice if there were a daemon that could run in the background and watch for changes to certain paths, then run Restic every so often on the files/directories that have changed.  That could make Restic really fast and relatively lightweight for desktop backups.
Since Restic is so flexible in this regard, I don't know if it would even be necessary for such a daemon to be specific to Restic.  If not, there might already be some solutions available, in which case we don't need to do anything, but documenting it would be very helpful.  :)
What do you think?
Thanks for your work on Restic!

This is something I always thought about as well (I too used CrashPlan briefly), but honestly I haven't felt the need with restic, as you say it's very fast.

Thinking about this a bit more, I think it does need support in restic, because restic needs to use the same set of paths for each backup run without actually scanning them; it needs to do this so that the new snapshot will have the same paths and be considered the next snapshot of the same paths.  It needs to do this while receiving the list of actual files to backup from another source, so it doesn't have to scan all the paths and find the modified files.  Then it needs to merge that list of files with the existing snapshot.  In other words, the end result (the snapshot) needs to be the same, as if restic had run normally.
So maybe restic needs to:

Copy previous snapshot.
Add/replace/remove paths given by daemon to the new snapshot.
Save new snapshot.

The example scenario, for why this would be useful, is one in which there are thousands of directories in a backup set, but since the last snapshot, only a few files have been added, changed, or deleted.  In this case, it takes restic much longer to scan all of the directories to find the changes than it does to actually backup the changes.  A daemon could monitor the directories in real time with, e.g. inotify, and tell restic exactly what has changed.  Restic then "just" needs to merge those changes with a copy of the previous snapshot.
This could make restic the "killer" backup app.  I've been using restic to backup most of my homedir for a few days now in a cron job, and it's working well, but since not much data actually changes each time, the job as a whole takes a long time because of the scanning.

@alphapapa Why can't you just have it create a snapshot with whatever files and directories in it that were changed (in a consolidated format of course)? For example, every time it runs, it could be done as if you manually asked it to back up those top-level items that changed manually.
Yes, there will be many snapshots with many different files and directory names, but so what? They will just tell you exactly what was backed up (except when some parts of it has been consolidated, e.g. when all files in a directory have changed it should of course only list those directories). I don't really see why you should try to hide the fact that only this and that item was backed up.
If it works like this, then this could even be done completely outside of restic; A separate process could utilize the filesystem notifier, build the list of items to back up, and when appropriate ask restic to back them up. It could be smart, e.g. in such a way that if X% of the files in a folder were changed, it will instead of listing all those files to the restic backup command, list the directory they're in and use excludes to not back up the non-changed ones. It could also use e.g. --files-from along with stdin (assuming supported by restic) like --files-from - instead of listing target files/directories on the command line when spawning restic.

@rawtaz If you did that, you would suffer data loss when forgetting and pruning snapshots:

Snapshots are grouped by path.
You can manually group snapshots by tag instead.
If you do this, you will lose data when you forget snapshot B which contains files from directory Z which are not present in snapshots A and C.

This is the same issue Obnam has: it allows you to backup different sets of files to a single repository, but when applying retention policy, it does not group snapshots at all, so you will lose data when forgetting a generation that has a subset of files in other generations.
Besides that, you would make it virtually impossible to do a full restore of a set of directories.  You would have to restore every snapshot in the repo, in order, to ensure that you get all files.
On top of that, you would not be able to determine whether a file had been deleted or was merely unchanged from earlier snapshots and omitted from later snapshots.
In summary, no, that wouldn't work at all.

It works if you combine it with "full" runs regularly, and/or tags to control which snapshots you forget. But sure, I see what you're saying.

Now you're talking about moving from a snapshot model to a full/incremental backup model, which is taking us back to the previous generation of backup software.  If that's what you're after, you can already have that with, e.g. Duplicity.

I really am not. But whatever you say.

Perhaps an alternative solution to reach the goal you are seeking is to implement some type of --include and/or --include-file (that supports stdin) in restic, that makes it only process the files in the include. That way an external process dealing with filesystem notifications can run restic and tell it which files to process (out of those in the files/folders you list for the snapshot).

That's what I proposed in the first message.  The issue is that restic must merge the "included" files (including removing deleted ones) with the previous snapshot.  Each snapshot must be complete or it's no longer a snapshot.

This is amazing. I've started a thread in https://forum.restic.net/t/continuous-backup/593/13 starting from the same idea as @alphapapa - to create a separate tool driving restic - and the more I thought about it the more I realised it has to be a change to the way new snapshots are created in exactly the same rationale as here. I'd definitely want to see that happen.
Thank you for your work on restic!

Linux Storage, Filesystem, and Memory-Management Summit talk regarding kernel support for this kind of feature (also offers significant savings for incremental backups of large, but mostly-unchanging data sets):
https://lwn.net/Articles/755277/

Since looking at restic I have had the same thoughts as @alphapapa and @arikb
My performance problem with restic is not the actually backing up of files or dedup or (lack of) compression, but the fact it spends 99% of its time pointlessly scanning 100,000's of files to notice they haven't changed since an hour ago 😄 That's a lot of wasted effort and I/O that continuous solutions like CrashPlan and Carbonite avoid.
restic support backing up a list of specific files (--file-from). So my thought was to have a inotify wrapper/daemon that would build a list of modified files for 15-60 minutes, and the dispatch a restic backup with the accumulated file path list. Then once every 12-24 hours it would run restic with a full scan to ensure nothing is missed (inotify and NTFS events are not guaranteed, they use kernel memory and just get dropped if not picked up in time).
A continuous wrapper would reduce hourly restic backup times from ~30 minutes runtime to ~3 minutes. An greatly reduce the hourly I/O operations on the filesystem.
Has this been done already? Anyone seen a generic inotify wrapper that could do this or be adapted to do this?
I guess a shell script with inotifywait could do it for local filesystems.

Have you looked at lsyncd for the continuous monitoring by any chance? lsyncd is designed to watch the filesystem for changes, then call a sync process (supports direct copy, rsync, and ssh+rsync and supports custom backends.
I've been using it to make near-live copies of critical data to off site locations for years. The more I think about it, the more I think this could be what gets me off CrashPlan.

@OmenWild yep took a look at that. It looks like it work work. However there a two huge flies in my inotify soup.

It was pointed out to me that the file list each restic snapshot will just be the files accumulated in the inotify list. When you come to restore you'll have all these partial file lists, rather than point-in-time snapshots.

Network filesystems like NFS have no inotify/fsevents like service, so you have to be able to run e.g. lsyncd directly on the NFS servers - which is sometimes not possible, e.g. for AWS EFS.

What would be even better would be a "continuous backup" strategy, where backups are not made of specific "snapshots" necessarily but just consist of a continuous stream of all changes done to the disk. Very few backup programs work this way (for obvious performance reasons), but some manage to make this meaningfully and it's pretty powerful.
I thought this would be basically impossible forever on Linux, but a LWN article about Changed-block tracking and differential backups in QEMU changed my mind: what they do is they use the network block device (NBD) protocol to track changes (remotely) and do magic things on top of that.
I'm just brainstorming here (and I'm coming from a longer conversation on the same topic in borg, see https://github.com/borgbackup/borg/issues/325) so obviously there's no clear path there from here. But since it's the first time I have the slightest clue of how this could work, I figured I would share it with you folks as well. :)

@anarcat I don't think this sort of VM-specific block level backups are useful for what Restic does.
Restic would require one or more APIs which allow user space to be notified of changes to filesystem entries.
It might be worth one of the Restic developers asking on the Linux-Fsdevel mailing list about this, since it looks like the fanotify infrastructure is currently under active development by @amir73il and others.

This https://github.com/fsnotify/fsnotify/issues/114 provides a summary of the position as of 1 year ago i.e. this appears to be possible using fanotify, with the caveat that events are only available for an entire filesystem, and the filtering (e.g. if only part of the filesystem was backed up) would need to be carried out in user-space - the project linked is a go library, and so might also be relevant to this project.

@tim-seoss your understanding of what fanotify provides in kernel v5.9 seems correct.
I should also point out https://facebook.github.io/watchman/ as a service that collects fs notification (on all the platforms that Restic supports) for subscribers to read the list of changed files periodically.

@amir73il Thanks for your comments!  At the time of writing, the Linux manual page for fanotify says:
Currently, only a limited set of events is supported. In particular, there is no support for create, delete, and move events. - https://git.kernel.org/pub/scm/docs/man-pages/man-pages.git/tree/man7/fanotify.7#n32
Is the man page lagging behind improvements in the kernel's functionality?  Is the man page the preferred place to look for fanotify documentation?

There's a lot of talk about detecting file changes. That's probably the easiest part, what's arguably more important is how this would work with the way restic is currently designed on the repository side of things.

@tim-seoss This sentence in man page is stale. Thanks for pointing out. I will send a patch to remove it.
Otherwise, the man page is uptodate. See FAN_CREATE, FAN_DELETE, FAN_MOVE and the fanotify_fid.c example.

I should also point out https://facebook.github.io/watchman/ as a service that collects fs notification (on all the platforms that Restic supports) for subscribers to read the list of changed files periodically.

There are about half a billion (give or take a billion ;) such programs, and they mostly all rely on the same kernel primitives. I counted 14 here and I keep finding new ones...
that said:

There's a lot of talk about detecting file changes. That's probably the easiest part, what's arguably more important is how this would work with the way restic is currently designed on the repository side of things.

That. The problem is not waking up on changes: you could probably find some obscure Linux API to get a ping with every write on the filesystem. The problem is streaming that meaningfully into a backup system, which is why I raised the question of "continuous backups".
If that's out of scope for this issue, someone just needs to come up with a quick tutorial on how to use one of the aforementioned tools (inotify, watchman, whatever) and call restic on every change. But that's not continuous backups, and could be very hard to scale in restic considering how slow purges are with large numbers of snapshots...

The minimum useful functionality (which should also be reasonably easy to implement, and is useful for large seldom-changed data sets) would be a daemon that only triggered a restic backup if changes had been made to the filesystem.
This would be a useful addition, and wouldn't require any changes to restic.
Without knowing the restic internals in detail, it seems like a possible next step would be to modify to add an option to restic to enable it to accept list (e.g. via an inherited fd) of files which have change (i.e. skip the normal filesystem tree walk, and use the supplied list of paths instead).

@tim-seoss You can already do this outside of restic. Use some tool that records relevant file changes. Have it build a list of files to back up and run restic with the --files-from - or the next-release --files-from-verbatim or --files-from-raw equivalents. There's no need to put this into restic, especially considering how much platforms and operating systems differ on the detection part.

For a long time this functionality was blocked on missing kernel features (at least on Linux). This is no longer the case.
In the 3 years that this bug has been open, both Restic and external tools have gained functionality.
A reasonable resolution to this bug may now be to document (and if appropriate create wrapper scripts etc.) to combine external software with Restic.

@tim-seoss You can already do this outside of restic. Use some tool that records relevant file changes. Have it build a list of files to back up and run restic with the --files-from - or the next-release --files-from-verbatim - or --files-from-raw - equivalents. There's no need to put this into restic, especially considering how much platforms and operating systems differ on the detection part.

I originally thought these options might work, but they don't work on restore. The snapshot is then only for the files listed, so you can't restore using these snapshots, since the file list doesn't include the unchanged files from the previous snapshot?

In order to implement this as a wrapper around restic, restic probably needs the feature to manually pass a list of files that has been changed relative to the parent snapshot. When using this feature of restic backup, it's probably a good idea to make the explicit --parent flag mandatory, to make sure the wrapper software and restic agree on the parent snapshot.

@aawsome any plans to implement this in rustic?

When working with zfs/btrfs snapshots, a list of files that was changed (instead of using inotify) could also be obtained using tools that report the diff between two snapshots.

Not sure this is relevant but fyi https://github.com/inotify-tools/inotify-tools/pull/134 fsnotifywatch can now be used as a standalone watch program

@alphapapa check if my implementation in #4469 suits your needs.

Any progress on this topic?
I see a lot of potential in combining inotify and restic for very fast incremental backups of large filesystems (thousands of files that rarely change). Besides efficiency, when the backup is fast, it can be combined with fsfreeze to have consistent point-in-time backups of live systems - as an alternative for filesystems/block devices that do not natively support snapshots. Currently it can take minutes to scan a large filesystem, meaning that fsfreeze is not an option.


Added support for AIX

Added missing checks for AIX

Cannot merge because of conflicts but added them manually.
see 3e722331a64a15cca6dafb0f8c722510efed90c6


fix(api): the route 'get-payment-by-id' is in singular change to plural

fix(api): the route 'get-payment-by-id' is in singular. So I am changing to plural #234

I am not sure about these changes. @sinedied does this look good to you? Is switching from payment/** to payments/** break the payment API call?

@glaucia86 will you further update the OpenAPI .yaml specification in order to merge this?

@glaucia86 will you further update the OpenAPI .yaml specification in order to merge this?

Hi, @anfibiacreativa I can update it! Let me see here and I will keep you update!

@glaucia86 will you further update the OpenAPI .yaml specification in order to merge this?

Hi, @anfibiacreativa I can update it! Let me see here and I will keep you update!

@anfibiacreativa
It seems that @sinedied made this change in the openAPI.yaml file 8 months ago, specifying "payment" as plural, as shown in the image below.

Thank you @glaucia86


Fine tune stereo depth settings

Enabling census transform mean mode and alpha=0, beta=2 census transform parameters improves the overall disparity map by improving the descriptors generated by census transform.
Valid data, a.k.a. fill level (== % of values that are not invalid) of disparity map increases by ~20%.
400p:
Before:

After:

800p:
Before:

After:

There is more in census transform settings to squeeze out that I'm looking into.
CC: @saching13 for testing with your setup for depth measurement.

CC: @saching13
I think we should compare these changes to some close to ground by feeding the datasets through it like: http://www.cvlibs.net/datasets/kitti/, just to be able to see how this affects the accuracy, etc...

actually that's a great idea. Will do it.

Is there a Python version of this I can run or should I get C++ up/running?  Sorry if dumb question.

Is there a Python version of this I can run or should I get C++ up/running? Sorry if dumb question.

The images were taken actually with a python script, I just happened to add them to core PR, because core contains the proposed changes.
Install the following library:
python3 -m pip install --user --extra-index-url https://artifacts.luxonis.com/artifactory/luxonis-python-snapshot-local "depthai==<IP_ADDRESS>.dev+3fdf376f871d1144e6f677240050211a897d0ef5"

Use the following branch temp_depth_quality_test
https://github.com/luxonis/depthai-python/tree/temp_depth_quality_test/examples
They are temporary changes added to spatial_mobilenet.py + spatial_location_calculator.py, since we should keep them simple, adding all the configs to them is too much.

Thanks!  So @VanDavv - we should definitely get this slider functionality into some advanced UI in depthai_demo.py.  What would be ideal is to have an advanced checkbox that opens up a window with all these controlls.
Thoughts?


fix: handle SIGINT/SIGTERM for the start command

What kind of change does this PR introduce?
This PR introduces a graceful shutdown for SIGINT/SIGTERM signals for the start command.
What is the current behavior?
Not implemented yet
What is the new behavior?
Handling SIGINT/SIGTERM signals.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 9 (0.0%)  changed or added relevant lines in 1 file are covered.
1 unchanged line in 1 file lost coverage.
Overall coverage decreased (-0.05%) to 27.087%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

internal/start/start.go
0
9
0.0%

Files with Coverage Reduction
New Missed Lines
%

internal/start/start.go
1
0%

Totals

Change from base Build<PHONE_NUMBER>:
-0.05%

Covered Lines:
1369

Relevant Lines:
5054

💛  - Coveralls

Hi @PaulRosset , apologies for getting back to you late on this PR.
The bubbletea terminal already handles signals. What's missing is to propagate cancellation to the command context. I've updated your PR to handle this more consistently https://github.com/supabase/cli/blob/main/internal/db/remote/commit/commit.go#L57.