Storing multiple files of data

Thinking about this primarily in terms of Dask Arrays (after discussion with a colleague), but maybe this applies elsewhere as well.
A typical use case we encounter is a user has a directory full of files (usually TIFFs, but sometimes other things). They would like to load this data into a single Dask Array, which they would like to perform some operations on. At the end of this, they would like to spit out another directory of files (again usually TIFFs, but sometimes other things).
The current recommendation with loading is a user can call functions wrapped with delayed and convert those into arrays using known metadata, which they can then run stack and concatenate to their hearts content. A nice example of this procedure is in issue ( https://github.com/dask/dask/issues/672 ). Though hopefully these days users just use block. There are some things like dask-imread that do this for users. When it finally comes to storing data, the user is encouraged to use store, which works fine for things that support a NumPy-like interface.
However when it comes to storing Dask Arrays, there are only a handful of things that could store a large array via a NumPy-like interface in the way that Dask would like to. If the storage format you have in mind (normally determined by what else it needs to be used with) does not fit this interface, you're stuck. In particular some cases are better resolved by writing out a bunch of small files (in our case TIFFs work well). Certainly one could home brew a NumPy-like interface that allows this, but this seems unnecessary since the chunks that Dask Array already operates on are perfect candidates for being mapped one-to-one to files on disk. With such a mapping one is also guaranteed parallel writes, which is desirable combination with Dask. In fact users are already hacking things like map_blocks to get this behavior (even though that is not their intend use).
So how could we make this better for users. One thought would be an operation to convert Dask Array chunks into proper Dask Arrays as envisioned in issue ( https://github.com/dask/dask/issues/3274 ). This would allow one to use store semantics with chunks and individual files. Though there would still be some home brewing to use the NumPy API for each chunk. Not to mention users would need to come up with a naming scheme. Another thought would be to simply implement an improved map_blocks interface for this intent called store_blocks. This would handle optimization of a Dask Array and take a function that performs the storage operation. The function could take a block_id/chunk_id parameter to help with file naming. There may be other options I have not thought of that would work as well. Feedback welcome. :)

cc @d-v-b

Perhaps there is an API within skimage like imsave that we could replicate within a dask-image project?  The API within dd.to_csv('foo.*.csv') or db.to_textfiles('foo.*.txt') might be worth investigating.

Coming to Dask from Spark, my first intuition is to look for an apply_blocks() method on a Dask Array (like RDD.foreach() in PySpark) which allows the user to generate side effects, including but not limited to serializing data to disk. Side effects can be useful for debugging  — e.g., dumping a str(block.dtype()) to disk, or just having each worker run print(block.dtype()).
Since I can't guarantee that I'm always going to use the same format for my data, I'm a little leery of methods that assume a data format, e.g. to_$FORMAT(); presumably, if there are a lot of these methods, they are using the same API under the hood, and that's the API I'd like to work with directly.
With this generality in mind, when I spoke to @jakirkham offline I suggested that Dask Arrays have an apply_blocks() method that would apply a user-supplied function per block; map_blocks() would then be a special case of apply_blocks() where the mapped function must return a Dask Array. But there might be strong arguments against this that I don't immediately see.

I have no objection to having an operation like map_blocks (or some modified variant like apply_blocks) become more robust to not returning numpy arrays. We will have to consider what that function returns though.  A delayed object?  A dask.bag?

Maybe a dask array, each of whose chunks is a single-element numpy array that contains a single element with object dtype?

For the general case, why is a return value necessary at all?

As the action would not have occurred yet. The return value would depend a bit on scope.
With something like store_blocks described in the OP, a Delayed object would be sufficient if not desirable in my view (please feel free to correct me though). It can return None (i.e. not return) when computed. There is no real need to serialize values from the workers or collect them. It's pretty clear from the outside that it has side effects. Also the Delayed object provides a nice handle on the proposed computation, which can be submitted for computation, included in progress bars, etc.
The apply_blocks use case described above seems much more like map_blocks. In which case one does want it to return things and those things don't really have side effects (e.g. block limited reductions). If there are things that deviate from this description, would be interesting in hearing about them.
As a side note, since print came up, am thinking of that as calling map_blocks, which returns an object array of strings. On this object we can compute part of it or all of it and print whatever is needed. Though would suggest caution with this as we could swamp the buffer if we try to print too much at once.
Does dask.bag work well with distributed? Haven't explored this much personally so am curious. Also these are unordered, right? If so, then an object array does sound desirable.

Also cc-ing @djhoese in case this is of interest.

Thanks for the mention @jakirkham. This is interesting since I've been brainstorming visualization methods for my SciPy talk coming up for showing satellite imagery as it is computed. In my quick test that produced an HTML table of PNG images in a notebook, it was as you described a hacky array-like class that created a new PNG image for each block.
This also makes me think of https://github.com/dask/dask/issues/3255 where instead of a single file being created for each block you want the dask graph node (task?) to know whether or not it needs to close the file (multithreaded versus multiprocess). As mentioned in that issue we also use a wrapper class to provide an array-like interface to open a file once and use __setitem__ for writing the individual blocks using rasterio's block writing (https://github.com/pytroll/trollimage/blob/master/trollimage/xrimage.py#L73-L94).
Lasty, just for reference, a colleague (@mraspaud) implemented a special type of geotiff reader by creating a dask graph for opening and slicing the array loaded with GDAL (https://github.com/pytroll/satpy/blob/master/satpy/readers/safe_sar_c.py#L294-L305).

Is there anything concrete to do here?  Discussion seems to have stalled?

This use case is pretty important to a lot of people. So we should either keep it open or break out a few concrete suggestions from here into different discrete and actionable issues. I've tried to do the latter.
IMHO there are two suggestions that stick out to me as very important. Namely @d-v-b's apply_blocks suggestion ( https://github.com/dask/dask/issues/4148 ) and @djhoese's (reasonable) complaint about __setitem__ ( https://github.com/dask/dask/issues/4149 ). I've broken these out into two issue that I think are actionable and address the relevant concerns. Please feel free to add more to those.
If there are other issues outside of those, please feel free to raise them and xref them from here.


additional headers for real user IP

Q
A

Is bugfix?
yes/no

New feature?
yes/no

Breaks BC?
yes/no

Tests pass?
yes/no

Fixed issues
comma-separated list of tickets # fixed by the PR, if any

I didn't close this PR, but I opened new PR and this closed automatically

seems you removed the commits from the branch. You should have a new branch for every pull request.

Need I create new PR for this changes? Is it useful?

I think it is useful but a few changes need to be made, you can either create a new PR or create an issue describing what you want.


Help needed in applying isomorphic-relay

@denvned, thank you for your awesome work in the field of SSR in Relay :+1:
I tried to adapt your solution into my open-source project. My project consist of two repos:

server side: https://github.com/tommyJS/backend-for-react-relay-redux-example
frontend side: https://github.com/tommyJS/react-relay-redux-example/tree/dev

Unfortunately, when I run the project and hit localhost:3000, a page is loading infinitely. I run the project in the following way:

npm install
npm run build
npm run start

... and I don't have any errors. I tried to debug the app, however, with no avail.
If you could check what is wrong and what might the root cause of infinite loading, I would be very grateful. I assume that it is my fault, however, it is quite hard to me to spot it.

Can't open the links as the user's account have been deleted.


Add slugs to post URL's

For SEO, it's nice to have the post title in the URL.  This PR changes the canonical route for a post to be:
/posts/:_id/p/:slug

The "/p/" part is so that a post with slug "edit" doesn't conflict with /posts/:_id/edit.  Routes to /posts/:_id redirect to include the slug; though the index pages at least are updated to point directly to the slugified versions.

Just make it easy for everybody to configure their own slugify function and their route parser that return the id in some way.
Any idea if somebody is going to work on this?

It's certainly still on the roadmap, but I don't think anybody's working on it right now.

Hey, folks! Telescope has changed a lot in the past months, and there is a new React-based version, Telescope Nova (intro video). If you know Crater, it runs on the latest version.
The Legacy version is no longer supported. I'm closing this PR.
I invite you to check the new version, it's awesome!
Happy hacking! 🚀
Ps: by the way, don't hesitate to join the Slack to ask any question, you are welcome! 👍


ENH: fill missing variables during concat by reindexing

[x] Closes #508,
[x] Closes #3545
[x] Tests added
[x] User visible changes (including notable bug fixes) are documented in whats-new.rst
[ ] New functions/methods are listed in api.rst

This is another attempt to solve #508. Took inspiration from #3545 by @scottcha.
This follows along @dcherian's comment in the same above PR (https://github.com/pydata/xarray/pull/3545#issuecomment-586346637).
Update:
After review the variable order is estimated by order of appearance in the list of datasets. That keeps full backwards compatibility and is deterministic. Thanks @dcherian and @keewis for the suggestions.

There are no tests added currently. I'm just wondering, if that approach would work in general.
The current assumptions here:

works only on data variables, not coordinates
order is taken from first dataset, variables missing in first dataset are appended at the end
missing variables are filled with fill_value by reindexing (thanks @dcherian for the inspiration).

Thanks @Illviljan for activating the benchmark runs. Are those errors related to the changes? I'm not up to date with mypy, are these errors induced by changes here?

Benchmark is a numba issue, probably #7306.
Mypy is real, cannot getitem a object. Try out using isinstance instead of the try/except to narrow the typing.

OK, green light's now also on mypy. Looks like the approach would work in general. Trying to add some tests now.

This is ready for a first round of review. Thanks!

Hi, thanks for doing this @kmuehlbauer .  FWIW I'm no longer seeing the issue I was previously seeing when I submitted https://github.com/pydata/xarray/pull/3545 when I just run with released xarray v2022.12.0 (I haven't gone back further to see when the issue started going away so I'm not really sure if the old error has just been suppressed or if the single case I was seeing back then was resolved in a previous PR--or there is also a chance there is something which changed in my data over that long time period).
That being said I also applied this PR to my workflow and reran the concat code and it continues to pass correctly with this PR from what I've seen so far.
@kmuehlbauer  did you see the tests I created here? https://github.com/pydata/xarray/blob/03f9b3b85aee039f47dd693322492ab09f57fb73/xarray/tests/test_concat.py
Not all of them got to a passing state but there were several cases I tried to document with the tests there.

Thanks @scottcha for taking the time to testing things.
I'll have a look at your tests in more detail now. I've concentrated on my use case in the first place and hoped to get away with it :grinning:.

Some typing suggestions.

Thanks @Illviljan, your suggestions and help is much appreciated.

@scottcha I've found a glitch in the code due to your tests. Already pushed the changes here.
I'm going to cherry pick your tests here next.

@scottcha I've tried to cherry-pick, but ended up copy/pasting and adding your authorship to the commit.
I think the final problem is the order in:

test_concat_missing_multiple_consecutive_var
test_multiple_datasets_with_multiple_missing_variables

These tests are flaky. Sometimes the order is correct and sometimes not. Can't immediately see the root cause here.
@Illviljan I'll try to add typing to these additional tests. Should be good for learning that.

@Illviljan OK, I'm stuck now. I can't make anything out of the remaining mypy errors. Would be great if you could have another look here, thanks!

@scottcha I think I've managed to get along with your tests. It looks like everything is running now.
One thing which is still unresolved:

The order of data variables which are not available in the first dataset is not deterministic because of using set for gathering all variables. But maybe that can be neglected for now.

@Illviljan @dcherian This is ready for another round of review. Thanks for considering.

I was hoping to gain something by merging the variable order code with _parse_datasets, to only have to traverse the datasets once.
The current behaviour, and the best I've come up so far in terms of performance:

count number of variables while iterating datasets (_parse_datasets)
check if first dataset contains all wanted variables
2a. if that's the case, take the order from first dataset
check if the dataset with max count variables contains all wanted variables
3a. if that's the case, take the order from that dataset
if not 2a or 3a, take order from first dataset and append missing variables to the end

Finally, this is as far I could get with it. I'll leave it as is now. Looking forward for reviews and suggestions. Thanks @Illviljan for the great support!

Thanks for bringing benchmarks back to life here @Illviljan. It's good to see that the changes from this PR do not affect the performance AFAICT.
It might make sense to add a benchmark for concatenation a large number of datasets with a large number of variables (with and without missing variables in some datasets) to check for performance of those special cases.

Gentle ping @shoyer, as original issue author, and @dcherian for feedback. Any suggestions for improvement much appreciated. Thanks!
@Illviljan I'd prefer to handle benchmark in subsequent PR. Thanks for the links!

Thanks Kai. I can take a look early next week. Sorry for the delay here, I just got back from a long vacation

Thanks Deepak, I've already have some answers on your review. Will check the other things the next days.

@dcherian @keewis The resulting variables are now in order of appearance as suggested. This is backwards compatible and deterministic.
Needs decision:

convert fill_value to dict before calling align does not work
not sure how to use xarray.core.alignment.reindex_variables for reindexing

I pushed a commit. Turns out reindex_variables also handles dict & scalar fill_value so it fixes both :)

@dcherian rebased on latest main and fixed whats-new.rst. Should be good for another review.

@dcherian There slipped an old item from whats-new.rst back into. I've removed it. Should be OK now.
Great to see this functionality coming to next xarray version.


need onnx model

hi, i tried to transform the .pth model you offered  into .onnx model, but the test result is bad. so，if can you offer a .onnx model or pth2onnx.py? thank you

Sorry, I do not have the experience working on .onnx model and we have no plan to do this.

hi, i tried to transform the .pth model you offered into .onnx model, but the test result is bad. so，if can you offer a .onnx model or pth2onnx.py? thank you

ok

i have solved it， so  happy!

hi, i tried to transform the .pth model you offered into .onnx model, but the test result is bad. so，if can you offer a .onnx model or pth2onnx.py? thank you

I have solved this problem to get correct result, so happy!

hi, i tried to transform the .pth model you offered into .onnx model, but the test result is bad. so，if can you offer a .onnx model or pth2onnx.py? thank you

I have solved this problem to get correct result, so happy!

hi, i tried to transform the .pth model you offered into .onnx model, but the test result is bad. so，if can you offer a .onnx model or pth2onnx.py? thank you

congrats!

I have solved this problem to get correct result, so happy!

hi, i tried to transform the .pth model you offered into .onnx model, but the test result is bad. so，if can you offer a .onnx model or pth2onnx.py? thank you

Seem problem, could you please share your solution or .onnx model？tks

Can you provide Onnx program code, please, thank you


feature request: Reddit Post data type

Should be the post only, but optionally also the comments. That's the reason the web page format won't cut it.
I think for small quantities, the reddit API can be used without auth, we could leave auth up to the user via env vars.

Reddit has 70+ million daily users as of 2024 and contains incredible amount of data
Anyone interested to add this data source to embedchain?


Unexpected Handling of Duplicate Column Names in Cell Results

This is an edge case hardly worth mentioning, but I accidentally created a select query with duplicate column label in the results. Interestingly, the SQL Notebook cell results only showed one column, the one with the right-most value.  But it was still better than the results displayed by the SQLTools VS Add-In I used until I discovered your gem.   For comparison, the attached has a simple example with display output of both compared to MySQL Workbench.  Mostly thought you would be interested in knowing, irrespective of whether you think it's worth your time or energy.  Regardless, thanks for creating this great tool and best wishes, T

Thanks for this report and providing the detailed comparison. This is interesting indeed. I think we should be able to solve this by using the rowsAsArray: true option in the mysql driver to prevent key collisions for duplicate column names.
https://github.com/sidorares/node-mysql2/blob/ab3feb75bb4af1f6d288c0494a99aa2483ac24d5/index.d.ts#L173


@kanaabe => [Minor] Add 'overflow_fillwidth' support to embeds

In Article1, embeds had a layout over 'overflow', where all other containers use 'overflow_fillwidth'.  This adds 'overflow_fillwidth' to css for embeds, in advance of a backfill on embed layouts.

Thank you! I will follow up with the embed backfill today or tomorrow 👍


Email autocomplete

Email autocomplete as a prevention from typos. Implemented with email-autocomplete.
In case of positive feedback I can implement it also for frontend registration form.

This might be better suited as a type: email backend form field in October. Thoughts?

Yes, definitely :-) I will try to look at it

Closing as it has been over a month since any activity on this occurred and we are trying to figure out what issues are still relevant. If this is still something that you would like to see through to fruition please respond and we can get the ball rolling.


Should support truly prepared statement?

@mp911de I have a problem for prepared statement in MySQL.
I noticed that even mysql-connector-j is actually implemented using a text query statement when implementing a prepare statement. In other words, it will replace the illegal characters and directly spliced into the SQL statement.
If need to implement a truly prepared statement, will encounter the following challenges:

MySQL's official library, at least mysql-connector-j can not give any help, can only read the API documentation with less description
mysql-connector-j is doing very well in all production environments. (OK, let's forget the unpleasant things...)
Sequence id makes concurrent requests difficult, because truly prepared statement has multi frontend messages, and each message should wait last message response before it send (because the sequence id always needs to be increased whether message is inbound or outbound, until the OK message or error message is returned).

If we always use simple text query, it only has single request message which sequence id is 0 (ignore big data), and multi text result packets which sequence id from 1 to X (last packet must be OK or error), so we can make sure every request packet's sequence id is increment from 0. Then we could send multi-request without any wait.
The exit message is obviously a very important message that needs to be sent immediately. If need to wait for the last response, it may lead to some unpredictable problems.

Below is the code snippet and Socket fragment I intercepted:
try (Connection connection = dataSource.getConnection()) {
    try (PreparedStatement statement = connection.prepareStatement("SELECT COUNT(0) FROM work_base WHERE `name` = ?")) {
        statement.setString(1, "' OR 1 = 1 OR '' = '");
        statement.executeQuery();

        statement.setString(1, "' OR 0 = 0 OR '' = '");
        statement.executeQuery();
    }
}

Socket packets:
0000   49 00 00 00 03 53 45 4c 45 43 54 20 43 4f 55 4e   I....SELECT COUN
0010   54 28 30 29 20 46 52 4f 4d 20 77 6f 72 6b 5f 62   T(0) FROM work_b
0020   61 73 65 20 57 48 45 52 45 20 60 6e 61 6d 65 60   ase WHERE `name`
0030   20 3d 20 27 5c 27 20 4f 52 20 31 20 3d 20 31 20    = '\' OR 1 = 1
0040   4f 52 20 5c 27 5c 27 20 3d 20 5c 27 27            OR \'\' = \''
...
0000   49 00 00 00 03 53 45 4c 45 43 54 20 43 4f 55 4e   I....SELECT COUN
0010   54 28 30 29 20 46 52 4f 4d 20 77 6f 72 6b 5f 62   T(0) FROM work_b
0020   61 73 65 20 57 48 45 52 45 20 60 6e 61 6d 65 60   ase WHERE `name`
0030   20 3d 20 27 5c 27 20 4f 52 20 30 20 3d 20 30 20    = '\' OR 0 = 0
0040   4f 52 20 5c 27 5c 27 20 3d 20 5c 27 27            OR \'\' = \''

I guess this is why jasync can't send a request when exchanging, because it use truly prepared statement.

Appreciate to hear that you want to tackle such a complex issue. I'm not entirely sure I understood the message flow. Let me tell you what we have for SQL Server:
In SQL Server we can either send a statement directly or use cursors. A cursored message flow using parametrized statements requires prepared statement call (it either prepares or reuses a prepared statement), then we fetch data from the cursor and close the cursor after consuming all data.
The message flow is synchronous: We need to issue a command and await the response before we can continue with the next (read from cursor) message. Same for read from cursor response. We need to await that one before closing the cursor.
Take a look at RpcQueryMessageFlow.exchange() and its handleMessage(…) method that considers the message flow. Probably that's something you're looking for.
In R2DBC, a single connection can be used only by a single conversation. We cannot (should not) expect to issue multiple commands on a single connection because that's not the way databases work.
In SQL Server we also have the concept of a sequence Id (it's called PacketId, but sounds basically the same to me). I implemented it as PacketIdProvider that is bound to a connection. This makes it simple to atomically increment a counter. With the assumption that a connection is used synchronously by a single subscriber, we exclude the concurrency aspect.
Does this help?

r2dbc-mssql's packet id implementation is very useful for me, thanks very much.
The answer is yes, this driver should support truly prepared statement.


[Event Request] Codeunit 95 "Purch Post Batch via Job Queue" PrintToPDF

For the extensibility purpose it would be required to have an event for substituting PDF generating in the PrintToPDF procedure.
similar to #20487
    local procedure PrintToPDF(ReportId: Integer; RecRef: RecordRef; PurchaseHeader: Record "Purchase Header"; JobQueueEntry: Record "Job Queue Entry"): Boolean
    var
        ReportInbox: Record "Report Inbox";
        OStream: OutStream;
    // new code.start
        SuccessfulRun: Boolean;
        IsHandled: Boolean;
    // new code.stop
    begin
        ReportInbox.Init();
        ReportInbox."User ID" := JobQueueEntry."User ID";
        ReportInbox."Job Queue Log Entry ID" := JobQueueEntry.ID;
        ReportInbox."Report ID" := ReportID;
        ReportInbox.Description := CopyStr(StrSubstNo(PrintingDescriptionTxt, PurchaseHeader."Document Type", PurchaseHeader."No."), 1, MaxStrLen(ReportInbox.Description));
        ReportInbox."Report Output".CreateOutStream(OStream);
        // old code.start
        // if not Report.SaveAs(ReportId, '', ReportFormat::Pdf, OStream, RecRef) then
        //     exit(false);
        // old code.stop
        // new code.start
        OnPrintToPDFOnBeforeReportRun(ReportId, RecRef, OStream, SuccessfulRun, IsHandled);
        if not IsHandled then
            SuccessfulRun := Report.SaveAs(ReportId, '', ReportFormat::Pdf, OStream, RecRef);
        if not SuccessfulRun then
            exit(false);
        // new code.stop
        ReportInbox."Created Date-Time" := RoundDateTime(CurrentDateTime, 60000);
        ReportInbox.Insert(true);
        OnPrintToPDFOnAfterReportInboxInsert(ReportInbox, PurchaseHeader, RecRef);

        exit(true);
    end;

    // new code.start
    [IntegrationEvent(false, false)]
    OnPrintToPDFOnBeforeReportRun(ReportId: Integer; RecRef: RecordRef; var OStream: OutStream; var SuccessfulRun: Boolean; var IsHandled: Boolean)
    begin
    end;
    //new code.stop

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. Please do not reply to this, as we do not monitor closed issues. If you have follow-up questions or requests, please create a new issue where you reference this one.
Build ID: 47191.


Running the Horusec tool using Docker (Using version: 27.2.1, )

What happened:
--> I did all the pre-installations before running the Horusec tool like installing Docker, then installing horusec, and started running horusec tool using Docker in the application folder.
--> But I was getting the following error "level=error msg="{HORUSEC_CLI} Your docker version is below of: " error=19.3". Even though my current version is above the required version which was mentioned in the error message.
--> This was my current Docker version while using the tool "Docker version 27.2.1, build 9e34c9b"
What you expected to happen: To start the Horusec tool and show the vulnerabilities in the application folder.
How to reproduce it (as minimally and precisely as possible):
Anything else we need to know?: N/A
Environment:

Horusec version (use horusec version): v.2.8.0
Operating System: Linux
Network plugin / Tool and version (if this is a network-related / tool bug): tool bug
Others: N/A

Hi, I dont know if it helps you, but currently I am using a beta version in order as a workaround:
image="horuszup/horusec-cli:v2.9.0-beta.3"; \
docker pull $image ;\
docker run --rm \
	-v /var/run/docker.sock:/var/run/docker.sock \
	-v $(pwd):/src/horusec $image horusec start \
	-p /src/horusec -P $(pwd)

I encountered the same problem, and I tried to use the beta version. But I still got the same error
time="2024-09-23T09:06:23Z" level=warning msg="{HORUSEC_CLI} Config file not found"
Error: docker not found. Please check and try again
time="2024-09-23T09:06:24Z" level=error msg="{HORUSEC_CLI} Your docker version is below of: " error=19.3
Usage:
  horusec start [flags]

Examples:
horusec start

Flags:
  -t, --analysis-timeout int                 The timeout threshold for the Horusec CLI wait for the analysis to complete. The minimum time is 10 (default 600)
  -a, --authorization string                 Authorization token to use on Horusec server. Read more: https://docs.horusec.io/docs/tutorials/how-to-create-an-authorization-token (default "00000000-0000-0000-0000-000000000000")
  -C, --certificate-path string              Path to certificate of authority. Example -C="example/ca.crt"
  -P, --container-bind-project-path string   Project path in host to be used on Docker when running Horusec inside a container
  -c, --custom-rules-path string             Path with custom rules that should be used by Horusec engine
  -D, --disable-docker                       Run Horusec without docker. If enabled it will only run the following tools: horusec-csharp, horusec-kotlin, horusec-java, horusec-kubernetes, horusec-leaks, horusec-javascript, horusec-dart, horusec-nginx
  -G, --enable-commit-author                 Enable to search commit author of vulnerabilities
      --enable-git-history                   Run Gitleaks and search for vulnerabilities in all git history of the project https://github.com/zricethezav/gitleaks
  -w, --enable-owasp-dependency-check        Run Owasp Dependency Check tool https://github.com/jeremylong/DependencyCheck
  -j, --enable-shellcheck                    Run ShellCheck tool https://github.com/koalaman/shellcheck
  -F, --false-positive strings               Ignore a vulnerability by hash and set it to be false positive. Example -F="hash1, hash2"
      --headers stringToString               Custom headers to send on request to Horusec API. Example --headers='{"X-Auth-Service": "value"}' (default [])
  -h, --help                                 help for start
  -u, --horusec-url string                   The Horusec server address to send analysis results (default "http://<IP_ADDRESS>:8000")
  -i, --ignore strings                       Paths to ignore in the analysis. Example: -i="/path/to/ignore, **/*_test.go, **/assets/**" (default [*tmp*,**/.vscode/**])
  -s, --ignore-severity strings              The level of vulnerabilities to ignore in the output ("LOW"|"MEDIUM"|"HIGH"). Example: -s="LOW, HIGH" (default [INFO])
  -I, --information-severity                 Enable information severity vulnerabilities. Information vulnerabilities can contain a lot of false positives
  -S, --insecure-skip-verify                 Disable the certification validation. PLEASE, try not to use it
  -O, --json-output-file string              Output file to write analysis result. This flag should be used with --output-format
  -o, --output-format string                 Output format of analysis ("text"|"json"|"sarif"|"sonarqube"). For json, sarif, and sonarqube --json-output-file is required
  -p, --project-path string                  Path to run an analysis. If this value is not passed, Horusec will ask if you want to run the analysis in the current directory (default "/")
  -n, --repository-name string               Send repository name to Horusec server, by default sends the actual directory name (default "/")
  -r, --request-timeout int                  The timeout threshold for the request to the Horusec server. The minimum time is 10 (default 300)
  -e, --return-error                         Return exit code 1 if found vulnerabilities. Default value is false (exit code 0)
  -R, --risk-accept strings                  Ignore a vulnerability by hash and set it to be risk accept. Example -R="hash1, hash2"
      --show-vulnerabilities-types strings   Show vulnerabilities by types ("Vulnerability"|"Risk Accepted"|"False Positive"|"Corrected"). Example --show-vulnerabilities-types="Vulnerability, Risk Accepted" (default [Vulnerability])

Global Flags:
      --config-file-path string   Path of the configuration file (default "/horusec-config.json")
  -l, --log-file-path string      Path of log file (default "/tmp/horusec-2024-09-23-09-06-23.log")
      --log-level string          Set log level ("panic"|"fatal"|"error"|"warn"|"info"|"debug"|"trace") (default "info")


jitable: do not write to potentially existing folder

I get the below error message when calling .jitable(). The reason is that a different user is also running pytorch geometric - I cannot write files in their directory.
There are at least three solutions: 1) not specify a folder and let NamedTemporaryFile just write the file, 2) create a randomly named folder (and hope it doesn't exist), 3) do no specify a folder but provide a more meaningful suffix/prefix (e.g., _pyg_jit.py).
I'm happy to change this PR - currently it uses option 1).
~/.cache/pypoetry/virtualenvs/python-tools-XWCSm5JO-py3.8/lib/python3.8/site-packages/torch_geometric/nn/conv/utils/jit.py in class_from_module_repr(cls_name, module_repr)
      9     path = osp.join('/tmp', 'pyg_jit')
     10     makedirs(path)
---> 11     with TempFile(mode='w+', suffix='.py', delete=False, dir=path) as f:
     12         f.write(module_repr)
     13     spec = spec_from_file_location(cls_name, f.name)

~/local/lib/python3.8/tempfile.py in NamedTemporaryFile(mode, buffering, encoding, newline, suffix, prefix, dir, delete, errors)
    538         flags |= _os.O_TEMPORARY
    539
--> 540     (fd, name) = _mkstemp_inner(dir, prefix, suffix, flags, output_type)
    541     try:
    542         file = _io.open(fd, mode, buffering=buffering,

~/local/lib/python3.8/tempfile.py in _mkstemp_inner(dir, pre, suf, flags, output_type)
    248         _sys.audit("tempfile.mkstemp", file)
    249         try:
--> 250             fd = _os.open(file, flags, 0o600)
    251         except FileExistsError:
    252             continue    # try again

PermissionError: [Errno 13] Permission denied: '/tmp/pyg_jit/tmpii_e2lzy.py'

I see, I didn't think about multiple users trying to write to the same folder. I personally prefer a root-folder for all temporary files (to avoid clutter), so how about we create a folder based on a users' name, e.g., rusty1s_pyg_jit/.

@rusty1s done

Looks good. Thank you :)


Update README and example answerfile

That for address the changes on the locales PR, right? I'm approving it.


Save correct millage after recording endet

Sometimes same millage is saved after ending 2 different trips

The correct value is saved in the database; it just isn't displayed correctly.


Added Virtual WAN Snippet

Added a snippet for Azure Virtual WAN resource.
Contributing an example

[x] I have checked that there is not an equivalent example already submitted
[x] I have resolved all warnings and errors shown by the Bicep VS Code extension
[x] I have checked that all tests are passing by running dotnet test
[x] I have consistent casing for all of my identifiers and am using camelCasing unless I have a justification to use another casing style

Thanks for the contribution @StefanIvemo! I will review once we get https://github.com/Azure/bicep/pull/2307 merged into main.


moment().format('MMMM Mo, YYYY') is always giving me "January 1st, 2015"

Hi All,
Moment is awesome!, doing some test while my client reported a bug, seems that moment().format('MMMM Mo, YYYY') is returning "January 1st, 2015" even today that is 13.
Best Regards,
Alfredo.

I was wrong, Mo means month. sorry.


refactor: use keychain encrypt

📝Changed routes:

/

(view full size)

And 1 other route:

/api/create-secret

Commit f51b6f6709d9eee90b8d95c14d138b5f12c8b650 (https://secret-git-refactor-use-keychain-encrypt.bukinoshita.now.sh).


Support other shells

Hi :) Congrats on the first issue in this repo.
I really like xplr, thank you. Moved from nnn.
So the issue is that I am using nushell as a login shell, and ofc I launch xplr using nu. xplr doesn't pick neither current shell or login shell, because bash is just hardcoded (i think xplr should only care about currently running shell)
I would like to submit PR but I

found other mentions of bash in the code
not sure how to get currently running shell (only know that $SHELL is login shell)
not familiar with lua at all

Hi :) glad to know 👍
For the interactive shell, see https://github.com/sayanarijit/xplr/blob/1441275860f39a22f30655a747128075223be296/src/init.lua#L2080
This is how I update it: https://github.com/sayanarijit/.files/blob/c9fb9d40afe509fb718e18e073d39ce35b5ded8b/nixos/files/xplr/init.lua#L240


How to use SHM

Issue template

Hardware description: docker, Ubuntu 22.04
RTOS:
Installation type:
Version or commit hash: humble

Steps to reproduce the issue
Hey, i have created a publisher and subscriber chain with micro-ROS nodes in a docker container , to analyze latencies. It works fine when i use the Agent. Now i want to use SHM. Therefore, i added "-DUCLIENT_PROFILE_SHARED_MEMORY=ON" to my colcon.meta. Do i still have to build the agent and the other steps like:
ros2 run micro_ros_setup build_firmware.sh
source install/local_setup.bash
ros2 run micro_ros_setup create_agent_ws.sh
ros2 run micro_ros_setup build_agent.sh
source install/local_setup.bash
ros2 run micro_ros_agent micro_ros_agent udp4 --port 8888 -v6

When i build, the Agent is always is used.  But when i am not launching the Agent, the launched nodes fail.
I am not using a microcontroller, does this still work? I want to see how big the latency is with and without the Agent. Furthermore, I am using Cyclone DDS for the Agent,  is this maybe the problem?
All nodes are in the same container.
Are there any modifications I forgot to do ?

Move this to a micro-ROS repo, Cyclone DDS is not compatible with eProsima's Micro XRCE-DDS


Gemma2 and Mistral-nemo not running on ollama

What is the issue?
[147@Clevo ~]$ ollama run mistral-nemo:12b-instruct-2407-fp16
Error: exception error loading model hyperparameters: invalid n_rot: 128, expected 160
[147@Clevo ~]$ ollama run gemma2:27b-instruct-q8_0
Error: exception error loading model architecture: unknown model architecture: 'gemma2'
Can someone explain why this is happening?
It is the first time models do not run on my ollama v0.1.30
OS
Linux
GPU
Nvidia
CPU
Intel
Ollama version
0.1.30

Gemma2 needs ollama 0.1.47 or newer.  minstal-nemo needs ollama 0.2.8 or newer.

Thanks Rick

Thanks Rick

If your issue is resolved please consider closing it.

Thank you


Telegram bot does not receive notification when website is down

Describe the bug
Here is my repository, (version 1.23.13). I have setup telegram notification as per documentation's instruction:
NOTIFICATION_TELEGRAM: true
NOTIFICATION_TELEGRAM_BOT_KEY : (something looks like this:) 123456:kioASDdjicOljd_ijsdf
NOTIFICATION_TELEGRAM_CHAT_ID: (something looks like this:) 23141232
Also I referred to this StackOverflow answer and have ensured that the bot is working, I can use python's requests to send message to bot and can receive the message.
To Reproduce
With this setup, I modify test if telegram can work by doing the following:

add a new broken url in .upptimerc.yml under sites
commit and wait for Update Template CI and Uptime CI in Github Actions to complete.
Wait until a new issue is created because it detected the site is not up.

Expected behavior
I expect my telegram bot will send something to me. Nothing was sent

telegram bot still  have problem!

This is now (finally) fixed, thanks so much for your patience! The fix is available in<EMAIL_ADDRESS>and the automated fix should be in your repositories within 24 hours.
You can also run Update Template CI manually to update to the latest version immediately; by default this runs once every day.


android overrideLocale won't restart app

The doc stated switch language require to restart the app but  _android.foregroundActivity.finish(); just close the app and not restarting the app. How to make it restart the app and change to the language I selected ? After I open back the app again, the language didn't change at all, still default device's language.
I want to change from chinese to english or vice-versa and able to restart the app after changing the language and display the content with the language selected.
Another question, will it be showing english after I change from chinese if I kill the app and reopen again?
P/S: My default device's language is zh-CN.
Nativescript 6.7.8
Angular 8.2.0
tns-android 6.5.0

try to call :
int main.ts file :
on(launchEvent, (args) => {     if (args.android) {         androidLaunchEventLocalizationHandler();     } });
Your function to change language programmatically (ts file. example :  example.component.ts) :
`changeLanguage(lang){
const localeOverriddenSuccessfully = overrideLocale(lang);
if (!isIOS) {
androidLaunchEventLocalizationHandler();
} else {
}

}`


dbeaver/pro#3281 Support for synonyms in semantic analyses

SELECT * FROM elizabeth.yuuy ; -- synonym for employee table

SELECT * FROM "PUBLIC".customers ; -- public synonym for table employee

SELECT * FROM "ELIZABETH".employee e WHERE rowid > 3; -- real table

 -- synonyms are not allowed for ALTER tables
ALTER TABLE "ELIZABETH".yuuy ADD COLUMN x int

-- queries to check synonyms in metadata
SELECT S.* FROM ALL_SYNONYMS S WHERE S.SYNONYM_NAME = 'CUSTOMERS';
SELECT S.* FROM ALL_SYNONYMS S WHERE S.table_owner = 'ELIZABETH';

Verified


Order the Ruby require statements and the Gems

Remove unneeded blank lines from Markdown and Ruby.

They are ordered intentionally like this, thanks.


How do I set up my.coding.github?

I did the following.

pip install --user HPI (python 3)
hpi config create

Now inside ~/.config/my, I have.
└── my
    └── config
        ├── __init__.py
        └── __pycache__
            └── __init__.cpython-38.pyc

(The __init__.py is empty and I didn't bother looking at the cache stuff).
Now I want to set up my.coding.github.  I went to MODULES.org but didn't see anything about coding.github.  Then I tried to read github.py.. I saw it pulls in the config, but then went a few steps out to .export_dir, inputs(), _dal(), ghexport and got lost.  How do I configure it?

Hi, thanks for trying it!
Yep, the config structure is fine (the cache files are just supplementary files Python creates in runtime, don't worry about them!
No worries, I can see it's confusing at the moment! I'll write clean up the module a bit and writegenerate documentation tomorrow, in the meantime you can set up and run the exporter for github data, which HPI is using as inputs.

Ok, I got it to work, sorta.

Download https://github.com/karlicoss/ghexport
Use it to create an export json file.  Place it in /home/me/data/exports/github
Use the github export button in settings to make a huge archive, then unzip in /home/me/data/exports/github_initial
Make a blank cache dir /home/me/.config/my/my/config/cache

Now set up
__init__.py
class github:
    export_dir = "/home/me/data/exports/github"
    gdpr_dir = "/home/me/data/exports/github_initial"
    cache_dir = "/home/me/.config/my/my/config/cache"

Now I can run python -m orger.modules.github to make a Github.org.
The only issue is I ended up having to point gdpr_dir to a blank directory.  Apparently something in my export makes the thing crash (I guess this could be another issue if you want).
File "/home/me/.local/lib/python3.8/site-packages/orger/modules/github.py", line 11, in get_items
    for e in gh.get_events():
  File "/home/me/.local/lib/python3.8/site-packages/my/coding/github.py", line 258, in get_events
    return sorted(iter_events(), key=lambda e: e.dt)
  File "/home/me/.local/lib/python3.8/site-packages/my/coding/github.py", line 258, in <lambda>
    return sorted(iter_events(), key=lambda e: e.dt)
AttributeError: 'RuntimeError' object has no attribute 'dt'

Right, I'm back!
Thanks for organizations example, and yeah, good job -- you figured all this out!
But I'll explain anyway:

The RuntimeError thing
It is actually an error handling technique, but I forgot to handle it properly during the sort.
The idea is that errors are propagated all the way up (instead of immediately crashing, or silently ignoring them), and downstream applications can decide what to do with them. For example, orger now displays it as an 'error' heading, so the user is aware when/if errors happen. I guess I'll add some flag to control error display in org-mode files -- naturally I end up with very few as I'm the developer, but I can imagine it might flood files for some people :)

As for the module itself: I added the docs for github modules, but will elaborate anyway:

cache_dir
You can actually keep it as None if you prefer, then it would just use /tmp. Although I'm not 100% sure this behaviour is settled -- feels like it makes more sense to mean 'caching is disabled', and to use user cache dir by default.

gdpr_dir
I added support for empty string ('') as path -- that way it will be simply ignored, so there won't be need for empty directory!

export_dir
I renamed to export_path (it's more consistent with the other modules), but the old name works too.

And in case you're curious: there are quite a few changes in https://github.com/karlicoss/HPI/pull/61 -- what I did is splitting the module into my.github.gdpr and my.github.ghexport bits. That way it's a bit more modular, makes it easier to use only one of sources if someone prefers. It's similar to the logic for twitter that I documented before. But the old my.coding.github is still there and should be backwards compatible.

Ok, great.  Thanks for the detailed explanation and I'll give the new modules a shot when they're released.


Typo in article

https://docs.microsoft.com/en-us/sccm/osd/deploy-use/create-a-task-sequence-to-upgrade-an-operating-system
Location:
Set Windows default apps and associations
Typo in step 4.
DefaultAppAssocations.xml is missing an i.
Should be
DefaultAppAssociations.xml
(admin edit: adding metadata to link to article)

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 2cfe3442-b59d-0d77-d9af-f18e787ec99d
Version Independent ID: 60cb54a2-d1fd-92b5-b027-de8a85c7af64
Content: Create an OS upgrade task sequence - Configuration Manager
Content Source: sccm/osd/deploy-use/create-a-task-sequence-to-upgrade-an-operating-system.md
Product: configuration-manager
Technology: configmgr-osd
GitHub Login: @aczechowski
Microsoft Alias: aaroncz

Thanks @janvik207, good catch!
I just submitted a fix that will publish live in about an hour with our daily publishing process.
In the future, you can submit this change yourself! For more information, see How to contribute. 😉


DocsArchive-<IP_ADDRESS>Auto archive content

Please don't merge this PR until target archive repo PR https://github.com/OPS-E2E-PPE/docs-archive-test-source-version-target/pull/204 is merged into live branch.
Auto archive content to https://github.com/opstest2/docs-archive-test-source-version-target.git

Docs Build status updates of commit 7467026:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

.openpublishing.redirection.json
:white_check_mark:Succeeded

versioningArchiveSourceFolder/TOC.yml
:white_check_mark:Succeeded
View (sql-server-2014 || azuresqldb-current || azuresqldb-mi-current || azure-sqldw-latest || >= aps-pdw-2016 || >= sql-server-2016 || >= sql-server-linux-2017 || sql-server-previousversions || sqlallproducts-allversions)

versioningArchiveSourceFolder/docfx.json
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel


Packages deprecated

I downloaded your zip project and I did run 'npm install' but a lot of packages were deprecated.
willianzeni@pop-os:~/wiregui-master$ sudo npm install
[sudo] password for willianzeni:
npm ERR! code ERESOLVE
npm ERR! ERESOLVE unable to resolve dependency tree
npm ERR!
npm ERR! While resolving<EMAIL_ADDRESS>npm ERR! Found<EMAIL_ADDRESS>npm ERR! node_modules/react
npm ERR<EMAIL_ADDRESS>from the root project
npm ERR!   peer<EMAIL_ADDRESS>from<EMAIL_ADDRESS>npm ERR!   node_modules/@chakra-ui/icons
npm ERR<EMAIL_ADDRESS>from the root project
npm ERR!   2 more (@emotion/react, @emotion/styled)
npm ERR!
npm ERR! Could not resolve dependency:
npm ERR! peer react@">=18" from<EMAIL_ADDRESS>npm ERR! node_modules/@chakra-ui/system
npm ERR!   peer<EMAIL_ADDRESS>from<EMAIL_ADDRESS>npm ERR!   node_modules/@chakra-ui/icons
npm ERR<EMAIL_ADDRESS>from the root project
npm ERR!
npm ERR! Fix the upstream dependency conflict, or retry
npm ERR! this command with --force, or --legacy-peer-deps
npm ERR! to accept an incorrect (and potentially broken) dependency resolution.
npm ERR!
npm ERR! See /root/.npm/eresolve-report.txt for a full report.
npm ERR! A complete log of this run can be found in:
npm ERR!     /root/.npm/_logs/2023-01-21T11_11_48_161Z-debug-0.log

weird, you should have been able to install the current packages, maybe it tried to update some packages because there's no package-lock.json
try installing with yarn, since we have yarn.lock it should work


Always allow Write Field editor in sidebar

When loading a record that has been deprecated, the new Spelunker GUI removes the ability to edit the feature in Write Field.
Which is probably a good thing (tm), but then there's this place: https://spelunker.whosonfirst.org/id/1292942521 which didn't get it's mz:is_current set correctly to 0 along the way.
I can edit it manually... but maybe we always allow editing?
If we do allow editing, then Write Field probably needs to also add a CAUTION: Depreciated place banner warning at the top (like the Spelunker has).

This is happening client-side in the JS here:
https://github.com/whosonfirst/go-whosonfirst-spelunker-httpd/blob/main/static/javascript/whosonfirst.spelunker.id.init.js#L271


Extend the admin interface to support requesting CDS, RDS sync/reload

I have a use case where I need to add a new cluster and as set of routed to the envoy configuration and then signal envoy to consume the new configuration so I can use new configuration immediately after.
I wanted to use CDS and RDS for this but currently envoy queries xDS periodically which means that there will be some un-deterministic time before the new clusters and routes take effect.
I think it would be useful to have /clusters/reload /routes/reload  etc.  functionality in the admin interface which allows deterministic reload/update of the configuration.
I think these methods should return/complete after the reload has completed not just when a sync has been initiated.
Thanks

Would gRPC streaming in the v2 API suffice for you use case?

Could you elaborate on what you mean by

gRPC streaming in the v2 API

I would like if it is possible to send a simple REST request possibly containing JSON body. I would rather avoid having to rely on protocol that requires using a library for encoding the request.
Either requesting a resync over RDS, CDS or directly adding the new configuration over REST would work for me.
Thanks

Although this feature is possible, it's a large amount of work, and IMO not worth it when the v2 APIs have been designed to be streaming with ACKs as @htuch mentions. @georgi-d can you please take a look at the v2 streaming APIs to see if that works for you. I think it's unlikely that any of the core Envoy developers are going to add this so if it did get added it would be a (complicated) community addition.

I read into the v2 streaming API and as far as I understand it it should cover what I want to achieve.
I will attempt the solution with the v2 API.
Thanks for the reply.

OK going ahead and closing. Thank you.


refactor: remove operators reexport

Ensures users won't run into #2812
I think we should consider merging this before pushing 5.5.0. Due to tooling (Webpack, Uglify, etc), if a user does:
import {map, merge} from 'rxjs/operators';

Their bundle size will increase significantly. On a very small sample project, the above import (as opposed to importing each directly) makes about 120kb change in the final bundle (almost doubling the bundle).
In order to avoid this problem, we shouldn't publish the re-export until tooling properly supports importing through re-exports.

This will not go green until #2879 is merged and rebased.

Closing this PR. I think we can fix this issue by running the output package through the build optimizer (https://github.com/angular/ngo). The problem seems to be that all classes imported through the entry point are retained. ngo helps fix that by marking downleveled classes as __PURE__ which allows Uglify to remove them.

Re-opening this PR. I created a test build of rxjs after running it through the Angular Build Optimizer. While the optimizer helps in final application bundle size, it doesn't allow Uglify to get rid of all unused symbols. This results in about 40k of unminified bloat. It's better than the previous number (which was actually well north of 120k), but is still far too much to be justified.
Therefore I think we should remove the reexport until the problem of pulling in unused code can be solved. This may be solved by Webpack doing better module resolution.

So... webpack is why we can't have nice things?

I don't think this is a direction we want to go, honestly... The direction that we let inadequate (or possibly just hard to configure) tooling dictate how we structure the ergonomics of a project. Esp. when there's no reason it shouldn't work.

@benlesh So pulling in the Build Optimizer from the Angular project (and making a few hacks to get it to run against the older version of TypeScript we're currently using), I can produce a build where the final application bundle size is really close between using direct deep imports and importing from rxjs/operators. I think the size will be acceptably close, and as the tools get better we will see an even further reduction.
Of note is this will rely on running through UglifyJS. Most projects do as it's the default for Webpack. Also, there are some configuration options that should be added to Uglify configuration in Webpack to get the best bundle size. I'll note all this in the PR with the updated build.

@jasonaden webpacks module resolution is working just perfectly. The only flaw is that we implemented es module support via the spec which states that all modules even reexports that aren't used, must be evaluated in the situation of potential side effects.
Luckily for you, webpack 4 will optimize this and prune unused. Please see our medium publication medium.com/webpack and check out our logbook for more details.

@TheLarkInn Sorry... my explanation above was a bit brief.
Yes, Webpack 4 should solve some of these issues by pruning unused imports. But I feel like we need a solution that works for people now, rather than publishing something that will be fixed in the future, but add lots of bloat until users upgrade.
But we found a way to do this with the Angular Build Optimizer and adding all the needed /**PURE*/ annotations. This gets us almost all the way there, but Webpack 4 should get us the rest of the way.

But I feel like we need a solution that works for people now, rather than publishing something that will be fixed in the future, but add lots of bloat until users upgrade.

Contributions welcomed. Happy if you want to accelerate the release process we need testing and early tried against webpack @next branch.

@jasonaden importing lettable operator from rxjs/operators produce a large bundle size even with build-optimizer, the new option sideEffects introduced in Webpack 4 removed the unused operators but only when importing from _esm5  folder
++ import { filter } from 'rxjs/_esm5/operators';
-- import { filter } from 'rxjs/operators';

Note: tested with rxjs: 5.5.0-beta.7

@aitboudad Because by default (rxjs/operators), you will import CJS versions of these operators, you will get all of Rx by running this import.
I'll be writing down the steps to get tree shaking with this new import, but if you import the path-mapping file as showing in this gist, you should get tree shaking back.
I haven't written this up yet because we found a few environments where using this path-mapping doesn't seem to result in tree shaking, so still debugging these use cases.
cc @benlesh @IgorMinar


F.E.A.R. Platinum - no offset_match

Hi! Thanks for working on this script.
Are some games not compatible?
$ ./gogextract.py setup_fear_platinum_collection_<IP_ADDRESS>-1.bin fear/
Traceback (most recent call last):
  File "./gogextract.py", line 28, in <module>
    script_lines = int(offset_match.group(1))
AttributeError: 'NoneType' object has no attribute 'group'

I tried increasing to 100kb and it does not seem to find the script.
beginning = game_bin.read(102400).decode("utf-8", errors="ignore")
offset_match = OFFSET_RE.search(beginning)
print (offset_match)

Prints None

Sorry, I must be blind, I was sure it is for Windows installers.

Hey, I thought it was for Windows installers as well, glad this is here.


is it possible to build libchromiumcontent out of a regular chromium build ?

Hi,
i'm trying to answer that question - in debian, it would be nice to be able to have the chromium package build spit out that shared library in the process, so that brightray could (dynamically) link to a libchromiumcontent debian package built out of chromium.
It would avoid maintenance of several versions of chromium.
Thanks for any info.
Jérémy.

I'm closing this since it is not possible.


Dwifft 0.3 Is Not Available in the CocoaPods Spec Repo

According to the releases on GitHub, 0.3 is the latest version, but only versions up to 0.2 are available via CocoaPods (see: https://cocoapods.org/pods/Dwifft). A Podfile like this will reproduce the issue.
pod 'Dwifft', '0.3'

The likely solution to this issue is running pod trunk push Dwifft.podspec.

Ah, sorry about that! I've pushed a 0.3.1 version (Cocoapods was 500'ing
when I tried pushing 0.3.0, so I bumped the version and that appeared to
fix things.)
On Sat, Jan 23, 2016 at 9:24 AM, Paul Rehkugler<EMAIL_ADDRESS>wrote:

According to the releases on GitHub, 0.3 is the latest version, but only
versions up to 0.2 are available via CocoaPods (see:
https://cocoapods.org/pods/Dwifft). A Podfile like this will reproduce
the issue.
pod ‘Dwifft', ‘0.3'
The likely solution to this issue is running pod trunk push Dwifft.podspec
.
—
Reply to this email directly or view it on GitHub
https://github.com/jflinter/Dwifft/issues/9.

Yup, that worked. Thanks!


Extracting progress parameters for reporting

What is your current rclone version (output from rclone version)?
v1.45
What problem are you are trying to solve?
Currently there is the possibility to display progress at the end of each transfer in the command line. I want to automate my backups and therefore use the summary for my reportings. In order to achieve that, there has to be the possibility to extract the parameters of the total files transferred, the file sizes, the duration, the transfer speed etc.
How do you think rclone should be changed to solve that?
The parameters could be saved to a file for further use or simply be returned as array or so.

If you use the -v flag, rclone will log how it processes each file.
This isn't exactly what you want but it might be helpful.

@ncw Thanks for your answer! I am planning to use the logging-function and a python script which extracts those informations from the log. But that's not really the clean way to do it. It will be super-difficult to decode the summary because it's formatted with spaces etc.
I thought the implementation of having the ability to extract the parameters would be easier. Are there any future plan on this?

There are some issues about rclone using a structured logging output which would probably help with your plan, but none about producing a report I don't think.

@ncw Is it so complicated to implement a new argument which is working like the logging function but with the output formatted as json? It doesn't sound like a huge project but unfortunately I have no programming skills to solve this. It would be awesome if you could solve this issue which will be important for future extensibility.

The relevant issues are

https://github.com/ncw/rclone/issues/2459
https://github.com/ncw/rclone/issues/2453

To use a structured logger.  That is probably a better way of going than just using an ad-hoc JSON dump


Further explanation request

Seeing no other vehicle for comments and questions, I submit this as an "issue."  I apologize in advance if this is inappropriate protocol.
I seek summed audio latency reduction to <30Ms for music performance on Dell notebook

.  I already have control of buffer size in my various VST (output) and audio production (input) software and hardware and know that reducing buffer size it too much (often around 128) results in intermittent crackling, yet still fails to solve my current Audio Latency issues (~60Ms).  It is impossible to use a piano keyboard as an input under this condition.  Any comment?
Thank you.
Tom Stacy
<EMAIL_ADDRESS>

I'm guessing you're using a DAW, in which case I would suggest having exclusive audio mode turned on. Unfortunately, REAL is of no use here because it only affects applications that are in shared audio mode.
Besides that, I'm not sure if I'll be able to help you with the latency issues.

No I use numerous VST playerswithinput from sequencing software (Finale 25)
and from keyboard controllers, etc.
On Mon, Dec 9, 2019 at 3:10 PM mini)(ant<EMAIL_ADDRESS>wrote:

I'm guessing you're using a DAW, in which case I would suggest having
exclusive audio mode turned on. Unfortunately, REAL is of no use here
because it only affects applications that are in shared audio mode.
Besides that, I'm not sure if I'll be able to help you with the latency
issues.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/miniant-git/REAL/issues/5?email_source=notifications&email_token=AMG4VPEZ54P6C6IUZ775LYLQX2Q3RA5CNFSM4JWWUL6KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEGKQM2I#issuecomment-563414633,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AMG4VPFLAVLDQ46BKNVCH53QX2Q3RANCNFSM4JWWUL6A
.

--
Tom Stacy


Support for flap locking?

Just for confirmation, as I couldn't find anything in the documentation related to this - but there isn't support for controlling the locking status of a flap, is there?

Not being a PHP or Python developer myself, I could probably figure this all out on my own but it would far longer than I would want it to.  I'm pleased to find someone working on a simple REST API for it.  I'm mainly looking for something that can control locking so that I setup more complex routines than what the official apps allow.  I'm content to wait, and I'm happy to help test.  I have a DualScan with multiple pets.

You described the intention of this project - a simple, easy-to-use and well documented REST API for SureFlap products. I will have a look into the implemenation of setting the different locking states.

Hey @kionei, take a look at the newly added PATCH endpoint "/devices/{device_id}/control", which let you set the lock mode of a specific flap to one of the 4 available lock modes (Only in, Only out, both in and out, and none). On my flap, it is working fine. Please let me know if it does for you as well.
In addition I published the entire OpenAPI Documentation publicly at https://fabieu.github.io/sureflap-api/. There you can find additional example how to call this endpoint correctly.

Unfortunately I can't test it yet.  I locally built a new Docker image and while the container runs the API is unavailable and the container logs are just repeating:
[Errno 2] No such file or directory: b'/bin/--'
at /usr/local/lib/python3.10/os.py:607 in _execvpe
603│         path_list = map(fsencode, path_list)
604│     for dir in path_list:
605│         fullname = path.join(dir, file)
606│         try:
→  607│             exec_func(fullname, *argrest)
608│         except (FileNotFoundError, NotADirectoryError) as e:
609│             last_exc = e
610│         except OSError as e:
611│             last_exc = e

Author

Sorry for the inconvenience, I tested all REST API features, but not the docker images - lessons learned. I realeased a fix for this issue just now. Feel free to share, if its working for you now.

Confirmed that the Docker image is running as expected, and the device control endpoint is working.  Personally I find the mode backwards, e.g. out = lock pets out, in = lock pets in, etc., but that may just be my perspective.  Regardless, I can work with that.
I appreciate the quick work and updates!

Confirmed that the Docker image is running as expected, and the device control endpoint is working. Personally I find the mode backwards, e.g. out = lock pets out, in = lock pets in, etc., but that may just be my perspective. Regardless, I can work with that.
I appreciate the quick work and updates!

Good point, with the lock mode. I will take a look at this!


Noise algorithms

Problem to solve
We closed #1263 since we don't want to duplicate functionality already provided by other packages in the dart ecosystem and since it wasn't necessary for anything internally in the engine (yet). However, we are not sure whether https://github.com/frankpepermans/fast_noise/ is actively maintained so that we could add 1d noise in there that @st-pasha wanted.
Proposal

Contact the author of fast_noise to see whether they would be up for accepting PRs or not. (@frankpepermans)
If 1 doesn't work, possibly fork that library under blue fire and release it with some other name.

This will give more value to the Flutter+Dart community than having it bundled into Flame.

As it turns out, we already have implementation of one kind of noise available: the SimplexNoise class in the vector_math_64 library (also adopted from Stefan Gustavson's work). Unfortunately, they don't have 1d noise either.

Hey guys,
Wouldn't mind helping out if still needed, gladly accept PRs for fast_noise.

That's aweseme @frankpepermans, thanks for the reply.

Closing this since we now have flame_noise which we can continue to add more algorithms to.


Implementation for the new react frontend

Hi @pleia2 @arshPratap, as discussed earlier in the issue #120, this is the current implementation of the react front-end which I worked upon during my LFX mentorship. I've also attached the link to my UI mock-ups for reference. Thanks!
MOCK-UP LINK -
https://www.figma.com/file/honJeXgg8gj24IudSlQSNf/lfx-mockup?type=design&node-id=0-1&mode=design&t=hPbLuXNIoojrHXIB-0

Just gone through this PR and HeroSection and Footer is not completed yet should i proceed with creating that
cc @aashish-khatri @pleia2

Since this has not been completed and has some issues like text overflowing, can i proceed with creating the remaining part?

Can i start working on the remaining part? and fix issues such as text overflowing

I appreciate the enthusiasm! But since this is specifically part of the mentorship this summer, I'd prefer that people just make comments here about what should be addressed, rather than working on it directly. Please feel free to look at any other parts of the project to make improvements :smile:

Oh okay sure, is there any slack channel we can join as the one provided in this repo is private

I appreciate the enthusiasm! But since this is specifically part of the mentorship this summer, I'd prefer that people just make comments here about what should be addressed, rather than working on it directly. Please feel free to look at any other parts of the project to make improvements 😄

Oh okay sure, is there any slack channel we can join as the one provided in this repo is private

@savarbhasin I just confirmed again that the channel is not private, you should be able to join. Were you able to join the Slack organization via https://join-slack.openmainframeproject.org/ ? If so, let me know your nickname on Slack and I'll invite you to the channel.

Oh okay sure, is there any slack channel we can join as the one provided in this repo is private

@savarbhasin I just confirmed again that the channel is not private, you should be able to join. Were you able to join the Slack organization via https://join-slack.openmainframeproject.org/ ? If so, let me know your nickname on Slack and I'll invite you to the channel.

Thanks a lot! I have joined the slack.

Interested in it. Where can i help on it.

Interested in it. Where can i help on it.

Welcome, @Stan370 ! This is part of our summer mentorship program this year so we're not actively making changes until that begins in June. If you're interested in the mentorship program, please apply! :smile: and you're welcome to look around the project and see if there are other issues that you think you could work on.


[ICD] Test ICD Identification during auto-commissioning via all-cluster-app with and without ICD code

@erjiaqing
For the without ICD code, this will be done with the all-cluster-app as the server
For the with ICD, this will be done with the lit-icd-app as the server
See #30291 30291

https://github.com/project-chip/connectedhomeip/pull/29427/files


allowedGroups sintax

Hello, i have a problem with allowedGroups module in plugin.
Have config:
http:
  middlewares:
    ldap-auth:
      plugin:
        ldapAuth:
          Attribute: "uid"
          BaseDn: "cn=users,cn=accounts,dc=org,dc=local"
          Enabled: true
          Url: "ldap://freeipa.org.local"
          LogLevel: "DEBUG"
          allowedGroups:
            - cn=test-group,cn=groups,cn=accounts,dc=org,dc=local

I have testuser in test-group. But traefik cant authenticate testuser:
Aug 03 06:59:13 infra-internal-lb traefik[3187004]: DEBUG: ldapAuth: 2023/08/03 06:59:13 restricted.go:52: No session found! Trying to authenticate in LDAP
Aug 03 06:59:13 infra-internal-lb traefik[3187004]: DEBUG: ldapAuth: 2023/08/03 06:59:13 restricted.go:51: Host: freeipa.org.local
Aug 03 06:59:13 infra-internal-lb traefik[3187004]: DEBUG: ldapAuth: 2023/08/03 06:59:13 restricted.go:51: Connect Address: freeipa.org.local:389
Aug 03 06:59:13 infra-internal-lb traefik[3187004]: DEBUG: ldapAuth: 2023/08/03 06:59:13 restricted.go:51: Running in Bind Mode
Aug 03 06:59:13 infra-internal-lb traefik[3187004]: DEBUG: ldapAuth: 2023/08/03 06:59:13 restricted.go:51: Authenticating User: uid=testuser,cn=users,cn=accounts,dc=org,dc=local
Aug 03 06:59:13 infra-internal-lb traefik[3187004]: DEBUG: ldapAuth: 2023/08/03 06:59:13 restricted.go:51: Searching Group: 'cn=test-group,cn=groups,cn=accounts,dc=org,dc=local' with User: 'uid=testuser,cn=users,cn=accounts,dc=org,dc=local'
Aug 03 06:59:13 infra-internal-lb traefik[3187004]: INFO: ldapAuth: 2023/08/03 06:59:13 restricted.go:51: LDAP Result Code 12 "Unavailable Critical Extension": Bad search filter
Aug 03 06:59:13 infra-internal-lb traefik[3187004]: DEBUG: ldapAuth: 2023/08/03 06:59:13 restricted.go:51: User: 'testuser' not found in Group: 'cn=test-group,cn=groups,cn=accounts,dc=org,dc=local'
Aug 03 06:59:13 infra-internal-lb traefik[3187004]: ERROR: ldapAuth: 2023/08/03 06:59:13 restricted.go:51: LDAP Result Code 12 "Unavailable Critical Extension": Bad search filter
Aug 03 06:59:13 infra-internal-lb traefik[3187004]: User 'testuser' does not match any allowed users nor allowed groups.
Aug 03 06:59:13 infra-internal-lb traefik[3187004]: DEBUG: ldapAuth: 2023/08/03 06:59:13 restricted.go:52: [LDAP Result Code 12 "Unavailable Critical Extension": Bad search filter
Aug 03 06:59:13 infra-internal-lb traefik[3187004]: User 'testuser' does not match any allowed users nor allowed groups.]

i made a workaround using searchmode with
searchFilter: (&(memberOf=cn=test-group,cn=groups,cn=accounts,dc=org,dc=local)({{.Attribute}}={{.Username}}))
and it works, but would be great to understand why normal way doesn't work.
Maybe it's because of "memberOf" field? Didnt see this in sources.

Hello, @plakun
Thanks for reporting this issue. I think this problem was added in PR #40.
I can replicate the problem using ldapsearch and FreeIPA demo LDAP server.
Without the member:1.2.840.1135<IP_ADDRESS>1: filter the search works:
$ ldapsearch -x \
  -b "cn=users,cn=accounts,dc=demo1,dc=freeipa,dc=org" \
  -H ldap://ipa.demo1.freeipa.org \
  -D "uid=manager,cn=users,cn=accounts,dc=demo1,dc=freeipa,dc=org" \
  -w Secret123 \
  "(uid=manager)"
# extended LDIF
#
# LDAPv3
# base <cn=users,cn=accounts,dc=demo1,dc=freeipa,dc=org> with scope subtree
# filter: (uid=manager)
# requesting: ALL
#

# manager, users, accounts, demo1.freeipa.org
dn: uid=manager,cn=users,cn=accounts,dc=demo1,dc=freeipa,dc=org
givenName: Test
sn: Manager
uid: manager
cn: Test Manager
displayName: Test Manager
initials: TM
gecos: Test Manager
krbPrincipalName<EMAIL_ADDRESS>objectClass: top
objectClass: person
objectClass: organizationalperson
objectClass: inetorgperson
objectClass: inetuser
objectClass: posixaccount
objectClass: krbprincipalaux
objectClass: krbticketpolicyaux
objectClass: ipaobject
objectClass: ipasshuser
objectClass: ipaSshGroupOfPubKeys
objectClass: mepOriginEntry
objectClass: ipantuserattrs
loginShell: /bin/sh
homeDirectory: /home/manager
mail<EMAIL_ADDRESS>krbCanonicalName<EMAIL_ADDRESS>ipaUniqueID: 33a91070-df83-11ed-919b-0674ebdae396
uidNumber:<PHONE_NUMBER>
gidNumber:<PHONE_NUMBER>
krbLastPwdChange:<PHONE_NUMBER>5652Z
ipaNTSecurityIdentifier: S-1-5-21-562557311-290852036-180187086-1003
memberOf: cn=ipausers,cn=groups,cn=accounts,dc=demo1,dc=freeipa,dc=org
memberOf: cn=employees,cn=groups,cn=accounts,dc=demo1,dc=freeipa,dc=org
memberOf: ipaUniqueID=d73ae6ae-df7f-11ed-8ae9-0674ebdae396,cn=hbac,dc=demo1,dc
 =freeipa,dc=org
memberOf: cn=managers,cn=groups,cn=accounts,dc=demo1,dc=freeipa,dc=org
krbPasswordExpiration:<PHONE_NUMBER>5652Z

# search result
search: 2
result: 0 Success

# numResponses: 2
# numEntries: 1

But using AD nested groups:
$ ldapsearch -x \
  -b "cn=users,cn=accounts,dc=demo1,dc=freeipa,dc=org" \
  -H ldap://ipa.demo1.freeipa.org \
  -D "uid=manager,cn=users,cn=accounts,dc=demo1,dc=freeipa,dc=org" \
  -w Secret123 \
  "(|(uid=manager)(member:1.2.840.1135<IP_ADDRESS>1:=uid=manager))"
# extended LDIF
#
# LDAPv3
# base <cn=users,cn=accounts,dc=demo1,dc=freeipa,dc=org> with scope subtree
# filter: (|(uid=manager)(member:1.2.840.1135<IP_ADDRESS>1:=uid=manager))
# requesting: ALL
#

# search result
search: 2
result: 12 Critical extension is unavailable
text: Bad search filter

# numResponses: 1

When you use a custom searchFilter and don't add allowedGroups, then no authorization is performed
https://github.com/wiltonsr/ldapAuth/blob/c6f6066dfd95b1cb019b25b71f9cf6a587462a31/ldapauth.go#L268-L271
And this piece of code that adds the AD nested groups filter doesn't run
https://github.com/wiltonsr/ldapAuth/blob/c6f6066dfd95b1cb019b25b71f9cf6a587462a31/ldapauth.go#L326-L331
You can make a test using v0.0.22 and report here if everything works as expected. This specific version doesn't have the AD nested groups feature.
If we confirm this is the problem, I will submit a fix with a param to explicitly enable the AD nested groups feature.

@fcinqmars fyi

@wiltonsr
I think you indeed found the culprit. I can confirm that PR#40 introduced this bug by using ipa.demo1.freeipa.org as it works with 0.0.22 but not with any further versions.
That being said, I would be curious to know what the difference is with that LDAP server. I tested against the one you usually use (forumsys.com) as well as with AD and they both work. ldap.forumsys.com and ipa.demo1.freeipa.org are both using the same LDAP version (LDAPv3).
ldapsearch -x   \
-b "dc=example,dc=com" \
-H ldap://ldap.forumsys.com  \
-D "cn=read-only-admin,dc=example,dc=com" \
-w password \
"(|(uid=euclid)(member:1.2.840.1135<IP_ADDRESS>1:=uid=euclid))"

# extended LDIF
#
# LDAPv3
# base <dc=example,dc=com> with scope subtree
# filter: (|(uid=euclid)(member:1.2.840.1135<IP_ADDRESS>1:=uid=euclid))
# requesting: ALL
#

# euclid, example.com
dn: uid=euclid,dc=example,dc=com
uid: euclid
objectClass: inetOrgPerson
objectClass: organizationalPerson
objectClass: person
objectClass: top
cn: Euclid
sn: Euclid
mail<EMAIL_ADDRESS>
# search result
search: 2
result: 0 Success

# numResponses: 2
# numEntries: 1

That being said, I would be curious to know what the difference is with that LDAP server.

Looks like a specific server feature to be enabled/configured.
Anyway, I think that the PR #46 will fix the problem.


Argentina holiday improvements

Proposed change

Argentina holiday improvements (test)
Type of change

[ ] New country holidays support (thank you!)
[x] Supported country holidays update (calendar discrepancy fix, localization)
[ ] Existing code/tests/processes improvement (best practice, refactoring, optimization)
[ ] Dependency upgrade (version update)
[ ] Bugfix (non-breaking change which fixes an issue)
[ ] Breaking change (a code change causing existing functionality to break)
[ ] New feature (adds functionality to python-holidays in general)

Checklist

[x] I've followed the [contributing guidelines][contributing-guidelines]
[x] This PR is filed against beta branch of the repository
[x] This PR doesn't contain any merge conflicts
[x] The code style looks good (make pre-commit)
[x] I've added tests to verify that the new code works and all tests pass locally (make test)
[x] The related [documentation][docs] has been added/updated (check off the box for free if no updates is required)

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

47 of 47 (100.0%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 100.0%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
7158

Relevant Lines:
7158

💛  - Coveralls


Make PyPI release from CI

Noticed you were having some issues with making PyPI releases from CI workflows.
You can take a look how we did it in this project:
https://github.com/EIDOSLAB/torchstain/blob/main/.github/workflows/build.yaml
Or this project:
https://github.com/andreped/GradientAccumulator/blob/main/.github/workflows/build_release.yml
Works great!

If you want to make github release AND make PyPI release in the same CI workflow, this is probably what you want:
https://github.com/smistad/FAST/blob/master/.github/workflows/CI-ubuntu.yml#L129
Hope it works out!

Thank you, sorry for spamming your github feed I hadn't thought of that.
I'll check out your workflows to see what you did.
I have the basic push to PyPi on release with the tag as version number working as of a minute ago though :) https://pypi.org/project/DeepSlice/

Yes no more 11 releases a day for the immediate future 🙃


Building EF7 database model (beta 8) fails under x64 but works under x86

We have in our uwp-application now installed EntityFramework.Sqlite 7, Beta8 and have dicovered (by chance) a strange problem:
Our EF-model works fine under x86 but crashes under x64, very strange.
Under x64 we get the following error:
"System.InvalidOperationException: The navigation property 'ArticleConnections' cannot be added to entity type 'Opacc.Mof.Client.Service.DataModel.Article'
because its CLR type 'System.Collections.Generic.List`1[[Opacc.Mof.Client.Service.DataModel.ArticleConnection, Opacc.Mof.Client.Service.DataModel, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=null]]'
does not match the expected CLR type 'Opacc.Mof.Client.Service.DataModel.ArticleConnection'.
It looks like that under x64 EF wants to create a Article. ArticleConnections as ArticleConnection instead ofIList
The exactly same model works when we build it under x86.
For me this sounds very strange but perhaps somebody has an idea for this ‚bug‘ (or ever seen some differences between x86/x64).
I would like to attach a test-application which reproduces the problem. The solution includes our two database-access projects and is therefore a little bit large but this shouldn't be bothersome.
But i don't know how to attach here zip-files: attach files let me only attach single text-files. I think it's not possible to reproduce this error with simple code snippets.

But i don't know how to attach here zip-files

You can just create a new repository under your account where you push your example project and then people can look at it.

Thanks poke for the hint, i have now under my account a repository created and uploaded the example solution which reproduces the bug.

@KarlAndenmatten we fixed some issues related to x64/x86 in RC1, can you retry on the latest preview and see if the issue still exists.

@rowanmiller I have now installed RC1 and it seems that the bug is fixed, thanks!


Reveal export in finder notification not working

macOS version: 10.14.4
Kap version: master
Steps to reproduce
With notifications enabled, Kapture and export to local, then click the notification to reveal the completed export
Current behaviour
Clicking the notification currently focuses the export history window.

I can't reproduce this in the current master


Update version number

Codecov Report

Merging #303 (f16f680) into master (fb70cca) will not change coverage.
The diff coverage is 0.00%.

@@            Coverage Diff            @@
##             master     #303   +/-   ##
=========================================
  Coverage     83.90%   83.90%
  Complexity     1101     1101
=========================================
  Files           148      148
  Lines          3206     3206
  Branches        393      393
=========================================
  Hits           2690     2690
  Misses          421      421
  Partials         95       95

Impacted Files
Coverage Δ
Complexity Δ

src/main/java/tutorspet/MainApp.java
0.00% <0.00%> (ø)
0.00 <0.00> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update fb70cca...f16f680. Read the comment docs.


💥Feat: Remaining Payment Method Integration UI improved

Related Issue
“None”
Fixes:  #4274
Description
This PR fix Remaining payment methods UI to provide users with a seamless checkout experience.
Type of PR

[x] Feature enhancement

Screenshots / videos (if applicable)
https://github.com/user-attachments/assets/36b0a1c8-8a84-4179-9c81-eb7c0bbe581a
Checklist:

[x] I have made this change from my own.
[x] My code follows the style guidelines of this project.
[x] I have performed a self-review of my own code.
[x] I have commented my code, particularly in hard-to-understand areas.
[x] I have made corresponding changes to the documentation.
[x] My changes generate no new warnings.
[x] I have tested the changes thoroughly before submitting this pull request.
[x] I have provided relevant issue numbers and screenshots after making the changes.

@abhi03ruchi please review and merge it under gssoc level 3 label


404 in Left Sidebar of zh-cn doc

https://www.tensorflow.org/addons/api_docs/python/tfa/layers/InstanceNormalization
Check the GELU changed into GeLU if you change the language from en to zh-CN

Because the url is case-sensitive, it only works with GELU, so its a 404 in zh-cn docs. I tried to bring a fix but only found the toc file in v0.7.1 branch instead of master.

Hi @Ir1d thanks for the issue! I'm not quite following why this hyperlink would be different for the CN version. I tried changing my language and the link looks correct?

++ @MarkDaoust @lamberta to see if this makes sense to them

I believe this is somehow fixed? I can't reproduce either.

I sent off a translation job for the _book.yaml files (nav) that just came back the other day. Usually translations can lag behind a bit. Should be up-to-date now.


Inconsistent spacing in user authorization menu

In the agent view user auth menu, the spacing is different for users with and without Github profiles

Addressed in a35c891


Create alternate metric view for non-visual users

Because plotly charts are svgs and plotly does not expose tooling in this area, it is going to be very difficult to make our metrics & other charts compliant. I propose a view with two horizontal tabs where users can switch between a graphical (plotly) and tabular (accessible) view.

duplicate of #1123


Undefined behavior in signal handler?

Hi @bombela and maintainers in backward-cpp!
I'm working with an issue in a bit of a complicated setup where I'm trying to handle error printing in a multithreaded app using backward-cpp. This errors triggers here:
https://github.com/bombela/backward-cpp/blob/dc8b8c76822dcbc8918f032171050ad90e11eb7f/backward.hpp#L4298
Specfically, it looks like this at the tip of the stack:
#11   Object "/usr/lib/x86_64-linux-gnu/libc.so.6", at 0x7fffdbe4251f, in
#10   Source "../lib/backward.hpp", line 4299, in sig_handler [0x5555557118cf]
       4296:             handleSignal(signo, info, _ctx);
       4297:
       4298:             // try to forward the signal.
      >4299:             raise(info->si_signo);
       4300:
       4301:             // terminate the process immediately.
       4302:             puts("watf? exit");
#9    Source "../sysdeps/posix/raise.c", line 26, in raise [0x7fffdbe42475]
#8  | Source "./nptl/pthread_kill.c", line 89, in __pthread_kill_internal
    | Source "./nptl/pthread_kill.c", line 78, in __pthread_kill_implementation
      Source "./nptl/pthread_kill.c", line 43, in __pthread_kill [0x7fffdbe96a7b]
#7    Object "/usr/lib/x86_64-linux-gnu/libc.so.6", at 0x7fffdbe4251f, in
#6    Source "../lib/backward.hpp", line 4299, in sig_handler [0x5555557118cf]
       4296:             handleSignal(signo, info, _ctx);
       4297:
       4298:             // try to forward the signal.
      >4299:             raise(info->si_signo);
       4300:
       4301:             // terminate the process immediately.
       4302:             puts("watf? exit");
#5    Source "../sysdeps/posix/raise.c", line 26, in raise [0x7fffdbe42475]
#4  | Source "./nptl/pthread_kill.c", line 89, in __pthread_kill_internal
    | Source "./nptl/pthread_kill.c", line 78, in __pthread_kill_implementation
      Source "./nptl/pthread_kill.c", line 43, in __pthread_kill [0x7fffdbe96a7b]
#3    Object "/usr/lib/x86_64-linux-gnu/libc.so.6", at 0x7fffdbe4251f, in
#2    Source "../lib/backward.hpp", line 4299, in sig_handler [0x5555557118cf]
       4296:             handleSignal(signo, info, _ctx);
       4297:
       4298:             // try to forward the signal.
      >4299:             raise(info->si_signo);
       4300:
       4301:             // terminate the process immediately.
       4302:             puts("watf? exit");
#1    Source "../sysdeps/posix/raise.c", line 26, in raise [0x7fffdbe42475]
#0  | Source "./nptl/pthread_kill.c", line 89, in __pthread_kill_internal
    | Source "./nptl/pthread_kill.c", line 78, in __pthread_kill_implementation
      Source "./nptl/pthread_kill.c", line 44, in __pthread_kill [0x7fffdbe96a7c]

si_signo in this specific case is SIGSEGV, and I'm expecting the program to terminate. Instead it's infinitely recursing in a signal handler, where it looks like a segfault is occurring in __pthread_kill_internal, as a result of the exception being raised from within the exception handler. While this is SIGSEGV, I'm pretty sure the same behavior occurs with other signals.
The original exception source is a background thread: Thread 26 "[core] bkg" received signal SIGSEGV, Segmentation fault.
The initial occurrence of SIGSEGV is correct behavior, as I am committing an access violation to trigger this behavior. The issue at hand is the infinite loop occurring after it (and I've seen it in other circumstances before.)
Looking at cppreference.com (https://en.cppreference.com/w/c/program/signal), I'm finding this quote:
If the signal handler is called as a result of [abort] or [raise], the behavior is undefined if the signal handler calls [raise].
Is it viable to remove this raise() call, or should there be some special consideration when the exception is occurring off the main thread?

Actually, this might be that another library, our embedded javascript engine, is causing the issue by registering a secondary signal handler and calling it when the engine doesn't have an active context. I'm a bit uncertain - but the issue almost looks like there's some kind of pingpong going on between a valid and invalid error handler:
    at /lib/x86_64-linux-gnu/libc.so.6
#12 __pthread_kill_implementation (no_tid=0, signo=11, threadid=140733503758336)
    at ./nptl/pthread_kill.c:44
#13 __pthread_kill_internal (signo=11, threadid=140733503758336)
    at ./nptl/pthread_kill.c:78
#14 __GI___pthread_kill (threadid=140733503758336, signo=signo@entry=11)
    at ./nptl/pthread_kill.c:89
#15 0x00007fffdbe42476 in __GI_raise (sig=11) at ../sysdeps/posix/raise.c:26
#16 0x00005555557118d0 in backward::SignalHandling::sig_handler(int, siginfo_t*, void*) (signo=<optimized out>, info=0x7fff127d1c30, _ctx=<optimized out>)
    at ../lib/backward.hpp:4300
#17 0x00007fffdbe42520 in <signal handler called> ()
    at /lib/x86_64-linux-gnu/libc.so.6
#18 __pthread_kill_implementation (no_tid=0, signo=11, threadid=140733503758336)
    at ./nptl/pthread_kill.c:44
#19 __pthread_kill_internal (signo=11, threadid=140733503758336)
    at ./nptl/pthread_kill.c:78
#20 __GI___pthread_kill (threadid=140733503758336, signo=signo@entry=11)
    at ./nptl/pthread_kill.c:89
#21 0x00007fffdbe42476 in __GI_raise (sig=11) at ../sysdeps/posix/raise.c:26
#22 0x00005555557118d0 in backward::SignalHandling::sig_handler(int, siginfo_t*, void*) (signo=<optimized out>, info=0x7fff127d29f0, _ctx=<optimized out>)
    at ../lib/backward.hpp:4300
#23 0x00007fffdbe42520 in <signal handler called> ()
    at /lib/x86_64-linux-gnu/libc.so.6
#24 __pthread_kill_implementation (no_tid=0, signo=11, threadid=140733503758336)
    at ./nptl/pthread_kill.c:44
#25 __pthread_kill_internal (signo=11, threadid=140733503758336)
    at ./nptl/pthread_kill.c:78
#26 __GI___pthread_kill (threadid=140733503758336, signo=signo@entry=11)
    at ./nptl/pthread_kill.c:89
#27 0x00007fffdbe42476 in __GI_raise (sig=11) at ../sysdeps/posix/raise.c:26
#28 0x00005555558d3435 in ...::call_raise_sigsegv() (this=0x555557292ca0)
    at ../src/...

Would it be worthwhile (or even possible considering exception handling rules) to detect when sig_handler is already in the stack and if so not do this? :smile:

The ping pong is likely because you do have multiple signal handlers. raise works only once I think? You can subclass and override the behavior. Or just write your own signal handler class of course. This one is merely a reasonable default to start with.
It would be possible to modify the code to not raise on SIGABRT. Then, not calling raise recursively is much harder, if not downright impossible.
The reason for calling raise btw, is so that if you have your own signal handler, you could recover.
For example, consider a segfault, in a thread, and you want to catch it and do something about it. backward-cpp will print a trace, then your very own handler can recover the situation.
Otherwise, SignalHandling would terminate your application without your control. With that said, again, it is mostly a default/example to get started, and you shouldn't be afraid of writing your own or subclassing it.