Challenge Extension

The Challenge Extension to the tus protocol proposed here allows clients to authenticate requests targeting an upload resource following the resource's creation through  creation or creation-with-upload.
This extension originates from the original Client-Tag PR and has been factored out here to allow separate discussion.
Compared with its original version, the proposal here includes various improvements:

it makes changes to the computation of Upload-Challenge to extend it to more HTTP methods and block re-use for different purposes (i.e. to prevent the reuse of a captured Upload-Challenge from a HEAD request for a DELETE request)
it allows the use of Upload-Challenge for requests referencing more than one upload resource (to support the Concat extension)
it adds a section with examples

Looking forward to your feedback 🙂

Can we explicitly state here again why we are sending a hashed version of the Upload-Secret in the requests instead of just the Upload-Secret? Maybe that was already discussed somewhere else but I would like to bring the reason up, so we think about which security goals we try to reach. Is the hash intended to avoid tampering by a MITM? Or is the hash intended so that the Upload-Challenge cannot be reused for other requests?
Also, do we know of a similar security mechanism in other applications or protocols? This would allow us to compare and see if we are missing anything.

Can we explicitly state here again why we are sending a hashed version of the Upload-Secret in the requests instead of just the Upload-Secret? Maybe that was already discussed somewhere else but I would like to bring the reason up, so we think about which security goals we try to reach. Is the hash intended to avoid tampering by a MITM? Or is the hash intended so that the Upload-Challenge cannot be reused for other requests?

Think of it as a shared secret:

the request that initiates an upload needs no protection, for it initiates the upload in the first place. Therefore, it can also send the Upload-Secret along in plain text.
all follow-up requests do need protection, so the Upload-Challenge is computed from the shared secret and the variables Upload-Offset, Content-Length and HTTP Method that should differ for every follow-up request. The idea here is that:

only someone in possession of the Upload-Secret will be able to compute a valid Upload-Challenge
the Upload-Challenge can't be used to recover the Upload-Secret, keeping anyone but the original uploader out

Also, do we know of a similar security mechanism in other applications or protocols? This would allow us to compare and see if we are missing anything.

PKCE is using a similar approach, but computes the Challenge as a simple SHA256 of the Secret. Which is ok, because it only needs the Challenge for a single request.
However, for multiple requests, a Challenge value should not simply be reusable. To protect against simply reusing a single hash value, I added Upload-Offset, Content-Length and HTTP Method to the computation.

@felix-schwarz Thank you very much for the additional explanations!

@smatsson @Acconut Thanks for your comments and suggestions! Once we agree on the still discussed points, I'd update the PR accordingly.

FYI, I have a few commits lined up for this PR and will push them in a few hours!

I pushed a few changes to the PR:

The server can now support multiple algorithms using the Tus-Challenge-Algorithm header
The Upload-Secret header is now ASCII encoded, just as the Upload-Tag header, so clients can reuse the code for generating random strings
The Upload-Challenge header is now Base64 encoded and has the same format as the Upload-Checksum header, so servers can reuse their parsing code
The Upload-Challenge header does not include the Content-Length header in its hash anymore since this information is not always available
Include a list of requests that currently would require an Upload-Challenge header to make it easier for implementations

Let me know if you have any feedback! Otherwise, I see this as ready to be merged into the 1.1 branch.

@Acconut Looks good to me. Before merging I think it would be beneficial to implement a POC, similar to the one for Upload-Tag, so that we do not miss anything. I'll see if I can get one up and running this week.

POC is now available at https://tustemp.azurewebsites.net/files/
To calculate the checksums I used this bash command (also works on Windows):
openssl dgst -sha256 -binary challenge.txt | openssl enc -base64 | clip
# challenge.txt should contain the data to hash, e.g. DELETE#R290Y2hhISBVc2UgYW4gYWN0dWFsIGhpR290Y2hhISBVc2UgYW4gYWN0dWFsIGhp

Restrictions:

One can use basic auth (any username/password is accepted) to lock an upload tag to a specific client
Unauthenticated users must provide a Upload-Secret to be able to use upload tag. The implementation is very naive and will just accept the initial POST and then return 404 for the HEAD request if no Upload-Secret was set.
Client-Tags can be resolved for any user, anonymous or not, regardless if they own the file or not, if the correct Upload-Challenge is provided. This is probably less than ideal and I will change this for the final release.
File sizes are limited to 1 MB
All files not written to in 30 minutes will be deleted (regardless of if it's completed or not)
No check that Upload-Secret is not sent with other requests than POST.
Downloads does not implement Upload-Challenge.

Feel free to try it out 😊
Issues I found while implementing (in no particual order):

Examples are still using zero instead of # in a lot of places
We should specify what casing of http method to use ("patch" vs "PATCH"). tusdotnet uses lower case internally which did not work very well with the examples given in the spec.
Concatenation feels off. I have written down the issues below.

Issues with concatenation

The algorithm does not feel as spot on as for other requests:

"sha256" + " " + SHA256("sha256 sXfhFCwyWMjnH1DMPkArsByfa4FEGtpf3LsAt6uDkTU=" + "sha256 JbVm0kH59MDQfGtzjJ3s9oBjzHp+Yqtv7O2/OzYTUqg=")

Do we need "sha256" for each of the partial file checksums? Since this checksum is calculated on each request we use the provided Upload-Challenge to determine the algorithm (i.e. we take "sha256" from the Upload-Challenge header). Would it be better to have something like this?
"sha256" + " " + SHA256("sXfhFCwyWMjnH1DMPkArsByfa4FEGtpf3LsAt6uDkTU=JbVm0kH59MDQfGtzjJ3s9oBjzHp+Yqtv7O2/OzYTUqg=")

What status code to return when Upload-Challenge is wrong when doing a POST with Upload-Concat: final /files/a;/files/b? It seems odd to return 404 as this would indicate that the actual creation endpoint does not exist. 400? 403? Not sure.

Is the client allowed to use Upload-Secret for when doing a POST with Upload-Concat: final /files/a;/files/b together with the Upload-Challenge? If yes, we should probably document it.

What if one of the partial files used does not contain an upload secret? E.g. file A uses an upload secret but file B does not. How would the final checksum be calculated? The  POC just adds an empty string as checksum for file B but I think this needs to be documented in the spec.

@Acconut @smatsson Thanks for the updates and the POC implementation!

Issues I found while implementing (in no particual order):

Examples are still using zero instead of # in a lot of places

Fixed that with the latest commit.

We should specify what casing of http method to use ("patch" vs "PATCH"). tusdotnet uses lower case internally which did not work very well with the examples given in the spec.

Since the HTTP methods are specified in upper case in RFC 2616, which also states that The method is case-sensitive., I was always under the assumption that these are only used in upper-case.
But in fact get / HTTP/1.1 and GET / HTTP/1.1 both seem to work (get / http/1.1 returns an error, though). TIL! 🤯
The PR didn't assume any case, so if the server used the method as received from the client for computation, that should also work. That assumes, however, that proxies in-between would pass the method on unaltered - which can't be taken for granted. I therefore added a clarification.
Out of interest: is there any particular reason why you're using lower case HTTP methods in tusdotnet?

 I have written down the issues below.

Issues with concatenation

The algorithm does not feel as spot on as for other requests:

"sha256" + " " + SHA256("sha256 sXfhFCwyWMjnH1DMPkArsByfa4FEGtpf3LsAt6uDkTU=" + "sha256 JbVm0kH59MDQfGtzjJ3s9oBjzHp+Yqtv7O2/OzYTUqg=")

Do we need "sha256" for each of the partial file checksums? Since this checksum is calculated on each request we use the provided Upload-Challenge to determine the algorithm (i.e. we take "sha256" from the Upload-Challenge header). Would it be better to have something like this?
"sha256" + " " + SHA256("sXfhFCwyWMjnH1DMPkArsByfa4FEGtpf3LsAt6uDkTU=JbVm0kH59MDQfGtzjJ3s9oBjzHp+Yqtv7O2/OzYTUqg=")

I agree the latter looks better.
However, I think it is more consistent to keep the sha256  portion even here. That way, the same piece of code that computes Upload-Challenge for all other requests for that upload can just be used, without special case handling.

What status code to return when Upload-Challenge is wrong when doing a POST with Upload-Concat: final /files/a;/files/b? It seems odd to return 404 as this would indicate that the actual creation endpoint does not exist. 400? 403? Not sure.

Semantically, I agree. The idea of the 404 is to not give any hints to the existence of a resource to anyone without the Upload-Secret. And going by that, I'd keep the 404.

Is the client allowed to use Upload-Secret for when doing a POST with Upload-Concat: final /files/a;/files/b together with the Upload-Challenge? If yes, we should probably document it.

Good catch! Since Concatenation also returns an upload resource, I feel it should be included as well. I just added it next to Create and Create-with-Upload in the latest commit.

What if one of the partial files used does not contain an upload secret? E.g. file A uses an upload secret but file B does not. How would the final checksum be calculated? The  POC just adds an empty string as checksum for file B but I think this needs to be documented in the spec.

Good point! I've added a paragraph on this to the PR, but wrote that the path to the resource should be used instead of an empty string. Otherwise, the Upload-Challenge header of the request could be reused while inserting any number of additional uploads in-between the ones secured with Upload-Secret. Or it would be possible to replace those resources without Upload-Secret with any other resource:
For upload resources that were created without an `Upload-Secret`, the path to the upload resource
MUST be used as `Upload-Challenge` for the purpose of computing the `Upload-Challenge` for
requests that reference multiple upload resources - of which at least one MUST have been created with
an `Upload-Secret`.

@Acconut @smatsson From my side, the PR is ready to be merged at this point.

Out of interest: is there any particular reason why you're using lower case HTTP methods in tusdotnet?

Mostly chance I think. I needed to support X-Http-Method-Override that could be in any casing I just needed to go with one casing and it ended up as lower case. ASP.NET supports any casing (pAtCH) and will interpret it correctly anyway.

However, I think it is more consistent to keep the sha256 portion even here. That way, the same piece of code that computes Upload-Challenge for all other requests for that upload can just be used, without special case handling.

I was actually thinking that we should simplify it even more as we still would need special casing for the cases where we do not have an upload secret for a file. I think it's a fair point to re-use code though.

Semantically, I agree. The idea of the 404 is to not give any hints to the existence of a resource to anyone without the Upload-Secret. And going by that, I'd keep the 404.

I understand why we want to return 404 but since I as a client would know that the /files endpoint does exist, it is still clear that I have the wrong Upload-Challenge. This would probably be OK if we updated the concatenation spec to include that the server should return 404 if one of the partial files does not exist (tusdotnet already does this).

Good catch! Since Concatenation also returns an upload resource, I feel it should be included as well. I just added it next to Create and Create-with-Upload in the latest commit.

Perfect! 👍

Good point! I've added a paragraph on this to the PR, but wrote that the path to the resource should be used instead of an empty string. Otherwise, the Upload-Challenge header of the request could be reused while inserting any number of additional uploads in-between the ones secured with Upload-Secret. Or it would be possible to replace those resources without Upload-Secret with any other resource.

Fair point 👍 Could we add an example of how to use this? The path is the same value included in the Upload-Concat header, e.g. /files/file1?

Semantically, I agree. The idea of the 404 is to not give any hints to the existence of a resource to anyone without the Upload-Secret. And going by that, I'd keep the 404.

I understand why we want to return 404 but since I as a client would know that the /files endpoint does exist, it is still clear that I have the wrong Upload-Challenge. This would probably be OK if we updated the concatenation spec to include that the server should return 404 if one of the partial files does not exist (tusdotnet already does this).

Good point. Right now I can't seem to find any direction in the spec on what to do when a non-existent file is referenced. And this would clarify that case.
I'm not sure where best to add that, however, below the Concatenation heading or below the Upload-Concat header section.

Good point! I've added a paragraph on this to the PR, but wrote that the path to the resource should be used instead of an empty string. Otherwise, the Upload-Challenge header of the request could be reused while inserting any number of additional uploads in-between the ones secured with Upload-Secret. Or it would be possible to replace those resources without Upload-Secret with any other resource.

Fair point 👍 Could we add an example of how to use this? The path is the same value included in the Upload-Concat header, e.g. /files/file1?

Yes, it'd be /files/file1. I've added an example.

of which at least one MUST have been created with an Upload-Secret.

What should be returned if this is not fulfilled? Do we need to require this or could we just ignoring the Upload-Challenge header in that case? I mean technically it does nothing but I think that we should either define the error code (e.g. 400 Bad Request) or just remove the requirement that a file must have an Upload-Secret.

My intention there was more to clarify that Upload-Challenge headers should not be used if none of the files actually was uploaded with an Upload-Secret, because it provides no security benefit.
I've changed that sentence now to (hopefully) make that clearer.

@felix-schwarz Thanks for the updates! Looks good to me and I think this PR is ready for merge. @Acconut Any input on where to put the error handling examples for concatenation? Concretely to respond 404 when a partial file does not exist when creating a final file?
@Acconut @nigoroll Any more input on this extension? :)

@felix-schwarz @smatsson Thank you very much for the updates in the last two weeks! I went over all changes and they all look good :+1:

Any input on where to put the error handling examples for concatenation? Concretely to respond 404 when a partial file does not exist when creating a final file?

A 404 sounds good to me. The response body can be used to describe the error in more detail (i.e. that one of the uploads to be concatenated wasn't found)

I think this PR is ready for merge.

I agree! Let's merge it! If one of you (or @nigoroll) has more input, we can open another PR but let's close this one for now :)


Include quoted index names in describe test

See CASSANDRA-12847

Is there a compelling reason that this is here, and not in cqlshlib tests? Patch LGTM, except will this still work on 2.1?

I've trigged a CI on branch to make sure.

Is there a compelling reason that this is here, and not in cqlshlib tests?

Only that the DESCRIBE test is already here and the cqlshlib tests are more focused on the layout and stying, rather than content.

Patch LGTM, except will this still work on 2.1?

That's a good point. I've pushed a second commit to ensure that we only expect the new behaviour on 3.0.11+ and 3.10+

Is this ready to merge, @beobal ?

Not yet, waiting for Cassandra ticket to get committed, shortly!

sorry, I missed the notifications for this PR. It needed a rebase as pretty much all tests were failing against trunk due to an outdated version of ccm. I've done that, force pushed, and kicked off one more round of CI. When that's happy I'll commit the C* ticket & merge this.  Apologies again for the latency :)

Bump, @beobal ?

CI looks good now, just giving heads up!

@ptnapoleon the C* ticket has been committed now, so cqlsh_tests will fail on 3.0+ until this is merged.


Paywall crashes on iOS when used in a ModalBottomSheet

[x] I have updated Purchases SDK to the latest version
[x] I have read
the Contribution Guidelines
[x] I have searched the Community
[x] I have read docs.revenuecat.com
[x] I have searched
for existing Github issues

Describe the bug
A clear and concise description of what the bug is. The more detail you can provide the faster our
team will be able to triage and resolve the issue. Do not remove any of the steps from the
template below. If a step is not applicable to your issue, please leave that step empty.

Environment

Platform: iOS
SDK version: 1.3.6+13.13.1
OS version: 17.0
IDE (e.g. Android Studio, Xcode, Fleet): Xcode
IDE version: 15.4
Device and/or emulator/simulator:

[ ] Device
[x] Emulator/simulator

Environment:

[x] Closed testing / Sandbox
[ ] TestFlight
[ ] Production

How widespread is the issue. Percentage of devices affected. 100%

Debug logs that reproduce the issue

*** Terminating app due to uncaught exception 'UIViewControllerHierarchyInconsistency', reason: 'child view controller:<RCPaywallViewController: 0x10e85d600> should have parent view controller:(null) but actual parent is:<ComposeHostingViewController: 0x10cb1f910>'
*** First throw call stack:

Steps to reproduce, with a description of expected vs. actual behavior
Attempt to show Paywall when inside a ModalBottomSheet

Works as expected
    val options = remember {
        PaywallOptions(dismissRequest = { }) {
            shouldDisplayDismissButton = true
        }
    }
    Paywall(options)

Crashes
    val options = remember {
        PaywallOptions(dismissRequest = { }) {
            shouldDisplayDismissButton = true
        }
    }
    ModalBottomSheet(onDismissRequest = {}) {
        Paywall(options)
    }

Other information (e.g. stacktraces, related issues, suggestions how to fix, links for us to
have context, eg. stackoverflow, etc.)

Likely a compose multipaltform bug with view interop but thought it was worth raising as showing paywall in this manner feel like a common way to want to show it. I can work around so its not a huge priority to be fixed for me anyway.
Thanks

Hi @Stuart-campbell, thanks for reporting this! Just curious, which version of Compose Multiplatform are you using? And may I ask what your workaround is? Thanks in advance!


HIVE-25373 : Modify buildColumnStatsDesc to send configured number of stats for updation

What changes were proposed in this pull request?
Group the stats objects into different batches to avoid large message size to HMS.
Why are the changes needed?
If the number of stats exceeds limit, the operation may fail.
Does this PR introduce any user-facing change?
No
How was this patch tested?
Unit test

LGTM. +1


Add ElasticSearchDom W/Out EncrypAtRestOpt #1089

Elastic Search Domain should have encrypt data at rest.
Closes #1089

LGTM ✅


Remove commercial technology targets

@kasanovic This is trivial.  I can rewrite history if you'd like, but that's a bit more of a pain.

This level of description needs to be private?

Here's Krste's email, that's why I tagged him as a reviewer
From<EMAIL_ADDRESS>To<EMAIL_ADDRESS>Subject:    removing tech path names from public repo
Date:       Tue, 04 Apr 2017 03:40:04 PDT (-0700)
Message-ID<EMAIL_ADDRESS>

Palmer,

The following files have path names into NDA tech files - I realize it
technically doesn't expose much because only the file names are given,
but we'll show up on searches (that TSMC does regularly now after the
Stanford tech file incident) and legally this is grey ground.

https://github.com/ucb-bar/plsi/blob/bar/src/technologies/tsmc28hpm.tech.json
https://github.com/ucb-bar/plsi/blob/bar/src/technologies/st28fdsoi.tech.json

I realize this makes the stuff harder to use, but that's life using
NDA tools.  The public library versions are fine to leave in as
examples to help guide people write these files.

This is urgent, and the bad thing is git holds these in repo history,
so would ideally scrub from there too.

Thanks,
Krste

Thanks for the info.

What are the "public library versions"?

The ones I didn't delete (SAED32 and OSU180)
On Apr 4, 2017 12:06 PM, "edwardcwang"<EMAIL_ADDRESS>wrote:

What are the "public library versions"?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/ucb-bar/plsi/pull/24#issuecomment-291600561, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ABUaXmmYRPdDk3PMIx02TipKREqMTnCNks5rspRWgaJpZM4MzCwW
.


[FEAT] Integrate variance code in Nada Numpy

All the links in this issue point to the specific files that need to be modified to achieve this result
Thank you for taking the time to review the issues related to Nada Numpy. This particular issue involves integrating an existing implementation of variance functionality into Nada Numpy. The source code for the variance can be found here: Nada by Example link.
To complete this task, please follow these steps:

Integrate the Variance Function:

In nada_numpy/array.py, integrate the variance class as a new member function. This will allow the variance to be called as array.var().

Add a Wrapper in nada_numpy/funcs.py:

In Nada Numpy, functions can also be called in the form na.var(array). To support this, add a wrapper in nada_numpy/funcs.py. You can refer to the existing functions in this file to see how they simply wrap around array.var() in this context.

Testing:

The tests/nada-tests folder contains the testing infrastructure for nada_numpy. You will need to create one or more scripts to test the variance functionality. Follow these steps for testing:

Add a script to tests/nada-tests/nada-project.toml.
Place your test script in tests/nada-tests/src/, where it will verify the expected behavior.
Generate the test file by running nada generate-test --test-name variance variance and placing it in tests/nada-tests/tests/.
Finally, add the script to the TESTS array in tests/test_all.py to integrate it with the CI/CD pipeline.

can help with this over week. are you generally happy with the implementation of variance here?

Please take a look at PR 75.

Thanks a lot for your PR. I have reviewed it. It still requires a bit of work, but I look forward to seeing your improvements on it.


Explicit reflection support

Use case
I know reflection is disabled in Flutter for optimizing performance and app size. But using code generators for json (de)serialization actually s***s. I know it works but that's not enough. The code becomes pretty ugly which makes its hard to understand as codebase grows. My proposal is adding a feature for explicitly enabling serialization for specific classes we create.
Proposal
For example we could use annotations to annotate classes to be reflectable (not exactly but like [Serializable] attribute in C#).
@serializable()
class Product {
  String name;
  double price;
}

In that way we'll not have to generate de/serialization code each time we change base class fields without sacrificing performance or app size (dart compiler will have to only save reflection data for classes annotated with @serializable() annotation).
I know this proposal probably will be declined but I have hope. Cause without this kind of features it's hard to write productive code. I don't want to feel lazy because of boring build_runner / code generation process.
What about 3rd party packages?
I think reflection could be limited to first party (project related) code only. Because otherwise lots of packages will opt-in for using reflection without even if app doesn't uses that feature.

For reference https://github.com/flutter/flutter/issues/1150.

This must be filed in the Dart issue tracker.


Validation of SSL cert

https://github.com/IndieSquare/counterwallet/blob/master/app/lib/require/network.js#L17 - should this be "TRUE"?

Sorry my late reply.
You are right. It was way to avoid some errors on our development environment.
(Even in this case, it will keep HTTPClient run over HTTPS.)
We resolved it. In the next update, it will always be true.
Thanks.

Updated.


developer mode : some steps are missing in the authorization code flow when we use the PAR

some steps of the flow are missing:

wallet sends a push authorization request as a POST (ok)
wallet receives a request_uri as the response of the POST (missing)
wallet redirect to the issuer authorization endpoint with the request uri as arguments (missing)
wallet receives a code as argument in the redirect to its callback endpoint  (missing)
wallet calls the token endpoint as a POST (ok)

Try with this configuration
https://wallet-provider.talao.co/configuration/webpage?login=guest@EUDI&password=guest&wallet-provider=https://wallet-provider.talao.co
and  OIDC4VCI Test 11

Step  1
replace title of the screen AUTHORIZATION REQUEST by PUSH AUTHORIZATION REQUEST


support change tun interface name

The "tun0" interface is too common, leading to potential conflicts.

thanks for your contribution


Add logic to fetch and maintain CRL cache for external CAs

When using external CAs for chain validation we need the CRLs to verify the chains.
See also #776

Implemented via external wrapper (EE only)


Import SimpleWallet

Pull Request Test Coverage Report for Build 464

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 93.031%

Totals

Change from base Build 448:
0.0%

Covered Lines:
366

Relevant Lines:
398

💛  - Coveralls

Commit wrong branch


feat: rustls feature for reqwest and any other relevant dependencies

+1, this fixes the issue of openssl being required to compile the todo_app_sqlite_axum example for me.


Use python-cloudflare library for Cloudflare pack?

CloudFlare has published python-cloudflare, a wrapper for the Cloudflare API. https://blog.cloudflare.com/python-cloudflare/
Wonder if we should move towards using this for the cloudflare pack?
Thoughts @jjm ?

Worth a short, happy for this to be assigned to me while I look at it.


Switch doc generation to use 6.0

NB: we always build docs with the latest release version of Swift by default. So once Swift 6 is released this change will be redundant and moreover would pin doc gen to 6.0 even once 6.1 released and should be reverted by then.
Tested to be working ok with our doc gen mechanism:

@swift-ci please test


Add --log-level flag to print compiler logs

Fixes: #142

With the env_logger and log crates there's no need to pass a log level into the program at all. It's conventional to use those two crates together and have the caller set RUST_LOG to get logs.


[Enhancement] Optimize the memory usage of SegmentZonemap in all Null scenarios

Fixes #issue
Currently if the column type is varchar(length) and the values is all Null, a string of length will be written to the Segment file, causing the loaded metadata to occupy a large amount of memory.
This PR optimizes mem usage of reading first, and the memory usage of write and segment file size will be optimization on next pr.
CREATE TABLE `t1` (
  `c1` int(11) NULL COMMENT "",
  `c2` varchar(65533) NULL COMMENT "",
  `c3` int(11) NULL COMMENT ""
) ENGINE=OLAP
DUPLICATE KEY(`c1`, `c2`)
DISTRIBUTED BY HASH(`c1`) BUCKETS 192
PROPERTIES (
"replication_num" = "1",
"in_memory" = "false",
"enable_persistent_index" = "false",
"replicated_storage" = "true",
"compression" = "LZ4"
);

mysql> select count(*) from t1;
+-----------+
| count(*)  |
+-----------+
| 143999468 |
+-----------+

Before opt: 12M
mysql> select count(c2) from t1;
+-----------+
| count(c2) |
+-----------+
|         0 |
+-----------+
1 row in set (0.09 sec)

mysql> admin execute on 10005 'System.print(GlobalEnv.GetInstance().segment_zonemap_mem_tracker().consumption())';
+----------+
| result   |
+----------+
| 12647616 |
+----------+
1 row in set (0.00 sec)

after opt: 73K
mysql> select count(c2) from t1;
+-----------+
| count(c2) |
+-----------+
|         0 |
+-----------+
1 row in set (0.08 sec)

mysql> admin execute on 10005 'System.print(GlobalEnv.GetInstance().segment_zonemap_mem_tracker().consumption())';
+--------+
| result |
+--------+
| 73256  |
+--------+
1 row in set (0.01 sec)

What type of PR is this:

[ ] BugFix
[ ] Feature
[x] Enhancement
[ ] Refactor
[ ] UT
[ ] Doc
[ ] Tool

Checklist:

[ ] I have added test cases for my bug fix or my new feature
[ ] This pr will affect users' behaviors
[ ] This pr needs user documentation (for new or modified features or behaviors)

[ ] I have added documentation for my new feature or new function

Bugfix cherry-pick branch check:

[x] I have checked the version labels which the pr will be auto-backported to the target branch

[x] 3.1
[x] 3.0
[x] 2.5
[x] 2.4

[FE PR Coverage Check]
:heart_eyes: pass : 0 / 0 (0%)

[FE PR Coverage Check]
:heart_eyes: pass : 0 / 0 (0%)

[FE PR Coverage Check]
:heart_eyes: pass : 0 / 0 (0%)

[FE PR Coverage Check]
:heart_eyes: pass : 0 / 0 (0%)

@Mergifyio backport branch-3.1

@Mergifyio backport branch-3.0

@Mergifyio backport branch-2.5

@Mergifyio backport branch-2.4


Initial implementation of screencasting support

Tries to implement screencasting support as described in #189
It is still a bit janky but works. I'll try to get it improved.

Damn, that was quick! :smiley:

Thanks, @auchenberg, fixed. Also, do you know is the ideal framerate we should use?

By looking at the implementation there doesnt seem to be a "magic" framerate predefined, but you can listen to the rate of screencastFrameAck's and adjust the frame-rate thereafter.

At a high level I'm a bit concerned about the possible performance impact this may have on apps being inspected.  Also, how is this feature enabled in the DevTools UI?

This is really exciting stuff, great work!

This feature can be accessed in the top right corner of the devtools window, there's a little phone icon where you can toggle screencasting on/off (see the screenshot above) - I didn't know it existed either :)
I indeed noticed a small impact on the app performance while the screencast is running, but I suppose that is somewhat expected. I'm sure there's more optimizations we can do to reduce the impact. This was just my first solution.

This is a power feature that was to be enabled in the DevTools, and the screencapturing isn't started before requested, so I don't see a performance-risk in adding the feature.
It's highly useful, as you don't have to look at the screen of the device.

@felipecsl / @auchenberg I agree that because the optional nature of this we can regress perf a bit.  I think it's totally fine to just get this working with perhaps a warning to logcat when it turns on that it may regress app performance.  I just wanted to make sure we're paying attention to little details like that :)

I'm still seeing OOMs after screencasting after a couple minutes and apparently the memory doesn't get reclamed even after stopping the screencast, so we might have a leak somewhere. I'll get a heapdump later

@felipecsl have you been able to work on a solution to the memory leaks yet?  I've very interested in getting this merged in but obviously we can't just leak memory until we OOM -- that's a pretty frustrating experience for a debug tool...

@jasta definitely! I didn't have time to touch it recently but I'm still plan to fix the OOMs. I'll update it soon!

Alright just had another stab at this, memory consumption looks much better now that we manage (and reuse) the bitmap ourselves (instead of using View.getDrawingCache()) however it's still pretty memory hungry. It doens't seem to be leaking memory or causing OOMs anymore but it obviously needs more thorough testing. Also fixed issues with the size of the bitmap that we sent back to match the requested maxWidth and maxHeight.
A couple screenshots of the current state below:

I don't think we can easily avoid the step of converting the byte array into a char array, since both JSONObject  and SimpleSession require a String to work. If we wanted to send the byte array directly into the wire, that would require a lot of changes.

bump Are we waiting on anything else here? :)

i'm not sure either :)

Okay here's another small round of feedback, mostly nits. Otherwise it looks mostly ready to go from what I can tell

alright comments addressed and commits squashed

Looks pretty good except there are still 2 commits to be squashed :)

DONE!

Awesome :) I will take this for a spin tomorrow and then merge it if it all looks good.

Okay unfortunately I found some serious issues in the code after reviewing it more thoroughly, along with a reliable crash when I tried it out.

no worries, I'm sure it's not 100% solid yet. Lets keep iterating

ok all issues taken care of (except for the multiple peers). I'll get back to it once I have more time this week.

Thanks for the thorough review

@felipecsl thanks for sticking with this PR BTW!  I'm really excited to have this in and I think you and @rickbrew are on the right track trying to massage this into a state where it will be stable and high quality for users.

thank you :)

@rickbrew regarding the multiple peers support, this may sound weird, but I got 2 devtools windows open, both with stetho connected to the same device and it seemed to worked fine. I was able to see the device screen on both devtools windows and they seemed identical in functionality.
Here's a screenshot. Am I missing something?

Wait, are those considered to be the same peer because they are running on the same chrome instance?

I'm not entirely sure :)
I should be able to look at this on Wednesday

Since this PR is getting quite big (and old), I'd propose moving the multiple peers support into a separate follow-up PR. I'd be happy to do that after this one is merged (and also other stuff like support for emulateTouchFromMouseEvent to allow interacting with the app from DevTools ;))

Yeah I'm perfectly happy to defer multi-peer support, so long as we have protection against it in the current PR (which it looks like you added!).

@rickbrew I postponed the 1.2.0 release to after I get back from vacation (I set the new milestone to be Aug 31).  I'd love to get this feature in that release, so I'll defer to you to just merge at will.

This looks good other than my 1 comment above about the name of the getBitmap method. I just tested and it looks like things work pretty well! There's 1 issue I found but I think it's Chrome's fault: element highlighting stops working when screencasting is enabled. DOM.highlightNode just isn't getting called. We can file an issue to track that separately though. Might be "by design."

5 months and still counting :)

sorry I was travelling the last few weeks and got distracted. Will get it finished tonight in time for the Aug 31th milestone :) thanks for the bump, @auchenberg

Alright, method renamed and commits squashed. #shipit!

lgtm, thanks so much!

:muscle: :clap: :tada: :balloon:
Bam! Happy to see this landed, after 5 months of work. Congrats!

@auchenberg glad to see this get in as well, I think it's a huge new step forward for Stetho's feature set.  In particular I really want to add keyboard and mouse support and maybe even add features similar to @koush's latest Chrome screencasting plugin offering.  Though I understand his solution is paid so it's unlikely we'll be able to integrate directly.

finally :beers:
I'd love to get mouse support as well, let's see if I can hack something together again :laughing:

@jasta Agreed. Mouse/Keyboard/Touch would be the ultimate experience, no need to pick-up the device again :)
You mean this, https://seo-michael.co.uk/mirror-screencast-app-for-android-by-koush/?
For the controlling logic, there's probably some inspiration to take from https://github.com/openstf/stf

nice work! does anyone know what the timeframe is for releasing this feature?

I meant this: http://www.vysor.io/
From the user point of view it works similar to @felipecsl's screencasting feature but it is better optimized by utilizing the same machinery as screenrecord and directly pumps input events to the input manager service (I presume).  It won't work for us for security reasons however, unless we wanted to require an additional daemon be running on your device via a Chrome extension or something.  I don't know that we want to go that far, however.

re:vysor…
The screencasting that's part of http://openstf.io/ might be the best I've seen. I was earlier urging Koush (of vysor) to adopt it.  https://github.com/openstf/minicap
We're planning on using the same core technique for screencasting v2 for chrome, as well.

That said, awesome to see screencasting here in stetho! very nice.

@paulirish Yeah, I dug into minicap. Minicap sends a high frequency jpegs across in a stream. Vysor is using MediaCodec to send an h264 stream. Minicap requires OS and architecture specific native binaries as well.
Latency is higher with Vysor's h264 pipeline (though still sub 100ms), and the host PC requirements are steeper, but that lets me keep the required bitrate under 500kbps and still get acceptable quality on a Nexus 6. This bitrate is tunable. Minicap requires a couple MBps. Varies by device resolution obviously.
Chose this route for Vysor Share specifically for remote testing and remote debugging, ie, non-local usage.


Quick proofreading

Hopefully there will not be merge conflicts with #2.

Thank you @chrunchyjesus . Great as usual 👍


Kalendar Firey: Year doesn't change

Describe the bug
The year always stays the same.
To Reproduce

Create a basic Kalendar Firey instance: Kalendar(kalendarType = KalendarType.Firey)
Click on the "next month" button until you cross a year boundary (e.g. "December" > "January")
The year stays the same and the calendar just loops the initial year

Expected behavior
The year should increase/decrease when crossing year boundaries.

Hey, this will be fixed in the new release


Failed to download plugin since 1 hour or so

Since an hour or so Im unable to down load the plugin anymore.
Getting:
 SEVERE: Failed to install Slack Notification jenkins    | java.io.IOException: Failed to download from http://updates.jenkins-ci.org/download/plugins/slack/2.15/slack.hpi (redirected to: http://archives.jenkins-ci.org/plugins/slack/2.15/slack.hpi) jenkins    | 	at hudson.model.UpdateCenter$UpdateCenterConfiguration.download(UpdateCenter.java:1182) jenkins    | 	at hudson.model.UpdateCenter$DownloadJob._run(UpdateCenter.java:1719) jenkins    | 	at hudson.model.UpdateCenter$InstallationJob._run(UpdateCenter.java:1982) jenkins    | 	at hudson.model.UpdateCenter$DownloadJob.run(UpdateCenter.java:1693) jenkins    | 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) jenkins    | 	at java.util.concurrent.FutureTask.run(FutureTask.java:266) jenkins    | 	at hudson.remoting.AtmostOneThreadExecutor$Worker.run(AtmostOneThreadExecutor.java:112) jenkins    | 	at java.lang.Thread.run(Thread.java:748) jenkins    | Caused by: java.io.IOException: Failed to load http://updates.jenkins-ci.org/download/plugins/slack/2.15/slack.hpi to /var/jenkins_home/plugins/slack.jpi.tmp jenkins    | 	at hudson.model.UpdateCenter$UpdateCenterConfiguration.download(UpdateCenter.java:1148) jenkins    | 	... 7 more jenkins    | Caused by: java.io.FileNotFoundException: http://archives.jenkins-ci.org/plugins/slack/2.15/slack.hpi jenkins    | 	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) jenkins    | 	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) jenkins    | 	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) jenkins    | 	at java.lang.reflect.Constructor.newInstance(Constructor.java:423) jenkins    | 	at sun.net.www.protocol.http.HttpURLConnection$10.run(HttpURLConnection.java:1944) jenkins    | 	at sun.net.www.protocol.http.HttpURLConnection$10.run(HttpURLConnection.java:1939) jenkins    | 	at java.security.AccessController.doPrivileged(Native Method) jenkins    | 	at sun.net.www.protocol.http.HttpURLConnection.getChainedException(HttpURLConnection.java:1938) jenkins    | 	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1508) jenkins    | 	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1492) jenkins    | 	at hudson.model.UpdateCenter$UpdateCenterConfiguration.download(UpdateCenter.java:1141) jenkins    | 	... 7 more jenkins    | Caused by: java.io.FileNotFoundException: http://archives.jenkins-ci.org/plugins/slack/2.15/slack.hpi jenkins    | 	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1890) jenkins    | 	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1492) jenkins    | 	at sun.net.www.protocol.http.HttpURLConnection.getHeaderField(HttpURLConnection.java:3051) jenkins    | 	at java.net.URLConnection.getHeaderFieldLong(URLConnection.java:629) jenkins    | 	at java.net.URLConnection.getContentLengthLong(URLConnection.java:501) jenkins    | 	at java.net.URLConnection.getContentLength(URLConnection.java:485) jenkins    | 	at hudson.model.UpdateCenter$UpdateCenterConfiguration.download(UpdateCenter.java:1125) jenkins    | 	... 7 more
or
13:22:47 Failed in the last attempt (curl -sSfL --connect-timeout 20 --retry 3 --retry-delay 0 --retry-max-time 60 https://updates.jenkins.io/2.150/latest/slack.hpi -o /usr/share/jenkins/ref/plugins/slack.jpi)

I'm guessing this has maybe something todo with the release from 45 minutes ago?

2.14 is the latest at the update centre
http://archives.jenkins-ci.org/plugins/slack/
It takes time and sometimes there's infra problems

I tried in the UI which I think i version specific and reverting to 2.150.
I also tried it with the plugin.sh and using version slack:latest. For some reason this also takes 2.150 and not the dir latest from the link you provided.
I guess we will have to wait until after the weekend. Thnx for the update

@timja It seems its there now.

Yup

It does seem not to be backward compatible in combination with the https://github.com/jenkinsci/configuration-as-code-plugin.
We where still using the token option  (yes I know we shouldn't) and this got removed :(. I though a minor version upgrade normally implies it should be backward compatible.
Have a nice weekend

the token should get automatically migrated, the fields aren't removed and getters and setters are still there
since you're using configuration-as-code should be an easy fix to change anyway?

@timja you migration does call save so it does indeed break downgrading, as plugin maintainer I would prefer to call save, so you don't have to carry around the deprecation forever.

Weird, I upgraded from 2.14 to 2.15 and was getting a deprecated token exception and jenkins wouldn't boot up. Will stick with 2.14 for the weekend and will look into it next week.

I would love not to carry it around forever 👍
@jorkzijlstra I think that's a feature of jcasc
https://github.com/jenkinsci/configuration-as-code-plugin/blob/master/plugin/src/test/resources/io/jenkins/plugins/casc/impl/configurators/SelfConfiguratorTest.yml#L1-L5
Because you're using a deprecated attribute it forces you to update, if you want to continue on the new version and not have it fail you can configure it here:
https://github.com/jenkinsci/configuration-as-code-plugin/blob/master/plugin/src/test/resources/io/jenkins/plugins/casc/impl/configurators/SelfConfiguratorTest.yml#L1-L5

Oh your right @timja he has to use the JCasC feature for only warning about deprecation :smile: but as I said it my review comments it is best for the deprecated code to behave as the new attribute.

will look at it next week most likely unless someone beats me to it.
The issues don't look critical

@timja and @casz Thnx for the quick replies

@jorkzijlstra its not a major version because it is compatible
migration code handles it, but jcasc is of the opinion that you shouldn't use deprecated fields.

@timja thnx, I now understand. Thnx for setting me straight :).


Update rank_combination.py

In my test in Python 3.8.10, This will prevent "AttributeError: 'str' object has no attribute 'decode'".

Thank you for your report. The reason for this error is that I added universal_newlines=True, causing the output of communicate() to be of type str instead of byte. Please remove the unrelated change, and I will proceed with the merge process afterwards.

OK, I have removed the new line at end of the file.


Added some conditional rendering and fixed some conditional rendering issues

Description
Added some conditional rendering and fixed some conditional rendering issues
Type of change
Please delete options that are not relevant.

[x] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)

Change status

[x] Complete, tested, ready to review and merge

How Has This Been Tested?
Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

[x] Tested using a local version of the app

Checklist:

[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code
[x] My code has been reviewed by at least one peer
[x] I have commented my code, particularly in hard-to-understand areas
[x] I have made corresponding changes to the documentation
[x] My changes generate no new warnings
[x] I have added tests that prove my fix is effective or that my feature works
[x] New and existing unit tests pass locally with my changes
[x] There are no merge conflicts

you guys are killing it this morning. Excited for the presentation 💯


WebRTC Multiplayer peer, documentation

In this PR:

Add WebRTCMultiplayer  (high level multiplayer implementation)
Normalize WebRTCDataChannel properties across browsers.
Add documentation for WebRTC classes.

The multiplayer implementation is a full p2p mesh, and support a "server emulation mode" where it expects to connect to a peer with id 1 and emits the server connection/disconnection events (see docs for more info).
There is a fork of the multiplayer bomber demo using WebRTC here:
https://github.com/Faless/bomber-rtc
You can try a live version here
Signaling server is the one from:
https://github.com/Faless/gd-webrtc-signalling
Running the game on native platforms requires usual WebRTC GDNative plugin -> https://github.com/godotengine/webrtc-native/releases/tag/0.3
Closes #14627, closes #12108

@StraToN thanks for the review! Updated with your suggestions :)

Thanks!


Fix contact form persistence

Email and message are persisting via submission/returning to page but name isn't, potentially animate the email icon with a sending/success icon with ajax submission too

Fixed in 92de74e84ae70a9991550b82c24a03e4e62ced9c


Update state of development

Description
Now that SFML 3 is released, there is little reason to mention the SFML 2 branch. Users can simply focus on version 3.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>5
Details

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 57.016%

Totals

Change from base Build<PHONE_NUMBER>5:
0.0%

Covered Lines:
12058

Relevant Lines:
19988

💛  - Coveralls

#3355 should be merged first


add python 3.12 job

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (303ee18) 99.00% compared to head (58590b4) 99.00%.

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@           Coverage Diff           @@
##           master      #16   +/-   ##
=======================================
  Coverage   99.00%   99.00%
=======================================
  Files           7        7
  Lines         303      303
  Branches       53       53
=======================================
  Hits          300      300
  Misses          2        2
  Partials        1        1

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Usage examples not working.

On macOS, I tried to run the command line examples from the usage section. However, none of them worked for me.
For example,
vulkan_samples swapchain_images should really be ./build/mac/app/bin/Release/x86_64/vulkan_samples sample swapchain_images (The original example missing the option sample)
and
vulkan_samples sample afbc --benchmark 5000  would report that 5000 is not expected.

Thanks for noticing. We recently switched argument handling and haven't updated the readme yet.

The benchmark mode currently doesn't properly work. @TomAtkinsonArm can you take a look?

Ah these will need updating.
Plugins attempted to introduce better CLI practices which in turn left a more explicit CLI and most features CLI options where changed.


Add atom-hclfmt package to README.md

I built a little package for atom to run hclfmt to run it automatically. :-)

That's great 👍


[doctest] updating to version 1.2.0

Testing:

https://ci.appveyor.com/project/ingenue/hunter/build/1.0.1731
https://travis-ci.org/ingenue/hunter/builds/233546102?utm_source=github_status&utm_medium=notification

Testing

Some toolchains stop working. Was fine before:

https://travis-ci.org/ingenue/hunter/builds/216186171

android-* and analyze toolchains stop working. I guess Android lacking of C++11 support. Analyze report some issue, need to investigate is it something valid or not... Feel free to send new fixed version or just exclude toolchains from testing by sending pull request to<EMAIL_ADDRESS>branch. Pull request should contains only .travis.yml/appveyor.yml update, see example:

https://github.com/ingenue/hunter/pull/56/files

thats unfortunate... In version 1.2 all calls to printf or snprintf got prefixed because of this PR - and I include the C headers like <cheader> instead of <header.c>...... so the android toolchain is at fault here.
And I analyze my code with clang-tidy-4 but I suppress some things - so I'm not sure what to do about the builds which fail now with clang-analyze - I feel such testing should be left up to the project itself

I created a PR for that like you suggested

Thanks.
Testing:

https://ci.appveyor.com/project/ingenue/hunter/build/1.0.1740
https://travis-ci.org/ingenue/hunter/builds/233593160?utm_source=github_status&utm_medium=notification

Uploading:

https://ci.appveyor.com/project/ingenue/hunter/build/1.0.1741
https://travis-ci.org/ingenue/hunter/builds/233600171?utm_source=github_status&utm_medium=notification

Released: https://github.com/ruslo/hunter/releases/tag/v0.18.62


Allow for SUFFIXED in confluent_kafka_acl

We have a need to support wildcarding and suffixes when creating confluent_kafka_acls.
We've tested using *_some_suffix using the PREFIXED pattern type but that doesn't work.
We have about 21 topics that already have a suffix that we'd like to key off of, changing them to using PREFIX doesn't seem prudent if there's support for SUFFIX.

Not sure whether the acl can given based on regular expression?

👋 it seems like the question more about Kafka REST API and not about TF Provider for Confluent.
That said, based on

https://stackoverflow.com/questions/54864129/kafka-acl-wildcard-suffixed-topic-name-support
https://kafka.apache.org/21/javadoc/org/apache/kafka/common/resource/PatternType.html

It looks like the answer is SUFFIXED  / using a similar wildcard abc_* this is not supported - MATCH is only for listing ACLs and the only modes available for creating them are "LITERAL" and "PREFIXED":


Support INT16 quantisation for RESIZE_NEAREST_NEIGHBOR

Per issue #43064 I'm requesting the RESIZE_NEAREST_NEIGHBOR of int16 implementation.
RuntimeError: Quantization to 16x8-bit not yet supported for op: 'RESIZE_NEAREST_NEIGHBOR'.
Thanks.

@peter197321
please share the code anf tf version or colab gist for us to analyse.

Hi @Saduf2019
this is the code we have been experimenting with https://gist.github.com/macenpav/404e65a23affc193a428d96a7260a2f4
There is a workaround using tf.lite.OpsSet.TFLITE_BUILTINS, but that creates quantize/dequantize nodes, i.e. f32 layer because RESIZE_NEAREST_NEIGHBOR is not supported (in the UpSampling layer).
On tensorflow==2.3.0 it prints just:
RuntimeError: Quantization to 16x8-bit not yet supported for op:
On tf-nightly (tf_nightly-2.4.0.dev20200910-cp37-cp37m-win_amd64.whl) it shows which op is not supported:
RuntimeError: Quantization to 16x8-bit not yet supported for op:
RESIZE_NEAREST_NEIGHBOR

Hi @peter197321 ! Above Runtime error is not replicating in the 2.8 version now. Can we move this issue to closed status now?


Use picture of Uppsala as "hero" background

It's there now, but the color of the text and logo need to be fixed. They're kind of unreadable right now 😢

replaced it with a darker one so that the white text is visible


Causes kitty to crash when changing window size

Not sure how to get meaningful logs but I'm able to reproduce a crash after scrolling up/down and toggling fullscreen in succession.
kitty: 0.29.0
neovim: v0.9.1
image.nvim: https://github.com/3rd/image.nvim/commit/24c312191ca6bc04e45610a7bcb984d3bf208820

@kovidgoyal thanks for joining, but most importantly thanks for Kitty, it's the software I spend most of my time with, absolutely love it! (userland)
I noticed some weird rendering artifacts as well after upgrading Kitty, last week one instance got into an interesting state where rendering any image would make the whole background become transparent, and clearing the images would bring it back to normal.

 And as always I am always happy to investigate bugs if you can provide reproducers. An easy way to do so is to use --dump-bytes when running kitty and perform the minimum steps needed to reproduce Then along with the terminal window size the dump should allow me to repro.


Defers the WorkingCopyService.onDidContentChange event

Previously dealing with working copies was taking about 1.5% of the total runtime for the keypress function, this reduces it to around 0.2% by deferring all handling of the event until the text has rendered. Since the onDidContentChange event data is not used, this should be a low risk change.
Part of #161622

Testing methodology outlined in https://github.com/microsoft/vscode/issues/161622#issuecomment-1279172489
Before:

After:

Closing as it's a bit stale and busy enough with other stuff


Update readme.md

Modified readme.md

can you provide the issue number?

@rish-singhal Please Close this PR its not of any USE.


⚠ Bofrost.fr has degraded performance

In e8bf302, Bofrost.fr (https://www.bofrost.fr) experienced degraded performance:

HTTP code: 200
Response time: 1508 ms

Resolved: Bofrost.fr performance has improved in 1276170.


Crash after installing pod  for Vuukle

Hi,
I need to integrate Vuukle SDK in my project written in Objective C. The project has been developed from a lower version of Xcode . But now I have updated Xcode to Version 8.2.1 . I have installed the pod file as per the documentation.
platform :ios, '9.0'
target 'My_Project' do
use_frameworks!
pod 'Vuukle', :git => 'https://github.com/vuukle/vuukle_iOS_SDK.git', :branch => 'swift_3'
end
post_install do |installer|
installer.pods_project.targets.each do |target|
target.build_configurations.each do |config|
config.build_settings['SWIFT_VERSION'] = '3.0'
end
end
end
After the pod installation, when I run the project, the app crashes.
This following is displayed:
dyld: Library not loaded: @rpath/Alamofire.framework/Alamofire
Referenced from: /var/containers/Bundle/Application/590FFFB5-2740-4948-AB8F-5AEEE6C402A1/MyProject.app/MyProject
Reason: image not found

We can't reproduce this issue and as crash reason you wrote is "Alamofire can't find image"
So please add more details when this crash happens (during command "pod install" or after running a project)
Also if you have a possibility, please record a broadcast of your screen.
Alamofire it's the other library which we use as dependency for you framework
Please make sure you placed the image in "Alamonfireimage" folder and not in "Alamonfire" folder.

@lesukk After successful pod installation when I run the project the following  crash" dyld: Library not loaded: @rpath/Alamofire.framework/Alamofire" occurred.
So I tried to solve  this crash by deintegrating and re-installing the pod file. But it was a temporary fix only.
For the first time when I run the project, the project builds and runs successfully. Afterwards, after few successful times running the app , the application crashes randomly ("dyld: Library not loaded: @rpath/Alamofire.framework/Alamofire").
To recover from this  I have to delete the app from the device/simulator and reinstall and it works fine. And this lasts for few times so I can run the app and again it crashes and I have to uninstall and reinstall the app.

Did you add alomofire to your project podfile ?
If yes then you should remove it.

No, i am not using alomofire. we cant reprouduce it.


Migrate coverage from v8 to c8

Clear and concise description of the problem
The latest stable version of vitest 0.34.6 uses v8 for that flavor of coverage. However, from v8,

This package has been deprecated
Author message:
This tool was built when c8 package wasn't ready to be used, c8 has now more functionality and works better that this module. So you should really use c8.

I tried to install @vitest/coverage-c8 manually but this version of vitest won't use it, and just wants to install v8.
Suggested solution
Go back to c8
Alternative
Suggest to install c8, but use c8 or v8 - whichever one is available
Additional context
Pull request that includes the motivation for moving from c8 to v8 https://github.com/vitest-dev/vitest/pull/3339
Validations

[X] Follow our Code of Conduct
[X] Read the Contributing Guidelines.
[X] Read the docs.
[X] Check that there isn't already an issue that request the same feature to avoid creating a duplicate.

Workaround: pin vitest version to 0.31

Vitest never used that package. To use v8 coverage, install @vitest/coverage-v8.


New Glassmorphism Button

Glassmorphism Button Documentation
The Glassmorphism Button is a modern and attractive button design that incorporates the glassmorphism effect, creating a glass-like appearance with subtle shadows and transparency. This documentation provides a step-by-step guide on how to implement and customize the Glassmorphism Button.
Table of Contents

Introduction
Getting Started

Prerequisites
Installation

Usage

HTML Structure
CSS Styling

Customization

Colors
Font
Button Size
Button Animation

Browser Compatibility
Examples
FAQ
License

Introduction
The Glassmorphism Button is a design element commonly used in web interfaces to create visually appealing and interactive buttons. It follows the principles of the glassmorphism trend, which combines a frosted glass-like effect with subtle shadows and transparency to give the illusion of depth and dimension.
This button design is versatile and can be used in various contexts, such as call-to-action buttons, navigation buttons, or interactive elements within a webpage.
Getting Started
Prerequisites
To use the Glassmorphism Button, you need the following:

An HTML file for your webpage.
A CSS file to style the button.

Installation

Create an HTML file and name it as desired (e.g., index.html).
Create a CSS file and save it in the same directory as your HTML file (e.g., styles.css).

Usage
HTML Structure
In your HTML file, you need to include the following structure for the Glassmorphism Button:
<!DOCTYPE html>

Button Text

You can add multiple buttons by duplicating the
...
 section within the .container div.
CSS Styling
To style the Glassmorphism Button, open your CSS file (styles.css) and add the following CSS code:
@import url("https://fonts.googleapis.com/css2?family=Poppins:wght@400;500;600;700;800;900&family=Roboto:wght@400;500;700&display=swap");

* {
  margin: 0;
  padding: 0;
  box-sizing: border-box;
  font-family: "Roboto", sans-serif;
}

/* Add your own styles here */

The @import rule is used to import the Google Fonts needed for the button styling. You can modify the imported fonts as per your preference.
Customization
The Glassmorphism Button can be customized to suit your design needs. Here are some aspects you can customize:
Colors
You can change the background colors of the buttons by modifying the background property for the .container .btn a selector. Choose the desired color or use hexadecimal, RGB, or HSL values.
.container .btn a {
  background: rgba(255, 255, 255, 0.05);
}

You can adjust the transparency (0.05) or change the color values to achieve the desired appearance.

Hi, are you still working on this?

Closed due to no activity, please create a fresh PR if you're still working on this


Broken Test Case [Responsive Web Design - Basic CSS - Import a Google Font]

Affected page
Responsive Web Design - Basic CSS - Import a Google Font
Your code

<link href="https://fonts.googleapis.com/css?family=Lobster" rel="stylesheet" type="text/css">

CatPhotoApp

Click here to view more cat photos.

Things cats love:

cat nip

laser pointers

lasagna

Top 3 things cats hate:

flea treatment

thunder

other cats

  <form action="https://freecatphotoapp.com/submit-cat-photo">
    <input type="radio" name="indoor-outdoor" checked> Indoor
    <input type="radio" name="indoor-outdoor"> Outdoor

    <input type="checkbox" name="personality" checked> Loving
    <input type="checkbox" name="personality"> Lazy
    <input type="checkbox" name="personality"> Energetic

    <input type="text" placeholder="cat photo URL" required>
    <button type="submit">Submit</button>
  </form>

Expected behavior
Should pass all test cases 100%, but instead fails on the third test case "You should only use an h2 element selector to change the font".
System (please complete the following information):

Device: Laptop
OS: Windows 8.1 Pro Latest Update
Browser: Firefox, Chrome, Opera
Version: Latest All

Additional context
The bug exists at the rule declaration colon where adding a space breaks the test fixtures. The following piece of CSS doesn't pass the tests, despite being valid and correct:
h2 {
    font-family : Lobster; // Extra space before colon
  }

While the following passes:
h2 {
    font-family: Lobster; // Removed space before colon
  }

Per the W3C Syntax and basic data types documentation, the declaration : property S* ':' S* value; regex is used to verify CSS rule declarations. I'd like to work on a fix for this if this will be fixed.

The test is here. The regex doesn't allow spaces there.
/\s*[^\.]h2\s*\{\s*font-family\s*:\s*(['"]?)Lobster\1\s*(;\s*\}|\}) should work

@moT01 Okay so, there's more than just that. I mentor a student based on the freeCodeCamp curriculum and a few of this bug have turned up so far. What is the Regex flavor? I tried running the Regex you provided in the fix using JavaScript and it fails.

Should I open another issue that's more general and mention this or continue in this one?

Is there still an issue of concern here @KL13NT?

There's a number of other lessons that suffer the same Regex bug, yes. @moT01

There's a number of other lessons that suffer the same Regex bug, yes. @moT01

Hey @KL13NT , I suggest you open a new issue with a list of the affected challenges if you are up to the task.

Sure thing.


pvlib.iotools.get_pvgis_hourly's surface_azimuth parameter doesn't use pvlib's azimuth convention

Nearly everything in pvlib represents azimuth angles as values in [0, 360) clockwise from north, except pvlib.iotools.get_pvgis_hourly:
https://github.com/pvlib/pvlib-python/blob/3def7e3375002ee3a5492b7bc609d3fb63a8edb1/pvlib/iotools/pvgis.py#L79-L81
This inconsistency is a shame.  However, I don't see any way to switch it to pvlib's convention without a hard break, which is also a shame.  I wonder how others view the cost/benefit analysis here.
See also https://github.com/pvlib/pvlib-python/pull/1395#discussion_r1181853794

Since get_pvgis_hourly is only using surface_azimuth to pass a value to a pvgis keyword aspect, we could add aspect with the South=0 convention and deprecate surface_azimuth as is. Then later, we can deprecate aspect and add surface_azimuth back with its pvlib meaning. Now that I write that out, seems better just to make the breaking change.

This issue discusses that surface_azimuth in get_pvgis_hourly should be change to clockwise from north, but the documentation in pvgis.py mentions counter-clockwise from north, even though the implementation 'aspect': surface_azimuth-180 matches what was discussed in this issue.
I would have a PR at the ready if this is not a misunderstanding on my side.

Hi @tongpu This got fixed in a recent release of pvlib. Please check the What's New section for the specific version.


vignette on data preparation

create a separate vignette on how to prepare your data - import to df, rename columns, compute solar.time, light, DO.sat, check against mm_data() formals, look for duplicates & misordering, etc.

suggestion by email:

consider a short section on formatting data for the package.
For example,
Date and time should be formatted as:
solar.time
7/30/15 0:00
7/30/15 0:05
Convert solar.time into POSIXct using:
dat$solar.time<- as.POSIXct(x=strptime(dat$solar.time,"%m/%d/%y %H:%M”), tz=“UTC")
Add units to the data using:
library(unitted)
dat <-unitted(dat,c("","mgO2 L^-1","m", "degC", "umol m^-2 s^-1","mgO2 L^-1”))


Fix up and reorder search filters reference

Follow up on the documentation changes in #23690, we should update the search operators (a.k.a. search filters, see #23767) in-app reference:

[ ] Replace email with user wherever it's mentioned, as the current search UI has user pills.
[ ] Change the order of filters to match the updated help center page, as indicated on this spreadsheet.

@zulipbot claim

@zulipbot claim


Haxe compiler hanging on compilation of generated code

When any field/type is used of math/big at the moment, for compiled Haxe code the compiler hangs. Using the --verbose flag the last typing is Typing stdgo.math.big._Big.Big_Fields_._karatsubaSqr
It's not very practical to look through the entire function without any specifics from the compiler of where exactly the compiler is looping/stopped etc and other information.
function _karatsubaSqr(_z:T_nat, _x:T_nat):Void {
        var _n:GoInt = (_x != null ? _x.length : ((0 : GoInt)));
        if ((((_n & ((1 : GoInt))) != ((0 : GoInt))) || (_n < _karatsubaSqrThreshold)) || (_n < ((2 : GoInt)))) {
            _basicSqr((_z.__slice__(0, ((2 : GoInt)) * _n) == null ? null : _z.__slice__(0, ((2 : GoInt)) * _n).__copy__()), (_x == null ? null : _x.__copy__()));
            return;
        };
        var _n2:GoInt = _n >> ((1 : GoUnTypedInt));
        var _x1:T_nat = (_x.__slice__(_n2) == null ? null : _x.__slice__(_n2).__copy__()), _x0:T_nat = (_x.__slice__(((0 : GoInt)), _n2) == null ? null : _x.__slice__(((0 : GoInt)), _n2).__copy__());
        _karatsubaSqr((_z == null ? null : _z.__copy__()), (_x0 == null ? null : _x0.__copy__()));
        _karatsubaSqr((_z.__slice__(_n) == null ? null : _z.__slice__(_n).__copy__()), (_x1 == null ? null : _x1.__copy__()));
        var _xd:T_nat = (_z.__slice__(((2 : GoInt)) * _n, (((2 : GoInt)) * _n) + _n2) == null ? null : _z.__slice__(((2 : GoInt)) * _n, (((2 : GoInt)) * _n) + _n2).__copy__());
        if (_subVV(_xd, _x1, _x0) != ((0 : GoUInt))) {
            _subVV(_xd, _x0, _x1);
        };
        var _p:T_nat = (_z.__slice__(_n * ((3 : GoInt))) == null ? null : _z.__slice__(_n * ((3 : GoInt))).__copy__());
        _karatsubaSqr((_p == null ? null : _p.__copy__()), (_xd == null ? null : _xd.__copy__()));
        var _r:T_nat = (_z.__slice__(_n * ((4 : GoInt))) == null ? null : _z.__slice__(_n * ((4 : GoInt))).__copy__());
        (_r == null ? null : _r.__copy__());
        _karatsubaAdd((_z.__slice__(_n2) == null ? null : _z.__slice__(_n2).__copy__()), (_r == null ? null : _r.__copy__()), _n);
        _karatsubaAdd((_z.__slice__(_n2) == null ? null : _z.__slice__(_n2).__copy__()), (_r.__slice__(_n) == null ? null : _r.__slice__(_n).__copy__()), _n);
        _karatsubaSub((_z.__slice__(_n2) == null ? null : _z.__slice__(_n2).__copy__()), (_p == null ? null : _p.__copy__()), _n);
    }

Any help would be greatly appreciated. (not target specific either from what I can tell)

A serious problem, clearly, hitting some internal Haxe limitation. Is there some internal Haxe error code you could view?
The obvious thing would be to reduce the complexity of the generated code in some way, does turning off in-lining improve the situation?

@elliott5 potentially but I think it would require me to build the Haxe compiler from source which is what I plan to do, since I agree it is a serious problem.
Reducing the complexity of the generated code is a good idea, there is some ways to make the situation much better, but I'm hesitant in case it doesn't change the hanging outcome, I'll try it though, and I'm not sure what in-lining you are referring to (unless you mean all of the in place nil checks)

If rebuilding the Haxe compiler from source gives better error indications, it may be worth it.
Yes, the nil/null checks are what stand out as repeated code ... but I appreciate it may not be easy to replace the approach without significant re-writing.
But...
What does the following line in the generated code achieve? The result does not seem to be assigned.
   (_r == null ? null : _r.__copy__());

The matching Go code it was generated from seems to be:
	copy(r, z[:n*2])

Is this a code generation error?

@elliott5 Oh yes it is! Nice catch, oh shoot 😓 I must have removed it as I thought it was a value copy rather then a slice copy. I'll revert back the Go.copy macro function and rename it to avoid confusion as I had for it, thanks!

Still hanging unfortunately after change, but a great change none the less.


introduce ZyteAPITextResponse and ZyteAPIResponse to store raw Zyte Data API Response

Fixes #9

The approach of storing the Zyte API response inside the response.meta was avoided since it will break the expectation in Scrapy that response.meta comes directly from response.request.meta, as what this code snippet embodies:
@property
def meta(self):
    try:
        return self.request.meta
    except AttributeError:
        raise AttributeError(
            "Response.meta not available, this response "
            "is not tied to any request"
        )

In addition, this is what the official Scrapy docs says about response.meta:

Unlike the Response.request attribute, the Response.meta attribute is propagated along redirects and retries, so you will get the original Request.meta sent from your spider.

Thus, this PR introduces these new classes which inherits from Scrapy's Response and TextResponse respectively:

ZyteAPITextResponse
ZyteAPIResponse

This enables users to retrieve the raw Zyte Data API response as a dict via response.zyte_api_response.
TODO:

[ ] more tests
[x] update README
[x] update CHANGELOG

I was going to suggest changing the class names, because eventually we probably want to allow non-browser responses to be TextResponse subclasses (#15).
However, now I am thinking this naming is OK, and what needs to change to address #15 is only that we use ZyteAPITextResponse for non-browser responses as well when those are plain-text responses.
I imagine there is a case for users wanting to know if the response content comes from browser or non-browser data, but I don’t think having different classes so that this can be checked with isinstance is the right way to go. If anyone requests such a feature, maybe we could add one more property for that (e.g. response.source == "browserHtml").

ZyteAPI*Response directly maps browserHtml if available, else httpResponseBody, as proposed. In the future maybe we can allow users to override this behavior through a plugin-specific zyte_api meta subkey.

Having httpResponseBody and browserHtml at the same time isn't allowed by the API as they're mutually exclusive. This is currently the set behavior in the PR wherein ZyteAPITextResponse is used when browserHtml is present, otherwise ZyteAPIResponse.
However, I might be missing something about the notion of "mapping the browserHtml when available". Is this related to your last two points?

Whether ZyteAPIResponse or ZyteAPITextResponse does not depend on whether the body comes from browserHtml or httpResponseBody, but whether the response is a text response or not.

That's a good point. I'll update https://github.com/scrapy-plugins/scrapy-zyte-api/pull/10/files#diff-ed75145a6134504c5e20e23c0958e4f4929703abac118e0968f055d51a5fde7bR88 on this one.

ZyteAPI*Response.api (rename suggestion for .zyte_api_response) is an instance of ExtractResult as proposed in zytedata/python-zyte-api#12, which is a dict with the API response plus some helpers.

Shortening it is indeed better. I just realized that the *_response suffix is already redundant (i.e. response.zyte_api_response).
I think we can rename it into zyte_api instead as it's more clear where the source is coming from. It could be the case that a Scrapy project is using other APIs. Having an explicit zyte_api name may alleviate some confusion.

ZyteAPIResponse.http and ZyteAPIResponse.browser are regular Scrapy response objects (which class depends on their content and headers) mapping the corresponding content from Zyte Data API, or None when not received. They would be lazy-loaded, so that if you do not use .http it never gets even base64-decoded.
ZyteAPI*Response.replace works on the top-level response data, but .http and .browser remain unchanged. (I have not thought too hard about this one, maybe there are scenarios where this is problematic)

I'm afraid I might be missing something here. Would you mind providing some use cases for having http and browser?

Having httpResponseBody and browserHtml at the same time isn't allowed by the API as they're mutually exclusive.

This is a limitation which will be addressed in future.

This is a limitation which will be addressed in future.

@kmike I see, thanks for confirming. This answers some of questions from @Gallaecio points above. 👍

The fact that browserHtml and httpResponseBody will be supported simultaneously is what makes https://github.com/zytedata/python-zyte-api/pull/12 such a puzzle, and could affect this as well.
I would not want it to block this PR, but I am worried about keeping backward compatibility if we move on with this PR in a way not compatible with the outcome of https://github.com/zytedata/python-zyte-api/pull/12.

Whether ZyteAPIResponse or ZyteAPITextResponse does not depend on whether the body comes from browserHtml or httpResponseBody, but whether the response is a text response or not.

@kmike @Gallaecio I just realized after working on the above suggestion that we'd have no direct way of knowing if the response is of type text (at least in the current setup).
For example, scrapy.Request("https://example.com", callback=self.parse) without Zyte API would lead to a response.headers[b'Content-Type'] of value b'text/html; charset=UTF-8'. We'd directly know that it's an HTML Text and not an image, file, etc.
But if we have scrapy.Request("https://example.com", callback=self.parse, meta={"zyte_api": {"httpResponseBody": True}}), we'd have no access to the headers unless we set httpResponseHeader=True.
Thus, by default, I think we should always set httpResponseHeader=True for proper content decoding. Moreover, making the headers always available would make the transition to Zyte API easier for Scrapy projects that relies heavily on accessing the headers.
Any thoughts on this one?

On the other hand, as @Gallaecio has pointed out in https://github.com/zytedata/python-zyte-api/pull/12/files#r876821518, we currently have the opportunity to expose some functionalities from the underlying client.
For example, perhaps we could have ExtractResult.is_text() to easily check if its a text response.

hey! I think that here we should focus on exposing raw API response, as JSON. Auto-filling httpResponseHeaders also looks out of scope, it's a separate discussion.
@Gallaecio's questions:

ZyteAPI*Response directly maps browserHtml if available, else httpResponseBody, as proposed. In the future maybe we can allow users to override this behavior through a plugin-specific zyte_api meta subkey.

This sounds good to me. I'm not sure we would even need to override this behavior in future.

Whether ZyteAPIResponse or ZyteAPITextResponse does not depend on whether the body comes from browserHtml or httpResponseBody, but whether the response is a text response or not.

That's correct. Though browserHtml responses are always text responses.

ZyteAPI*Response.api (rename suggestion for .zyte_api_response) is an instance of ExtractResult as proposed in https://github.com/zytedata/python-zyte-api/pull/12, which is a dict with the API response plus some helpers.

I'd keep it out of scope here. We can add more helpers later, and discuss it separately.

ZyteAPIResponse.http and ZyteAPIResponse.browser are regular Scrapy response objects (which class depends on their content and headers) mapping the corresponding content from Zyte Data API, or None when not received. They would be lazy-loaded, so that if you do not use .http it never gets even base64-decoded.

I think that's fine not to implement it now, and focus on returning the raw data.

ZyteAPI*Response.replace works on the top-level response data, but .http and .browser remain unchanged. (I have not thought too hard about this one, maybe there are scenarios where this is problematic)

Let's keep this out of scope.
What do you think about implementing the following behavior?

For any Zyte API response user should be able to access the raw returned data (decoded from JSON) as .zyte_api_response attribute.
If there is browserHtml in response, user should be able to run css/xpath selectors, and the response should be a subclass of TextResponse.
When only httpResponseBody is in the output (but not httpResponseHeaders), user can't run selectors, and response is not a subclass of TextResponse.
When both httpResponseBody and httpResponseHeaders are in response, there should be some kind of content type detection (probably relying on default Scrapy's). If data is text, response should be a subclass of TextResponse, and user should be able to run css/xpath selectors. I guess (4) is what can be tricky to implement; if so, some alternative can be explored for (4).

Sounds good to me as well. I've got an idea on how to implement point 4 but testing it with a lot of difference cases is indeed quite tricky. I'll have an update tomorrow after the test cases are sorted out, to see if we're okay with approach.

For any Zyte API response user should be able to access the raw returned data (decoded from JSON) as .zyte_api_response attribute.

Yes, this one is already implemented. Albeit, .zyte_api_response has been renamed to .zyte_api to prevent redundancies in naming like response.zyte_api_response.

If there is browserHtml in response, user should be able to run css/xpath selectors, and the response type should be a subclass of TextResponse.

This is implemented but I've added some tests in https://github.com/scrapy-plugins/scrapy-zyte-api/pull/10/commits/910085bfb749cdc9bec0c7092de3d69c280386b5. Thanks for the reminder!

When only httpResponseBody is in the output (but not httpResponseHeaders), user can't run selectors, and response type is not a subclass of TextResponse.
When both httpResponseBody and httpResponseHeaders are in response, there should be some kind of content type detection (probably relying on default Scrapy's). If data is text, response type should be a subclass of TextResponse, and user should be able to run css/xpath selectors. I guess (4) is what can be tricky to implement; if so, some alternative can be explored for (4).

Implemented this on https://github.com/scrapy-plugins/scrapy-zyte-api/pull/10/commits/e3214d80178169998b06903d71e2b999f25d4eb6

🎉 thanks @BurnzZ! 🚀


graphviz tests dont actually run because the binaries are not in PATH on the build machines

Investigate why default-graphviz-program doesn't seem to find the executables and fix it for the build machines.

@erg is this fixed now that you added find-in-standard-login-path

Yes, fixed!


feat: add CustomComicId and CustomEpId

Allows user to customise comics and page numbers read by comic reading task.
No need to know which comic was read, so instead of getting it, just remove from log. It can be done again if an issue calls for it.
Allow edits by maintainers.

Notice: I'm not going to test this pull request because I think it will just fail due to user error.

WARN
Changes were just pulled from develop branch to resolve merge conflict, but unexpectedly CodeQL reported that build failed.

WARN Changes were just pulled from develop branch to resolve merge conflict, but unexpectedly CodeQL reported that build failed.

Maybe you can try pull the latest codes from upstream, then rebase your feature branch to develop branch.


Investigate coding style automatic check and enforcement

Suggested enhancement
Investigate how we could check automatically our coding style and have a tool to format C code according to our coding style.
Stories:

CI runs a coding style check script and the associated job fails if the check script finds an issue in the PR code changes (check of the diff not of whole modified files).
The project contains a coding style check and formatting script that developers can use to check and enforce the coding style before to submit their code to GitHub.

Coding style change ?
To achieve this the current coding style could be modify to simplify things but this is not mandatory. To not disturb too much our current coding practices a coding style quite close to the current one would probably be better.
Level of enforcement?
The scripts do not need to reach 100% enforcement if it is too complicated to do so.
Tooling?
In preliminary discussions clang-tidy was mentioned as a possibly suitable tool.
Justification
Mbed TLS needs this because this would save development and review time.

Clang Tidy offers a variety of style checks, such as

option
style/convention

darwin-
Checks related to Darwin coding conventions.

fuchsia-
Checks related to Fuchsia coding conventions.

google-
Checks related to Google coding conventions.

hicpp-
Checks related to High Integrity C++ Coding Standard.

linuxkernel-
Checks related to the Linux Kernel coding conventions.

llvm-
Checks related to the LLVM coding conventions.

llvmlibc-
Checks related to the LLVM-libc coding standards.

Why don't we use the existing style, instead of a custom one?

That's definitively an option. Is there one that is quite close to our current coding style ?

That's definitively an option. Is there one that is quite close to our current coding style ?

I'm looking into it.

Unfortunately existing coding styles are quite similar to each other, and all differ from ours significantly.
linux kernel:
int fun(int a)
{
        int result = 0;
        char *buffer;

        buffer = kmalloc(SIZE, GFP_KERNEL);
        if (!buffer)
                return -ENOMEM;

        if (condition1) {
                while (loop1) {
                        ...
                }
                result = 1;
                goto out_free_buffer;
        }
        ...
out_free_buffer:
        kfree(buffer);
        return result;
}

darwin kernel:
Boolean
OSCompareAndSwap8(UInt8 oldValue, UInt8 newValue, volatile UInt8 *address)
{
	return (Boolean)os_atomic_cmpxchg(address, oldValue, newValue, acq_rel);
}

google (and fuchsia)
class Shape {
 public:
  virtual void Rotate(double radians) = 0;
};

class Circle : public Shape {
 public:
  void Rotate(double radians) override;
};

void Circle::Rotate(double /*radians*/) {}

llvm-libc
LLVM_LIBC_FUNCTION(int, sigaction,
                   (int signal, const struct __sigaction *__restrict libc_new,
                    struct __sigaction *__restrict libc_old)) {
  struct sigaction kernel_new;
  if (libc_new) {
    copySigaction(kernel_new, *libc_new);
    if (!(kernel_new.sa_flags & SA_RESTORER)) {
      kernel_new.sa_flags |= SA_RESTORER;
      kernel_new.sa_restorer = __restore_rt;
    }
  }
  ...
}

gnu
for (i = 0; i < num_elements; i++)
  {
    foo[i] = foo[i] + 42;

    if (foo[i] < 35)
      {
        printf ("Foo!");
        foo[i]--;
      }
    else
      {
        printf ("Bar!");
        foo[i]++;
      }
  }

In summary:

no existing style is using 4-space indent, it's either 2 or 8.
no existing style is using a space between function's paranthesis and it's arguments,
such as  int do_calc_length( const unsigned char *str )

Perhaps it's necessary to actually write our own clang-tidy config file, however I would suggest leaning towards existing styles when it comes to indentation or space placement.
btw - what is the reason behind mbedTLS using it's own coding style?

I don't know why Mbed TLS has its own coding style. It's already the case in the first git commit from PolarSSL (ancestor project of Mbed TLS) but not in the most recent version I have from XySSL (ancestor project of PolarSSL). In any case, I want to move away from this style to a standard style.
Here are my requirements for the coding style:

Must be reasonably standard so that people can easily read and write code. I propose to start with K&R style, because that's a well-established point of reference, and any deviation must be justified.
Must be mostly enforced by a tool so that we don't waste effort in reviews.
Must make the code easy to read and debug without special tooling.

No tabs because you will end up with different people using different tab widths.
Avoid or forbid multiple instructions on the same line because it makes it harder to step through in a debugger.
Braces are preferred or mandatory even for single-line loop/conditional blocks to avoid goto fail bugs.
Empty statements need a comment because a lone {} or ; is easily lost in refactoring.
80 columns is recommended because that's the standard width, but not enforced because it's extra work to make and review refactorings when they introduce line breaks in random places.
Avoid mandating too many blank or almost-empty lines (e.g. lines with only {) because vertical space is expensive — many people are stuck with wide but short monitors.

I think we have a consensus on using clang-tidy as the tool. So I propose to write a script that reindents the whole code base with a given style, and look at the output for the standard styles. Then we pick one and tweak it if desired.
All contributors should be able to run the tool, so we shouldn't require a version that's too recent. Note that our CI still has systems with Ubuntu 16.04 where the version of clang-tidy is 3.8, but we can live with only running the tool on Ubuntu 18.04 with Clang 6.0.

I found the clang-format file that linux kernel is using.
https://github.com/torvalds/linux/blob/master/.clang-format
I believe this could be tweaked a bit to achieve desired outcome.

Attached a PR with aes.c modified by
clang-format library/aes.c -i


Fix explorer URLs in examples

Description of change
Adds Network::resolver_url and corrects the examples to print both the explorer URL for messages and identity-resolver URL for DIDs, including the Wasm examples.
This fixes examples like create_did which print the wrong URL for messages.
E.g.
DID Document Transaction > https://explorer.iota.org/mainnet/identity-resolver/message/<message_id>

should be
DID Document Transaction > https://explorer.iota.org/mainnet/message/<message_id>

Type of change
Add an x to the boxes that are relevant to your changes.

[x] Bug fix (a non-breaking change which fixes an issue)
[ ] Enhancement (a non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] Documentation Fix

How the change has been tested
Unit tests and Wasm examples pass locally.
Change checklist

[x] I have followed the contribution guidelines for this project
[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[x] I have added tests that prove my fix is effective or that my feature works
[x] New and existing unit tests pass locally with my changes

Ideally we would move the explorer stuff out of Network completely, since it's really only for the examples.

Thanks for the review and I definitely like the ExplorerUrl code. The only change I would make is to allow the creation of a custom ExplorerUrl for the private Tangle examples.
I'm going to leave that for a future PR as it's a nice-to-have improvement currently.


"database is locked" SQLite error with clean_duplicate_history command

I noticed every once in a while my app would log a "database is locked" error:
[2022-01-26 03:45:25,580] [22345] [  ERROR] Internal Server Error: /cards/
[2022-02-02 04:45:51,686] [26440] [  ERROR] Internal Server Error: /sessions/
[2022-02-05 18:45:49,874] [31872] [  ERROR] Internal Server Error: /stats/usage/
[2022-02-08 22:45:57,367] [978] [  ERROR] Internal Server Error: /stats/usage/
[2022-02-10 04:45:49,758] [15725] [  ERROR] Internal Server Error: /stats/usage/
[2022-02-19 20:45:46,361] [28267] [  ERROR] Internal Server Error: /rest-auth/login/
[2022-03-09 12:45:45,934] [27711] [  ERROR] IPN route - OperationalError - database is locked
[2022-03-14 10:45:43,600] [27708] [  ERROR] IPN route - OperationalError - database is locked
[2022-03-20 00:45:27,309] [13434] [  ERROR] Internal Server Error: /stats/usage/
[2022-03-23 02:45:46,301] [13435] [  ERROR] Internal Server Error: /stats/usage/
[2022-03-30 02:45:56,876] [13435] [  ERROR] Internal Server Error: /stats/usage/
[2022-03-30 03:45:44,279] [13432] [  ERROR] Internal Server Error: /stats/usage/
[2022-03-31 15:45:36,220] [4628] [  ERROR] Internal Server Error: /stats/usage/
[2022-04-01 02:45:26,084] [4629] [  ERROR] Internal Server Error: /stats/usage/
[2022-04-09 17:45:46,624] [24482] [  ERROR] Internal Server Error: /stats/usage/

When it happened, it was always at the 45th minute. This is when I run the clean_duplicate_history command with cron:
45 * * * * /opt/[snip]/python /opt/[snip]/manage.py clean_duplicate_history -m 60 --auto >> /var/log/[snip]-cron.log 2>&1

This is a difficult situation because:

the project is so small it's not worth migrating off SQLite
I've already increased the database timeout to 20 seconds. I don't want to increase it further because I don't want requests to take that long.
running the command once per day at night isn't much better because I could miss a PayPal transaction

I suspect this atomic transaction is what's taking so long. Would it be possible to break it into pieces so that other threads could access the database?
Or is there a different solution you guys recommend? Thanks!

I realize this is an old issue but here is what is happening:

You have a corn that runs every 45 minutes.  It executes a management command that creates a database connection
Your process is failing at minute 45
This is because your first command is still running while the crontab has invoked another execution
Since SQLite can only handle a single connection at a time, and since crontab is spawning another connection, you are seeing a database lock error

Thanks for the reply. It's not practical to stop my app to run the clean_duplicate_history command each night because I might miss an important request like a PayPal payment notification. It needs to stay running.
The clean_duplicate_history was locking the database for too long, which is why I filed this issue.

You're misunderstanding what I m saying. What RDBMD are you using?

Yes, I understand that. My issue described one solution:

I suspect this atomic transaction is what's taking so long. Would it be possible to break it into pieces so that other threads could access the database?

SQLite is actually fine for production use: https://www.sqlite.org/whentouse.html
Each writer is supposed to hold a lock for a short period of time, not many seconds like the  clean_duplicate_history command. Other writers queue up and wait their turn. The "database is locked" exception is raised when another writer waits for longer than a set timeout (in my case 20 seconds).
Note that the Django docs specifically call this out:

Performance considerations
Open transactions have a performance cost for your database server. To minimize this overhead, keep your transactions as short as possible. This is especially important if you’re using atomic() in long-running processes, outside of Django’s request / response cycle.


Update Excel Export to Include Low 7Q2 Data - WALLY (High #2)

As a WALLY user I want the Low 7Q2 data exported into the Excel so that I am able to further analyze EFN (Environmental Flow Needs) risk or make further adjustments as necessary.
Acceptance Criteria

[ ] Given a WALLY user selects Low 7Q2 data data to export, when they export the data, then they are given all data and summary information in a format compatible with Excel.
[ ] Export should have this data in the Summary tab

Data comes from here:

Should export into Excel here:

Additional Context

Low 7Q2  means rolling average 7 day flow every 2 years

Data to be included in the Excel export that is not currently being exported and is manually added into Excel to evaluate EFN risk

Export function: This function allows users to export to Excel (or as a shapefile).  However, the current output from WALLY into Excel only contains partial information that is found in the online staging environment.

Low 7Q2 is not calculated and is pulled directly from a spreadsheet.

There are multiple tickets that pertain to updating the Excel export function.

As I understand the exact value of Low7Q2 from the Wally Watershed web page should be added to the exported Excel file in the Summary tab before Glacial coverage %. Is that right?

This data has been added to PR #732 and will be released with Ticket #707
The data has been added to the Summary page in the excel document.
Private Zenhub Image


[WIP] Implement loss recovery from the current WG draft

Fixes #498 and will hopefully go a long way towards fixing the many
flaky tests.

I think we need to check if a packet is retransmittable in sentPacketHandler.SentPacket() (it probably makes sense to define this as a method on the Packet.

Agreed, but I'd rather do this as a separate PR later. Filed #505.

Codecov Report

Merging #502 into master will increase coverage by 0.06%.
The diff coverage is 95.93%.

@@            Coverage Diff             @@
##           master     #502      +/-   ##
==========================================
+ Coverage   90.76%   90.82%   +0.06%
==========================================
  Files          81       81
  Lines        5421     5446      +25
==========================================
+ Hits         4920     4946      +26
  Misses        321      321
+ Partials      180      179       -1

Impacted Files
Coverage Δ

ackhandler/packet.go
100% <ø> (ø)
:arrow_up:

session.go
87.5% <100%> (+0.03%)
:arrow_up:

ackhandler/sent_packet_handler.go
95.65% <95.76%> (+1.18%)
:arrow_up:

flowcontrol/flow_controller.go
97.67% <0%> (-0.05%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 959a8d0...3b79ecf. Read the comment docs.

Does this resolve the flakiness of our integration tests? If so, I'd be very happy if we could remove the travis_retry hack.


Please add the support for Export Item API

Microsoft exchange Introduced new API ExportItems which is used to get all content related to mail.

Is this even implemented in the .NET API? It wasn't last I knew, but that may have changed. The Export Item will just return a huge Base64 string containing all properties of an item, if I'm not mistaken. You'd need to be able to extract all of the properties from that binary data to make any use of it. This can be done (I built something similar a while back to extract properties from the forward recipients binary property), but it's not trivial, and to fully implement such a system would require some significant and potentially confusing changes to the API, ie, the recipients table would have extra properties not available in the vanilla API calls, which might mean extending various objects (ie, Attendees) to hold these various properties. Simply building in support for the call to return the binary data wouldn't be difficult, but would seem like a pretty half-baked solution. The fully-baked solution would require considerably more work.

I want JAVA API.
How to get mail content.


Problems passing a glob to input param.

I am trying to compile all the templates in the "template" folder
export default {
    input: './templates/**/*.html',
    output: {
        file: 'assets/js/bundle.js',
        format: 'iife'
    },

I am getting this error instead.
[!] Error: Could not resolve entry (./templates/**/*.html)

Can someone tell me what I am doing wrong? Thanks

Use https://github.com/kei-ito/rollup-plugin-glob-import to glob all html files. Despite the Readme you can also glob directories.

Closing as unrelated to this plugin


fix(vscode): windows pathe format

Linked issue: #1644
Linked Commit: #1632
function pathe:dirname will change the slash of path
https://github.com/unjs/pathe/blob/main/test/index.spec.ts#L86
This cause unequal on Windows and could not load config file.

@chris-zhu do you think it's necessary to remove pathe.


Don't create native drags out of an array, and make them enumerable

The Object.create() thing was a weird one. It really meant to be a plain object, I think.
Then I made the properties enumerable so we can do stuff like { ...monitor.getItem() } more easily 😄

I'll try to make some time this week to test it out. It looks reasonable at first glance though

LGTM!