Suggestion: Documentation tests

With Python and Rust I can put examples into my doc-comments and run them as tests via pytest or cargo test respectively. Is there any chance of Doxide helping to extract these examples from the doc-comments and generate unit-test files from them?
This would obviously be more complex for C++ because nobody can agree on a single unit test framework to use. Having a configurable header and footer for the output files, and for each test case, might be enough.
You'd also probably want something like Rust has where only some lines of the code block are shown in documentation, but all of them are output to the test file. Rust uses a # prefix on lines not included in the docs, but that would be awkward for C++.
#include directives would need to be extracted and moved to the top of the output test file.

Thanks for the suggestion @Tim-Evans-Seequent . I think I'm seeing two aspects here:

Showing code examples/tests in the documentation that can be run. The "Run this code" buttons on cppreference.com are backed by the Coliru online compiler, so that might be an approach.
Extracting those examples/tests from the doc comments to build into unit tests. Could be a heavy lift, and risk of inventing yet another unit test framework for C++, so I'm less enthusiastic about this one, but also... why not? If you or someone else has the bandwidth to put together a compelling design and implementation then happy to take contributions. The complexity, to my mind, is in building and running the tests given the user will have a preferred build system. But perhaps just extracting the tests to source files is enough to start with, and the user can integrate into their build system as they please from there.

I hadn't even thought about the first case, but I can see the value in it now, especially if users can also edit that code. That's getting into "Jupyter Notebook but C++" territory and could be a deep rabbit hole.
For 2 I think the one absolutely necessary feature would be some way of hiding lines of code in the documentation. With Rustdoc for example lines starting with # are executed as part of the test but not shown in the final docs. That character would be confusing in C++ so maybe @ would be better. That would let me write example code as Catch2 (or anything else) unit tests like so:
/// Calculates factorials
///
/// ```c++
/// @ TEST_CASE("Factorial example", "[doctest]") {
///     REQUIRE(factorial( 1) == 1);
///     REQUIRE(factorial( 2) == 2);
///     REQUIRE(factorial( 3) == 6);
///     REQUIRE(factorial(10) == 3'628'800);
/// @ }
/// ```
uint32_t factorial( uint32_t number ) {
    return number <= 1 ? number : factorial(number-1) * number;
}

Actually extracting the example code and transforming it into a runnable unit test could be done by a separate tool. Doxide could help with by making it easier to get metadata like what class/function is being documented for example, but it isn't necessary. Even once that information is extracted it could just be passed to a script to process, making it independent of the testing framework used.
I'm not sure if I will have time to work on this, but I'll post a patch if I end up writing it. I haven't found any other good tools for doing this yet.


Following the instructions for "CONTRIBUTE.md" leads to a warning during local CI validation

I forked the repo to contribute and was reading through CONTRIBUTE.md, one of the steps instructs you how to run CI validation. When I do so I get this error:

Which rustc version are you using? If you are using beta or nightly clippy probably added a new lint or changed one to trigger in more cases.

Which rustc version are you using? If you are using beta or nightly clippy probably added a new lint or changed one to trigger in more cases.

You're right, I was using the nightly (1.56). When I switched back to 1.54 the error went away.
The getting started guide does list switching to nightly as a way to improve compilation times, so maybe this should still be fixed? I'm still new-ish to rust so wasn't sure how to fix the warning, google wasn't any help.

Yep, we should have a warning that small things of this nature may be broken when using nightly. That belongs in the book IMO, over on https://github.com/bevyengine/bevy-website.

As of the last rust update the false positive for collapsible-match has been fixed. I update my pr to remove the #allows, and performing the steps from CONTRIBUTING.md no longer gives any errors:
cargo fmt --all -- --check (remove --check to let the command fix found problems)
cargo clippy --workspace --all-targets --all-features -- -D warnings -A clippy::type_complexity


Issues with network-app fuzzing

Hey @mxmssh thanks so much for the commit I couldn't resist not to try this feature!
Unfortunately, am having some issues with the network fuzzing.
Here's the output after literally 4mins of fuzzing:
[-] PROGRAM ABORT : connect() failed with error code : 10055
         Location : send_data_tcp(), c:\projects\winafl\afl-fuzz.c:2534

And on the server:
Recv failed with error: 10053
Am playing with some toy binaries just to get familiar, any ideas what may causing this?
Thanks,
Symeon

Hi Symeon, sorry for the long response (DEF CON). Could you clarify please this "Here's the output after literally 4mins of fuzzing". Does it mean that winAFL worked for the first 4 mins? What about stability (low/high)?

Ok, I got it :) It is simple:
PROGRAM ABORT : connect() failed with error code : 10055
"Error 10055 means that Windows has run out of TCP/IP socket buffers because too many connections are open at once."
So, it looks like your program is not closing connection with the server.

Hey Maksim,
That's very interesting! However, it looks like the the connection gets closed, it's vulnserver which I'm planning to do a blog post once I figure it out :)

Thanks!

Hm, could you post vulnserver's code and command line you run winAFL? It looks like something wrong with instrumentation...

Yeap, it's Lupin's code: http://www.thegreycorner.com/2010/12/introducing-vulnserver.html from 8 years ago! The command line I've used is the following:
afl-fuzz.exe -p 9999 -a <IP_ADDRESS> -w 1000 -i in -o out -D C:\DRIO7\bin32 -t 20000 -- -target_module vulnserver.exe -target_offset 0x12D0 -coverage_module vulnserver.exe -fuzz_iterations 50000 -nargs 1 -- C:\Users\ida\Desktop\vulnserver\vulnserver.exe
That offset is pretty much main's function:

Cheers and thanks!

Ok, it looks like I start understanding the root of the problem. Basically, we have three things here:

You need to specify your argument for vulnserver when you are starting winAFL. For example
afl-fuzz.exe -p 9999 -a <IP_ADDRESS> -w 1000 -i in -o out -D C:\DRIO7\bin32 -t 20000 -- -target_module vulnserver.exe -target_offset 0x12D0 -coverage_module vulnserver.exe -fuzz_iterations 50000 -nargs 1 -- C:\Users\ida\Desktop\vulnserver\vulnserver.exe 9999

You don't need to run vulnserver from a separate console, winAFL will run everything for you (that's why stability is 0 here).

If you want to use persistent mode (argument -fuzz_iterations 50000), the function __main is not a good choice :). Try to run it without persistency (just specify -fuzz_iterations 1) first and check whether it works or not. If it works (performance should be super slow), we need to modify source code to be able to enable persistent mode.

Brilliant, will try that tomorrow cause it's 00:30 here, and get back to you!
Thanks, (btw vulnserver runs by default on 9999 unless you specify another port!)

Actually, you don't need to modify source code, try to run it as is. -nargs should be 2. So the resulting command is:
afl-fuzz.exe -p 9999 -a <IP_ADDRESS> -w 1000 -i in -o out -D C:\DRIO7\bin32 -t 20000 -- -target_module vulnserver.exe -target_offset 0x12D0 -coverage_module vulnserver.exe -fuzz_iterations 50000 -nargs 2 -- C:\Users\ida\Desktop\vulnserver\vulnserver.exe 9999

Yeah sure, although from IDA it looks like we are passing three arguments.
Generally from my experience with the exception of a few custom functions I didn't have luck instrumenting them, winafl would fail. I will test it again thoroughly and post back with a more detailed description ok?
Many thanks!

Sounds good. Yes it looks like we need to set -nargs 3.

Ok, now the fuzzer stats looks much healthy :) Could you try to disable persistent mode (```-fuzz_iterations 1)?

It would be easier to debug if you can redirect those printf into file. We could better understand what's going on in the server.

Alright, with persistent mode disabled am getting almost 0.25exec/s and exactly same error as before. I think your idea of redirecting those printfs to a file is brilliant, will do that once I get some time!
Thanks so much Maksim!

Hey Maksim!
I've finally had the chance to modify the printfs to fprintf (had to use a x32 bit version of cygwin!),
am attaching the log file. It looks decent frankly as in:
Received a client connection from <IP_ADDRESS>:61180
Connection closing...
Received a client connection from <IP_ADDRESS>:61181
Connection closing...
Received a client connection from <IP_ADDRESS>:61182
Connection closing...

I think it would be a good idea to add those fprintfs, to the TRUN method that I am fuzzing cause this one does not help us much..
Here's the link for the file (sorry github is failing!)
Once again thanks!

Hi,
Did you change all printfs to fprintfs? It is pretty strange.

Hi Maksim,
Update my side: Indeed the connections were not closed properly (took me some time to figure this out!). Here's the latest error after 9min fuzzing:
[-] PROGRAM ABORT : connect() failed with error code : 10055
Location : send_data_tcp(), c:\users\ida\desktop\winafl\afl-fuzz.c:2553

After a bit for researching it turns out:
WSAENOBUFS | 10055
No buffer space available.An operation on a socket could not be performed because the system lacked sufficient buffer space or because a queue was full.

and then following this article here, I've modified the MaxUserPort parameter and increased it to 0xFFFE, however winafl still complains. Have you ever this seen?
Thanks!

Hi, I have tried your client and had the same issue as you had before, error code: 10061. It was happened because there is some problem with instrumenting _main routine using DynamoRIO (may be my version is out of date or something else). Which version of DR are you using ? Anyway, I see several other problems with this vulnserver:

vulnserver assumes that after you send your input you should receive output from them, however, winAFL doesn't support that (I have created a new pull request where it is possible to define a custom logic via third-party DLL). So, you either need to comment out all send in the vulnserver or pull new winAFL functionality and implement a custom client/server logic)
Infinite loop, as it is described in the winAFL README file, target should exit/close connection after processing test case. In vulnserver, they use infinite loop to receive new incoming connections and create a new thread for each one. That's why you are experiencing this problem with no buffer space available. You have to remove infinite loop as well.
I have modifed the vulnserver and managed to make fuzzer work for me. You can find source code file in the attachment.
vulnserver.zip

Hm, it looks like there is some problem with instrumentation (target exiting/crashing) even before executing target function. Does it work without winAFL at all?

Yes Maksim, it does work, in fact I was able to crash it by sending a large string...
I guess I need to hook up a debugger and understand how this instrumentation works/why am getting this error.

Could you copy/paste here cmd you are using to execute winAFL ?

Debug version:
afl-fuzz.exe -p 9999 -a <IP_ADDRESS> -w 1000 -i in -o out -D C:\DRIO7\bin32 -t 20000 -- -target_module vulnserver.exe -target_offset 0x17E0 -coverage_module vulnserver.exe -fuzz_iterations 500 -nargs 1 -- C:\Users\ida\Desktop\vulnserver-winafl\Debug\vulnserver.exe

and release:
afl-fuzz.exe -p 9999 -a <IP_ADDRESS> -w 1000 -i in -o out -D C:\DRIO7\bin32 -t 20000 -- -target_module vulnserver.exe -target_offset 0x1320 -coverage_module vulnserver.exe -fuzz_iterations 500 -nargs 1 -- C:\Users\ida\Desktop\vulnserver-winafl\Release\vulnserver.exe

results in:
[+] You have 2 CPU cores and 0 runnable tasks (utilization: 0%).
[+] Try parallel jobs - see docs\parallel_fuzzing.txt.
[*] Checking CPU core loadout...
[+] Found a free CPU core, binding to #0.
[*] Setting up output directories...
[+] Output directory exists but deemed OK to reuse.
[*] Deleting old session data...
[+] Output dir cleanup successful.
[*] Scanning 'in'...
[+] No auto-generated dictionary tokens to reuse.
[*] Creating hard links for all input files...
[*] Attempting dry run with 'id_000000'...

[-] The program took more than 20000 ms to process one of the initial test cases.
    Usually, the right thing to do is to relax the -t option - or to delete it
    altogether and allow the fuzzer to auto-calibrate. That said, if you know
    what you are doing and want to simply skip the unruly test cases, append
    '+' at the end of the value passed to -t ('-t 20000+').

[-] PROGRAM ABORT : Test case 'id_000000' results in a timeout
         Location : perform_dry_run(), c:\users\ida\desktop\winafl\afl-fuzz.c:2991

process 8104 is not running under DR
0 processes nudged

Is this offset point to the main function ? If it is, could you try -fuzz_iteration=1 and some other function (I tried ConnectionHandler and it worked for me).
BTW, if you compile your target with debug information you can use -target_method ConnectionHandler instead of offset.

Ah interesting, I thought I need to export the symbol (am aware of the -target_method which didn't try at all), but reading at this code, it looks like winafl first tries the exported symbols and then the debug symbols as well!
Yeah, my function is indeed the ConnectionHandler...
The -target_method gives me the Can't find specific method in fuzz_module but I'll try to export it and give it another try

It worked for just 1min with crazy slow speed (back at 0.75 again!) and gave me back the old 10061 error.
The exported file was this one:
Dump of file C:\Users\ida\Desktop\vulnserver-winafl\Debug\vulnserver.exe

File Type: EXECUTABLE IMAGE

  Section contains the following exports for vulnserver.exe

    00000000 characteristics
    FFFFFFFF time date stamp
        0.00 version
           1 ordinal base
           1 number of functions
           1 number of names

    ordinal hint RVA      name

          1    0 00011366 _ConnectionHandler@4 = @ILT+865(_ConnectionHandler@4)

and the command:
afl-fuzz.exe -p 9999 -a <IP_ADDRESS> -w 1000 -i in -o out2 -D C:\DRIO7\bin32 -t 20000 -- -target_module vulnserver.exe -target_method _ConnectionHandler@4 -coverage_module vulnserver.exe -fuzz_iterations 1 -nargs 1 -- C:\Users\ida\Desktop\vulnserver-winafl\Debug\vulnserver.exe

Well, It is very strange, the last guess I can make that I've used a new patch for winAFL. You can try to use this branch https://github.com/ivanfratric/winafl/pull/134 while we are merging it with master.

So I've tried your latest pull request and still not getting lucky.
I've used the following command:
set AFL_CUSTOM_DLL_ARGS=-p 6666 -a <IP_ADDRESS> -w 1000
and fuzzed it with:
afl-fuzz.exe -s -i in -o out -D C:\DRIO7\bin32 -t 20000 -- -target_module vulnserver.exe -target_method ConnectionHandler -coverage_module vulnserver.exe -fuzz_iterations 500 -nargs 1 -- C:\Users\ida\Desktop\vulnserver-winafl\Debug\vulnserver.exe 6666
I haven't exhausted all the posibilities yet, I have to read a bit about your DLL harness and try that as well! Am closing the issue and maybe re-comment if I get it to work. Thanks so much!

Hi @symeonp , Hope you are doing fine, I came across the 10055 error and I solved it so wanna share what I have done.
my problem was cased by 2 issues one in the server I was fuzzing and the other was in the send_tcp_data in the fuzzer.
the server was setting the below socket option
setsockopt(s, SOL_SOCKET, SO_REUSEADDR, (char*)&opt, sizeof(int));

When winafl restart the server, all unfinished connections were turned to WAIT_TIME state waiting for the final ACK which will never come, this leads to a huge number of WAIT_TIME connections that consume the system buffer. and at some point, winafl will fail to create a new socket to send the mutant data.
To solve this we need to set the SO_LINGER option, which seems to kill the WAIT_TIME connections when the server is closed
setsockopt(s, SOL_SOCKET, SO_LINGER, (char*)&opt, sizeof(int));

This issue solve half the problem, But I still had the 10055 error after some time, to make sure that the connections are closed I added socketclose call to the send_tcp_data after the shutdown
    if (shutdown(s, 0x1) == SOCKET_ERROR)
      FATAL("shutdown failed with error: %d\n", WSAGetLastError());
    if(closesocket(s)== SOCKET_ERROR)
      FATAL("closesocket failed with error1: %d\n", WSAGetLastError());

This solved the problem completly for me, I hope it will solve the problem for you too.

Hey Mostafa, that's very interesting, thanks so much, I'll definitely give it a try!
Cheers,

Hey @MostafaSoliman, it is cool. Could you commit this change ?

Hi @mxmssh I created a pull request with the change, Is this enough, Also it is better to add the note about the target server socket option in the README.md under the network fuzzing section.


Enable automated builds with Travis CI

Get the swagd build process working with Travis CI and Maven

Travis-CI builds are now online, using mvn compile as a pre-build action and mvn test as the script action.  This will ensure the launch4j plugin does not fire, as it will not work properly on travis-ci.


only camera test mAP==0?

hello ,thanks for your code.
We used your code for step-by-step training. In the test, we only tested the camera stream and found that the result of the test was mAP=0.
We trained using the pre-trained weights you provided, but the test map for the camera stream is 0, but the test for the point cloud stream is fruitful.
In the meantime, we downloaded the camera stream weights you provided for testing, and the result was OK.
Are which parameters are incorrect during training? We have not changed the config file you provided.

I encountered the same problem that the loss suddenly increased during 12 epoch and the test result is mAP=0. Do u find the reason causing this problem now?

I encountered the same problem that mAP = 0, do you solve it ?

I encountered the same problem that mAP = 0, do you solve it ?plzzzz


Allow changing color of the text

When the status bar is set to a light color, the white text is not readable.
It looks like API 23 supports a light/dark text theme for the status bar:
https://developer.android.com/reference/android/R.attr.html#windowLightStatusBar

:+1:

I have same issue with light background.Is there any solution?


Convert Parameter to number

According to the documentation Expressions and functions in Azure Data Factory
"@pipeline().parameters.myNumber" should return a number, but when I execute the following expression on a query on ADF
'@{ addminutes(pipeline().parameters.StartTime, -"@pipeline().parameters.Minutes" )}'
I get: The expression is not valid: the value '-' at position '59' cannot be converted to number
Notice that the pipeline parameter pipeline().parameters.Minutes contains a number.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 76e31411-f1a2-710b-94c0-39acf9e7314b
Version Independent ID: d5cefffe-dee0-2d2e-f3cb-8eefd5d40d17
Content: Expression and functions in Azure Data Factory
Content Source: articles/data-factory/control-flow-expression-language-functions.md
Service: data-factory
GitHub Login: @sharonlo101
Microsoft Alias: shlo

@giusbi
Thanks for your question. Please provide us with the link to the documentation you are referring to so we can better assist. Also, please elaborate more on your question so that we can better assist.

@giusbi Any update?

@giusbi The '-' is not evaluated as -1*pipeline().parameters.Minutes. Hence, when the function tries to parse the second operand, it's encountering this error. You can use the following expression to solve your problem :
@{ addminutes(pipeline().parameters.StartTime, sub(0,pipeline().parameters.Minutes) )}
The sub function will be used here to subtract Minutes parameter from 0, which will be equal to -Minutes.

@ChiragMishra-MSFT That works fine, the only thing I had to change in order to parse the parameter as a number was:
@{ addminutes(pipeline().parameters.StartTime, sub(0, int(pipeline().parameters.Minutes)) )}

@giusbi Glad to know that it worked for you. I will now proceed to close this issue. Feel free to still comment and tag me in case you have any other queries.


extract-bc on Mac

Downloaded the latest Xcode today. It comes with a new version of otool that
does not respect the -X flag. As a result, extract-bc does not work anymore
for me.

This looks to be a bug in otool, since it claims -X  will  "print no leading addresses or headers".
That said I could provide a better  parser that will handle this. First I need to get the old otool
working so I can compare...
Thanks for diagnosing this.

Fixed by throwing away any lines of output from otool that isn't whitespace and hex.


SPFx Extension

To whom it may concern,
While the article clearly explains the process for Web Parts, I would advocate expanding upon the above with some of the extra steps for debugging a SPFx Extension.  Providing specifics on how once opened the workbench browser can be used test on different pages in a SharePoint site while still letting you step though your Extension code would be very useful.  I eventually figured it out myself, but having that extra info readily available in the article would be much more efficient.
Cheers,
-John

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 959a8968-baa1-e94c-10af-a52604aef21e
Version Independent ID: cecd4be2-5717-d282-587e-7a93910a03a3
Content: Debug SharePoint Framework solutions in Visual Studio Code
Content Source: docs/spfx/debug-in-vscode.md
Service: unspecified
GitHub Login: @spdevdocs
Microsoft Alias: spdevdocs

I agree with John.  It would be nice to see example of extension debugging.

@john88benson maybe you could propose a pull request on the steps you took to accomplish this? I’m interested in the steps myself.

@sparkitect nope... no reason why you can't just change the start URL. In this doc, the web part is used as an example as you just need to point to the start URL. Extensions are no different.
In fact, it's nothing special about SPFx... any web dev works like this when debugging with VSCode.

This issue was never closed from merging PR #1884... closing now


npm: Enforcing 2FA for high-impact projects

Summary
As part of our ongoing commitment to npm ecosystem security, we have already rolled out enhanced login verification to all publishers on the npm registry. The next step in securing the accounts of publishers on the registry is to enforce the use of 2FA for all accounts with publishing rights to high-impact packages. The third phase in this is to roll out 2FA for all "High-Impact Projects".
Intended Outcome
All "High-Impact Projects" are enrolled in enforced 2FA
How will it work?
Once mandatory 2FA is enforced accounts with publish rights to the top-100 packages on the registry, by dependents, will have limited access to the registry and npmjs.com until they register a 2FA device.

this has shipped! https://github.blog/changelog/2022-11-01-high-impact-package-maintainers-now-require-2fa/ closing out this issue


[blog] 2022 Developer survey public report

Summary
Opening as a draft to collect initial feedback for the public survey report.
Preview
https://deploy-preview-38318--material-ui.netlify.app/blog/2022-developer-survey-results/
Original Charts
https://lph4m3.csb.app/

Pushed a commit that standardizes all the image's width (adding the little container border + border-radius) and adds the post's open graph image. If any change needs to happen until it's ready to go, let me know so I can jump in and change stuff, or feel free to jump on Figma and do it yourself! :)


Niloofar / Bug - Fix Typescript errors in derivx

Changes:

...

Type of change

[x] Bug fix
[ ] New feature
[ ] Update feature
[ ] Refactor code
[ ] Translation to code
[ ] Translation to crowdin
[ ] Script configuration
[ ] Improve performance
[ ] Style only
[ ] Dependency update
[ ] Documentation update
[ ] Release

Well Done! 👍
Approved 🤗


External OAuth Integration forcing update in place when no change

Provider Version
0.53.0
Terraform Version
1.3.6
Describe the bug
Terraform forcing update in place when there is no change to the snowflake_external_oauth_integration.
It looks to be because I have multiple strings for the jws_keys_url and the token_user_mapping_claims. The reason I have two values is that we set it up for Power BI and then for an External OAuth requirement.
Expected behavior
After I run terraform apply and there are no changes to the code - terraform thinks it needs to update in place the existing external_oauth_integration when there were no changes.
Code samples and commands
Terraform Code:
resource "snowflake_external_oauth_integration" "INTEGRATION_POWERBI" {
  name                             = "POWERBI"
  type                             = "AZURE"
  enabled                          = true
  snowflake_user_mapping_attribute = "login_name"
  token_user_mapping_claims        = ["upn", "sub"]
  any_role_mode                    = "ENABLE"
  audience_urls                    = ["https://analysis.windows.net/powerbi/connector/Snowflake", "api urn removed for security"]
  issuer                           = "removed for security"
  jws_keys_urls                    = ["removed for security", "removed for security"]
  comment                          = "OAuth Integration for Power BI and AAD Oauth"
}

Terraform plan output:
Terraform will perform the following actions:

  # snowflake_external_oauth_integration.INTEGRATION_POWERBI will be updated in-place
  ~ resource "snowflake_external_oauth_integration" "INTEGRATION_POWERBI" {
        id                               = "POWERBI"
      ~ jws_keys_urls                    = [
          - " removed for security",
          - "removed for security",
          + "removed for security",
          + "removed for security",
        ]
        name                             = "POWERBI"
      ~ token_user_mapping_claims        = [
          - " sub",
          + "sub",
            # (1 unchanged element hidden)
        ]
        # (10 unchanged attributes hidden)
    }

Additional context
I think this might be because of a leading space that is somehow being placed when terrafrom is performing some action. As you can see in the TF plan output it is putting a space before the existing token_user_mapping_claims and the jws_keys_urls

Yes, i have noticed this issue with several commands, that are using list of values.

I see this bug was submitted in January and is still open.
@sfc-gh-swinkler Curious, do you know the improvement cadence for this repo?  Is it still actively maintained?  Thanks!

@chrisweis we are currently working on provider stability for long term maintainability. That means rewriting much of the code that generates the code for making SQL commands to Snowflake and adding new unit and integration tests (sdk package). It took some time to design, but we are working through each resource one at a time. Issues like this are on my radar, in fact I have a jira ticket assigned to me. After I finish failover groups I was going to work on oauth integration. The current version of the provider is too difficult to maintain and debug issues like this, which is why we are investing time in this refactor project. Thank you for your patience

Hey @culpgrant  @cpmub1 @chrisweis    👋
The new version v0.93.0 provides adjusted external_oauth_integration. Now, these lists are handled correctly. Please follow the migration guide.

@sfc-gh-jmichalak thanks for taming this wild Snowflake Terraform beast! Really appreciate the work you’re doing! 🙏 We’ve been making manual changes to our integrations over the past year but I’ll revisit upgrading and using this provider for it next time we have a change. Thanks!


only run 1 vcpkg instance

Currently, many vcpkg instances can be run in msbuild if parallel builds are enabled, and many projects with no shared dependencies use this approach.
We should add some sort of blocking semaphore as this blocking behavior is a vcpkg requirement it seems, and should be enforced seamlessly for the caller.

addressed


Request based CORS policies

In order to get CORS working over signalr for my content management system that has a list of supported domains in the DB needs a means to examine the request and auto respond to OPTIONS requests with the current domain being allowed so that the follow on call can decide what is and isn't actually allowed for the current user on the requested domain.
The background here is that my CMS hosts multiple sites in the same db with a table containing the list of sites supported (which isn't small).
An api call can be made to add a new site / domain so I can't hard code a list of known domains on start and to my knowledge there's no guidance on how to dynamically add new CORS policies at runtime during a HTTP request.
For performance reasons I can't keep going back to the database on every options request anyway so I'd like to default to all options requests telling the client that their domain is allowed for cross domain calls with credentials.
I would like a means to say something like this ...
app.UseCors(builder =>
{
    builder.AllowAnyHeader();
    builder.AllowAnyMethod();
    builder.AllowRequestedOrigin();
    builder.AllowCredentials();
});

.... or maybe something like ...
app.UseCors(builder =>
{
    builder.AllowAnyHeader();
    builder.AllowAnyMethod();
    builder.AllowOrigin(request => true);
    builder.AllowCredentials();
});

... since what i effectively want is "No Cors failures for any request on any domain with credentials" but no browser will accept that and the framework won't allow me to configure that due to the way the standard works I accept here that I am deliberately asking for what might be considered a "security risk" from the standards point of view but I am mitigating that in the actual request so I am not concerned about this.
Others have suggested putting things like a list of domains in to the config file but this would simply bloat out my configs to an unusable level due to the number of domains being manged by the application.
background information ...
I have a single API application that responds to calls on "api.mydomain.com" then I have a front end application that accepts calls from any domain forwarded to the server that makes API calls to get details of a page to render.
The rendered page then has javascript on it that may fire up signalr components using standardised hubs in the API.
For this reason I can't "pre-configure" a set of accepted domains as is documented,
Hitting the db for a request that I will always accept is pointless.

builder.AllowOrigin(request => true);

This is supported with SetIsOriginAllowed

Oh ok ... did not know that!
Side question ...
In configure services I will now have ...
services.AddCors(options => options.AddDefaultPolicy(builder => {
                builder.AllowAnyHeader();
                builder.AllowAnyMethod();
                builder.SetIsOriginAllowed(origin => true);
                builder.AllowCredentials();
            }))

and in configure i have ...
app.UseCors(builder =>
{
    builder.AllowAnyHeader();
    builder.AllowAnyMethod();
    builder.SetIsOriginAllowed(origin => true);
    builder.AllowCredentials();
});

... do i need both or is this not just basically repetition at least at some level?

YMMV on this one. If you are using attribute based config then policy will take effect. Pure middleware can be configured through use of a policy or ad hoc as in your example.

IdentityServer4 does something like this by implementing ICorsPolicyProvider. Here's relevant pieces of code:

https://github.com/IdentityServer/IdentityServer4/blob/44651bea9b02c992902639b21205f433aad47d03/src/IdentityServer4/src/Hosting/CorsPolicyProvider.cs#L17
https://github.com/IdentityServer/IdentityServer4/blob/56eaba86deb8709f16604dd3188a7c4bfe83e758/src/EntityFramework.Storage/src/Services/CorsPolicyService.cs

I would generally advise against all origins when you allow credentials. It has a known XSS vulnerability: https://github.com/aspnet/AspNetCore/issues/3106

Closing as this has been answered.


Implement a example project w/ unit & automated tests and benchmark

depends #18
will create a project using this lib to test and use as example

https://github.com/lucoiso/renderer-application


Resource starvation due to reconnect spam

Logs in prod seem to be cluttered with the following messages, which causes a lot of disk pressure causing pods to evict:
2022/05/06 12:09:00 worker 9688xxxxxx: Message channel closed. Trying to reconnect...
2022/05/06 12:09:00 worker 9688xxxxxx: Message channel closed. Trying to reconnect...
2022/05/06 12:09:00 worker 9688xxxxxx: Message channel closed. Trying to reconnect...
2022/05/06 12:09:00 worker 9688xxxxxx: Message channel closed. Trying to reconnect...
2022/05/06 12:09:00 worker 9688xxxxxx: Message channel closed. Trying to reconnect...
2022/05/06 12:09:00 worker 9688xxxxxx: Message channel closed. Trying to reconnect...
2022/05/06 12:09:00 worker 9688xxxxxx: Message channel closed. Trying to reconnect...
2022/05/06 12:09:00 worker 9688xxxxxx: Message channel closed. Trying to reconnect...
2022/05/06 12:09:00 worker 9688xxxxxx: Message channel closed. Trying to reconnect...
2022/05/06 12:09:00 worker 9688xxxxxx: Message channel closed. Trying to reconnect...
2022/05/06 12:09:00 worker 9688xxxxxx: Message channel closed. Trying to reconnect...
2022/05/06 12:09:00 worker 9688xxxxxx: Message channel closed. Trying to reconnect...
2022/05/06 12:09:00 worker 9688xxxxxx: Message channel closed. Trying to reconnect...
2022/05/06 12:09:00 worker 9688xxxxxx: Message channel closed. Trying to reconnect...
2022/05/06 12:09:00 worker 9688xxxxxx: Message channel closed. Trying to reconnect...
2022/05/06 12:09:00 worker 9688xxxxxx: Message channel closed. Trying to reconnect...
2022/05/06 12:09:00 worker 9688xxxxxx: Message channel closed. Trying to reconnect...

I should either reduce logging or figure out why the same worker is trying to reconnect so many times over and over again.

The problem is we keep retrying to connect on error (e.g. aternos is temporary down). We store a max reconnects value and return an error instead.


Android 12

System info
The issue occurs on Android 12
Plugin name: connectivity_plus
Plugin version: 2.0.2
Hi, I tried this package on the android 12 emulator and it couldn't detect the connection please check and fix this issue

it was my emulator issue, I fixed it now thanks for your time :)


[buildinfo] Download package and signature at once

Signed-off-by: Juergen Hoetzel<EMAIL_ADDRESS>

This also speeds up the package download process a bit

Nice, thanks!


what's meaning of snabbdom

Why did you name it?in other words,wht's meaning of snabbdom？

Great question. Here you go:
https://github.com/snabbdom/snabbdom/issues/172#issuecomment-301305142


Implementation of ABFS

Implemented list directories and a small amount of test cases for list directories
Test cases may have to be changed later because it relies on ABFS already having directories.

@travisle22 I believe https://github.com/cloudera/hue/pull/932 had everything. Can we close?


Trigger animation by adding class

Hi,
I'm wondering if its possible to play an animation when an element enters the viewport. Lets say I were using something like Scrollspy to detect when a div enters the viewport, a class would added to the div such as "in-visible". Is it possible to start the animation once the class is added? Any thoughts on how to play an animation based on a class being added?
Thanks
G

It's probably best to trigger the animation through a callback instead. Monitoring a class using a polling or DOM monitoring method seems quite heavy.

I have got a function that checks if an element is scrolled into view, then, in simple if-statement I check if the element is in viewport, if yes, I trigger an animation.
Have a look at my code below:

  function isScrolledIntoView(elem) {
      var docViewTop = $(window).scrollTop();
      var docViewBottom = docViewTop + $(window).height();
      var elemTop = $(elem).offset().top;
      var elemBottom = elemTop + $(elem).height();
      return ((elemBottom <= docViewBottom) && (elemTop >= docViewTop));
  }

 $(window).scroll(function () {
       if (isScrolledIntoView('section#therapy .praluent-flex .box-1') === true) {
        var relativeOffset = anime.timeline();

        relativeOffset
            .add({
              targets: '.praluent-flex-anime .box-1',
              translateY: -200,
              easing: 'easeOutExpo',
            })
            .add({
              targets: '.praluent-flex-anime .box-2',
              translateY: -200,
              easing: 'easeOutExpo',
              offset: '-=600' // Starts 600ms before the previous animation ends
            })
            .add({
              targets: '.praluent-flex-anime .box-3',
              translateY: -200,
              easing: 'easeOutExpo',
              offset: '-=800' // Starts 800ms before the previous animation ends
            });
       }
  });

@agnieszkabugla What about intersectionObserver?
https://developers.google.com/web/updates/2016/04/intersectionobserver

@equinusocio That's also an option 👍  :)

Any update to support IO natively with the lib? I find very useful a way to trigger each element animation only if in viewport using the intersection observer.
For example, it's very useful in lists with infinite elements, where each item should be animated only when scrolled in (applied also for containers with overlflow.)

@juliangarnier I agree with @equinusocio please provide support for IO this thing seems very promising!


Updating Hint for The Problem.

As most of the students make the mistake of just going recursively left and right, which is not correct as they ignore the basic condition of B.S.T.

[x] I have read freeCodeCamp's contribution guidelines.
[x] My pull request has a descriptive title (not a vague title like Update index.md)
[x] My pull request targets the master branch of freeCodeCamp.
[x] None of my changes are plagiarized from another source without proper attribution.
[x] All the files I changed are in the same world language (for example: only English changes, or only Chinese changes, etc.)
[x] My changes do not use shortened URLs or affiliate links.

Closes #XXXXX

Hi, Please read our Guidelines for Contributing before opening a PR.
We are marking this as invalid and closing.
Thanks.


Show message text in bottom sheet

In the redesigned message bottom sheet #90 from @terpimost, the top of the message sheet shows the text of the message you're acting on (cutting it off at two lines). This is helpful when the message itself isn't visible, which becomes more and more likely as we add more actions to the bottom sheet.
I'm splitting this out from the rest of #90, because the concept of "the text of the message" is actually a bit slippery and raises some product/design questions.

We could use the message's Markdown source. But that isn't great for this UI context: it may have a bunch of markup that obscures what the message actually says.

Consider in particular the case where the message starts with a quote-reply.
Consider also a message that starts with one short paragraph like "Yes.", and then has more details in a second paragraph. The first two lines of the Markdown source will be just "Yes." followed by a blank line. That's not very informative, and we might want to show more text than that.

If we decide we're OK with that, there's a second pitfall which we'll need to pick a response to: we don't normally keep around the Markdown source for all the messages we have, only the rendered HTML. On the occasions where we need Markdown source (like for generating a quote-and-reply), we make a separate request to the server to get it. We can do that here… but then we need to decide what to show when that request hasn't completed yet, which will be at least a frame or two and often more.

We could delay the whole bottom sheet. That seems bad.
We could show nothing; when the text arrives, the top edge of the bottom sheet jumps up to make room.
We could show a blank space; when the text arrives, it pops into existence in the top area of the bottom sheet. Until it arrives, that area looks oddly blank.
We could show a CircularProgressIndicator, or another loading indicator. Might feel noisy; this message text isn't meant to be a point of focus, just a helpful bit of context in the nature of a breadcrumb.

We could use the message's HTML. That will definitely have a bunch of cluttery-feeling markup, more so than the Markdown. On the other hand we have it immediately and don't have to wait.

We could introduce some new notion of a "text summary" of a message's content: we'd walk the content parse tree, and try to generate something like its plain text without markup. This will require some work to define just what we want and to implement it.

Another option is to take a screenshot of the rendered message on which user has clicked and show top part of that, cutting out the rest:

It might be ok to copy that with a background:

Hmm yeah, that's an interesting idea. That'd be a good experiment to try.


Validation not done for month and year

In 'month' field, it can accept any string. Instead, it must check if the entered text is a valid month or not.
Similar is the case with the 'year' field.

Resolved #1 . Thanks for your help. :+1:


Another attempt to stabilize MacOS GA runner caused by QEMU / Colima / Docker issues

Description
Another attempt to stabilize MacOS GA runner caused by QEMU / Colima / Docker issues
Related Issues
N/A (multiple pull request check are failing sporadically)

Check List

[X] New functionality includes testing.

[X] All tests pass

[X] New functionality has been documented.

[X] New functionality has javadoc added

[X] API changes companion pull request created.
[X] Failing checks are inspected and point to the corresponding known issue(s) (See: Troubleshooting Failing Builds)
[X] Commits are signed per the DCO using --signoff
[X] Commit changes are listed out in CHANGELOG.md file (See: Changelog)
[X] Public documentation issue/PR created

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@reta Is there an issue tracking the mac runner issues?  I'm curious how we are evaluating the different changes.

@reta Is there an issue tracking the mac runner issues? I'm curious how we are evaluating the different changes.

No, there is no issue nor explicit changes done by us, but we know that (post-factum):

Github changed runners (and few times)
New versions of emulation (QEMU / Colima / Lima) have been released

From our perspective - no changes but MacOS build checks became very unstable :(

What is failing? Is that something where we could modify this PR to run the failing workflow 10 times and compare against the original setting?

What is failing?

In the nutshell: Github Actions never supported Docker on MacOS (and never will), so the workaround is to use QEMU / Colima / Lima over homebrew, it just fails to initialize, example here https://github.com/opensearch-project/OpenSearch/actions/runs/9318725963/job/25651653560?pr=13900

Is that something where we could modify this PR to run the failing workflow 10 times and compare against the original setting?

There are no settings that we have changed

Github Actions never supported Docker on MacOS (and never will)

@reta I would be in favor of disabling the mac workflows, what do you think?  IMO A flaky failing test is worse than a disabled test.  I think this would prompt a deeper conversation about mac support.
@opensearch-project/engineering-effectiveness Have ya'll been looking into these issues, any recommendations on how to proceed?

@reta I would be in favor of disabling the mac workflows, what do you think? IMO A flaky failing test is worse than a disabled test. I think this would prompt a deeper conversation about mac support.

No objections, let me play with it today and if no success - we'll disable it

@peternied as one might expect, (murphy law), the assemble steps haven't been failing for last 10 builds ....

see if its periodically succeeding - @reta are you saying that you'd check in at that level? If you are willing then I am OK to move forward with this kind of change.

Thanks a lot @peternied , yes, it needs to be checked and I could / will do that on all my pull requests till it gets stable enough to be enabled again, thanks!

@peternied may I ask you please to reapprove? thank you!

@reta Feel free to merge when ready.

Thanks @peternied , I am looking into one hiccup with the checks - the workflow was cancelled but no logs it was actually failed the step [1], not sure why but have an idea how to test it to be sure we are not introducing another variation of unstable workflows.
[1] https://github.com/opensearch-project/OpenSearch/actions/runs/9373499168/attempts/1?pr=13900


Added configUI menu system and aura options

There's some whitespace stuff in /lib/utils.js, but I think we covered that already.

This should close #64 as well.


blob type not supported

It seems blob types are not supported. I'd be happy to fix the code if someone could point me in the right direction. I think it just needs an option to download the blob as some specified format?

I’ve added a new type in SQLColumn.php, to just return it as a string, which improved the error. It at least loads the page, but has an exception in a modal window “in for each”.

Hey @ehallein unfortunately I've not been able to maintain this framework for a while, but I do intend to clean it up and modernize things a bit and add more, now that I have some free time. I will definitely be adding a BLOB type to download it as a file.
In the meantime if you're looking at shipping it within a few weeks, I would suggest to try out a few other packages which offer this functionality.


Add it to awesome-go

Add it to https://github.com/avelino/awesome-go.

Happy for someone to help with a PR

I'll add it when we close all PR and issues and update the README with all the docs we can (I will start it soon).
https://github.com/avelino/awesome-go/blob/master/CONTRIBUTING.md


Introduce a "Reveal" action on password fields

Part of https://github.com/opencollective/opencollective/issues/6337
See https://dev.to/detoner777/show-hide-password-on-toggle-in-react-hooks-1lph on how to achieve that
Steps:

[ ] Extract password input to a new PasswordInput component
[ ] Update all references (in Sign in, User Security and Reset password) to make sure they all use the same component
[ ] Introduce the "Reveal" feature, using an icon from https://styled-icons.dev/?s=eye

@ShreyaPrasad1209 Sure :+1:

Hi @ShreyaPrasad1209, do you still want to look at this one?

@Betree I did some analysis and it seems like creating a different component (namely PasswordInput) for Reveal action is not possible because it will update the state of existing StyledInputs and this key is dependent on these specific files like UserSecurity.js. So this Reveal feature can be added inside these files and not as a separate component.

@Betree I did some analysis and it seems like creating a different component (namely PasswordInput) for Reveal action is not possible because it will update the state of existing StyledInputs and this key's value is dependent on specific files like UserSecurity.js, reset-password.js etc.

Doesn't sound like a blocker, we could still pass the key prop to this new component right?

@Betree I did some analysis and it seems like creating a different component (namely PasswordInput) for Reveal action is not possible because it will update the state of existing StyledInputs and this key's value is dependent on specific files like UserSecurity.js, reset-password.js etc.

Doesn't sound like a blocker, we could still pass the key prop to this new component right?

Right, but they're also updating each other's state. I'll need to look into the implementation.

This would be amazing!!


Fix broken headings in Markdown files

GitHub changed the way Markdown headings are parsed, so this change fixes it.
See bryant1410/readmesfix for more information.
Tackles bryant1410/readmesfix#1

Coverage remained the same at 100.0% when pulling<PHONE_NUMBER>d4c4bf44fe372566768deaa9cc5df7 on bryant1410:develop into 76307eb4cb527112e10116bfe2a4afa9d46be791 on ajwhite:develop.

Good catch, thanks!


Wrapping devenv shell in a FHS environment?

Hi!
I'm working on a Rust Steam Play tool, and I'm trying to wrap devenv shell in an FHS environment, for Proton. It doesn't run otherwise.
I've tried using pkgs and inputsFrom to get this working, to no avail.
Is it possible to wrap devenv into a FHS environment?
Thanks.

Actually, I think I've found the issue. It was my Nix derivation.
Regardless, using devenv as a shell wrapped in FHS might be useful. Is it worth opening a feature request for?

I don't think anyone has really tried this.
You can override shell and replace the default mkShell implementation with buildFHSEnv.
{ pkgs, lib, config, inputs, ... }:

{
  env.HELLO = "WORLD";

  packages = [
    pkgs.hello
  ];

  shell = lib.mkForce (pkgs.buildFHSEnv {
    name = "devenv-shell";
    targetPkgs = _: config.packages;
    runScript = "bash";
    profile = ''
      ${lib.optionalString config.devenv.debug "set -x"}
      ${config.enterShell}
    '' + lib.concatStringsSep "\n" (lib.mapAttrsToList (name: value: ''
      export ${name}=${value}
    '') config.env);
  }).env;
}

This seems to work as expected:
which hello
/usr/bin/hello

But direnv fails to launch this shell, so the question is what else does this break.
Another approach would be to wrap just the binary in a buildFHSEnv. Maybe there's a way to leverage scripts for this?

Yeah, I get the following error from direnv - do you as well?
Would it be a good idea to open a feature request?
direnv: loading ~/projects/git.shymega.org.uk/DeckCheatz/DeckCheatz/.envrc
direnv: loading https://raw.githubusercontent.com/cachix/devenv/95f329d49a8a5289d31e0982652f7058a189bfca/direnvrc (sha256-d+8cBpDfDBj41inrADaJt+bDWhOktwslgoP5YiGJ1v0=)
direnv: using devenv
direnv: .envrc changed, reloading
• Using Cachix: devenv
[1/0/7 built] building python3.11-aiohttp-3.9.3 (buildPhase): cython -3 -o aiohttp/_websocket.c aiohttp/_websocket.pyx -I aiohttp -Werrodirenv: ([/nix/store/ic11r65mm23k23jf6k1bcnvxwssl4sk7-direnv-2.34.0/bin/direnv export zsh]) is taking a while to execute. Use CTRL-C to give up.
direnv: updated devenv shell cache
pre-commit-hooks.nix: updating /home/dzrodriguez/projects/git.shymega.org.uk/DeckCheatz/DeckCheatz repo
pre-commit installed at .git/hooks/pre-commit
✨ devenv 1.0.8 is newer than devenv input in devenv.lock. Run `devenv update` to sync.
mkdir: cannot create directory ‘’: No such file or directory
/etc/profile: line 154: export: `-C': not a valid identifier
/etc/profile: line 154: export: `link-arg=-fuse-ld=mold': not a valid identifier
/etc/profile: line 156: export: `-C': not a valid identifier
/etc/profile: line 156: export: `link-arg=-fuse-ld=mold': not a valid identifier
✨ devenv 1.0.8 is newer than devenv input in devenv.lock. Run `devenv update` to sync.
bash: export: `-C': not a valid identifier
bash: export: `link-arg=-fuse-ld=mold': not a valid identifier
bash: export: `-C': not a valid identifier
bash: export: `link-arg=-fuse-ld=mold': not a valid identifier

I got:
/home/sandydoo/.cache/direnv/cas/77ef1c0690df0c18f8d629eb003689b7e6c35a13a4b70b258283f9622189d6fd:1997: DISPLAY: unbound variable
direnv: error exit status 1

Would it be a good idea to open a feature request?

You can keep this issue open as well.
@domenkozar and I briefly discussed this and we're unlikely to want to implement and maintain this because of the lack of support on macOS. Maybe we can come up with docs on how to work around this, either using the above shell override or by providing instructions on how to wrap specific binaries.

I'd also like to be able to do that. I have a project with pre-commit configured (through .pre-commit-config.yaml). Every hook works fine except the ruff hook, because of the linker.
Switching to devenv's pre-commit system is not an option, because my colleagues don't use nix (yet ;-) ).
I have experimenting with nix-ld but without full success (running ruff directly in the commandline using the absolute path in ~/.cache/precommit works fine, but for some reasons it fails with error -11 when executing it through pre-commit run or through the hook).
My idea would have been to fix that using a FHS env.

I hope FHS could be supported， cause its tedious to setup those lsp, and python conda also depend on fhs env and there are some dependencies requirements I meet before just cant resolve by poetry.

I think, if macOS is the blocker, could we add a conditional to the shell side of things that checks the pkgs.system attribute, and enables/disables based on this?

Alright then let's get it in, as long as it errors out with a clear message that macOS isn't supported and what are the alternatives.

SGTM.


fix for N+1 message length problem

this seems to fix #38, I think it was assumed call_count was evaluated at function def time, but it evaluated when the function is callend, call_count is changed in the loop.
however there may be deeper problems with the way split_input_array is used...and it does not look particularly efficient for call_counts=1

also this needs a test, but I have not yet reproduced it independent of Fi (the bug is not fi related)

I have simplified some more (I think call_count=1 is actually an error), but some of it crossed with 994a593, I think it merged ok, but please double check..

it could be I am misunderstanding the reason for the call_count=1 case, and oversimplifying. tests are ok here I think, so if that is the case a test case is needed...


[feature request] add bitsearch

hello good job with the torrent search api could you add this site to the list
https://bitsearch.to/

@yadianluffy yeah sure

ohhh thank you very much appreciated

@yadianluffy added bitsearch scrapper Check Here

thank you you are the best

thank you you are the best


Does it support "xls" file parsing?

@vela666
Yes there is this plan, but it is a bit difficult, originally I wanted to xls format withxlsxthis library to parse, and then the data structure of their own organized into the same as now, but the implementation of the workload is quite a lot, if you start from scratch, although you can refer to the source code of xlsx, but one person to do is really difficult, if there are more people together or more to give me some encouragement and support may be able to accelerate the word, now I'm a bit difficult, sorry.


Lightning Reflexes are not being properly added to Dex for turn order

As the subject says, Lightning Reflexes' bonus DEX are not being added in when the action order is being created.

Right now there is nothing that takes Lightning Reflexes into account for turn order, but I am interested in adding it.
That would only apply to a specified action correct?

From the rules:
A character can purchase Lightning Reflexes to affect all of his Actions, a group of Actions, or a single Action. The cost is indicated on the accompanying Lightning Reflexes Table.
When a character uses Lightning Reflexes to increase his effective DEX, he may only execute the specific Action or maneuver he purchased Lightning Reflexes for. For example, if a character has +4 Lightning Reflexes that only work with his Defensive Strike, he may only use his Defensive Strike (no movement, acrobatics, or other Actions) in a Phase when he uses the +4 effective DEX.

Lightning reflexes have been added to the combat tracker. Please let me know if it looks correct rules wise. Haven't had a campaign that uses lightning reflexes yet

I'll check it tomorrow. Have you joined the Hero System Discord? There's a Foundry VTT channel there. That would be a great resource for testing for you as well.

Yeah I joined it recently


etcd failure with no apparent reason

CoreOS: 717.3.0, etcd: 2.0.10
Last night two out of our three dedicated machines had etcd restart and in the process etcd as a cluster became unavailable.  The restarts were 4 hours apart, so at any one time there should have been 2 instances left in the cluster.  On the second and final restart on machine etcd3 fleet finally retracted all units across the cluster and restarted everything, a situation thats highly undesirable.
Here are the logs (restart occurs at 07:08:05):
-- Logs begin at Tue 2015-06-16 17:52:26 UTC, end at Fri 2015-08-28 13:17:42 UTC.
Aug 28 07:08:05 etcd3 systemd[1]: Started etcd2.
Aug 28 07:08:05 etcd3 systemd[1]: Starting etcd2...
Aug 28 07:08:05 etcd3 etcd2[12236]: 2015/08/28 07:08:05 etcd: already initialized as member before, starting as etcd member...
Aug 28 07:08:05 etcd3 etcd2[12236]: 2015/08/28 07:08:05 etcd: listening for peers on http://<IP_ADDRESS>:2380
Aug 28 07:08:05 etcd3 etcd2[12236]: 2015/08/28 07:08:05 etcd: listening for peers on http://<IP_ADDRESS>:7001
Aug 28 07:08:05 etcd3 etcd2[12236]: 2015/08/28 07:08:05 etcd: listening for client requests on http://<IP_ADDRESS>:2379
Aug 28 07:08:05 etcd3 etcd2[12236]: 2015/08/28 07:08:05 etcd: listening for client requests on http://<IP_ADDRESS>:4001
Aug 28 07:08:05 etcd3 etcd2[12236]: 2015/08/28 07:08:05 etcdserver: datadir is valid for the 2.0.1 format
Aug 28 07:08:05 etcd3 etcd2[12236]: 2015/08/28 07:08:05 etcdserver: discovery token ignored since a cluster has already been initialized.  --
Aug 28 07:02:18 etcd3 etcd2[577]: 2015/08/28 07:02:18 etcdserver: start to snapshot (applied: 331083256, lastsnap: 331073252)
Aug 28 07:02:18 etcd3 etcd2[577]: 2015/08/28 07:02:18 etcdserver: compacted log at index 331083256
Aug 28 07:02:18 etcd3 etcd2[577]: 2015/08/28 07:02:18 wal: segmented wal file /var/lib/etcd2/member/wal/0000000000008146-0000000013bbedf9.wal is created
Aug 28 07:02:18 etcd3 etcd2[577]: 2015/08/28 07:02:18 etcdserver: saved snapshot at index 331083256
Aug 28 07:02:21 etcd3 etcd2[577]: 2015/08/28 07:02:21 filePurge: successfully removed file /var/lib/etcd2/member/snap/0000000000000071-0000000013bb2a9d.snap
Aug 28 07:02:23 etcd3 etcd2[577]: 2015/08/28 07:02:23 filePurge: successfully removed file /var/lib/etcd2/member/wal/0000000000008141-0000000013bb2a9e.wal
Aug 28 07:08:05 etcd3 etcd2[12236]: 2015/08/28 07:08:05 etcd: already initialized as member before, starting as etcd member...
Aug 28 07:08:05 etcd3 etcd2[12236]: 2015/08/28 07:08:05 etcd: listening for peers on http://<IP_ADDRESS>:2380
Aug 28 07:08:05 etcd3 etcd2[12236]: 2015/08/28 07:08:05 etcd: listening for peers on http://<IP_ADDRESS>:7001
Aug 28 07:08:05 etcd3 etcd2[12236]: 2015/08/28 07:08:05 etcd: listening for client requests on http://<IP_ADDRESS>:2379
Aug 28 07:08:05 etcd3 etcd2[12236]: 2015/08/28 07:08:05 etcd: listening for client requests on http://<IP_ADDRESS>:4001
Aug 28 07:08:05 etcd3 etcd2[12236]: 2015/08/28 07:08:05 etcdserver: datadir is valid for the 2.0.1 format
Aug 28 07:08:05 etcd3 etcd2[12236]: 2015/08/28 07:08:05 etcdserver: discovery token ignored since a cluster has already been initialized. Valid log found at "/var/lib/etcd2/member/wal"
Aug 28 07:08:05 etcd3 etcd2[12236]: 2015/08/28 07:08:05 etcdserver: recovered store from snapshot at index 331093257
Aug 28 07:08:05 etcd3 etcd2[12236]: 2015/08/28 07:08:05 etcdserver: name = etcd3
Aug 28 07:08:05 etcd3 etcd2[12236]: 2015/08/28 07:08:05 etcdserver: data dir = /var/lib/etcd2
Aug 28 07:08:05 etcd3 etcd2[12236]: 2015/08/28 07:08:05 etcdserver: member dir = /var/lib/etcd2/member
Aug 28 07:08:05 etcd3 etcd2[12236]: 2015/08/28 07:08:05 etcdserver: heartbeat = 100ms
Aug 28 07:08:05 etcd3 etcd2[12236]: 2015/08/28 07:08:05 etcdserver: election = 1000ms
Aug 28 07:08:05 etcd3 etcd2[12236]: 2015/08/28 07:08:05 etcdserver: snapshot count = 10000
Aug 28 07:08:05 etcd3 etcd2[12236]: 2015/08/28 07:08:05 etcdserver: discovery URL= https://discovery.etcd.io/62a8042dce61fd971f2303595385aa7b
Aug 28 07:08:05 etcd3 etcd2[12236]: 2015/08/28 07:08:05 etcdserver: advertise client URLs = http://<IP_ADDRESS>:2379
Aug 28 07:08:05 etcd3 etcd2[12236]: 2015/08/28 07:08:05 etcdserver: loaded cluster information from store: etcd1.c.socialrank-prod.internal=http://<IP_ADDRESS>:2380,etcd2.c.socialrank-prod.internal=http://<IP_ADDRESS>:2380,etcd3=http://<IP_ADDRESS>:2380
Aug 28 07:08:05 etcd3 etcd2[12236]: 2015/08/28 07:08:05 etcdserver: restart member fa4f34253575aa7f in cluster ca0f831f8a418a5b at commit index 331093486
Aug 28 07:08:05 etcd3 etcd2[12236]: 2015/08/28 07:08:05 raft: fa4f34253575aa7f became follower at term 114
Aug 28 07:08:05 etcd3 etcd2[12236]: 2015/08/28 07:08:05 raft: newRaft fa4f34253575aa7f [peers: [3ce697aa7b025ce0,e9a3c498216f66af,fa4f34253575aa7f], term: 114, commit: 331093486, applied: 331093257, lastindex: 331093486, lastterm: 114]
Aug 28 07:08:05 etcd3 etcd2[12236]: 2015/08/28 07:08:05 raft: fa4f34253575aa7f [logterm: 0, index: 331093891] rejected msgApp [logterm: 114, index: 331093891] from e9a3c498216f66af
Aug 28 07:08:05 etcd3 etcd2[12236]: 2015/08/28 07:08:05 raft.node: fa4f34253575aa7f elected leader e9a3c498216f66af at term 114
Aug 28 07:08:05 etcd3 etcd2[12236]: 2015/08/28 07:08:05 raft: fa4f34253575aa7f [logterm: 0, index: 331093892] rejected msgApp [logterm: 114, index: 331093892] from e9a3c498216f66af
Aug 28 07:08:05 etcd3 etcd2[12236]: 2015/08/28 07:08:05 raft: fa4f34253575aa7f [logterm: 0, index: 331093893] rejected msgApp [logterm: 114, index: 331093893] from e9a3c498216f66af
Aug 28 07:08:05 etcd3 etcd2[12236]: 2015/08/28 07:08:05 raft: fa4f34253575aa7f [logterm: 0, index: 331093893] rejected msgApp [logterm: 114, index: 331093893] from e9a3c498216f66af
Aug 28 07:08:05 etcd3 etcd2[12236]: 2015/08/28 07:08:05 rafthttp: starting client stream to e9a3c498216f66af at term 114
Aug 28 07:08:05 etcd3 etcd2[12236]: 2015/08/28 07:08:05 raft: fa4f34253575aa7f [logterm: 0, index: 331093896] rejected msgApp [logterm: 114, index: 331093896] from e9a3c498216f66af
Aug 28 07:08:05 etcd3 etcd2[12236]: 2015/08/28 07:08:05 raft: fa4f34253575aa7f [logterm: 0, index: 331093895] rejected msgApp [logterm: 114, index: 331093895] from e9a3c498216f66af
Aug 28 07:08:05 etcd3 etcd2[12236]: 2015/08/28 07:08:05 raft: fa4f34253575aa7f [logterm: 0, index: 331093895] rejected msgApp [logterm: 114, index: 331093895] from e9a3c498216f66af
Aug 28 07:08:05 etcd3 etcd2[12236]: 2015/08/28 07:08:05 raft: fa4f34253575aa7f [logterm: 0, index: 331093897] rejected msgApp [logterm: 114, index: 331093897] from e9a3c498216f66af
Aug 28 07:08:05 etcd3 etcd2[12236]: 2015/08/28 07:08:05 etcdserver: published {Name:etcd3 ClientURLs:[http://<IP_ADDRESS>:2379]} to cluster ca0f831f8a418a5b

Besides using journalctl -f -t etcd2 to get possible missing log when process stopped (though there is no missing log in this case), you should also check:

whether the machine is hard killed/rebooted at that time
whether etcd process is killed by linux OOM killer

There may have other possibilities in system that kill etcd suddenly or miss last few lines of log. You could check this by yourself.
From etcd side, we would like to accept bug report about this if you could reproduce it in shell. Follow https://github.com/coreos/etcd/blob/master/Documentation/reporting_bugs.md to report so we could reproduce this at our side.

Close due to low activity. Please open new issue if this is a etcd problem.


"pip install tiktoken" fails when I execute "docker build" for PyPy and amd64 on an M1 Mac.

The command "pip install tiktoken" fails when I execute "docker build" for PyPy and amd64 on an M1 Mac.
We cannot deploy.
I suspect the issue may be related to cross-compilation, but I'm unable to identify the exact cause.

It succeeds on an x86 Mac.
It works when the platform option for the build is aarch64.
Trying the --cmd PyPy written in the PyO3 docs did not change the error below.
https://docs.rs/pyo3/latest/pyo3/#rustc-environment-flags

Environment:

Apple-M1 MacOS 13.0
Rancher desktop 1.7.0 (Memory 10GB)

Build command:
nerdctl -n k8s.io build --platform linux/amd64 -f dockerfile .
# nerdctl -n k8s.io is a substitute for docker build.

dockerfile:
# https://github.com/openai/tiktoken/issues/23#issuecomment-1437463984
# copied from 'wheels for ARM64 Linux #23'
FROM pypy:3.9 as builder
RUN apt-get update && apt-get install -y gcc curl
RUN curl https://sh.rustup.rs -sSf | sh -s -- -y && apt-get install --reinstall libc6-dev -y
ENV PATH="/root/.cargo/bin:${PATH}"
RUN pip install --upgrade pip && pip install tiktoken

Try it without "docker build":
nerdctl -n k8s.io run --platform linux/amd64 -it --name pypy3_container pypy:3 bash
apt-get update && apt-get install -y gcc curl
curl https://sh.rustup.rs -sSf | sh -s -- -y && apt-get install --reinstall libc6-dev -y
PATH="/root/.cargo/bin:${PATH}"
pip install --upgrade pip && pip install tiktoken

Error log:
  158.9       copying tiktoken/registry.py -> build/lib.linux-x86_64-pypy39/tiktoken
  #0 158.9       copying tiktoken/core.py -> build/lib.linux-x86_64-pypy39/tiktoken
  #0 158.9       copying tiktoken/model.py -> build/lib.linux-x86_64-pypy39/tiktoken
  #0 158.9       creating build/lib.linux-x86_64-pypy39/tiktoken_ext
  #0 158.9       copying tiktoken_ext/openai_public.py -> build/lib.linux-x86_64-pypy39/tiktoken_ext
  #0 158.9       running egg_info
  #0 158.9       writing tiktoken.egg-info/PKG-INFO
  #0 158.9       writing dependency_links to tiktoken.egg-info/dependency_links.txt
  #0 158.9       writing requirements to tiktoken.egg-info/requires.txt
  #0 158.9       writing top-level names to tiktoken.egg-info/top_level.txt
  #0 158.9       reading manifest file 'tiktoken.egg-info/SOURCES.txt'
  #0 158.9       reading manifest template 'MANIFEST.in'
  #0 158.9       warning: no files found matching 'Makefile'
  #0 158.9       adding license file 'LICENSE'
  #0 158.9       writing manifest file 'tiktoken.egg-info/SOURCES.txt'
  #0 158.9       copying tiktoken/py.typed -> build/lib.linux-x86_64-pypy39/tiktoken
  #0 158.9       running build_ext
  #0 158.9       running build_rust
  #0 158.9       <jemalloc>: MADV_DONTNEED does not work (memset will be used instead)
  #0 158.9       <jemalloc>: (This is the expected behaviour if you are running under QEMU)
  #0 158.9       <jemalloc>: MADV_DONTNEED does not work (memset will be used instead)
  #0 158.9       <jemalloc>: (This is the expected behaviour if you are running under QEMU)
  #0 158.9           Updating crates.io index
  #0 158.9       cargo rustc --lib --message-format=json-render-diagnostics --manifest-path Cargo.toml --release -v --features pyo3/extension-module --crate-type cdylib --
  #0 158.9       error: `cargo rustc --lib --message-format=json-render-diagnostics --manifest-path Cargo.toml --release -v --features pyo3/extension-module --crate-type cdylib --` failed with code -9
  #0 158.9       [end of output]
  #0 158.9
  #0 158.9   note: This error originates from a subprocess, and is likely not a problem with pip.
  #0 158.9   ERROR: Failed building wheel for tiktoken
  #0 159.0   Building wheel for regex (pyproject.toml): started
  #0 199.5   Building wheel for regex (pyproject.toml): finished with status 'done'
  #0 199.5   Created wheel for regex: filename=regex-2023.5.5-pp39-pypy39_pp73-linux_x86_64.whl size=307847 sha256=d83dcfd2f298f85a445de8f3d10dd92bdf276be4ba49caa8d015181931dbd3a6
  #0 199.5   Stored in directory: /root/.cache/pip/wheels/cb/b2/be/aa7f39218e250941dba541de7dbb51f057c4b55a4021f95311
  #0 199.5 Successfully built regex
  #0 199.5 Failed to build tiktoken
  #0 199.5 ERROR: Could not build wheels for tiktoken, which is required to install pyproject.toml-based projects

If you're trying cross compilation, maybe try setting CARGO_NET_GIT_FETCH_WITH_CLI=true. This fixes some weird issue that causes OOMs and you are seeing a -9 exit code.

Thank you very much for your help.
The build was successful.


Single posts are not being created by react

First of all, thank you for making the project available.
I implemented my project by calling information from a Wordpress installed on my server at Godaddy. My interest is to use my Godaddy server for the Wordpress panel and host the Gatsby presentation layer on Netlify.
I managed to implement the local tutorial calling from my server, but in the part of displaying the Single posts the content is called by the on the blog page, but individually is not displayed.

I forward the error image displayed in the browser as an attachment.
The terminal also displayed the error:
(node: 15552) [DEP0148] DeprecationWarning: Use of deprecated folder mapping "./" in the "exports" field module resolution of the package at
C: \ Users \ ... \ node_modules \ postcss \ package.json.
Update this package.json to use a subpath pattern like "./*".
Is there a relationship with the non-display of single posts?
Sorry, the question seems out of place, but I have no idea why this error is occurring.
Thank you for your attention

hey, @low-bin — thanks for your patience here! I missed this in my sea of GitHub notifications
the issue appears to be a double slash in your code for creating pages (see the bottom, where it's showing /blog//hello-world/ as a page that exists). most likely that extra slash is in your gatsby-node.js
hope that helps!


Update polypoly URLs

Summary
Update URLs of existing entry to reflect domain changes of polypoly web site.
Quality Assurance Checklist

Review Items
Y/N

Site has a publicly known bug bounty or vulnerability disclosure program
Y

Disclosure policy is publicly available
Y

Public URL
Y

Submission follows the Diodb schema
Y

Looks good, thank you!


Minor refactorings

See each commit for the details. :wink:

Hi @ikesyo
Great series of tweaks, thanks :+1:
If you're interested in helping me maintain this repo I'd be happy to give you write access. What do you think?

@andreamazz Thank you for the message, I'm interested in contributing to the repo and the write access. :sparkles: It's great if you add me as a collaborator!


Additions in Installation Guidelines

Targets #{1016}

I have added 3 new guidelines which are essential for the installation of the portal project
These additions help to counter a list of specific errors which one might encounter while installing the project
Checklist:

[ ~] I have performed a self-review of my own code.

@GauravGusain98 @rathorevaibhav , Since you guys had requested a change on this PR, it would require approval from your end as well. Please take a look and let's get it merged if everything looks fine.


Suggested for 'plot'

Enable adding a title to the plot.

Crop better the image so it doesn't plot in two rows when not necessary.

Fix slices='84x84x1' so it gets simply slices=84, or better slices=84:90, or slices=c(20,32,56) for multiple images.

also color bar ....
this simple example should also work much better
mni<-antsImageRead( getANTsRData("ch2") , 3 )
plot( mni )

btw, just updated to latest antsr and plot now diplays stretched images.

The numeric vector for specifying slices should work.
No idea where the squished images came from.  I'll take a look at that.

Squished images should be fixed now, and a basic version of colorbar works.  Still working on getting the proportions done reasonably.

It is possible to add a title to both the image and colorbar now, but location is still an issue.  Automatic image cropping also works now.

It's working now, thanks.
I am having an issue with images cropped to a single slice. They should appear full slice on the image, but they still appear as a raw with only a slice at the beginning. Not sure this needs a fix, the problem is cropImage and cropIndices, which produce still a 3D image even when a single slice is selected. Will post this on another issue.

Another suggestion from Dorian: Separate auto-slice selector from cropping
the image, so you can have one without the other.
2015-03-14 22:42 GMT-04:00 dorianps<EMAIL_ADDRESS>
It's working now, thanks.
I am having an issue with images cropped to a single slice. They should
appear full slice on the image, but they still appear as a raw with only a
slice at the beginning. Not sure this needs a fix, the problem is cropImage
and cropIndices, which produce still a 3D image even when a single slice is
selected. Will post this on another issue.
—
Reply to this email directly or view it on GitHub
https://github.com/stnava/ANTsR/issues/19#issuecomment-80813069.

Change plotting to stack slices in a 4:3 format instead of a single line.

Title and colorbar are not in correct position.

Also title doesn't work at all unless colorbar is enabled.
2015-03-20 16:35 GMT-04:00 dorianps<EMAIL_ADDRESS>

Change plotting to stack slices in a 4:3 format instead of a single
line.
2.
Title and colorbar are not in correct position.
—
Reply to this email directly or view it on GitHub
https://github.com/stnava/ANTsR/issues/19#issuecomment-84138049.

i think most of this is addressed now.


Add editor setting for instantiated scenes to have editable children by default

Part of the usability changes for https://github.com/godotengine/godot-proposals/issues/3248
I added a new editor setting that is disabled by default, so this is entirely opt-in.

I added code for this in 3 places, so that it would work with the Instantiate button, with dragging into the scene tree, and with dragging into the viewport.
https://user-images.githubusercontent.com/43449832/210794693-01890b1b-e6c3-49eb-8ba1-e8bfcc4b1ebe.mp4
Let me know what you think, I am open to feedback.

@realkotob this looks great - are you still maintaining it or is it abandoned though?


Add Intel E810 PF 1591 Network Adapter

The Intel series of E810 cards with a 1591 PF ID need to be added to the supported card listing in OpenShift.  This PCI ID belongs to Intel.
Cards have Vendor ID 8086, PF 1591, and VF of 1889.
Note: Silicom TimeSync cards (STS2, STS4, etc) contain this embedded E810 in those particular network adapters.
Cards were tested internally in Red Hat and found to be passing.  Request of this PR is to get them to the supported list of devices:
https://bugzilla.redhat.com/show_bug.cgi?id=1919361 -- add STS2 & STS4 cards support

/cc @Eoghan1232

just added a comment, name change.
Otherwise LGTM.


Select different values file during deployment

Is your feature request related to a problem? Please describe.
Zarf.yaml requires values file to be pre-defined. I have a use case where i need to use the same images with different configuration files in different environments. I do not want to create separate packages for each enviornment
Describe the solution you'd like
Considering zarf uses helm SDK which provides capability to override values file, can this be made available during zarf package deploy?
Describe alternatives you've considered
Alternate option is to create separate zarf yaml per enviornment and separate packages which i do not want to do as it introduces overheads for product managers
Additional context
Add any other context or screenshots about the feature request here.

@pam2014 have you looked into using https://docs.zarf.dev/examples/variables/? You have to define the interfaces but can provide options on deploy (settable with config files, --set or interactive prompts).
We have been looking at improving some of the flow between variables and Helm (https://github.com/defenseunicorns/zarf/pull/2131) but decided earlier to not expose the raw Helm values to reduce the variable surface and thus the chance for user error when packages go into an air gapped environment.


revert: switching to history mode

Reverts the relevant changes that switched the vue router to history mode.
Signed-off-by: Philipp Rudloff<EMAIL_ADDRESS>

So this is not needed anymore: https://github.com/kumahq/kuma/issues/5306 ?

@lahabana I’d still love to get https://github.com/kumahq/kuma/issues/5306 in place. Working with hash-based URLs is not great and they prevent us from ever using anchors anywhere else in the application. It’s easy enough (looking at the minimal changes) to redo this later so for now reverting this is more important than waiting for a possible fix.

I guess this can now be canned right?


"tarrget" instead of "target"?

https://github.com/Cazka/diepAPI/blob/bd3eaea5b77f041903dca6b4d0d95d8b45f9657e/src/entity_manager.ts#L252

why is this an issue instead of a pr lmao


Add UI mockup

Codecov Report
Base: 72.15% // Head: 72.15% // No change to project coverage :thumbsup:

Coverage data is based on head (2ecf5b3) compared to base (7657157).
Patch has no changes to coverable lines.

Additional details and impacted files
@@            Coverage Diff            @@
##             master      #48   +/-   ##
=========================================
  Coverage     72.15%   72.15%
  Complexity      399      399
=========================================
  Files            70       70
  Lines          1232     1232
  Branches        125      125
=========================================
  Hits            889      889
  Misses          311      311
  Partials         32       32

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


[refactor] kubectl changes to go with those submitted in kraken-lib

What this PR does / why we need it: Refactors code to the changes in line with changes in https://github.com/samsung-cnct/kraken-lib/pull/854. Allows for deterministic patters of input and correct handling of arguments passed.
Which issue this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged): fixes #
Special notes for your reviewer:
Release note:

going to dismiss @joejulian's review as he is blocking on my review.


Racon hanging on aligned overlap on unix

Hello,
There are some other issue reports that are similar such as  racon stuck at generating consensus and I'm not sure if Racon is stalled or just taking a long time, but I wanted to reach out as the issue I am seeing is a bit different and is stalling at a different step.
I am running racon version 1.3.2 on a 28-core, 256Gb RAM unix server running CentOS 7. When I run racon, the program loads the sequences and the overlaps just fine, but seems to stall out/run very slow on the aligned overlap step. I have also seen this issue using v1.3.1.

When I check the CPU usage of racon using top, it seems the program has gone to sleep or into some sort of dormancy.

If I watch top, it will sometimes spike up to 0.2% CPU usage, but stays at 0.0% usage most of the time. Racon will continue to align overlaps, but it moves incredibly slowly, sometimes taking multiple hours.
If I run the exact same data downloaded from the unix server on a 4-core MacBook Pro with 16Gb of RAM, also using version 1.3.2 of racon, the program will complete in under an hour and will run full usage of the cores I give it. The same command is being used on both systems, just with a change in the threads parameter. I have also recreated this issue with multiple datasets now.
The command I am using is:
racon -t 8 CombinedIlluminaReads.fq align.sam assembly.fasta > assembly_racon.fasta
I have tried building racon on the unix server using the git clone method outlined in the README, as well as using the release package tar.gz. Additionally, I have built the tool using the -Dracon_build_tests=ON flag and racon_tests passes all of the tests.
Any thoughts on what is going on?

Hi,
could it be that it is hitting swap due to memory shortage?
Best regards,
Robert

Hi Robert,
I would be shocked if this was hitting swap memory. The current example I posted above is running now and there is more than 28Gb of memory indicated as free. This also doesn't account for the memory bound up by the system that is freed when more memory is needed.
The dataset itself is not very big. The genome size is 2.7Mb and we have a little over 100x coverage for the Illumina reads. The assembly is from Oxford Nanopore data that has been nanopolished, and is now going through subsequent iterative racon polishes to get as clean of an assembly as possible.
One other note, I am using your script to combine the paired-end Illumina data.
Thanks for your help!

Would you maybe be willing to share one of your datasets that behave as described above? It would make it easier for me to solve this issue.

Absolutely. Which files would you like and is there a good way for me to transfer this to you directly?

It would be the best if I get all files you were using (reads, overlaps, contigs) and the exact command. Share with google drive/dropbox and send the link via mail?

Will do. Pulling it all together now. Thank you again!


Christoph's Vorschlag: Split up config and private data

Seperation of config and private data

Done in #227


pkgs.fetchgit not deterministic when fetching submodules

Description
When I try to use pkgs.fetchgit with fetchSubmodules = true and deepClone = true on a repo that has a submodule that references the same submodule as root repo I get a hash mismatch error and different hashes for different nix-build runs:
hash mismatch in fixed-output derivation '/nix/store/lyis1l5zbsi9kfhlsg16pqxs12y518jn-nix-fetchgit-debug-f0e389d':
  wanted: sha256:0qmrg2a79ynjdwcw8jgkd0qdy880rwn366p78qr9yj7hrnznpzci
  got:    sha256:0brc139p24m1b4hrxkkcra5fjlv8dl1662a9r355jlhpp41km033
error: build of '/nix/store/zz6cbs4440yiqdb007ddq7yldkzwgd38-nix-fetchgit-debug-f0e389d.drv' failed

Reproduction
You can reproduce this behavior by building the derivation in this debug repo:
https://github.com/status-im/nix-fetchgit-debug
Additional context
This seems to be caused by differences in Git Packfiles when comparing results of the derivation built multiple times:
 > diff -r \
     /nix/store/lyis1l5zbsi9kfhlsg16pqxs12y518jn-nix-fetchgit-debug-f0e389d \
     /nix/store/9vylzwxxi52bvyr8dcrsaaxj5ppwrid2-nix-fetchgit-debug-f0e389d

diff -r /nix/store/lyis1l5zbsi9kfhlsg16pqxs12y518jn-nix-fetchgit-debug-f0e389d/nim-faststreams/.git/objects/info/packs /nix/store/9vylzwxxi52bvyr8dcrsaaxj5ppwrid2-nix-fetchgit-debug-f0e389d/nim-faststreams/.git/objects/info/packs
1c1
< P pack-2a516976bb18daa498c8296b7de9c3bca3a61dba.pack
---
> P pack-348a1af74c1229307cad55ddc9af3b06f4496d20.pack
Only in /nix/store/lyis1l5zbsi9kfhlsg16pqxs12y518jn-nix-fetchgit-debug-f0e389d/nim-faststreams/.git/objects/pack: pack-2a516976bb18daa498c8296b7de9c3bca3a61dba.idx
Only in /nix/store/lyis1l5zbsi9kfhlsg16pqxs12y518jn-nix-fetchgit-debug-f0e389d/nim-faststreams/.git/objects/pack: pack-2a516976bb18daa498c8296b7de9c3bca3a61dba.pack
Only in /nix/store/9vylzwxxi52bvyr8dcrsaaxj5ppwrid2-nix-fetchgit-debug-f0e389d/nim-faststreams/.git/objects/pack: pack-348a1af74c1229307cad55ddc9af3b06f4496d20.idx
Only in /nix/store/9vylzwxxi52bvyr8dcrsaaxj5ppwrid2-nix-fetchgit-debug-f0e389d/nim-faststreams/.git/objects/pack: pack-348a1af74c1229307cad55ddc9af3b06f4496d20.pack
diff -r /nix/store/lyis1l5zbsi9kfhlsg16pqxs12y518jn-nix-fetchgit-debug-f0e389d/nim-waku/vendor/nim-faststreams/.git/objects/info/packs /nix/store/9vylzwxxi52bvyr8dcrsaaxj5ppwrid2-nix-fetchgit-debug-f0e389d/nim-waku/vendor/nim-faststreams/.git/objects/info/packs
1c1
< P pack-7a7a30973ad008980ac4954722689eb55b1941df.pack
---
> P pack-b56d89b8eafd21f484dc6af50a7771bbf28ed92c.pack
Only in /nix/store/lyis1l5zbsi9kfhlsg16pqxs12y518jn-nix-fetchgit-debug-f0e389d/nim-waku/vendor/nim-faststreams/.git/objects/pack: pack-7a7a30973ad008980ac4954722689eb55b1941df.idx
Only in /nix/store/lyis1l5zbsi9kfhlsg16pqxs12y518jn-nix-fetchgit-debug-f0e389d/nim-waku/vendor/nim-faststreams/.git/objects/pack: pack-7a7a30973ad008980ac4954722689eb55b1941df.pack
Only in /nix/store/9vylzwxxi52bvyr8dcrsaaxj5ppwrid2-nix-fetchgit-debug-f0e389d/nim-waku/vendor/nim-faststreams/.git/objects/pack: pack-b56d89b8eafd21f484dc6af50a7771bbf28ed92c.idx
Only in /nix/store/9vylzwxxi52bvyr8dcrsaaxj5ppwrid2-nix-fetchgit-debug-f0e389d/nim-waku/vendor/nim-faststreams/.git/objects/pack: pack-b56d89b8eafd21f484dc6af50a7771bbf28ed92c.pack

The cause of this seems to be the fact that both the root repo and the nim-waku submodule reference the same nim-faststreams repo. Also both this repo and nim-waku use the same nim-faststreams commit: 5df69fc6.
Metadata
 - system: `"x86_64-linux"`
 - host os: `Linux 5.7.19, NixOS, 20.03.git.95d979819fa (Markhor)`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.3.6`
 - channels(sochan): `"nixos-20.03.3107.067d8e6c9f4, nixos-unstable-21.03pre246062.420f89ceb26"`
 - channels(root): `"nixos-20.03.3112.08d429920bc"`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nix

Pings
@Mic92 @bhipple @jtojnar @MarcWeber @bjornfor

I think that's because .git needs to be included in case of a deepClone, and so it cannot be guaranteed that it is reproducible. If this is the case, such option should not exist in a Nixpkgs builder and should in my opinion be removed.

Another option would be to remove the .git/objects/info/packs and .git/objects/pack files here:
https://github.com/NixOS/nixpkgs/blob/d74573d8ae6f02ef0ac1299c862d1b762ba0aad1/pkgs/build-support/fetchgit/nix-prefetch-git#L244-L247

This is still and issue and not stale.

I've been working on some reproducible builds in Debian which use submodules, and have based that code off of the prefetch routine referenced here. The following script successfully creates a usable and deterministic clone for a repo with submodules of any depth and reference location in my tests - feel free to test and adapt as needed.
#!/usr/bin/env bash

set -euf -o pipefail

#
# Enumerates the various nondeterministic data objects in a git repository that
# should be deleted.
#
declare -ra NONDETERMINISTIC_DATA=(
    "logs"
    "hooks"
    "index"
    "FETCH_HEAD"
    "ORIG_HEAD"
    "config"
    "refs/remotes/origin/HEAD"
)

#
# Cleans the submodules in the given directory, removing nondeterministic data.
# @param MODULES_DIR The submodule directory to clean
#
function clean_submodules() {
    local -r MODULES_DIR="${1}"

    for module_dir in $(find "${MODULES_DIR}" -mindepth 1 -maxdepth 1 -type d)
    do
        for d in "${NONDETERMINISTIC_DATA[@]}"; do
            rm -rf "${module_dir}/${d}"
        done

        # Clean nested submodules
        if [[ -d "${module_dir}/modules" ]]; then
            clean_submodules "${module_dir}/modules"
        fi
    done
}

#
# Cleans the given git repository, removing nondeterministic data and repacking
# its objects.
# @param REPO The local git repository to clean.
#
function clean_repo() {
    local -r REPO="${1}"

    cd "${REPO}"

    # Clean submodules, if any
    if [[ -d .git/modules ]]; then
        clean_submodules .git/modules
    fi

    # Remove files that contain timestamps or other nondeterministic properties
    for d in "${NONDETERMINISTIC_DATA[@]}"; do
        rm -rf ".git/${d}"
    done

    # Remove remote branches
    git branch -r | while read branch; do
        git branch -rD "${branch}" >&2
    done

    # Remove unreachable tags
    local -r MAYBE_TAG="$(git tag --points-at HEAD)"
    git tag --contains HEAD | while read tag; do
        if [[ "${tag}" != "${MAYBE_TAG}", ]]; then
            git tag -d "${tag}" >&2
        fi
    done

    # Do a full repack. Must run single-threaded or determinism is lost.
    git -c pack.threads=1 repack -A -d -f
    rm -f .git/config

    # Garbage collect unreferenced objects
    git gc --prune=all --keep-largest-pack
}

#
# The main entry point of the program.
# @param REPO The local git repository to clean.
#
function main() {
    local -r REPO="${1}"

    # Run in subshell to not touch working directory
    ( clean_repo "${REPO}" )
}

main ${@}

I elected not to delete .git/objects/info/packs and .git/object/pack for the sake of backwards compatibility, and it appears to work fine without it.

Any news on it? It would be great to have reproducible submodule fetch.

@Nihlus when in the build process do you call that script?

@kjeremy Hey, sorry about the long wait - things have been busy here and I kept forgetting to check.
Essentially, I call it right after the clone from a specific tag and branch, and then I pack the results into a deterministic tarball. It boils down to this (assuming UPSTREAM_BUNDLE is a git bundle made from a detached head checkout with submodules updated and initialized):
$(UPSTREAM_TARBALL): $(UPSTREAM_BUNDLE)
    git clone ${UPSTREAM_BUNDLE} -b ${PACKAGE_VERSION} --single-branch ${TMP_CLONE}
    make-deterministic.sh ${TMP_CLONE}
    tar -C ${TMP_CLONE}/../ --sort=name --mtime=@${SOURCE_DATE_EPOCH} --owner=0 --group=0 --numeric-owner --pax-option=exthdr.name=%d/PaxHeader/%f,delete=atime,delete=ctime -cf - ${BUILD_NAME} | gzip -n > ${UPSTREAM_TARBALL}

This also breaks flakes which use submodules in one of their inputs

This issue has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/handling-git-submodules-in-flakes-from-nix-2-18-to-2-22-nar-hash-mismatch-issues/45118/1


Contribute is_power_of_ten?

I'd like to contribute a high performance implementation of is_power_of_ten. Is that an appropriate function for this crate?

I'm open to it, if we can present it in a generic trait befitting the rest of the crate.  That doesn't mean that you have to have a generic implementation, just that it's designed such that things like BigUint can implement it too.  Take the Roots trait, for example.
The unsigned primitive integers already have is_power_of_two, and a related next_power_of_two.  It would be good to also abstract this in a trait, while we're doing it for 10.  Something like:
pub trait Power2: Integer {
    fn is_power_of_two(&self) -> bool;
    fn next_power_of_two(&self) -> Self;
}
pub trait Power10: Integer {
    fn is_power_of_ten(&self) -> bool;
    fn next_power_of_ten(&self) -> Self;
}

Or maybe just combine those into a single trait Powers.  And feel free to suggest a better name.
Could we reasonably implement checks for arbitrary powers of n?  It wouldn't have to be as fast, but I'm wondering if that's feasible and useful to have too.

A separate trait is consistent with the way I was thinking about it.
Curious, why don't signed primitive integers implement is_power_of_two? (obviously, all negative numbers would return false).
How would we abstract over arbitrary powers on n? Extra parameter to the method? I think 2 and 10 sufficiently interesting to have their own trait.

Curious, why don't signed primitive integers implement is_power_of_two? (obviously, all negative numbers would return false).

I'm not sure, but probably just because negative inputs are useless if they're all false.
Similarly, what would next_power_of_two do with a negative -- jump all the way to 1?

How would we abstract over arbitrary powers on n? Extra parameter to the method?

Yeah, fn is_power_of_n(&self, n: u32) -> bool, or maybe n: usize or n: &Self.  I'm not sure if this is really useful anywhere, just musing.

Alright, that gives me enough to move this over to a PR. Expect something in the next few days. :smile:


Is this being deprecated ?

With MS recommending Flows over Workflows and The ability to add a Workflow/action from Power platform is it being deprecated ?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 57349746-f059-70c3-49fb-976189f7e46c
Version Independent ID: d39f4298-e77a-2629-0ebb-e0f1e8af3651
Content: Workflow processes
Content Source: ce/customerengagement/on-premises/customize/workflow-processes.md
Service: crm-online
GitHub Login: @Mattp123
Microsoft Alias: matp

@pavansarma93 - For information about deprecated features, see Important changes. Classic workflows are currently not included in the list. Closing issue.


Walker does not stay in bounds for non-square maps

If you change the map size to be a rectangle, not a square, the walker will often "escape" the bounds of the map you gave it.

My bad, I broke the borders when I refactored.

My bad, I broke the borders when I refactored.

Hey no worries. It happens. Let me know if you have any questions.


Utterly miserable experience

Hi,
Your insdtructions at https://devblogs.microsoft.com/visualstudio/creating-a-react-todo-app-in-visual-studio-2022/and the cloned source faile miserably to be either folloew-able or to work.  Tell me, why on earth do you not test stuff you put up as instructional before you publish it ?  Miserable.

Hello, sorry about that I made a lot of changes to the code and I must have missed updating the post. I'm going through and updating it today.
What problems did you have when you cloned the repo here? The code here should be working, but if not, I will get it fixed.


"wrong adaptation size" error

Get "mpegts: wrong adaptation size" error for the stream added to go2rtc.
Stream works ok in vlc

You should provide stream dump if you want this fixed.

First, it needs to understand what kind of stream source you have. You haven't provide any info.

@AlexxIT, this is an output from ffmpeg -i:
Input #0, mpegts, from 'https://streamer126.tattelecom.ru/intercom_8102/mpegts?token=XXX
Duration: N/A, start: 58366.001667, bitrate: N/A
Program 1
Stream #0:0[0xd3]: Video: h264 (Main) ([27][0][0][0] / 0x001B), yuvj420p(pc, bt709, progressive), 1920x1080, 25 tbr, 90k tbn, 180k tbc
Stream #0:1[0xdd]: Audio: aac (LC) ([15][0][0][0] / 0x000F), 8000 Hz, mono, fltp, 37 kb/s

OK. You can try curl or wget for this url with some timeout. Can you handle it?

https://github.com/AlexxIT/go2rtc#source-ffmpeg

thanks

But if you send me a dump from curl or wget, I can try to fix this for http source

It seems that problem was on my side. I tried again and no it works


Active admin closes #18

Coverage remained the same at 100.0% when pulling 7e703f335788785eb99cdd144a6c1639d2af6523 on active-admin-#18 into 291e93fea7b755b99fa1d516748b956d4f9ae142 on develop.


'make install' does not create 'tf/Tensorflow.so' but instead create 'tf'

When we do 'make install' step, the system will do something like:
/usr/bin/install -c -m 0755 Tensorflow.so /usr/local/rvm/rubies/ruby-2.2.4/lib/ruby/site_ruby/2.2.0/x86_64-linux/tf
However if the tf/ does not exist thus instead of creating Tensorflow.so in tf/, it just creates Tensorflow.so
The easiest bypass for now is to create 'tf/' in the directory that the install will be prior to doing 'make install'.
Investigating better possibilities.

@Arafatk One way to elegantly resolve the 'make install' issue is to restructure the 'ext/' directory, which I have done.
I have also restructure the 'lib/' directory so that Graph, Session, Tensor, etc. are now all prefixed with Tensorflow::. Hopefully this keeps the global namespace clean; much like in Python tensorflow.Session is used instead of just Session, with this patch we can use Tensorflow::Session.
Any thoughts? Please let me know how to make it better.

Closing because #9 merged.


Gerrit Build Fails using intellij-CE-2019.1 plugin.

I have installed default IJ-2019.1 CE edition.
Install the Bazel plugin latest supported version for this IJ, which was: v2019.08.05 from the official Bazel Jetbrains repository.
Bazel build works fine for the project.
But when you try to import the project / sync it with the workspace it always fails with this error:
ERROR: /home/trevor/development/vanilla/gerrit/gerrit-2.16/tools/BUILD:11:1: in @intellij_aspect//:intellij_info_bundled.bzl%intellij_info_aspect aspect on py_binary rule //tools:merge_jars:
Traceback (most recent call last):
	File "/home/trevor/development/vanilla/gerrit/gerrit-2.16/tools/BUILD", line 11
		@intellij_aspect//:intellij_info_bundled.bzl%intellij_info_aspect(...)
	File "/home/trevor/.cache/bazel/_bazel_trevor/d727a15682ad095c2677225881c6eee5/external/intellij_aspect/intellij_info_bundled.bzl", line 54, in _aspect_impl
		intellij_info_aspect_impl(target, ctx, semantics)
	File "/home/trevor/.cache/bazel/_bazel_trevor/d727a15682ad095c2677225881c6eee5/external/intellij_aspect/intellij_info_impl.bzl", line 889, in intellij_info_aspect_impl
		collect_py_info(target, ctx, semantics, ide_info, <2 more arguments>)
	File "/home/trevor/.cache/bazel/_bazel_trevor/d727a15682ad095c2677225881c6eee5/external/intellij_aspect/intellij_info_impl.bzl", line 294, in collect_py_info
		struct_omit_none(sources = sources, launcher = py_l..., <2 more arguments>)
	File "/home/trevor/.cache/bazel/_bazel_trevor/d727a15682ad095c2677225881c6eee5/external/intellij_aspect/intellij_info_impl.bzl", line 298, in struct_omit_none
		_get_python_srcs_version(ctx)
	File "/home/trevor/.cache/bazel/_bazel_trevor/d727a15682ad095c2677225881c6eee5/external/intellij_aspect/intellij_info_impl.bzl", line 270, in _get_python_srcs_version
		getattr(ctx.rule.attr, "srcs_version", defau...")
unexpected keyword 'default', for call to function getattr(x, name, default = unbound)

This appears to be because the getattr, version in the official release, has removed "default" from being a named parameter, to being purely a positional param.
Now I can see that you have indeed fixed this issue in later versions of the IJ plugin for 2019.2 / .3, by this PR change. https://github.com/bazelbuild/intellij/commit/3b0fcdd90f42922c8deab01c0d5666e30fac5504#diff-3a3a2e2292228746349df940a4d3d83b
Would it please be possible to pull this back into earlier code stream and publish another update for CE2019.1 if possible.   thanks.

Would it please be possible to pull this back into earlier code stream and publish another update for CE2019.1 if possible. thanks.

The team isn't shipping any more updates for 2019.1. Please update to 2019.2, or manually build the plugin with the aforementioned cherrypick.