Support for custom genesis via EIP1085

Continuation of https://github.com/ethereum/py-evm/pull/1247
fixes https://github.com/ethereum/py-evm/issues/722
What was wrong?
Trinity should be able to run custom chains beyond just mainnet and ropsten.
How was it fixed?
Now mainnet and ropsten are done in the same way that a custom chain would be, via an EIP1085 genesis file.  There is now a new cli option --genesis which can be used to specify either the raw JSON string or a filesystem path to a JSON document.
Cute Animal Picture

Don't bother looking at this.  It's piling up into 3-4 separate PRs that need to be untangled.

@pipermerriam from what I read, EIP1085 doesn't seem to be finalised.
Do you have any timeline for this PR to land. And even if it lands do you think the spec will remian unchanged? I'm asking because I'd like to integrate this into eth-teser as part of https://github.com/ethereum/eth-tester/issues/129.

@voith no idea on either (this PR or finalization of spec).  I don't think that should block you however from moving forward with using it in eth-tester.  Make sure the API is marked in some way as experimental and subject to breaking changes.

More updates on this:
@carver and @cburgdorf
This is now yielding some interesting and fundamental flaws with how we currently do multiprocess communication, but I think it's a good thing.  The concise issue is that with the introduction of custom genesis configurations, we end up with un-pickleable VM classes since they must be dynamically constructed based on the genesis configuration (this is primaily due to the dao fork, but even without this we'd have the same problem with POA networks since we'll have to inject the proper consensus engine).
That means all of Chain.get_vm_configuration(), Chain.get_vm, Chain.get_vm_class_for_block_number and probably a number of others can no longer be called across the multiprocessing boundary.  I think the proper way to fix this is to do some or all of the following.

Remove the chain object from the things served up by the database process and require local chain instances in each process (thought the underlying database connection may still be across process boundaries)
Switch to RocksDB which allows for multiple database connections at which point we can potentially get rid of the database process entirely.

I see the potential for this to break some things, specifically surrounding our assumptions about where processing will happen and thus which calls are safe to make within the networking process without risking blocking too long and timing out with all of our peers.

require local chain instances in each process

This sounds right to me. I believe I've done all the necessary work on Chain to make this fairly painless (no local state, all state depends on the database). The only necessary change was removing the canonical head attribute into the MiningChain.

Working on rebasing and splitting this up but also pleased to report that a basic test of this code seems to be working right  now.

Pulled out 6 things from this. will rebase when they are merged:

https://github.com/ethereum/py-evm/pull/1433
https://github.com/ethereum/py-evm/pull/1434
https://github.com/ethereum/py-evm/pull/1435
https://github.com/ethereum/py-evm/pull/1436
https://github.com/ethereum/py-evm/pull/1437
https://github.com/ethereum/py-evm/pull/1438

@carver looking for feedback, still not sure this is really "done" but I'd like to get some review before I take it any further.  I think I can still yank out maybe 100-200 lines from the diff into smaller pull requests but it's getting harder.

grumble,  There is no reason I can tell for the integration test to be failing.  It passes locally and passed in python3.7

grumble, There is no reason I can tell for the integration test to be failing. It passes locally and passed in python3.7

Huh, no idea. It looks like it's just hanging, huh?

I can reproduce the integration failure reliably, although I'm using Python 3.7 (not 3.6 as in current failure).
The reason seems to me as follows. The explanation may be bogus. If correct, though, then it's not strictly caused by changes in this PR.

During a regular (non-test) run of
trinity --light --tx-pool

the error message
   ERROR  11-11 17:15:46   BasePlugin#TxPlugin  Transaction pool not available in light mode

is printed after being caught as an exception; the fact that it's an exception allows trinity to back-track the start-up procedure, and shut down gracefully.

During a test run of
pytest "tests/trinity/integration/test_trinity_cli.py::test_txpool_deactivated[command0]"

the exception seems to be "suppressed" by pytest somehow, which is subsequently reflected in
ERROR  11-11 17:17:46             plugin.py  Task exception was never retrieved

The fact that the exception doesn't reach where it's supposed to prevents graceful shutdown.

Rebasing against master to resolve current conflict with trinity/plugins/builtin/tx_pool/plugin.py gives me the same result as before rebasing.

The weird thing is that running command1 instead of command0 with pytest, which amounts to
trinity --light --ropsten --tx-pool

passes successfully.
On Ropsten, my geth lightserv always has free capacity, so I'm assuming there's no problem finding a peer; and the server is in trinity's bootnode list...
On main-net, that is not the case...
However, I find it extremely weird that finding a peer is a prerequisite of determining that both --light and --tx-pool have been specified, and shutting down. X_X

Found culprit. See https://github.com/pipermerriam/py-evm/pull/2

I'm unlikely to be able to pick this back up for at least another 1-2 weeks.  Anyone else should feel free to take this over if they choose, otherwise, I'll pick it back up when I can.

I picked this up today to move it forward. Great work @pipermerriam !
I just applied some basic cleanup so far.
More importantly though, I took a deep dive into the changes to understand them better. I think the most important thing to understand is what @pipermerriam pointed out already in a comment.

I see the potential for this to break some things, specifically surrounding our assumptions about where processing will happen and thus which calls are safe to make within the networking process without risking blocking too long and timing out with all of our peers.

To make it clear, this PR makes quite a drastic change in which the Chain object that is passed into the Server does not continue to be a proxy anymore but instead is a local chain of that process. That means that all called APIs (importing blocks, executing transactions (regular sync)) will run in the same process as the rest of the networking code.
Previously, all this work happened outside of the networking process in the database process.
There are two possible scenarios now:

Turns out, doing this work in the networking process isn't a problem

Doing this work in the networking process is a problem because and we can not keep up with the networking traffic anymore eventually resulting in a loss of connections.

It's probably a good idea to dig deeper to gain clarity about this but my gut feeling says that it will most likely be a problem (after all, there's a reason why these things were separated in the first place)
Assuming that it will be a problem, here are some things to try out.

Try to achieve the previous architecture again that relied on proxies. This has become problematic because the ChainConfig is now dynamic and non-pickable. I believe it should be possible to hook into the serialization to simply teach it that the ChainConfig does not need to be serialized. After all, we do not care about the serialization of the  ChainConfig (as it can be reconstructed in every process) and only want to continue to have a ProxyChain to call methods on remotely. I think that should be doable.

Try to build a new isolated plugin that serves as process for block processing and gets it work items feed through the EventBus. This should be fairly straight forward (the JSON-RPC Server does something roughly related when in light mode)

Extract the entire syncing logic to become an isolated plugin. I believe this is a long term goal anyway but it I think it's not really a relaxed afternoon task as it basically means we would need to detangle, node, peer pool and syncing logic to all go into separate processes (isolated plugins) that use the EventBus as a first class communication pattern.

I'm intrigued to spend some time to look into the second option as it seems to be a nice stepping stone towards the third option (aka the end game) but less frightening :)
@carver @pipermerriam thoughts?

So it seems import_block and maybe validate_receipt are the only potentialy costly methods that are called as part of syncing.
I started working on an isolated block importer and I think I like it.
https://github.com/ethereum/py-evm/pull/1299/commits/eb9e208ca66a1fa96be1d73a01cf5e9f078282f9

@cburgdorf I don't have strong enough intuition about this to know what is best so use your best judgement.  Feel free to consider anything that steps us closer towards the goals outlined in #1485 if it seems to fit.

@pipermerriam ok, I'm currently fiddling with the second approach where we move the expensive chain interaction of the syncing in a dedicated process provided by an isolated plugin. That said, the really costly stuff doesn't affect fast sync so much. So, if we move forward without that, we would most likely only see things becoming a problem when we go into regular sync (which appears to be broken currently anyway)
I will fiddle with that a bit longer to form a better opinion.

@pipermerriam I moved that exploratory work out of this into https://github.com/ethereum/py-evm/pull/1497
I think we should take this as is because the performance issues will most likely not hit us too hard during fast-sync and we aren't quite at a point where regular sync is our main prio yet. So, I think we should move this forward so that hive integration / görli can advance forward.
I will still follow up on the performance issues as (hence #1479) but I don't think they should be blocking this from getting merged.

(CI is green, rate limit issue as usual)

@pipermerriam Apparently I can't ping you for a review since it is your PR. I approved it (you've written most of it after all) but I would wait on another :+1: before I merge.

PR still missing Cute Animal Pic.

@veox fixed!

@carver Thanks for reviewing this!

Obviously needs a rebase and/or squashing since there are dirty commits.

Yep, will just squash this all together into one commit that adds custom genesis support

The only thing that might need more discussion is: what's happening in the transaction validator? At best, something is unclear, but it appears broken.

I think you are right. Will add tests for this (tomorrow)

Ping me please when custom genesis / config is ready to play with. I am adding Trinity to Hive.

I didn't branch things out into multiple PRs but I sliced semantic commits that can be reviewed in isolation and should make it easier to follow.
I also added tests for the transaction validation stuff and corrected the functionality (it was indeed buggy).
I did not change the is_full_mode / is_light_mode stuff yet because the SYNC_LIGHT / SYNC_FULL constants are used at multiple places across the code base and if we refactor that to use an enum (I think we should) I would bloat this PR even more. I can send a follow up PR to refactor that.
@carver do you like to give this another review?

I did not change the is_full_mode / is_light_mode stuff yet ...  would bloat this PR even more

Yup, totally reasonable.
It's Thanksgiving, so I'll be mostly offline. I'll give a optimistic :+1: since you mentioned that this merge is needed to let master run a regular sync.

@FrankSzendzielarz This is now merged. Let me know if you run into any issues trying it out :tada:


MDNS example not working (mdns_query_a unable to resolve host name to IP) (IDFGH-7528)

Environment

Development Kit:      ESP32S3-DevKitC
Kit version (for WroverKit/PicoKit/DevKitC): v1
Module or chip used:  ESP32-S3-WROOM-1
IDF version (run git describe --tags to find it): v4.4.1
Build System: idf.py
Compiler version (run xtensa-esp32-elf-gcc --version to find it): xtensa-esp32-elf-gcc (crosstool-NG esp-2021r2-patch3) 8.4.0
Operating System: Windows
(Windows only) environment type: ESP Command Prompt
Using an IDE?: No
Power Supply: USB

Problem Description
mdns_query_a unable to resolve to IP address
Expected Behavior
mdns_query_a should resolve to valid IP address
Actual Behavior
mdns_query_a always returns ESP_ERR_NOT_FOUND
Steps to reproduce

Modify the \examples\protocols\mdns\main.c in the function check_button:

         // query_mdns_hosts_async("esp32-mdns");
        query_mdns_host("esp32-mdns");

Compile the example
Press GPIO_0 (BOOT) to trigger query
Repeat step 3 many times

// If possible, attach a picture of your setup/wiring here.
Code to reproduce this issue
static void check_button(void)
{
    static bool old_level = true;
    bool new_level = gpio_get_level(EXAMPLE_BUTTON_GPIO);
    if (!new_level && old_level) {
        // query_mdns_hosts_async("esp32-mdns");
        query_mdns_host("esp32-mdns"); //not working
        query_mdns_service("_http", "_tcp"); //not working
    }
    old_level = new_level;
}

Other info:
sdkconfig.txt

Host is reachable via ping esp32-mdns for Windows or ping esp32-dns.local for MAC.
Host appears in the network via dns-sd -B for MAC.
Same issue appears for Huawei and ASUS router.
Tried the solution described in https://github.com/espressif/esp-idf/issues/2507 but nothing works

Hi @ble0
Are you trying to resolve our own host name? I'm sorry, but this will not work, the packets that bounce back are suppressed.

I’m just curious. Then why some systems send mdns query their own hostname?

Then why some systems send mdns query their own hostname?

While querying their own names gets resolved correctly on some platforms (such as avahi), it's not supported in Espressif's mDNS library.
This was a design decision to keep the implementation simple and practical (and per mDNS spec: not to pollute the local network), and the way how the low level multicasts on WiFi are implemented.

do you mean the "known suppression" mechanism?

No, this is unrelated to mDNS's definition of known suppression. This is how multacasts are treated on physical network interfaces (esp. WiFi)
And also in mdns sources:
https://github.com/espressif/esp-idf/blob/01d014c42def8d0c19e1ce55c07de6761e092ffa/components/mdns/mdns.c#L3307-L3322
PS: If you compile the mDNS library for a different platform (and remove the conditions above), you might be able to resolve the advertized service (you can check this linux compilation example)
PPS: I'm sorry, this is not well documented that the mdns library doesn't resolve its own advertised names and services. Will update the docs.

Hi David, thanks. Yep keep me posted when these doc updates are pushed.


Refactoring RPC & Archive

@unnamedd thanks for review! I'm going to use your suggestions in future refactoring

@unnamedd thanks for review! I'm going to use your suggestions in future refactoring

if you want, I can review the other PRs in this repo. Will be a pleasure.


Update scrape.py

The output list of get_anime() will now be sorted based on weekday and time.
anime button will also show time in sorted order


Extension seems to crash on startup

Hello i went through the manual installation and the settings/ the button to enable and disable at the bottom appeared as expected.
I added my node path "/usr/bin/node" but im still not getting any completion suggestions.
when opening the Console it shows an error being thrown at startup (Uncaught Error: write EPIPE)
obsidian.md-1715152018123.log
I also never received the ready to use notice stated in step 4 of the installation.
Any guidance on what to try from here?

Hi @timo-x30xe, thanks for your interest!
From your log, it seems that the node process is not running as expected when we try to communicate with it.
This possibly comes from a problem with your node path or your node version. Can you give more information about your node version?
If you installed the plugin by cloning the project, can you enabled the debugging mode (turn the setDebug to true l.30 in the main.ts file), it will give use more detail about the child process crash.

@timo-x30xe I've updated the code to add a "Test the path" button near the node path input in the plugin settings.
It will check for you if the node path is accessible, executable and if the node version is correct.
You can get this update by pulling the last commit or downloading the last released files in the Release 1.0.0.

Hi, thanks for the update. I am using just the release file instead of building it from source.
Using the "Test the Path"-Button throws an error but nothing pops up in the Obsidian log (see Screenshot)

The path is the same that is being returned by "which node" and the version is 20.12.0

There also now appears to be a "Copilot has stopped" notification popping up when enabling the extension.

I just realised it might have to do with Obsidian being a flatpak installation for me.
so i tried adding "/var/run/host/" in front of my path which gives me a different error

@timo-x30xe I've implemented the sign-in process in case you need it. You can download the files from the release 1.0.0.
If you're not signed in, a popup will appear with instruction to follow.
Don't hesitate to tell me if it works for you or not

@Pierrad Thank you! The Method you described worked flawlessly. Didn't expect you to go through that trouble reproducing it with a VM for such a niche issue. I also used the Sign-in Process as it popped up and it worked without any issues
Thanks for the fast response and the great work on the plugin!


[CLI] Timeout command #66

PR for #66

Coverage increased (+0.43%) when pulling a56654d788c21761f4c5e5fae9ed5d83062601f5 on tmilos:I-66 into 3ded898798a15d7cbe348d7d7b0753adb4bad462 on aerialship:master.


Make exporter pod PSA ready

Signed-off-by: Alex Kalenyuk<EMAIL_ADDRESS>
What this PR does / why we need it:
AFAIK there is no reason to not make this pod fully PSA ready, most things were already there.
E1023 15:51:16.640185       1 util.go:72] pods "virt-export-vme-test-whttslbd8wvd" is forbidden: violates PodSecurity "restricted:v1.24": allowPrivilegeEscalation != false (container "vme-test-whttslbd8wvd" must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (container "vme-test-whttslbd8wvd" must set securityContext.capabilities.drop=["ALL"]), seccompProfile (pod or container "vme-test-whttslbd8wvd" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost")

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #
https://bugzilla.redhat.com/show_bug.cgi?id=2137123
Special notes for your reviewer:
Release note:

BugFix: Exporter pod does not comply with restricted PSA

/cc @awels @mhenriks

/retest

/retest

/retest

/retest

/retest

/retest

/retest

/test pull-kubevirt-e2e-k8s-1.22-operator

/retest

/retest

/cherrypick release-0.58


XSPEC interface failure running in Jupyter Notebook

When an XSPEC fit is run from a Jupyter Notebook, something funny goes on with the creation of the summary fits file. The actual XSPEC run and fit complete fine, but the script fails when it comes to write out the results.
Errors like this get thrown:
Unable to redirect prompts to the /dev/tty (at headas_stdio.c:152)
ERROR: No such device or address
Task ftcreate 0.0 terminating with status 6
The same fits work fine when run outside of Jupyter Notebook

I think I'm going to have to not use the fits creation command line tools supplied by HEASOFT, as I can't figure out why they're causing these errors with Jupyter Notebooks.
Its easily sidestepped, but entails an annoying amount of work re-rewriting how XGA reads in the fit results, and will make the fit directories much messier.
Possible that I will get XGA to read in the text files I'll have to make, then combine them into a fits and write that out. That way I won't have to change the way XGA looks for and loads in past fit results at all.


Create page on globaldevopsx.com that allows users to start the event

When the event date is there; show this page instead.
We first show that the event is started, so people need to log in with their GH account.
We then show name of venue the user is registered to and allow them to select a team. If already in a team, allow them to leave the team (in order to start over).
After selecting a team, we show an info/help/uitleg page.
We then go to the sharepoint portal to actually start the event.

Done


v0.3.2 backportapalooza

This bundles together all the relevant bug fixes, documentation changes, and refactors/tests that make sense for a patch release. Notably it includes all of these PRs

#91
#103
#106
#116
#109
#122
#132
#138
#147
#145
#150
#155
#154
#156
#166
#169
#171
#172

TODO: This has that zbus issue that causes color theme detection to fail on Linux. Check to see if that's fixed when updating deps Looks like the existing issue should be fixed and I'm now hitting https://bugzilla.redhat.com/show_bug.cgi?id=2239248 :upside_down_face:
TODO: Need to follow through all the regular release steps. Bump version, update deps, draft up a more coherent description of the changes, etc.

[x] Check for unused dependencies

$ cargo +nightly udeps

[x] Update rust-version in Cargo.toml

$ cargo msrv --min 1.60 -- cargo check

[x] Bump version in Cargo.toml
[x] Run cargo check to propagate changes to Cargo.lock
[ ] Merge changes through a PR or directly to make sure CI passes
[ ] Publish on crates.io

$ cargo publish

[ ] Publish on GitHub by pushing a version tag

$ git tag v{VERSION} (make sure the branch you are on is up to date)
$ git push upstream/origin v{VERSION}

[ ] Make a release announcement on GitHub after the release workflow finishes

Whelp... this will have to be on hold until I'm no longer sick :face_with_thermometer:

@AlphaKeks whenever you have time (no rush at all) could you check to see if this branch still fixes that panic you were experiencing that you recently fixed on main? I just want to make sure things are working before cutting the release

I can confirm that, with nix develop nixpkgs#inlyne followed by cargo build and patching the resulting binary, it does start.
What's weird is that nix run nixpkgs#inlyne works perfectly fine...
I can't tell why, or since when. The upstream package hasn't changed, so I assume this is something specific to my system, but I can't tell you what. It's weird because compiling inlyne manually requires this patchelf dance (and before I opened the original issue it also did with the upstream package!), but the upstream package works (now)...?
I'm honestly confused, but whatever it is, it's not inlyne's issue.

Well happy to hear that things are working at least!


Add async segment file download support from remote store within OpenSearch core

Description

This PR builds on top of #9592 and utilizes the new APIs within core
The APIs will be used within the RemoteSegmentStoreDirectory path to copy over segment files from the remote store.
Logically, the caller will not see any difference within the workings of download segments but the files will be parallelized and each file will support multiple streams for download due to the async nature of the APIs.

Related Issues
Resolves #8596

Check List

[x] New functionality includes testing.

[x] All tests pass

[x] New functionality has been documented.

[x] New functionality has javadoc added

[x] Commits are signed per the DCO using --signoff
[x] Commit changes are listed out in CHANGELOG.md file (See: Changelog)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/25026/
CommitID: 77ab43e
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

Git clone: https://github.com/kotwanikunal/OpenSearch.git with ref: 77ab43e4b5bbb0de42abde82c59e80ebe341cd88
rm: cannot remove 'search/qa/evil-tests/build': Directory not empty
rm: cannot remove 'search/server/build/testrun/test/temp/org.opensearch.index.shard.RemoteStoreRefreshListenerTests_7143E3A5038C66ED-001/tempDir-024/indices/_na_/0/translog': Directory not empty

https://build.ci.opensearch.org/job/gradle-check/25041/

Looks like Jenkins crashed again

java.lang.AssertionError: Failure at [snapshot.restore/10_basic:44]: expected [2xx] status code but api [snapshot.restore] returned [500 Internal Server Error] [{"error":{"root_cause":[{"type":"illegal_state_exception","reason":"Message fully read (request) but more data is expected for requestId [5436], action [cluster:admin/snapshot/restore]; resetting","stack_trace":"OpenSearchException[Message fully read (request) but more data is expected for requestId [5436], action [cluster:admin/snapshot/restore]; resetting]; nested: IllegalStateException[Message fully read (request) but more data is expected for requestId [5436], action [cluster:admin/snapshot/restore]; resetting]; nested: EOFException;\n\tat org.opensearch.OpenSearchException.guessRootCauses(OpenSearchException.java:708)\n\tat org.opensearch.OpenSearchException.generateFailureXContent(OpenSearchException.java:635)\n\tat org.opensearch.rest.BytesRestResponse.build(BytesRestResponse.java:172)\n\tat org.opensearch.rest.BytesRestResponse.<init>(BytesRestResponse.java:132)\n\tat org.opensearch.rest.BytesRestResponse.<init>(BytesRestResponse.java:109)\n\tat org.opensearch.rest.action.RestActionListener.from(RestActionListener.java:74)\n\tat org.opensearch.rest.action.RestActionListener.onFailure(RestActionListener.java:88)\n\tat org.opensearch.action.support.TransportAction$1.onFailure(TransportAction.java:122)\n\tat org.opensearch.action.support.RetryableAction$RetryingListener.onFinalFailure(RetryableAction.java:218)\n\tat

Test succeeds locally:
 Task :qa:mixed-cluster:v2.11.0#mixedClusterTest
Test cluster endpoints are: [::1]:33755,<IP_ADDRESS>:36025,[::1]:34745,<IP_ADDRESS>:43803,[::1]:35709,<IP_ADDRESS>:40707,[::1]:36683,<IP_ADDRESS>:45261
Upgrading one node to create a mixed cluster
Upgrade complete, endpoints are: [::1]:40973,<IP_ADDRESS>:36883,[::1]:34745,<IP_ADDRESS>:43803,[::1]:35709,<IP_ADDRESS>:40707,[::1]:36683,<IP_ADDRESS>:45261
Upgrading another node to create a mixed cluster
Upgrading complete, endpoints are: [::1]:40973,<IP_ADDRESS>:36883,[::1]:40459,<IP_ADDRESS>:35779,[::1]:35709,<IP_ADDRESS>:40707,[::1]:36683,<IP_ADDRESS>:45261

 [2.11.0] WARNING: A terminally deprecated method in java.lang.System has been called
WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.BootstrapForTesting (file:/local/home/kkotwani/workspace/OpenSearch/test/framework/build/distributions/framework-3.0.0-SNAPSHOT.jar)
WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.BootstrapForTesting
WARNING: System::setSecurityManager will be removed in a future release
WARNING: A terminally deprecated method in java.lang.System has been called
WARNING: System::setSecurityManager has been called by org.gradle.api.internal.tasks.testing.worker.TestWorker (file:/local/home/kkotwani/.gradle/wrapper/dists/gradle-8.3-all/6en3ugtfdg5xnpx44z4qbwgas/gradle-8.3/lib/plugins/gradle-testing-base-8.3.jar)
WARNING: Please consider reporting this to the maintainers of org.gradle.api.internal.tasks.testing.worker.TestWorker
WARNING: System::setSecurityManager will be removed in a future release

BUILD SUCCESSFUL in 5m 4s
203 actionable tasks: 203 executed

Looks like i was missing the commit https://github.com/opensearch-project/OpenSearch/commit/4ccecd6958e02287549bfcfd30bcc4088d18bf4b on my branch. Rebased.

#9869

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/25086/
CommitID: c59213f
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

https://build.ci.opensearch.org/job/gradle-check/25153/

#9787

I am more concerned about which threadpool does the await.

@Bukhtawar, here is my understanding of how the threading works. IndexShard.copySegmentFiles() is a synchronous API as currently implemented, so it blocks until all files are downloaded. I've attempted to diagram the flow (note some of the functionality is repository-specific, so I'm describing the S3 implementation here):
-> (calling thread) IndexShard.copySegmentFiles()
-> (calling thread) For each segment file:
  -> (calling thread) RemoteSegmentStoreDirectory.copyTo()
  -> (calling thread) AsyncMultiStreamBlobContainer.asyncBlobDownload()
  -> (calling thread) AsyncMultiStreamBlobContainer.readBlobAsync()
  -> (calling thread) S3.GetObjectAttributesRequest (blocks on result)
    -> (s3 async thread) for each part: S3.GetObject (_starts_ streaming the object part)
    -> (s3 async thread) upon completion of the last part, trigger ReadContextListener.onResponse
    -> (GENERIC thread) for each input stream: drain InputStream to file (FilePartWriter.run())
<- (calling thread) blocks until all files are completely downloaded

I'll try to describe this in prose as well: Starting from IndexShard.copySegmentFiles(), for each file the calling thread does an initial blocking call to get the object metadata, but then returns immediately while registering listeners to do the rest of the work. IndexShard.copySegmentFiles() blocks because it is synchronous and expects the files to be downloaded when it returns. As for the multi-threading work, the initial call to get the S3 InputStreams is done by the s3AsyncClient on a repository-specific thread pool. Once those futures are complete (which means that streaming is just beginning for each part), the work to drain the streams into a file is done on the generic thread pool. Meanwhile, the original calling thread is blocking until all files are completely downloaded.
I think the initial call must block unless we do a more major refactoring to make the code flows that call it asynchronous. Let me know if you have any concerns here.
I have a couple comments/concerns here:

That blocking call to get object metadata is probably not ideal. Can we register a whenComplete handler on the CompletableFuture so that we don't sequentially block when starting all the individual file downloads?
I'm not sure that the handoff to the generic thread pool is the right thing to do. In theory we could chain the logic to drain the streams to a file onto the s3 async thread that did the initial GetObject call and not involve another thread pool at all.

/cc @kotwanikunal @vikasvb90

I'm not sure that the handoff to the generic thread pool is the right thing to do. In theory we could chain the logic to drain the streams to a file onto the s3 async thread that did the initial GetObject call and not involve another thread pool at all.

We should create another threadpool with much bigger size purely to do I/O . I think we are already tracking it in https://github.com/opensearch-project/OpenSearch/issues/10106 .
Even for GetObjectAttributesRequest, we don't need to a Get and use CompleteableFutures here.

Refactor async blob read to avoid blocking calls, support non multipa… #10192

Updated the S3 reference here: https://github.com/opensearch-project/OpenSearch/pull/10192

@andrross Handoff to generic pool is definitely not right and we can also make metadata call non-blocking by chaining it with subsequent downloads. There is a way by which downloads can be made completely async and we wouldn't need any threadpool for multiple parallel downloads but that requires significant changes in stream processing layer (decryption/data integrity).
To add more on this, there are two types of IO happening in downloads - fetch from S3 and disk writes. Fetch from S3 is async even in the current state but we are not able to truly benefit from it because each part download happens within a separate thread. And this is done because we need some pre-processing work like decryption to be done before committing a buffer to disk.
We can make both disk writes and fetch from S3 async and still be able to do pre-processing work like decryption by implementing AsyncResponseTransfer similar to what FileAsyncResponseTransformer does but this requires decent amount of changes across s3 and decryption layer. What is lacking is a clean communication between these two layers which can be easily addressed in golang via channels.
We can still take up this as a follow up task thought and as @gbbafna mentioned, since we will only be doing IO in this new pool, for now we can assign it a large size. This would mean that we would still continue to bear context switch cost and minimal CPU provided stream processing work is further bounded in a small pool. I would still very much like to see downloads happening in async given we are making certain compromises.


Graduate popup to Open UI

Open UI CG took up <popup> as an incubation work item, with the intent of eventually filing PRs against the HTML Living Standard in WHATWG. This PR graduates the explainer, and issue porting will follow.

Thanks Travis!


fix invalid unsigned arithmetic.

your check is incorrect. since the variables are unsigned, it is i != j.
so I suggest a simple fix for the error.

j is signed. Why do you think this change is needed nevertheless?

j is signed.
yes. but https://wiki.sei.cmu.edu/confluence/display/c/INT02-C.+Understand+integer+conversion+rules

and
sizeArray = 60; (sizeArray - 10) = 50; true
sizeArray = 10; (sizeArray - 10) = 0; false
sizeArray = 1; (sizeArray - 10) =<PHONE_NUMBER>3709551607; true
sizeArray = 0; (sizeArray - 10) =<PHONE_NUMBER>3709551606; true
if (sizeArray - 10 > 0)

Better question and a fix - why is i unsigned (size_t)?

size_t i = numchars() https://github.com/tesseract-ocr/tesseract/blob/2ba69ae888ca0bdfd9b431163509c9740599f8e8/src/textord/cjkpitch.cpp#L700
numchars() returns chars_.size() https://github.com/tesseract-ocr/tesseract/blob/2ba69ae888ca0bdfd9b431163509c9740599f8e8/src/textord/cjkpitch.cpp#L336
genericvector.size() returns int https://github.com/tesseract-ocr/tesseract/blob/2ba69ae888ca0bdfd9b431163509c9740599f8e8/include/tesseract/genericvector.h#L71

Having both i and j as ints should not create a problem in i - j > 0 expr.

I agree this is also a valid way to fix the problem.

Having estimated the complexity, I again propose to accept my PR as an intermediate solution. and in the subsequent preparation of more global changes with type replacement.

I don't think that using int again like it was used in earlier versions goes into the right direction. I'd like to use std::vector or other standard types instead of GenericVector, and standard types use size_t for size() methods.

I don't think that using int again like it was used in earlier versions goes into the right direction. I'd like to use std::vector or other standard types instead of GenericVector, and standard types use size_t for size() methods.

Is it possible to remove generic vector completely?

Is it possible to remove generic vector completely?

Yes, I think so. STRING is the other type which I would like to replace completely.


I feel like I tried this before for something else.  Perhaps have it autoconverted? https://stackoverflow.com/questions/10718767/have-the-same-readme-both-in-markdown-and-restructuredtext

fixed in https://github.com/CityOfZion/neo-python/pull/317


[enhancement]: upgrade to lockfileVersion 2

Task name
No response
Describe your feature request here
Currently all the tasks use lockfileVersion 1 in their package-lock.json (except for NuGetAuthenticate@1, which uses lockfileVersion 2).
lockfileVersion 1 was introduced with npm v5^1, and lockfileVersion 2 was introduced with npm v7^2. Changes to lockfileVersion 2 allow for faster installs, while still being backwards compatible with lockfileVersion 1 and npm v6 (which shipped with Node v10^3).

Hi @JamieMagee  thanks for reporting! We are working on more prioritized issues at the moment, but will get back to this one soon.

This is not completed.


Incorrect bounding box for visfauna Yser

All occurrence records seem to fall outside the bounding box defined in the metadata: http://www.gbif.org/dataset/0ac24b3c-feb9-48d5-bf02-da4a103f024e. One of the two is incorrect, probably the metadata. This Yser dataset has a correct bounding box: best just copy that one?

Corrected in metadata and republished:


Update binder link in open_data module

Closes #179.
FIX the binder link

@htwangtw I've updated the instructions. Do you want to take a quick look at them ? The rest of the module is LGTM

yes all looks good. do you want to click the review button?


Use cgImage.bytesPerRow instead of width

Because the bitmap data allocated may be padded.

@YuAo can you provide more information? What do you mean by the data allocated may be padded?

When an image is stored in memory, the memory buffer may contain extra padding bytes after each row of pixels. bytesPerRow is number of bytes allocated for a single row of an image.
For example:
let cgContext: CGContext = CGContext(data: nil, width: 100, height: 100, bitsPerComponent: 8, bytesPerRow: 0, space: CGColorSpaceCreateDeviceRGB(), bitmapInfo: CGBitmapInfo.byteOrder32Little.rawValue | CGImageAlphaInfo.premultipliedFirst.rawValue)!
let image: CGImage = cgContext.makeImage()!
image.bytesPerRow //448
image.width * 4 //400

This image object here has the bytesPerRow of 448. The index of pixel byte at (x,y) should be y * bytesPerRow + x * 4 instead of ((width * y) + x) * 4.

Yup, that makes sense. I think this looks good. Can you also bump the version from 2.2.0 to 2.3.0?

Can you also bump the version from 2.2.0 to 2.3.0?

Done.

Ah, I should've specified the locations but we'll also need to update the version here: https://github.com/jathu/UIImageColors/blob/8b375489ae1e29fbc26a10ce2a584ba8038208ac/UIImageColors/Supporting Files/Info.plist#L18


Run linter independet of locally installed linter

Copied from cluster-service makefile

@ciaranRoche @ydayagi @pvasant @oriAdler @tzvatot PTAL

LGTM

looks like prow doesn't have podman installed. Also, we don't include any modules from internal gitlab so that flag can be removed

@zgalor can we close this?


Add tutorial to main

To make the tutorial materials more accessible, let's add them to main. There should be no functional changes to the code, except adding stream to benchmarks/examples and I think it's an useful example.
@ilectra please check that the post-processing files merge ok

I'm done with adding/checking the postprocessing stuff. Once we update the notes to final version, feel free to merge.
I'll run the whole tutorial on archer2 tomorrow, just in case...

@tkoskela I'm done updating the files needed for the post-processing part of the tutorial, as well as the tutorial.md in your hackmd. AFAIK, you can merge this once you bring the newest tutorial.md over.


odyssey-x86.coffee: Change flasher image to live image

Changelog-entry: Change odyssey-x86 flasher image to live image
Signed-off-by: Lakshantha Dissanayake<EMAIL_ADDRESS>

Can one of the admins verify this patch?

@resin-jenkins add to whitelist

@floion Thank you for the PR.
Can I know after how long the new image with the changes will appear on https://dashboard.balena-staging.com ?

Hello, I have a problem. I have several Odysseys and most of them are solder emmc. I can't flash the emmc directly, is there a solution to bypass this pr and use the classic operation to flash a usb key and install it in the emmc on the Odyssey?

Hello,
Could you please help to compile from source, use the "flasher image" and test it?
https://wiki.seeedstudio.com/BalenaOS-X86-Getting-Started/#q3-how-can-i-compile-balenaos-from-source-code
Please let me know your findings.


Update: Profiles (v1.1.5)

Post UsernameChanged events to make the plugin work with the farming timers and Bitwarden plugin. @Spedwards can take back the plugin when he finds time for development.

Since this plugin deals with credentials I don't want to transfer it without @Spedwards permission. As he hasn't given that for over a month I am closing this as stale.


Config example needs to work out-of-the-box

Issue: Example config does not run without error
For best user experience, the config.json.example should work out of the box without error
Including:

notifications
probe target
alerts
intervals

update notifications documents also, according to the new config.json.example

I think the only way an example config works out of the box is if the config only contains webhook notification which points to a server we maintain.
An idea to make Monika works almost out of the box is to have multiple config examples where each of the example only contains single notification:
config.smtp.gmail.example.json: contains an smtp notification configuration as follows
  "notifications": [
    {
      "id": "unique-id-smtp",
      "type": "smtp",
      "data": {
        "recipients": ["ENTER_YOUR_EMAIL_ADDRESS_HERE"],
        "hostname": "smtp.gmail.com",
        "port": 587,
        "username": "ENTER_YOUR_GMAIL_ADDRESS_HERE",
        "password": "ENTER_YOUR_GMAIL_PASSWORD_OR_APP_PASSWORD_IF_2FA_ACTIVE"
      }
    }
  ]

config.sendgrid.example.json: contains a sendgrid notification configuration as follows
{
       "id": "unique-id",
       "type": "sendgrid",
       "data": {
         "recipients": ["ENTER_YOUR_EMAIL_ADDRESS_HERE"],
         "apiKey": "GET_API_KEY_FROM_SENDGRID"
       }
     }

And so on.


Convert MasterLayout to controllable component

hybrid component: uncontrolled by default, but optionally controllable

I think that alone won't make the MasterLayout fully controllable, as CollapsibleItems also have a local state: https://github.com/vivid-planet/comet-admin/blob/master/packages/admin/src/mui/menu/CollapsibleItem.tsx#L41


Could not locate a DataRegistration for class class me.mrdaniel.adventuremmo.data.manipulators.MMOData

Hi,
Brand new vanilla sponge server being run through msm.
Crashes seem random yet frequent. Particularly when someone logs out. 3 Example crash logs attached to show frequency and details.
Description: Exception in server tick loop
org.spongepowered.api.data.DataRegistrationNotFoundException: Could not locate a DataRegistration for class class me.mrdaniel.adventuremmo.data.manipulators.MMOData
at org.spongepowered.common.data.SpongeManipulatorRegistry.getRegistrationFor(SpongeManipulatorRegistry.java:149)
at org.spongepowered.common.data.util.DataUtil.getRegistrationFor(DataUtil.java:429)
at org.spongepowered.common.data.util.DataUtil.getSerializedManipulatorList(DataUtil.java:132)
at net.minecraft.entity.Entity.writeToNbt(SourceFile:5044)
at net.minecraft.entity.EntityLivingBase.writeToNbt(SourceFile:2547)
at net.minecraft.entity.Entity.handler$onWriteToNBT$zci000(SourceFile:4948)
at net.minecraft.entity.Entity.func_189511_e(SourceFile)
at net.minecraft.server.management.PlayerList.handler$onPlayerLogOut$zhg000(SourceFile:1770)
at net.minecraft.server.management.PlayerList.func_72367_e(SourceFile)
at net.minecraft.network.NetHandlerPlayServer.func_147231_a(SourceFile:735)
at net.minecraft.network.NetworkManager.func_179293_l(SourceFile:382)
at net.minecraft.network.NetworkSystem.func_151269_c(SourceFile:215)
at net.minecraft.server.MinecraftServer.func_71190_q(SourceFile:1603)
at net.minecraft.server.dedicated.DedicatedServer.func_71190_q(SourceFile:335)
at net.minecraft.server.MinecraftServer.func_71217_p(SourceFile:562)
at net.minecraft.server.MinecraftServer.run(SourceFile:466)
at java.lang.Thread.run(Thread.java:748)
crash-2017-06-06_08.15.14-server.txt
crash-2017-06-06_13.17.26-server.txt
crash-2017-06-06_14.12.51-server.txt

Upgrading sponge vanilla to the latest beta seems to have fixed the problem.
No crashes anyhow which was my issue.


fix(Context-Based Restrictions): fix output to Operations

PR summary
Add on to previous work:
https://github.com/IBM/platform-services-go-sdk/pull/277
The output is improper due to generation from an older version of the swagger. The change should be very minor and will be propagated to the other languages' PRs.
PR Checklist
Please make sure that your PR fulfills the following requirements:

[X] The commit message follows the Angular Commit Message Guidelines.
[X] Tests for the changes have been added (for bug fixes / features)
[X] Docs have been added / updated (for bug fixes / features)

Current vs new behavior
Adds the "Type" output
Does this PR introduce a breaking change?

[ ] Yes
[X] No

Test screenshots:

Waiting on a review from someone on my side as well

Test Screenshot:

Examples Screenshot:

@padamstx should be good to merge whenever you're ready!

:tada: This PR is included in version 0.50.1 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Update claims.md

Greetings! Recently I found that Microsoft Docs has just a few translations in  ukrainian language. It's my native language so I decided to contribute a bit in the translations to make learning materials more availible

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.:x: hinterfa851 sign nowYou have signed the CLA already but the status is still pending? Let us recheck it.

Hello @hinterfa851 ... The documents here must remain in English for automatic and human translation systems to work properly.
It looks like from prior issues that they've decided not to translate into Ukrainian ...
https://github.com/MicrosoftDocs/feedback/issues?q=is%3Aissue+ukrainian+is%3Aclosed


Use both volume_name and description when applicable

See discussion on #89

Awesome, that was fast :)

I published 5.0.21 with the fix!

Fantastic, thanks for the quick turnaround!


Disable gateway CRDs

Welcome!

[x] Yes, I've searched similar issues on GitHub and didn't find any.
[x] Yes, I've searched similar issues on the Traefik community forum and didn't find any.

What did you expect to see?
Hi all,
is there any way to disable the following CRDs via values.yaml?

gatewayclasses.gateway.networking.k8s.io
gateways.gateway.networking.k8s.io
grpcroutes.gateway.networking.k8s.io
httproutes.gateway.networking.k8s.io
referencegrants.gateway.networking.k8s.io

The problem is that I see the following warning message in argocd and the gateway CRDs are not being used.
RepeatedResourceWarning
Resource apiextensions.k8s.io/CustomResourceDefinition//gatewayclasses.gateway.networking.k8s.io appeared 2 times among application resources.
an hour ago (Thu Dec 12 2024 10:43:29 GMT+0100)
RepeatedResourceWarning
Resource apiextensions.k8s.io/CustomResourceDefinition//gateways.gateway.networking.k8s.io appeared 2 times among application resources.
5 minutes ago (Thu Dec 12 2024 11:23:38 GMT+0100)
RepeatedResourceWarning
Resource apiextensions.k8s.io/CustomResourceDefinition//grpcroutes.gateway.networking.k8s.io appeared 2 times among application resources.
5 minutes ago (Thu Dec 12 2024 11:23:38 GMT+0100)
RepeatedResourceWarning
Resource apiextensions.k8s.io/CustomResourceDefinition//httproutes.gateway.networking.k8s.io appeared 2 times among application resources.
5 minutes ago (Thu Dec 12 2024 11:23:38 GMT+0100)
RepeatedResourceWarning
Resource apiextensions.k8s.io/CustomResourceDefinition//referencegrants.gateway.networking.k8s.io appeared 2 times among application resources.

This is actually an error when using kustomize. The gateway CRDs produce this error message:
Error: could not parse rnode slice into resource map: may not add resource with an already registered id: CustomResourceDefinition.v1.apiextensions.k8s.io/gatewayclasses.gateway.networking.k8s.io.[noNs]

We are currently unable to update Traefik...

I am not using kustomize but argocd and helm and the argocd app shows these warnings.


[cronjob] executing cronjob issue

Hi Jack
Thanks for the script, we do not have any issues running the script manually from the command line using this:
sudo /opt/google-compute-snapshot/gcloud-snapshot.sh
However, when we try to set the cronjob, it just will not execute
We are using Ubuntu 18.
To test we have set the cron to run every 10 min, but we do not see any results.
We are using
0 5 * * * root /opt/google-compute-snapshot/gcloud-snapshot.sh > /dev/null 2>&1
Editing the crontab located here
nano /etc/crontab
Also tried editing crontab -e but this has no effect.
Would you by any chance have an idea if we are doing something wrong?
Thanks
Matthew

Hi Matthew,
Have you reloaded your cron service? ($  sudo service cron reload)
Also, the example you provided runs at 5am - does your test one (that runs every 10 mins) look like this:
*/10 * * * * root /opt/google-compute-snapshot/gcloud-snapshot.sh > /dev/null 2>&1

You can also check for crontab errors here: /var/log/syslog
Thanks,
Jack

Hmmmm - okay, I'd try saving the output of the log to a file: https://github.com/jacksegal/google-compute-snapshot#saving-output-to-log
It only takes about 5 minutes to setup and then you'll be able to see the output from the script.
Good luck!

@jacksegal
so I have added the log which outputs this info
Listed 0 items. [2018-11-22 16:15:45]: End of google-compute-snapshot [2018-11-22 16:20:01]: Start of google-compute-snapshot /opt/google-compute-snapshot/gcloud-snapshot.sh: line 157: gcloud: command not found tput: No value for $TERM and no -T specified tput: No value for $TERM and no -T specified tput: No value for $TERM and no -T specified tput: No value for $TERM and no -T specified [ERROR]: No disks were found - please check your script options / account permissions.
I had a look through the code and do not understand why the instance is not being picked up.
Any ideas.
Matthew

The first error: gcloud: command not found suggests that CRON can't access gcloud.  Did you install gcloud manually on the server?
I had a quick google and found these on Stackoverflow:

https://stackoverflow.com/a/53416714/5953656
https://stackoverflow.com/q/31037279/5953656

Hope it helps!

Thanks @jacksegal


Persistence diagram and wasserstein distance using ripser vs torch_topological

Hello,

I have used ripser and torch_topological libraries to compute cubical persistence of the same image but the two two diagrams are different.
About the wasserstein distance, I have tried to input same persistence diagram to persim.wasserstein and torch_topological.nn.wasserstein distance with same setting but the answers are different.

Can you please guide me what is the reason behind this? Am I doing something wrong? Is there a way to resolve this issue?
Ripser/Persim
"diagram1 = np.array([[0., 1.]])
diagram2 = np.array([[0., 1.],
[0., 1.]])
distance = persim.wasserstein(diagram2, diagram1)
output: Wasserstein distance: 0.7071067811865476"
with torch_topological
I got "tensor(0.5000)"
Thanks in advance.

Please provide more information about the code you are using in torch_topological.


Remove IREE_ERROR_ON_MISSING_SUBMODULES for better integeration

Request description
When integrating only iree runtime in a project, IREE needs to be built with IREE_ERROR_ON_MISSING_SUBMODULES, otherwise it complains about not having submodules not required for runtime (llvm-project being a heavy one, which isnt needed for runtime). Example: https://github.com/nod-ai/sharktank/blob/c15b751327d0460fbdc14f257d195f32b64d72d2/libshortfin/CMakeLists.txt#L151
A good to have feature would be to get rid of this flag, and instead check a list of submodules required to build the current target (compiler/runtime).
What component(s) does this issue relate to?
CMake

FWIW, I haven't run into that error when using runtime-only submodules and -DIREE_BUILD_COMPILER=OFF in multiple projects.
You can either use https://github.com/iree-org/iree/blob/main/build_tools/scripts/git/update_runtime_submodules.sh or an explicit command like what is recommended at https://github.com/iree-org/iree-template-runtime-cmake?tab=readme-ov-file#cloning-the-repository to initialize just the runtime submodules, skipping LLVM, stablehlo, and torch-mlir:
git \
    -c submodule."third_party/llvm-project".update=none \
    -c submodule."third_party/stablehlo".update=none \
    -c submodule."third_party/torch-mlir".update=none \
    submodule update --init --recursive

The CMake code and check script look for the compiler being disabled here:
https://github.com/iree-org/iree/blob/60843ecad5fe49301e036a7c2690378f974ab07f/CMakeLists.txt#L666-L673 https://github.com/iree-org/iree/blob/60843ecad5fe49301e036a7c2690378f974ab07f/build_tools/scripts/git/check_submodule_init.py#L39-L54
Ah, your list of submodules is incomplete (pulling only what is needed for hip). The list at https://github.com/iree-org/iree/blob/main/build_tools/scripts/git/runtime_submodules.txt doesn't match your code:
  FetchContent_Declare(
    iree
    GIT_REPOSITORY https://github.com/iree-org/iree.git
    GIT_TAG candidate-20240904.1006
    # TODO: We shouldn't have to pull googletest when we are not building tests.
    #       This needs to be fixed with IREE.
    GIT_SUBMODULES "third_party/benchmark third_party/cpuinfo third_party/flatcc third_party/hip-build-deps third_party/googletest"
    GIT_SHALLOW TRUE
  )

+1 to Stella's comment from https://github.com/nod-ai/sharktank/pull/183#discussion_r1754370502

Side note: I would love to get rid of this flag and instead have a macro in iree that checks a submodule belt it is used. This flag has been a footgun for integration.

The existing check against the full list is too strict when you only want to enable certain components.


Update README.rst

Current coverage is 25.00%

Merging #1 into master will not affect coverage as of 70685e5

@@            master     #1   diff @@
=====================================
  Files            4      4
  Stmts          228    228
  Branches         0      0
  Methods          0      0
=====================================
  Hit             57     57
  Partial          0      0
  Missed         171    171

Review entire Coverage Diff as of 70685e5

Powered by Codecov. Updated on successful CI builds.


⚠ Bofrost.at has degraded performance

In 446c4df, Bofrost.at (https://www.bofrost.at) experienced degraded performance:

HTTP code: 200
Response time: 1280 ms

Resolved: Bofrost.at performance has improved in d8ca88a after 10 minutes.


Update troubleshooting-guide.md

Added troubleshooting section for subnet delegation errors.

@JackStromberg or @greg-lindsay

Can you review this PR?
IMPORTANT: When this content is ready to merge, you must add #sign-off in a comment or the approval may get overlooked.

@greg-lindsay - if this is still your article, can you update the author metadata value to the correct GitHub alias?
#label:"aq-pr-triaged"
@MicrosoftDocs/public-repo-pr-review-team

emailed assignees

#sign-off

Apparently the author information is not correct in this article. My github alias is not the same as my MSFT alias

We are going to work this issue with an internal PR. #please-close


Error caught: [NoMethodError] undefined method `symbolize_keys!'

Just when click on submit configuration

[----] I, [2016-02-24T12:45:37.402137 #22712:3f97bbeffac8]  INFO -- : Started POST "/ops/settings_update/server?button=save" for <IP_ADDRESS> at 2016-02-24 12:45:37 +0100
[----] I, [2016-02-24T12:45:37.438159 #22712:3f97bbeffac8]  INFO -- : Processing by OpsController#settings_update as JS
[----] I, [2016-02-24T12:45:37.438272 #22712:3f97bbeffac8]  INFO -- :   Parameters: {"button"=>"save", "id"=>"server"}
[----] F, [2016-02-24T12:45:37.535234 #22712:3f97bbeffac8] FATAL -- : Error caught: [NoMethodError] undefined method symbolize_keys!' for #<ActiveSupport::HashWithIndifferentAccess:0x007f2f21496d88> /home/Ladas/Projects/cfme/manageiq/lib/vmdb/configuration_encoder.rb:19:in block in symbolize!'
/home/Ladas/Projects/cfme/manageiq/lib/vmdb/configuration_encoder.rb:19:in each_key' /home/Ladas/Projects/cfme/manageiq/lib/vmdb/configuration_encoder.rb:19:in symbolize!'
/home/Ladas/Projects/cfme/manageiq/lib/vmdb/configuration_encoder.rb:34:in load' /home/Ladas/Projects/cfme/manageiq/lib/vmdb/configuration_encoder.rb:6:in validate!'
/home/Ladas/Projects/cfme/manageiq/lib/vmdb/config.rb:266:in save' /home/Ladas/Projects/cfme/manageiq/app/models/miq_server/configuration_management.rb:40:in set_config'
/home/Ladas/Projects/cfme/manageiq/app/controllers/ops_controller/settings/common.rb:403:in settings_update_save' /home/Ladas/Projects/cfme/manageiq/app/controllers/ops_controller/settings/common.rb:179:in settings_update'
/home/Ladas/.rvm/gems/ruby-2.2.3/bundler/gems/rails-22db455dbe9c/actionpack/lib/action_controller/metal/basic_implicit_render.rb:4:in send_action' /home/Ladas/.rvm/gems/ruby-2.2.3/bundler/gems/rails-22db455dbe9c/actionpack/lib/abstract_controller/base.rb:183:in process_action'
/home/Ladas/.rvm/gems/ruby-2.2.3/bundler/gems/rails-22db455dbe9c/actionpack/lib/action_controller/metal/rendering.rb:30:in process_action' /home/Ladas/.rvm/gems/ruby-2.2.3/bundler/gems/rails-22db455dbe9c/actionpack/lib/abstract_controller/callbacks.rb:20:in block in process_action'
/home/Ladas/.rvm/gems/ruby-2.2.3/bundler/gems/rails-22db455dbe9c/activesupport/lib/active_support/callbacks.rb:126:in call' /home/Ladas/.rvm/gems/ruby-2.2.3/bundler/gems/rails-22db455dbe9c/activesupport/lib/active_support/callbacks.rb:126:in call'
/home/Ladas/.rvm/gems/ruby-2.2.3/bundler/gems/rails-22db455dbe9c/activesupport/lib/active_support/callbacks.rb:506:in block (2 levels) in compile' /home/Ladas/.rvm/gems/ruby-2.2.3/bundler/gems/rails-22db455dbe9c/activesupport/lib/active_support/callbacks.rb:455:in call'
/home/Ladas/.rvm/gems/ruby-2.2.3/bundler/gems/rails-22db455dbe9c/activesupport/lib/active_support/callbacks.rb:455:in call' /home/Ladas/.rvm/gems/ruby-2.2.3/bundler/gems/rails-22db455dbe9c/activesupport/lib/active_support/callbacks.rb:101:in run_callbacks'
/home/Ladas/.rvm/gems/ruby-2.2.3/bundler/gems/rails-22db455dbe9c/activesupport/lib/active_support/callbacks.rb:750:in _run_process_action_callbacks' /home/Ladas/.rvm/gems/ruby-2.2.3/bundler/gems/rails-22db455dbe9c/activesupport/lib/active_support/callbacks.rb:90:in run_callbacks'
/home/Ladas/.rvm/gems/ruby-2.2.3/bundler/gems/rails-22db455dbe9c/actionpack/lib/abstract_controller/callbacks.rb:19:in process_action' /home/Ladas/.rvm/gems/ruby-2.2.3/bundler/gems/rails-22db455dbe9c/actionpack/lib/action_controller/metal/rescue.rb:27:in process_action'
/home/Ladas/.rvm/gems/ruby-2.2.3/bundler/gems/rails-22db455dbe9c/actionpack/lib/action_controller/metal/instrumentation.rb:31:in block in process_action' /home/Ladas/.rvm/gems/ruby-2.2.3/bundler/gems/rails-22db455dbe9c/activesupport/lib/active_support/notifications.rb:164:in block in instrument'
/home/Ladas/.rvm/gems/ruby-2.2.3/bundler/gems/rails-22db455dbe9c/activesupport/lib/active_support/notifications/instrumenter.rb:21:in instrument' /home/Ladas/.rvm/gems/ruby-2.2.3/bundler/gems/rails-22db455dbe9c/activesupport/lib/active_support/notifications.rb:164:in instrument'
/home/Ladas/.rvm/gems/ruby-2.2.3/bundler/gems/rails-22db455dbe9c/actionpack/lib/action_controller/metal/instrumentation.rb:29:in process_action' /home/Ladas/.rvm/gems/ruby-2.2.3/bundler/gems/rails-22db455dbe9c/actionpack/lib/action_controller/metal/params_wrapper.rb:248:in process_action'
/home/Ladas/.rvm/gems/ruby-2.2.3/bundler/gems/rails-22db455dbe9c/activerecord/lib/active_record/railties/controller_runtime.rb:18:in process_action' /home/Ladas/.rvm/gems/ruby-2.2.3/bundler/gems/rails-22db455dbe9c/actionpack/lib/abstract_controller/base.rb:128:in process'
/home/Ladas/.rvm/gems/ruby-2.2.3/bundler/gems/rails-22db455dbe9c/actionview/lib/action_view/rendering.rb:30:in process' /home/Ladas/.rvm/gems/ruby-2.2.3/bundler/gems/rails-22db455dbe9c/actionpack/lib/action_controller/metal.rb:190:in dispatch'
/home/Ladas/.rvm/gems/ruby-2.2.3/bundler/gems/rails-22db455dbe9c/actionpack/lib/action_controller/metal.rb:262:in dispatch' /home/Ladas/.rvm/gems/ruby-2.2.3/bundler/gems/rails-22db455dbe9c/actionpack/lib/action_dispatch/routing/route_set.rb:50:in dispatch'
/home/Ladas/.rvm/gems/ruby-2.2.3/bundler/gems/rails-22db455dbe9c/actionpack/lib/action_dispatch/routing/route_set.rb:32:in serve' /home/Ladas/.rvm/gems/ruby-2.2.3/bundler/gems/rails-22db455dbe9c/actionpack/lib/action_dispatch/journey/router.rb:39:in block in serve'
/home/Ladas/.rvm/gems/ruby-2.2.3/bundler/gems/rails-22db455dbe9c/actionpack/lib/action_dispatch/journey/router.rb:26:in each' /home/Ladas/.rvm/gems/ruby-2.2.3/bundler/gems/rails-22db455dbe9c/actionpack/lib/action_dispatch/journey/router.rb:26:in serve'
/home/Ladas/.rvm/gems/ruby-2.2.3/bundler/gems/rails-22db455dbe9c/actionpack/lib/action_dispatch/routing/route_set.rb:724:in call' /home/Ladas/.rvm/gems/ruby-2.2.3/gems/omniauth-1.3.1/lib/omniauth/strategy.rb:186:in call!'
/home/Ladas/.rvm/gems/ruby-2.2.3/gems/omniauth-1.3.1/lib/omniauth/strategy.rb:164:in call' /home/Ladas/.rvm/gems/ruby-2.2.3/gems/omniauth-1.3.1/lib/omniauth/builder.rb:63:in call'
/home/Ladas/.rvm/gems/ruby-2.2.3/bundler/gems/rails-22db455dbe9c/actionview/lib/action_view/digestor.rb:12:in call' /home/Ladas/.rvm/gems/ruby-2.2.3/gems/rack-2.0.0.alpha/lib/rack/etag.rb:25:in call'
/home/Ladas/.rvm/gems/ruby-2.2.3/gems/rack-2.0.0.alpha/lib/rack/conditional_get.rb:38:in call' /home/Ladas/.rvm/gems/ruby-2.2.3/gems/rack-2.0.0.alpha/lib/rack/head.rb:12:in call'
/home/Ladas/.rvm/gems/ruby-2.2.3/gems/rack-2.0.0.alpha/lib/rack/session/abstract/id.rb:220:in context' /home/Ladas/.rvm/gems/ruby-2.2.3/gems/rack-2.0.0.alpha/lib/rack/session/abstract/id.rb:214:in call'
/home/Ladas/.rvm/gems/ruby-2.2.3/bundler/gems/rails-22db455dbe9c/actionpack/lib/action_dispatch/middleware/cookies.rb:613:in `call'

@dclarizio ^ can you check it out?

@dclarizio damn, it's caused by new rails brought in here https://github.com/ManageIQ/manageiq/pull/6861

@Ladas
This is not related to rails 5, this code is https://github.com/ManageIQ/manageiq/blob/master/lib/vmdb/configuration_encoder.rb#L19 only works with Regular Hashes and doesn't work with HashWithIndifferentAccess. HashWithIndifferentAccess has never supported the symbolize_keys! that only works with regular hashes.
We have an embedded hash with the key 'api' that is being stored as HashWithIndifferentAccess when they should be 'Hash'. We would have to look at our UI code and see why it uses HashWithIndifferentAccess for 'api'

api => {"token_ttl"=>"10.minutes", "authentication_timeout"=>"30.seconds"}
comes in as ActiveSupport::HashWithIndifferentAccess

@mkanoor from my testing this doesn't occur on master, but occurs when I cherry-pick your commit, so I suppose something from there is causing that?

@Ladas Something is converting hashes to HashWithIndifferentAccess, one of the PR's that got merged from yesterday

The code in rails causing this was reverted, so we are green again :-)
https://github.com/rails/rails/commit/6216a092ccfe6422f113db906a52fe8ffdafdbe6


NGINX needs to prevent reverse proxy calls to clustercall.rpc

NGINX needs a new location to prevent calls to clustercall.rpc being reverse proxied to stroom.
Calls to clustercall.rpc will be direct, on port 8080. This port must not be exposed, but it is still the path and the port nginx reverse proxies to, so we need to block it specifically when going in on port 80, via NGINX.
This needs to be done in conjunction with https://github.com/gchq/stroom/issues/648

33756527b5a73699fdef1c87e572c53207d48862


Add CMakeLists.txt to streamvbyte

Currently, there is no option to install other than to
install the library, headers in a global scope. This doesn't play
nice with other build systems.
With this file a user can simply do:
mkdir build
cd build
cmake .. -DCMAKE_BUILD_TYPE=Release \
         -DCMAKE_INSTALL_PREFIX:PATH=/path/to/install
make -j8 install

In addition, I added tests/unit to the unit test framework of cmake
so that you can do ctest -V and you would get:
Test project /home/agallego/workspace/streamvbyte/build
Constructing a list of tests
Done constructing a list of tests
Updating test list for fixtures
Added 0 tests to meet fixture requirements
Checking test dependency graph...
Checking test dependency graph end
test 1
    Start 1: unit

1: Test command: /home/agallego/workspace/streamvbyte/build/unit
1: Test timeout computed to be: 9.99988e+06
1: Code looks good.
1: And you have a little endian architecture.
1: Warning: you tested non-vectorized code.
1/1 Test #1: unit .............................   Passed    0.02 sec

100% tests passed, 0 tests failed out of 1

Total Test time (real) =   0.02 sec

@lemire  also, let me know if you want me to update the readme as well.

One last thing!
 Can you please update the readme...
Note that, by design, it is safe to keep the makefile. So all that is needed is to copy your instructions in the readme. Don't hesitate to add your name to AUTHORS as well.

Sure, I just updated the readme, here is the rendering: https://github.com/senior7515/streamvbyte
Let me know if you need anything else for this patch.

@lemire  ping.

Great. Merging.


CPEs in package metadata for better vulnerability tracking

We should add a new standardized meta field called cpe that will contain a CPE (Common Platform Enumeration) string.
Why it's useful to have this
This enables tools like bombon that create SBOMs from arbitrary Nix derivations to insert this CPE for each package into the SBOM to enable (better) vulnerability tracking.
SBOMs that contain CPEs can be used to scan for vulnerabilities by tools like DependencyTrack, Grype, or nvdtools
This will also help Nixpkgs' upcoming security tracker.
Why we cannot automatically generate identifiers

sbomnix automatically generates CPEs and bombon automatically generates PURLs.
Automatically generating identifiers means guessing them (from pname, version, etc.) which is not sufficient for accurate vulnerability tracking.
Manually specifying them is the only way to achieve accurate enough results for useful tracking

Why CPEs over other identifiers

There are three main contender schemes: CPE, PURL, SWID
Compared to other identifiers (e.g. PURL or SWID), CPEs are canoncial. They are determined by the NVD (National Vulnerability Database) maintained by NIST. This makes it easy for contributors to manually determine the correct CPE (by searching the NVD) and add it to the metadata of a package.
CPEs do not depend on ecosystems (unlike PURL which requires a type field that identifies the package ecosystem, e.g. Debian or Fedora).
SWID seems to be hardly used and there is no CVE source that can actually be searched via SWIDs.

Implementation
I envision this to be an optional meta field. Something that maintainers who care about security can add but that does not have to be included. This field should be added to important packages over time by people that care about this information. I do not want this to be a hard requirement for new packages.
We could either populate this field directly by strings or add a library function (e.g. lib.mkCPE) that converts an attrset to the string representation.
Option 1:
meta = {
  cpe = "cpe:2.3:a:systemd_project:systemd:253:-:*:*:*:*:*:*";
};

Option 2:
meta = {
  cpe = lib.mkCPE {
    vendor = "systemd_project";
    product = pname;
    inherit version;
  };
};

Example CPE strings for systemd, glibc, openssl:
cpe:2.3:a:systemd_project:systemd:253:-:*:*:*:*:*:*
cpe:2.3:a:gnu:glibc:2.39:*:*:*:*:*:*:*
cpe:2.3:a:openssl:openssl:3.0.1:*:*:*:*:*:*:*

People that showed interest in this idea
@Mic92 @Lassulus @RaitoBezarius @arianvp @blitz

+1 on Option 2, -1 on Option 1.
I wonder if Option 2 should not be given more thought so that actually we have another data structure that is not called a CPE but something different and that synthesize a dumber CPE but in the future, may synthesize something richer for more advanced tools in the ecosystem.
paging @fricklerhandwerk who PM the security tracker.

Updated the Implementation section to include how I envision this meta field will be populated over time.
I view my proposal as very low risk and low ecosystem churn because this meta field does not have to be populated instantly but instead can be filled over time. Additionally it is absolutely fine if not all leaf packages contain CPE information. It's more important to get this info right for core packages (like glibc, systemd, openssl, etc.)
In general I favor iterative improvement over finding the best imaginable solution.

we have another data structure that is not called a CPE but something different and that synthesize a dumber CPE

I'm open to this but this still needs to somehow include weird CPE specific information (most notably their vendor field).
@RaitoBezarius do I understand you correctly that you want something like this:
meta = {
  sbom = {
    cpe = {
      vendor = "systemd_project"
      # Maybe the version or some other field needs to be manipulated for CPEs specifically
      # version = "v${version}"
    };
  };
};

And then a tool like bombon can synthesize a CPE from pname, version, and meta.sbom.cpe.vendor?

@RaitoBezarius do I understand you correctly that you want something like this:
meta = {
  sbom = {
    cpe = {
      vendor = "systemd_project"
      # Maybe the version or some other field needs to be manipulated for CPEs specifically
      # version = "v${version}"
    };
  };
};

More something like:
meta = {
    # names are tentative
    tracking-information.v1 = {
       vendor = "systemd_project";
       supportedSystems = platforms;
       featureSets = mesonFlagsToFeatureSets mesonFlags;
    };

on each drv
AND then, in a private detail meta API:
meta.cpe = lib.deriveCPE meta.tracking-information;

deriveCPE is private API that we control, tracking-information is a public API that we tell people to populate progressively.
v1: we list what we need to implement deriveCPE in the most useful way.

heavily appreciated!


iOS: library not found

Whenever I try to run the iOS app on a real device (or simulator, for that matter), I keep getting the "library not found" error for one of these 3 dependencies: AgoraRtcEnginerKit, AgoraRtcCryptoLoader or libcrypto.
I have tried unlinking and relinking them, but it hasn't helped.
It's only happening with Agora related dependencies, cam you guys help?

u can follow the documentation to install it.


Automated Package Upgrade

:tada: This PR is included in version 13.0.99 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


CON 3434 update cli schema

Update CLI schema
Files Changed
FLIR/conservator/generated/schema.py
api_version.txt
[Jira task]

This change is also included in PR 346


fix: Auto-complete/auto-suggest for Activity and Cards

closes #2330
Adding support of auto-complete/auto-suggest for Activity and Cards
Here is a demo:

this is great @cosmicshuai !

this is great @cosmicshuai !

Thanks. I think I can add a formatting command once user press enter to make sure the the ending ] is in the right place.


Un-deprecated command line short variant options of -c

Since 1.9, the following short variant options have been deprecated in favor of their long variants:
docker run -c (--cpu-shares)
docker build -c (--cpu-shares)
docker create -c (--cpu-shares)
docker update -c (--cpu-shares)
However, -c is still widely used and is considered as a convenient option for swarm (see #16271).
This fix undeprecated the command line short variant options of -c and updated the deprecated.md.
Signed-off-by: Yong Tang<EMAIL_ADDRESS>

LGTM, thanks!

LGTM (besides @thaJeztah's nit)

:( I don't think this is a good idea.

There was some discussion about this on the original PR https://github.com/docker/docker/pull/16271, so ping @vieux @abronan are there still objections against this?
If we decide to not remove these, we should un-deprecate them at least

@runcom @cpuguy83 @thaJeztah Thanks for the review. Let me know if we want to un-deprecate this option so that I could update the PR.

If @cpuguy83 doesn't like it, then probably we shouldn't do it :/

Had a quick chat with @cpuguy83 and he mentioned seeing a lot of people still using the shorthand -c option (despite the deprecation warning 😞). I guess we should take the sensible approach then, and un-deprecate the shorthand -c. It's only a minor cleanup, so
@yongtang can you update the PR to un-deprecate?

Thanks @thaJeztah. The pull request has been updated. Let me know if there are any issues.

LGTM, thanks @yongtang

LGTM

docs LGTM as well. Merging


Build Initialize command fails: "The OutputPath property is not set for project ..."

Steps to reproduce
No matter if I checkout release or dev branch when I attempt to build the solution using command "build initialize" I receive the same error.
The issue
Unable to build the solution
This is the end of the output of the command where the error is reported:
log  : Restore completed in 7447ms.
log  : Writing lock file to disk. Path: D:\PROYECTOS\EntityFramework\test\Microsoft.EntityFrameworkCore.Tools.Core.Tests\project.lock.json
log  : D:\PROYECTOS\EntityFramework\test\Microsoft.EntityFrameworkCore.Tools.Core.Tests\project.json
log  : Restore completed in 9482ms.
info: Run target: csproj-initialize
info: Exec
info:   program: C:\Program Files (x86)\MSBuild\14.0\Bin\MSBuild.exe
info:   commandline: src\Microsoft.EntityFrameworkCore\Microsoft.EntityFrameworkCore.csproj /t:GenerateProjectLockTargets /v:m /nologo /p:Configuration=Debug
info:   workingdir: D:\PROYECTOS\EntityFramework
C:\Program Files (x86)\MSBuild\14.0\bin\Microsoft.Common.CurrentVersion.targets(
724,5): error : The OutputPath property is not set for project 'Microsoft.Entity
FrameworkCore.csproj'.  Please check to make sure that you have specified a vali
d combination of Configuration and Platform for this project.  Configuration='De
bug'  Platform='x64'.  You may be seeing this message because you are trying to
build a project without a solution file, and have specified a non-default Config
uration or Platform that doesn't exist for this project. [D:\PROYECTOS\EntityFra
mework\src\Microsoft.EntityFrameworkCore\Microsoft.EntityFrameworkCore.csproj]
fail: Exit code 1 from C:\Program Files (x86)\MSBuild\14.0\Bin\MSBuild.exe
fail: Stack trace:
   en View351babc0efa143b89b401f954fd98949.<RenderViewLevel0>b__c7() en d:\Users\Sergio\AppData\Local\Temp\01f8281722ef4a7393f078b8bc031a9e-1.cs:línea 11060
   en Sake.Engine.Builder.BuilderBase`1.CallTarget(String name)
   en Sake.Engine.Builder.BuilderBase`1.StartingTarget(String name)
   en View351babc0efa143b89b401f954fd98949.StartingTarget(String name) en d:\Users\Sergio\AppData\Local\Temp\01f8281722ef4a7393f078b8bc031a9e-1.cs:línea 3603
   en View351babc0efa143b89b401f954fd98949.<RenderViewLevel0>b__64() en d:\Users\Sergio\AppData\Local\Temp\01f8281722ef4a7393f078b8bc031a9e-1.cs:línea 4059
   en Sake.Engine.Builder.BuilderBase`1.CallTarget(String name)
   en Sake.Engine.SakeEngine.Execute(Options options)
   en Sake.Engine.SakeEngine.Execute(String[] args)
   en Sake.Program.Main(String[] args)

Further technical details
EF Core version: no matter if I use dev or release branch
Operating system: Windows 10
Visual Studio version: VS 2015 Update 3 Version 14.0.25421.03

I suspect you have the PLATFORM environment variable set to x64 which is interfering with the build script. Try the following to see if it works.
SET PLATFORM=
build initialize

Triage: closing as we believe the question is answered. Feel free to reactivate if it isn't.

Yes it worked.
Thank you

@bricelam where do i have to make those changes?


Support for creating layers with ACI index or setting layer color index

https://forum.dynamobim.com/t/autocad-node-layer-create-color-doesnt-support-pen-tables/47772

Already solved in #31.


build(fee): define valid resources bounds enum

This change is

#506
#505
#504
#503  👈
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @nimrod-starkware and the rest of your teammates on  Graphite

Merge activity

Aug 25, 3:32 AM EDT: @nimrod-starkware started a stack merge that includes this pull request via Graphite.


[New] Edit geodatabase with transactions

Description
This PR implements Edit geodatabase with transactions in Edit and Manage Data category.
URL to README: README
Linked Issue(s)

swift/issues/5304

How To Test

Tap "Start" to start a transaction.
Tap the map to add features.
Tap "End" to commit or rollback the changes.
Tap "Requires Transaction" to apply edits directly.
Tap "Sync" to sync the geodatabase and feature service.

Screenshots

The sample currently downloads the geodatabase on each run. An alternative to that would be to store the geodatabase file in the documents folder, which is what the .NET implementation does. Let me know if you think that would be a better approach.

Thank you for your feedback!


feat: use .cjs extension for commonjs to support esm projects

Checklist

[ ] run make test
[ ] PR title must follow conventional-commit-standard
[ ] tests and/or benchmarks are included
[ ] documentation is changed or added
[ ] commit message and code follows the Code of conduct

Yep nextjs testapp works, default testapp works, vite testapp works (with both esm and commonjs).
Technically it doesn't change anything, just the file extension from .js to .cjs for all generated files in .wundergraph/generated.


New content to be added to the start page of the Create Measures journey

TP2000-256 New content to be added to the start page of the Create Measures journey
Why
This PR adds additional text to the create measures start page in order to inform a user of the information they need in order to create a measure.
This will prevent users from partially completing the form, only to find that they have to go back and lose all their work as they've missed an important piece of data, or didn't create something they needed for this process first.
Due to the nature of the tool not having drafts, or versioning for creating a measure, the next most appropriate measure is to ensure that the user is adequately prepared to complete this process in one sitting.
What
This PR adds text to the page to highlight what information is needed to create a measure. It goes from looking like this:

...To looking like this:

Looks good to me, it may be an idea to hold off until a fix for the test fail (not from this branch is ready)  ?

Codecov Report

Merging #537 (db12851) into master (0a3fe1d) will increase coverage by 0.02%.
The diff coverage is 95.36%.

@@            Coverage Diff             @@
##           master     #537      +/-   ##
==========================================
+ Coverage   92.62%   92.64%   +0.02%
==========================================
  Files         304      306       +2
  Lines       18769    18908     +139
  Branches     1493     1514      +21
==========================================
+ Hits        17384    17518     +134
- Misses       1107     1110       +3
- Partials      278      280       +2

Impacted Files
Coverage Δ

measures/tests/factories.py
92.30% <ø> (ø)

settings/common.py
77.24% <ø> (ø)

common/xml/util.py
89.23% <89.23%> (ø)

common/tests/test_xml_util.py
100.00% <100.00%> (ø)

common/tests/util.py
88.32% <100.00%> (+0.21%)
:arrow_up:

common/xml/namespaces.py
100.00% <100.00%> (ø)

conftest.py
95.54% <100.00%> (+0.09%)
:arrow_up:

exporter/apps.py
100.00% <100.00%> (ø)

importer/taric.py
80.26% <100.00%> (ø)

quotas/tests/test_xml.py
100.00% <100.00%> (ø)

... and 1 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5968b38...db12851. Read the comment docs.


Ability to set and reset default system prompt

Fixes #138, addresses part of https://github.com/tarasglek/chatcraft.org/issues/194#issuecomment-1661036662.
When we create a new chat, we always begin with a default system prompt.  The text of this system prompt is baked into the app's code.  We allow the user to edit it in the current message, but you can't use it again in a new chat.
This change allows the default system prompt to be overridden by a user (or reset back to the default).  The process is this:

User edits the current system prompt
In the 3-dots menu for the system prompt, a new set of options is shown when the system prompt has been changed (i.e., it is NOT shown if it's not needed):

Users can set the default system prompt to the current system prompt (i.e., override)
Users can reset the overridden system prompt back to the default (i.e., reset)

Here's a video showing how this works in practice (sorry it's small, GitHub won't let me post anything > 10M):
https://github.com/tarasglek/chatcraft.org/assets/427398/2491af38-6bca-4c89-b981-b7fbb69f9450
@steven4354 can you try this and comment?  NOTE: I'm not tackling the part about saving multiple custom prompts and choosing between them, mainly because I'm not sure how to do the UI.  I wonder if we should start with this and iterate?  Maybe you have ideas now.

For showing previous system prompts, I could easily query the db to get all previous prompts, maybe sorted by date used or something (i.e., I don't need to add any new storage for this to work).  But how do I show it?  A drop-down for something that fits in a textarea isn't really going to work (not enough context to pick them).

Just took a look at the video, that looks great and would solve the problem for me
A drop-down for something that fits in a textarea isn't really going to work (not enough context to pick them).
Not sure if I'm 100% on the same track (pls lmk if I am), but are you saying the drop-down of the prompt titles?
If so, I guess you would need a flow to "name" each system prompt as well right, maybe set a max length for the title and then only show in the dropdown the ones that they "named" + "saved"?
So "saving" and "naming" a system prompt is required to show in the dropdown

Thanks for the feedback.  Here's a concept to let the user choose from previous system prompts.  I'm borrowing UX from the way that AI Message versions work, giving the system message a dropdown in the heading:

Clicking that reveals your old system prompts, sorted by most recently used, and clicking one updates the current system prompt to use it:

There's a scrolling/height bug that this exposes, but it can be fixed in a follow-up.
I also realized that I need a way to set a different custom system prompt in my menu solution (i.e., if you already have a custom system prompt, but you want to use a different one, we need to somehow differentiate it).
Take a look at this and see if things make sense, or what you'd do differently.

I've pushed my updated code, if people want to try it out.  You might want to export/import your db from production so you can get the full effect (do that in the Settings modal).

I like the system prompt dropdown concept. Why not add a rename button in ChatCraft (System Prompt), so we could list them by name

Options 2 would be ok for me.

OK, I've reworked this to allow users to "star" a system prompt, and then only show the ones they've starred.  I also reworked the way the default system prompt works to let you do it via its own dedicated modal.  Trying to do this via the menu was too complicated, and I think this is simpler.

NOTE: this change needs #198 to get scrolling to not jump to the bottom when you star/unstar the system prompt.

I just tried this.

edited the system prompt
stared it
now I have no more default prompt in the menu. This is scary.

"new blank chat" brings the old prompt back, but menu still doesn't show it. We should star the default prompt.
Feels like a scary amount of state for a very corner-case feature...How about tagging shared chats as "chat-templates".

To clarify what I'm reading above, you're saying the default system prompt should also be in that drop-down menu?  If so, that's a good idea.

Rebased and updated to include the default system prompt as first item in list of starred entries.

I wonder about having this sort of complexity in indexeddb...For me it would be better to keep such a "personal prompt library" in cloud so i could use it across all my devices. Same goes for the /commands and @functions we are talking about in other issue

"keep...in the cloud" is scope creep.  If we want to explore that, let's not do it in this PR.
How about the functionality of this change?  What do I need to do in order to finish it?  Same question to @steven4354.

I def think this is sufficient. Not asking for scope creep, just noting that you are taking on a more complex client-side schema with this change.

lets land the last version you posted

Rebased, updated toast() calls to use new useAlert() hook, will land when this goes green...

Go forth and create reusable system prompts!


[WIP] Enable extending NonEquilibriumCyclingProtocols

This PR introduces the ability to extend a NonEquilibriumCycling protocol by including serialized and compressed OpenMM State, System, and Integrator in the main simulation units. When instantiating new NonEquilbriumProtocols which extend a previous result, these objects are used to bypass the normal function of the SetupUnit. Closes #2.

One limitation of GufeTokenizables, including ProtocolDAGResults: they must be JSON-serializable, and therefore can't have binary data. So any outputs of a ProtocolUnit._execute method must also be data types that are JSON-serializable.
Is this the case for the outputs added in this PR?

These are not json serializable. I can add another step in (de)serialization for converting to and from base64.

@ijpulidos any idea why my tests fail on license keys?

@ianmkenney My guess is that the secrets fail since it's coming from a fork? Let me dig around and see if there's a way to make this work.
One way would be making a separate PR in the repo, but I'll see if there's another way.

to kick off ci (you need write access to the repo for this to work)
$ gh pr checkout 44
$ git push origin feature-noneq_cycling_extends

This is currently blocked by #38.

@ianmkenney @dotsdl Is this ready to be reviewed? If so, can you mark it as such? Thanks!

@ijpulidos I think an initial review works, not sure if you want to wait for gufe/openfe v1.0.0 before merging though.

@ijpulidos will make another review now that #38 is in! Thanks @ijpulidos!

I was looking over these changes again, and I think we should take a closer look at how we’re supporting extensions. While this works, I’m not sure it will generalize well to other protocols we plan to host here. Probably worth a bit more discussion to make sure we are not limiting ourselves down the line.

From today's iteration meeting - let's put this on the agenda for the next protocol devs call.


Release Google.Cloud.CloudBuild.V1 version 2.13.0

Changes in this release:
New features

Add PrivateServiceConnect option to WorkerPool (commit 59748e6)

Created release for Google.Cloud.CloudBuild.V1-2.13.0


Handling content outside the map directory: incorrect links in nav toc

When I work with content outside the map directory and I set the parameter generate.copy.outer to 3, the nav-toc (collapsible in my case) in the output does not take this into account resulting in incorrect links.
Creating a default HTML5 output does not have this problem.
I attached a small sample map.
sample_map.zip

I did some further testing and this is a general HTML5 plug-in problem. I will log it in the dita-ot repo.

@PieterjanVandenweghe Thanks for reporting this, and for verifying that it also occurs in the default HTML5 output.
Since the problem isn't specific to the DITA Bootstrap plug-in, I'll close this issue and link to https://github.com/dita-ot/dita-ot/issues/4478.


How do you use the Laravel implementation?

There are no docs at all.

Thought it was self-explanatory. Guess not.

Include the file in your project.
Set the correct namespace.
From your controller call instantiate the class like so:
$vuetables = new EloquentVueTables; (import the file on top with a use statement)
return the result:
return $vuetables->get(new SomeModel, ['field_a','field_b'])

If you are using L5 you need to replace the Input class on line 12 with the Request one.

Okay, so i was on the right track. I just kept hitting errors and wondered if I was way off base somehow.
I do keep getting SQL errors though.
2/2
QueryException in Connection.php line 673:
SQLSTATE[42000]: Syntax error or access violation: 1064 You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '0' at line 1 (SQL: select `first_name`, `last_name`, `email` from `users` offset 0)

It seems the logic dictates that a limit must be set, though it isn't documented anywhere.

I hacked on it a bit and got the query to run, but the component's tbody is empty, even though it has all the matching data necessary.

limit IS sent, along with all the other params, from the client side, as specified in the docs.
Check two things:

Make sure you get a 200 response with the correct keys and data (data and count)
Use vue-resource version 0.9.0 or greater.

Data is correct and vue-resource is 0.9.0. Still no rows.

Are there any errors in the console?

main.js:19269 Uncaught (in promise) TypeError: Cannot read property 'forEach' of undefined(…)

Here's a more verbose look:

mmm. The guy from Issue #73 had the same problem.

What browser are you using? Which build tool?

Chrome and Browserify with Laravel-Elixir-Vueify.

Why is that Issue closed?

And what version of vue?

Vue 1.0.24

I'm using the exact same settings.
Is the component nested within another?

Nope, no nested components.

Can you paste in the relevant HTML and JS?

Are there any required options? I'm not setting any.
The template:
<v-server-table url="/admin/api/users" :columns="columns"></v-server-table>

The JS:
new Vue({
    el: '#app',
    data: {
        columns: ['first_name', 'last_name', 'email']
    }
});

The route:
    public function getUsers()
    {
        $vuetable = new EloquentVueTables();

        return $vuetable->get(new User, ['first_name', 'last_name', 'email']);
    }

No required options.
Can you add a console.log(this.customColumns) in the file add-custom-columns.js and paste the result?

An empty array [].

That is correct. What about this.data?

undefined

Ok. so there is a problem with assigning the data returned from the server.
Let's look at the raw response.
Log data on v-server-table.js line 59.

Response {url: "/admin/api/users?limit=2&query=&limit=10&orderBy=first_name&ascending=1&page=1&byColumn=0", body: "{"data":[{"first_name":"Apple","last_name":"Bees",…name":"Two","email":"three@four.com"}],"count":8}", headers: Object, status: 200, statusText: "OK"…}
body
:
"{"data":[{"first_name":"Apple","last_name":"Bees","email":"fake@italian.biz"},{"first_name":"Booker","last_name":"Washington","email":"jack@jackmcdade.com"},{"first_name":"Carver","last_name":"Fours","email":"carver@hackmcdade.com"},{"first_name":"Jack","last_name":"McDade","email":"jack@wilderborn.com"},{"first_name":"Jack","last_name":"One","email":"jack+test@jackmcdade.com"},{"first_name":"Joe","last_name":"TWO","email":"joe@jackmcdade.com"},{"first_name":"Me","last_name":"three","email":"me@you.com"},{"first_name":"One","last_name":"Two","email":"three@four.com"}],"count":8}"
data
:
"{"data":[{"first_name":"Apple","last_name":"Bees","email":"fake@italian.biz"},{"first_name":"Booker","last_name":"Washington","email":"jack@jackmcdade.com"},{"first_name":"Carver","last_name":"Fours","email":"carver@hackmcdade.com"},{"first_name":"Jack","last_name":"McDade","email":"jack@wilderborn.com"},{"first_name":"Jack","last_name":"One","email":"jack+test@jackmcdade.com"},{"first_name":"Joe","last_name":"TWO","email":"joe@jackmcdade.com"},{"first_name":"Me","last_name":"three","email":"me@you.com"},{"first_name":"One","last_name":"Two","email":"three@four.com"}],"count":8}"
headers
:
Object
ok
:
true
status
:
200
statusText
:
"OK"
url
:
"/admin/api/users?limit=2&query=&limit=10&orderBy=first_name&ascending=1&page=1&byColumn=0"

It looks like data is no longer an object when it goes through setData(). data.data is undefined.

Yeah that's definitely the source of the issue. If i do var data = JSON.parse(data.data); at line 60 there, the table loads up fine. All the features don't work though since that parse isn't at the component level.

I see that.
This is my log, using vue-resource 0.9.0. As you can see my data is an object.

So where does the issue lie? My GET response is a valid JSON object.

I am as baffled as you are...

I suppose I can temporarily add a hacky if statement to parse as json if the response is a string.

Might be the best choice until it can be sorted out more clearly.

What if you replaced calls to the getData() response where you reference data.data with data.json()? That's the Vue Resource method for ensuring JSON.

Yeah, i just tried that locally by updating v-server-table.js, order-by-column.js, search.js and set-page.jsand everything seems to be working properly.

Great. I will update and upload shortly.

lol..been having the same forEach error message as jack while browsing the repo for a solution...reading through this thread, while slowly realizing that this is happening right at this moment..hit F5 repeatedly to check if your comments timestamps were real...haha...never had this coincidence before...
HOWEVER...I do have the same error message as Jack...I hope I'm not hijacking this issue but it seems related...I am trying to use the server component and have the server (laravel 5) all set up...however it gives me this error:
Uncaught (in promise) TypeError: Cannot read property 'forEach' of undefined
Thanks!

Please install the fix (1.4.51). Hopefully it did the trick.

Boom. That it did.
I'll be sending you a PR for the Laravel implementation if you're up for it.

Bingo bongo! Does work for me now, too! Cheers guys, been a pleasure...

Sure. I wrote it for L4. It might need some extra tweaking.

hi, I use the lastest dist single file version with vue-resource 0.9.3, and get the same error when I call refresh or use the customFilters(used to work with the vue-resource 0.8.x and with the old vue-tables 1.4.0 version), but it was ok for the first time loading data from server (php backend, which return the data array and a count)
so I wonder if the dist file is not the lastest one... and would you please upload the full version of vue-tables.js too, Thanks. And the lastest one's MD5 is here:
MD5(vue-tables.min.js)= 257f2407982a82b340310346d5aa8efa
``

I have compiled the fix into the minified file. Just double checked by looking there for the latest change (calling data.json 4 times) and found it.
You can see that it was changed in the latest commit:
https://github.com/matfish2/vue-tables/commit/2dd28ea0c49f4283bc0ac95503cbf0bc3a6cf8b1

I've updated the minified version along with the last commit.
Probably a caching issue on your side.

I look into the error in Chrome just now, and found the problem:
https://github.com/matfish2/vue-tables/blob/master/lib/v-server-table.js#L84
this line also needs to be changed from data.data to data.json and it solved my problem
and after thecking the project, vue-tables/lib/methods/set-data.js is also still using the data.data
popozhu lib $ find ./ -name "*.js" | xargs grep 'data.data'
.//methods/set-data.js:  this.data = data.data;
.//v-server-table.js:        this.setData(data.data);

Thank you. You are right about the call in v-server-table.js, But the data.data in set-data.js is called on the the response data itself (i.e what data.json() returns) which is comprised of the data and count keys. I will upload a fix momentarily.


Getter and setter for object pointers missing casts

Brief Description
In the process of generating a C++ wrapper to a C library I experienced a error where the getters and setters generated for a pointer type would generate for the correct type (in this case global::TestState*) but the local variable would generate as a global::System.IntPtr. This causes a casting error where the build fails, adding correct casts manually fixes the issue.
C++ input: https://pastebin.com/LEYY8MkW
Generated C# output: https://pastebin.com/R3NweFnL
CppSharp log: https://pastebin.com/VFhZuSnQ
Errors:
Severity	Code	Description	Project	File	Line	Suppression State
Error	CS0266	Cannot implicitly convert type 'System.IntPtr' to 'TestState*'. An explicit conversion exists (are you missing a cast?)	Test	C:\Users\mattp\Documents\Flounder Workspace\FlounderSharp\FlounderSharp\Test.cs	122	Active

Severity	Code	Description	Project	File	Line	Suppression State
Error	CS0266	Cannot implicitly convert type 'TestState*' to 'System.IntPtr'. An explicit conversion exists (are you missing a cast?)	Test	C:\Users\mattp\Documents\Flounder Workspace\FlounderSharp\FlounderSharp\Test.cs	127	Active

OS: Windows
Target: MSVC
Driver: https://pastebin.com/vAWeTgua

Thanks for the well detailed issue report and repro.
This one should be easy to fix, we'll take a look when we can.