about using coc_medium

I try to run validate.py with coc_medium, it can not load the checkpoint.
error message like these.

A few changes seem to be required in the code
In models/context_cluster.py change
Cluster class:  self.f --> self.fc1     (rename)
self.proj --> self.fc2    (rename)
self.v --> self.fc_v     (rename)
Mlp class:      self.fc1 --> nn.Conv2d(in_features, hidden_features, kernel_size=1)                     (change linear to conv2d)
                     self.fc2 --> nn.Conv2d(hidden_features, out_features, kernel_size=1)                  (change linear to conv2d)
                     also remove all permutes in forward()

In validate.py change:
change line 155 to
load_checkpoint(model, args.checkpoint, args.use_ema, strict=False)
Although I'm not the author of this paper, these changes seemed to do the trick for me

@lai-pf @myeyeget Sry for the late reply.
We slightly re-organized the codes for fast training and inference. @myeyeget 's solution seems good.
You can also try the submission link here: https://anonymous.4open.science/r/ContextCluster/README.md
We will update the checkpoints soon. Thanks for your issue.

@ma-xu The submission link has expired. Could you please update it so that I can solve the mismatch problem?

@ErlebnisW @myeyeget @lai-pf
Thanks for your patience. The updated checkpoints are updated, with slightly improvements. Mainly two updates: 1) fix a bug mask.sum to sim.sum; 2) conv1x1 to Linear implementation.

Closed no further discussions. Feel free to reopen it if any further questions.


'cellular anatomical entity' is no longer a subclass of 'anatomical entity' (CARO or UBERON)

I see this issue in the 2023-05-18 snapshot, but not the 2023-05-10 release. Discovered this issue while looking into https://github.com/geneontology/noctua-form/issues/222

It looks like it happened in this commit: https://github.com/geneontology/go-ontology/commit/e8c9d87d0e4e0e7844148a55365f68bd5d79943e
I would guess it was a Protégé accidental click.

Thanks Jim, sorry about that !

Thanks Jim, sorry about that !

No problem—I was glad it was a very simple fix!


OutlineProxyController has initialization errors and Client enters loop of Proxy Initialization instead of Connection

Describe the bug
After Client starts and Connect button is pushed initialization process ends with success.
But in journalctl I can see error messages.
Than Connect button doesn't bring the connection up, instead it launches another initialization process.
To Reproduce
Start Client, try to Connect, after initialization try to Connect again
Expected behavior
Client connects to the endpoint
OutlineProxy Controller log
OutlineProxyController[8974]:<PHONE_NUMBER>.689736: [INFO] successfully set the tun device ip address
OutlineProxyController[8974]:<PHONE_NUMBER>.693377: [INFO] successfully added outline gateway routing entry
OutlineProxyController[8974]:<PHONE_NUMBER>.699736: [ERROR] Error: any valid prefix is expected rather than "".
OutlineProxyController[8974]:
OutlineProxyController[8974]:<PHONE_NUMBER>.700705: [WARN] unable to query the default route to the outline proxy
OutlineProxyController[8974]:<PHONE_NUMBER>.700935: [WARN] we could not detect the best interface, will try again at connect
OutlineProxyController[8974]: updated unix socket owner to<PHONE_NUMBER>,1002
Outline-Client log
Update for version 1.8.1 is not available (latest version: 1.8.1, downgrade is disallowed).
connecting to 2c1ec9d2-1b09-4c18-82f9-9781564a4bf4...
using tun2socks to check connectivity
[EXIT - tun2socks]: Exited with code 0
UDP support: true
could not connect: _ (routing daemon is not running)
[EXIT - tun2socks]: Killed by signal SIGTERM
tun2socks terminated due to Error: Process terminated by signal: SIGTERM
disconnected from 2c1ec9d2-1b09-4c18-82f9-9781564a4bf4
Client System (please complete the following information):

Outline Client Version [e.g. Linux 1.8.1]
Your Operating System: [e.g. Kubuntu 22.04.1]

@jyyi1 it seems there's an issue with the installation on Linux. Could you take a look since you touched that code recently?

same, code was taken from outline website

Hi @simakazi, thanks for reporting the issue. From the logs I can see:
OutlineProxyController[8974]: updated unix socket owner to<PHONE_NUMBER>,1002

This means we failed to get the current user's uid and grant the required permissions to rw /var/run/outline_controller. This needs further investigation (for example, maybe the uid has been changed). As a workaround, could you try adding your current user to the outlinevpn group (we should already created this group) and restart the machine?
usermod -aG outlinevpn "$USER"


feat: add assert/is-well-formed-string

This commit adds the assert/is-well-formed-string module for checking well-formed-strings.
Resolves #1065
Description
Added the is-well-formed-string package. It checks for four things to check if a string is well formed or not:

A low surrogate is present at the beginning.
A high surrogate is present at the last.
No low surrogate after a high surrogate.
No high surrogate before a low surrogate.

What is the purpose of this pull request?

This pull request:

Adds a package to check if a string is well-formed or not.

Related Issues

Does this pull request have any related issues?

This pull request:

resolves #1065

Questions

Any questions for reviewers of this pull request?

No.
Other

Any other information relevant to this pull request? This may include screenshots, references, and/or implementation notes.

Screenshot of tests:

Checklist

Please ensure the following tasks are completed before submitting this pull request.

☑  Read, understood, and followed the contributing guidelines.

@stdlib-js/reviewers

@Planeshifter
There is some error in linting the examples. I have written the correct outputs, but it is expecting something else, and thus giving errors. I checked the implementation of the functions, it is indeed correct.

@gunjjoshi Thanks for checking! I believe the issue is that the instanceof String check doesn't work across realms when there are multiple String constructors. To lint examples and their expected outputs, we spawn a separate process via vm2 that will have it's own String constructor. Here is how we make the object check in the assert/is-string package https://github.com/stdlib-js/stdlib/blob/develop/lib/node_modules/%40stdlib/assert/is-string/lib/object.js more robust to have it work across realms.
Would you mind updating this package accordingly?

@gunjjoshi Thanks for checking! I believe the issue is that the instanceof String check doesn't work across realms when there are multiple String constructors. To lint examples and their expected outputs, we spawn a separate process via vm2 that will have it's own String constructor. Here is how we make the object check in the assert/is-string package https://github.com/stdlib-js/stdlib/blob/develop/lib/node_modules/%40stdlib/assert/is-string/lib/object.js more robust to have it work across realms. Would you mind updating this package accordingly?

Sure, I'll update this package.

Great ! It got passed by the CI pipeline.

@gunjjoshi Will merge after CI clears; thanks again for your contribution!


dnsmasq: install launchd plist on darwin

It's just a static file in the contrib folder of the project's source, added as an output in System/Library/LaunchDaemons (I copied this from a few other packages), with its path corrected.

@LnL7 pointed out most packages put plist files in just $out/Library/LaunchDaemons, so I switched to that.

Package ‘xen-4.5.5’ in ‘/home/travis/build/NixOS/nixpkgs/pkgs/applications/virtualization/xen/generic.nix:210’ is not supported on ‘i686-linux’, refusing to evaluate.
The job exceeded the maximum time limit for jobs, and has been terminated.

What's that from?

Those seem to be the causes of the current build failures.


Update Wasm dependencies to fix WebGPU spec change crash in Firefox with Vello

Fixes running on firefox nightly (requires the latest wgpu to fix https://github.com/gfx-rs/wgpu/issues/6290#issuecomment-2383605551). This also required using vello with the git wgpu version.
We get a warning due to "use of deprecated method web_sys::CanvasRenderingContext2d::set_stroke_style". There is an alternative function but it is broken. I fixed it in https://github.com/rustwasm/wasm-bindgen/pull/4170 but it has not been released. There will apparently be a patch release "today".

It sounds like next week is when WGPU 0.23 is planned for release. I'll go ahead and merge this now but we should be able to update as soon as that comes out and Vello pushes an upgrade to it, letting us switch to the released WGPU and the official Git version of Vello instead of needing to use the Vello fork for now. But that'll all be an easy followup PR next week.


Compiling on OS X

Hi again.  I cloned the repo locally and changed a few files.  I can run make build just fine as well as make image but when I try running a container I get
ERROR: 500 Internal Server Error: Cannot start container 07e264978e89f23abce32540037d7a5a9040482113707d20bfe13026d0425502: [9] System error: exec format error
I'm running on OS X so I looked at the Makefile and changed the GOOS to darwin and the GOARCH to arm, 386, and arm64, none of which worked.  I'm pretty new to go so after none of those worked I didn't really know what to do.  I was wondering if you could point me in the right direction.

This is because Interlock runs in a container and you are building an OS X
binary.  Try setting GOOS=linux and GOARCH=amd64.  That should result in a
working image.  I had a "build-container" target in the past to build in a
container.  I will add that back.  For now, you can use the env var as a
workaround.
On Tue, Mar 22, 2016 at 9:38 PM, Kris<EMAIL_ADDRESS>wrote:

Hi again. I cloned the repo locally and changed a few files. I can run make
build just fine as well as make image but when I try running a container
I get
ERROR: 500 Internal Server Error: Cannot start container
07e264978e89f23abce32540037d7a5a9040482113707d20bfe13026d0425502: [9]
System error: exec format error
I'm running on OS X so I looked at the Makefile and changed the GOOS to
darwin and the GOARCH to arm, 386, and arm64, none of which worked. I'm
pretty new to go so after none of those worked I didn't really know what to
do. I was wondering if you could point me in the right direction.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub
https://github.com/ehazlett/interlock/issues/106

Sorry.  When I tried to make build it the first time I had GOOS=linux and GOARCH=amd64 and then ran make image I got the same error.

There was an issue with the Makefile.  Please pull and try again.

I'm still getting the same error

Ok pull again.  I just added a build-container target and set the default to build in a container.  This should result in a working image (I just tested on OS X).

It works!  Thanks again for you help!  I appreciate it.


HDDS-11843. Enhance DataNodeSafeModeRule to Use Pipeline DN List.

What changes were proposed in this pull request?

Background

We enhanced the DataNodeSafeModeRule to validate the registered DN based on the DN list stored in the Pipeline.
Currently, the DataNodeSafeModeRule in SCM requires users to manually configure the number of DataNodes in the cluster using the setting hdds.scm.safemode.min.datanode, with a default value of 1. If not configured, this value defaults to 1. However, since clusters are frequently scaled and the number of active DataNodes is uncertain, relying on a fixed value to determine whether SafeMode conditions are met is not reliable.
To address this issue, we propose a new rule:
allowing DataNodeSafeModeRule to retrieve the DataNode list from the registered Pipeline and use a configurable ratio to dynamically calculate the required number of DataNodes. This will provide a more accurate way to determine when the system can exit SafeMode.

The usage of the new rules

The following configurations need to be set simultaneously:

Configuration 1: Enable Pipeline Rule.
<property>
    <name>hdds.scm.safemode.datanode.use.pipeline.enabled</name>
    <value>true</value>
</property>

Configuration 2: Enable Pipeline Rule.
<property>
    <name>hdds.scm.safemode.reported.datanode.pct</name>
    <value>0.1</value>
</property>

Execution Flow of the New Rule

We will use the DN list from the Pipeline as the total list of DataNodes to be registered.
The total DN list will determine the required DataNodes for registration based on the hdds.scm.safemode.reported.datanode.pct value.
Once DataNodes continuously report and the number of registered DataNodes reaches the calculated threshold, the DataNodeSafeModeRule will exit automatically.
At the same time, we will list 5 DataNodes that have not been registered, in order to facilitate troubleshooting of DataNodes that failed to register.

Final Result Display

What is the link to the Apache JIRA
JIRA: HDDS-11843. Enhance DataNodeSafeModeRule to Use Pipeline DN List.
How was this patch tested?
Manual Testing.

@sadanand48 @siddhantsangwan @adoroszlai Could you help review the code? Thank you very much!
In this PR, we have enhanced the DataNodeSafeModeRule to determine how many DataNodes need to be registered based on the list of DataNodes stored in the pipeline, multiplied by a certain ratio. From my perspective, this approach is better than directly setting the number of DataNodes.

cc @nandakumar131

Hi @slfan1989 can you elaborate more on the use case for this feature?

However, since clusters are frequently scaled and the number of active DataNodes is uncertain, relying on a fixed value to determine whether SafeMode conditions are met is not reliable.

I can see how the default value of 1 may not be too helpful, but what are the practical issues that the current safemode configurations are causing in your cluster? The other rules that check for container replicas and pipelines should ensure that cluster is sufficiently operational when safemode is exited.
I have a few concerns with adding this. Pipeline list is not a definitive list of cluster membership, especially if new nodes are added with the restart, so the rule may be more or less an approximation depending on the situation. It also adds more configuration and complexity to safemode. FWIW HDFS does not have such a feature because it does not have fixed cluster membership by design. Ozone does not really have fixed cluster membership either. Persistent pipelines actually add more complexity to the system than they are worth IMO and I don't think we should tie more features into them.
I feel like a dashboard as described in HDDS-11525 would better address the concern about nodes not registering, and that the container and pipeline rules provide stronger guarantees about the cluster's readiness than node count or percentage.

Thanks @errose28 for the input. I agree with Ethan's point. We should try to simplify the SafemodeRules, not add complexity to it. Even with the additional complexity, this doesn't provide any value addition to the Safemode logic.
The Container and Pipeline Safemode rules ensures that the cluster is operational when it comes out of Safemode.

@errose28 @nandakumar131
Thank you very much for your response! However, please allow me to provide some clarification regarding this PR. My intention is not to increase the complexity of SafeMode, but rather to address some issues that have arisen during its use:
The reasons I added this feature are as follows:

The Ozone SCM does not maintain a complete list of DataNodes, which can lead to a problem. In large clusters, when we restart the SCM, some DataNodes may fail to register, and we won't be able to locate these DataNodes (since they are not included in the DataNode list).

Currently, my approach is to manually compare the DataNode lists from the two SCMs, identify the DataNodes that failed to register, track these DataNodes, and take appropriate actions (in most cases, this involves restarting the DataNodes).
The purpose of retrieving the DataNode list from the Pipeline List is to identify any unregistered DataNodes, as shown in my screenshot.

The reason for improving the DataNodeSafeModeRule is that this rule is difficult to apply effectively in real-world usage. Let me provide a specific example:

Adding DataNodes:
When our cluster reaches the 75% threshold, we need to expand the number of DataNodes, typically scaling from 100 machines to 120 or 130.

Reducing DataNodes:
Our cluster may include various types of machines, some of which have poor performance. In such cases, we may need to take some machines offline for replacement, which could result in a reduction in the number of DataNodes.

So, what value should we set for the parameter hdds.scm.safemode.min.datanode? It's quite difficult to assess. A default value of 1 is certainly not ideal, but when we consider a cluster with 100 machines, should we set it to 40, 50, or 60? This is also hard to determine.
However, if we set a proportion, the rule would become more flexible. For example, if we expect 60% of the DataNodes to register, this value would dynamically change over time.

Grafana is a good solution, but it also has some issues in large-scale cluster environments. The problem is that there are too many metrics for each DataNode. Even with a 30-second collection interval, a single DataNode generates a large of metrics. Currently, we have 5 clusters with over 3,000 DataNodes, which puts significant pressure on our collection system.  At the moment, we collect metrics for DataNodes every 5 minutes, which results in delays in reflecting the actual situation in real time.

I would really appreciate any suggestions you may have. Thank you once again!

Thanks @slfan1989 for sharing details.

Currently, my approach is to manually compare the DataNode lists from the two SCMs, identify the DataNodes that failed to register, track these DataNodes, and take appropriate actions (in most cases, this involves restarting the DataNodes).
The purpose of retrieving the DataNode list from the Pipeline List is to identify any unregistered DataNodes, as shown in my screenshot.

We could implement new (or improve existing) ozone admin command to help with that.  It may need backend changes, but not DataNodeSafeModeRule.

The reason for improving the DataNodeSafeModeRule is that this rule is difficult to apply effectively in real-world usage.

I think the sentiment here is that other safemode rules should help with that.  DataNodeSafeModeRule is kind of a "minimum bar" for the cluster to pass.

Related to this change, @nandakumar131 recently filed HDDS-11904. Maybe once that is fixed hdds.scm.safemode.healthy.pipeline.pct will better map to the percent of nodes that need to register before safemode is exited.

We could implement new (or improve existing) ozone admin command to help with that. It may need backend changes, but not DataNodeSafeModeRule.

Probably ozone admin datanode list is the command we are looking for here. This does not check against a master list of datanodes because Ozone does not attempt to maintain such a thing (nor does HDFS). If this is desired it must be stored externally.

So, what value should we set for the parameter hdds.scm.safemode.min.datanode? It's quite difficult to assess. A default value of 1 is certainly not ideal, but when we consider a cluster with 100 machines, should we set it to 40, 50, or 60? This is also hard to determine.

I see datanode safemode rule as a sort of smoketest that cluster wide configs are setup correctly so SCM and DNs can communicate. Security, DNS, hostnames, etc. are all working such that if at least one datanode registers then we have reason to believe that later safemode rules are failing for different reasons. I don't think the intent of this rule is really to enforce a cluster membership requirement before bringing the cluster out of safemode.

Even with a 30-second collection interval, a single DataNode generates a large of metrics. Currently, we have 5 clusters with over 3,000 DataNodes, which puts significant pressure on our collection system.

I don't have much experience in this area but it seems like the system being monitored should not be bound by the system being used to monitor it. If Ozone is being scaled horizontally then in theory the metrics collection system should be able to scale horizontally to match. If the datanodes are doing something egregious that is causing the size of metrics data to explode then we should try to optimize that, but we shouldn't remove information that could otherwise be useful.

@errose28 @adoroszlai Thank you very much for your response! The feature I submitted is relatively small, and I really appreciate your attention and the detailed explanation. I agree with your point that integrating some of the functionality into DataNodeSafeModeRule may not be ideal. However, we have already rolled out this feature internally, and for my part, it meets our needs. I understand that different users may have different requirements and perspectives on the system. I plan to close this PR and set the JIRA status to "Works for me." If other community members search for this JIRA and see our discussion, and find it helpful, that would be great.
HDDS-11525 can solve our issue, so I will focus on this JIRA and do my best to contribute. I look forward to HDDS-11904 bringing better results. If a PR is submitted for the related JIRA, I will also take a look.

We could implement new (or improve existing) ozone admin command to help with that. It may need backend changes, but not DataNodeSafeModeRule.

I will also consider this idea, as it's a good approach. However, as @errose28 mentioned, we would need to store this information elsewhere, which would still add complexity to the system.
Thank you all again for your time!
cc: @nandakumar131 @ChenSammi


[GCE E2E tests] Terraform is unable to remove routes created by CCM

The GCE E2E tests are failing because Terraform can't remove routes created by Kubernetes CCM.
Error: Error waiting for Deleting Network: The network resource 'projects/project-name/global/networks/network-name' is already being used by 'projects/project-name/global/routes/route-name'

We need to find a way to remove routes created by the CCM, so Terraform can proceed with deleting the network.
Besides that, it seems like we are reaching limits:
Error: Error waiting for instance to create: Quota 'IN_USE_ADDRESSES' exceeded.  Limit: <limit> in region <region>.

Acceptance criteria

Routes are removed before running Terraform
Check what we can do about the limits

/remove-lifecycle stale
/lifecycle frozen

Terraform is not able to handle those routes, and without that, we can't have GCE E2E tests. Alternatively, we could eventually have some scripts that would remove those routes, but the problem is that we didn't really like having some external script.
As discussed with @xmudrii, closing it for now.


Support add menu to search panel

I made a extension with a command search in git changed files, and want to make it a menu item in search panel menu:

https://github.com/tjx666/power-edit/commit/89cd3be5b86ddca7841d645fdc0c819e1d4db409#diff-f6bebb5f7443731836b3c64873eabf7338b0180ea95819e66d33dc3e6e229dbe

https://code.visualstudio.com/api/references/contribution-points#menu-example includes an example of how to contribute a command to any view.

thanks @gjsjohnmurray
"menus": {
            "view/title": [
                {
                    "command": "power-edit.searchInGitChangedFiles",
                    "when": "view == workbench.view.search",
                    "group": "navigation"
                }
            ]
        },


prime number detector

prime number

Close #2786

Checklist:

[x] I have mentioned the issue number in my Pull Request.
[x] I have commented my code, particularly in hard-to-understand areas
[x] I have created a helpful and easy to understand README.md

@MohitGupta121 Kindly review this PR.


Suggestion: Tweak fullScreenZenMode description

Testing #15647
"Controls if turning on Zen Mode also puts the workbench into full screen mode."

@chrisdias you can assign testing bugs straight to the test item owner.


Add test for concurrent recursive creates

This tests for bug https://github.com/Alluxio/alluxio/issues/8430 discovered in branch-1.8. The bug was accidentally fixed by master locking changes

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/2092/
Test PASSed.


[Archiving failing] [XCode 14.3] Multiple commands produce GoogleUtilities

Hi friends,
I have a problem suddenly by archiving my app in XCode 14.3 I'm getting this strange error:

Showing All Errors Only

Prepare build
note: Building targets in dependency order
error: Multiple commands produce '/Users/razvanu/Library/Developer/Xcode/DerivedData/Runner-aeiwqtydtcicmbcjpyzoxtfwhzig/Build/Intermediates.noindex/ArchiveIntermediates/Runner/IntermediateBuildFilesPath/UninstalledProducts/iphoneos/GoogleUtilities.framework'
    note: Target 'GoogleUtilities-00567490' (project 'Pods') has create directory command with output '/Users/razvanu/Library/Developer/Xcode/DerivedData/Runner-aeiwqtydtcicmbcjpyzoxtfwhzig/Build/Intermediates.noindex/ArchiveIntermediates/Runner/IntermediateBuildFilesPath/UninstalledProducts/iphoneos/GoogleUtilities.framework'
    note: Target 'GoogleUtilities-54e75ca4' (project 'Pods') has create directory command with output '/Users/razvanu/Library/Developer/Xcode/DerivedData/Runner-aeiwqtydtcicmbcjpyzoxtfwhzig/Build/Intermediates.noindex/ArchiveIntermediates/Runner/IntermediateBuildFilesPath/UninstalledProducts/iphoneos/GoogleUtilities.framework'

I tried all of the solutions from the web nothing works for me 😦 please help/guide me with this issue, thanks in advance!
P.S: I'm able to build but not archive the app!

hi, did you manage to solve the problem?

hi, did you manage to solve the problem?

I found that in some forum post that you have to add pod 'GoogleUtilities' into each of your target Pod file.
target 'Release' do
  pod 'GoogleUtilities'
  pod 'Firebase'
  use_frameworks! :linkage => podfile_properties['ios.useFrameworks'].to_sym if podfile_properties['ios.useFrameworks']
...
end

I hope this helps to someone.

any solution?

I think we'll need a reproduction shared to troubleshoot this one
Does it happen when you just use the native firebase-ios-sdk and compile it vs this precompiled repackaging?
(if it works using firebase-ios-sdk instead of this repo, you can still enjoy fast builds with a compiler cache strategy like explained here https://reactnative.dev/docs/build-speed#use-a-compiler-cache )

hi, did you manage to solve the problem?

I found that in some forum post that you have to add pod 'GoogleUtilities' into each of your target Pod file.
target 'Release' do
  pod 'GoogleUtilities'
  pod 'Firebase'
  use_frameworks! :linkage => podfile_properties['ios.useFrameworks'].to_sym if podfile_properties['ios.useFrameworks']
...
end

I hope this helps to someone.

This works for me. I added 'pod 'GoogleUtilities' to all targets.
Thank you @vborbely


Training Instructions

I'm trying to train the model on Pascal VOC.
I downloaded the dataset and converted them into TFRecords following the instructions on the README.
However I'm a little confused about how to train. My tfrecords files are in the ./datasets folder.
My ./logs folder has a fine-tune subfolder which is empty with the exception of some events.out.tfevent files that get created every time I attempt to train.
I'm not sure what I'm missing. Am I supposed to have some retrained ImageNets weights somewhere?
Any help is appreciated!
Here is my error message:
$ python ./train_model.py
INFO:tensorflow:Fine-tuning from None
INFO:tensorflow:Restoring parameters from None
INFO:tensorflow:Error reported to Coordinator: <class 'tensorflow.python.framework.errors_impl.InternalError'>, Unable to get element from the feed as bytes.
Traceback (most recent call last):
  File "./train_model.py", line 435, in <module>
    obj.run()
  File "./train_model.py", line 427, in run
    self.__start_training()
  File "./train_model.py", line 222, in __start_training
    save_interval_secs=self.save_interval_secs)
  File "/usr/local/lib/python2.7/site-packages/tensorflow/contrib/slim/python/slim/learning.py", line 732, in train
    master, start_standard_services=False, config=session_config) as sess:
  File "/usr/local/Cellar/python/2.7.13_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/contextlib.py", line 17, in __enter__
    return self.gen.next()
  File "/usr/local/lib/python2.7/site-packages/tensorflow/python/training/supervisor.py", line 964, in managed_session
    self.stop(close_summary_writer=close_summary_writer)
  File "/usr/local/lib/python2.7/site-packages/tensorflow/python/training/supervisor.py", line 792, in stop
    stop_grace_period_secs=self._stop_grace_secs)
  File "/usr/local/lib/python2.7/site-packages/tensorflow/python/training/coordinator.py", line 389, in join
    six.reraise(*self._exc_info_to_raise)
  File "/usr/local/lib/python2.7/site-packages/tensorflow/python/training/supervisor.py", line 953, in managed_session
    start_standard_services=start_standard_services)
  File "/usr/local/lib/python2.7/site-packages/tensorflow/python/training/supervisor.py", line 708, in prepare_or_wait_for_session
    init_feed_dict=self._init_feed_dict, init_fn=self._init_fn)
  File "/usr/local/lib/python2.7/site-packages/tensorflow/python/training/session_manager.py", line 281, in prepare_session
    init_fn(sess)
  File "/usr/local/lib/python2.7/site-packages/tensorflow/contrib/framework/python/ops/variables.py", line 654, in callback
    saver.restore(session, model_path)
  File "/usr/local/lib/python2.7/site-packages/tensorflow/python/training/saver.py", line 1548, in restore
    {self.saver_def.filename_tensor_name: save_path})
  File "/usr/local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 789, in run
    run_metadata_ptr)
  File "/usr/local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 997, in _run
    feed_dict_string, options, run_metadata)
  File "/usr/local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1132, in _do_run
    target_list, options, run_metadata)
  File "/usr/local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1152, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InternalError: Unable to get element from the feed as bytes.
ERROR:tensorflow:==================================
Object was never used (type <class 'tensorflow.python.framework.ops.Tensor'>):
<tf.Tensor 'init_ops/report_uninitialized_variables/boolean_mask/Gather:0' shape=(?,) dtype=string>
If you want to mark it as used call its "mark_used()" method.
It was originally created here:
['File "./train_model.py", line 435, in <module>\n    obj.run()', 'File "./train_model.py", line 427, in run\n    self.__start_training()', 'File "./train_model.py", line 222, in __start_training\n    save_interval_secs=self.save_interval_secs)', 'File "/usr/local/lib/python2.7/site-packages/tensorflow/contrib/slim/python/slim/learning.py", line 655, in train\n    ready_op = tf_variables.report_uninitialized_variables()', 'File "/usr/local/lib/python2.7/site-packages/tensorflow/python/util/tf_should_use.py", line 170, in wrapped\n    return _add_should_use_warning(fn(*args, **kwargs))', 'File "/usr/local/lib/python2.7/site-packages/tensorflow/python/util/tf_should_use.py", line 139, in _add_should_use_warning\n    wrapped = TFShouldUseWarningWrapper(x)', 'File "/usr/local/lib/python2.7/site-packages/tensorflow/python/util/tf_should_use.py", line 96, in __init__\n    stack = [s.strip() for s in traceback.format_stack()]']
==================================

Hi @alberto139, yes, you will need to download the pretrained vgg16 weights, please see the updated readme for more details.

Thanks @LevinJ! I was able to train the model by following the readme.

That sounds great.  @alberto139, do share with me if you find means to push the mAP beyond current 0.65 :)


Implicit dependency on pip

safety version: master, 1.7.0  , 0.5.1
Python version: any
Operating System: any

Description
Somewhat related to https://github.com/pyupio/safety/issues/90 , safety has a dependency on pip which is not declared in the requirements.
Instead requirements_dev.txt has pip==9.0.1
Currently the listed dependencies are

Click>=6.0
requests
packaging
dparse>=0.2.1

dparse master depends on pipenv, which depends on pip, but dparse latest release 0.2.1 does not include pip as a dependency, so on a brand new install of Python, running setup.py results in a missing dependency, unless ensurepip is used.  Not a very likely scenario, but still not desirable.
But more importantly, it is nice to declare dependencies in the package metadata.  Especially when that dependency has forever said that it isnt a library to be imported ;-)

But more importantly, it is nice to declare dependencies in the package metadata. Especially when that dependency has forever said that it isnt a library to be imported ;-)

Totally agree here, also on the relying on pip part. Any recommendations going forward?
Try/catch importing get_installed_distributions should work, but ideally I'd like to drop the dependency on pip all together. Help on that is greatly appreciated.

Well pip wants to be called as a cmdline app, so why not run pip list --some --args to get the desired list and then parse it.  If they dont have a cmd line that does what safety needs, we could ask them to create one.
Then a consideration is which versions of pip are needed to run the required command, and using that as the minimum requirement in safety dependencies.

Yep, that‘s a pretty good idea.

On 19. Mar 2018, at 12:51, John Vandenberg<EMAIL_ADDRESS>wrote:
Well pip wants to be called as a cmdline app, so why not run pip list --some --args to get the desired list and then parse it. If they dont have a cmd line that does what safety needs, we could ask them to create one.
Then a consideration is which versions of pip are needed to run the required command, and using that as the minimum requirement in safety dependencies.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.

pip is now a declared dependency: https://github.com/pyupio/safety/blob/master/setup.py#L21
The transitive pipenv dependency has been removed with dparse 0.4.1.
Calling pip through the commandline is going to be a major foo on windows. I'm going to try/catch the API breakage while continuing using the internal API.
Not ideal, but this is packaging. No one expected this to be easy.


Failure to download dataset

Thanks for publishing this great dataset.
I would like to download the data, but I am having trouble copying it using the Onedrive or Sharepoint functions. Also, I downloaded the files directly, but I can't extract them with 7zip, the files could be corrupted during the download process.
Has anyone been able to solve this problem in a similar situation?

@zinuoli

Hi, the OneDrive link is working fine and the zip files are good too. May you check your Internet environment?

Thank you for your confirmation.
May I have the embedded link for OneDrive download to download with wget?
Also, do you use 7zip to extract the 7z files?

Yes I do use 7zip. Did you download all of the zip files from  01 to 04? These files are connected together and cannot be extracted separately.

I was trying to extract files individually. Thank you very much.

No worries :)


documenting user account behavior during external auth setup

Doc updates related to this thread: https://github.com/rancher/rancher/issues/13684#issuecomment-391816121
General Questions:

Should we recommend making users accounts specifically for external auth authentication?
Before I started working on this issue, there was already a note about AD:  "Create a service account in Active Directory with read-only access. Rancher uses this account to verify group membership when a user makes a request using an API key." Is this note redundant to the accounts I refer to in bullet 1?
@cjellick , you mentioned you can only see GitHub users listed in Rancher if you're signed in as a GitHub user. Is that true for AD as well?

@cjellick , you mentioned you can only see GitHub users listed in Rancher if you're signed in as a GitHub user. Is that true for AD as well?

Yes, true of AD as well.
Regarding your other questions/comments, frankly there is a round of refactoring and enhancements that i think auth really needs that would change some of this. But, i dont know what the proper balance is between documenting as it is now and waiting for the refactor.

@cjellick , can I get one more review on this issue? Denise and I made some pretty big edits and it could use another technical review.


provide a canvas based renderer

The snabbdom based renderer performance is ok when drawing a few graphs. It seems to fall apart at a certain scale. In another project I'm rendering 50 timeline components with data coming in 60 times per second and rendering 500 samples per graph, and the vdom approach seems to choke the performance pretty hard in this case.
I'm thinking a canvas based renderer should offer a big performance gain here.

implemented via 72f7dd3b773a38d9ddaa7cccf210d4f29f71fd95


[BEAM-3063] Improve VoidCoder structural value to use a single shared instance.

Follow this checklist to help us incorporate your contribution quickly and easily:

[ ] Make sure there is a JIRA issue filed for the change (usually before you start working on it).  Trivial changes like typos do not require a JIRA issue.  Your pull request should address just this issue, without pulling in other changes.
[ ] Each commit in the pull request should have a meaningful subject line and body.
[ ] Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue.
[ ] Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
[ ] Run mvn clean verify to make sure basic checks pass. A more thorough check will be performed on your pull request automatically.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

R: @kennknowles

A fitting PR #4000 :-)


corrected spelling for the first three chapters

Thx for contribution. @SimplyDing
Just can't believe someone so hot is doing C++ , so welcome to this repo.


⚠ Empornium has degraded performance

In 3b4d0ca, Empornium ($EMP) experienced degraded performance:

HTTP code: 200
Response time: 1358 ms

Resolved: Empornium performance has improved in 14bab44 after 6 minutes.


Prism is brocken as well

bot_1      | Traceback (most recent call last):
bot_1      |   File "/usr/local/lib/python3.9/site-packages/telegram/ext/utils/promise.py", line 96, in run
bot_1      |     self._result = self.pooled_function(*self.args, **self.kwargs)
bot_1      |   File "/app/bot/mode.py", line 226, in cleanup_wrapper
bot_1      |     return cleanup_inner_wrapper(seconds, remove_cmd, remove_reply, args,
bot_1      |   File "/app/bot/mode.py", line 194, in cleanup_inner_wrapper
bot_1      |     result = func(*args, **kwargs)
bot_1      |   File "/app/bot/skills/prism.py", line 86, in show_top
bot_1      |     words = _eval_filter(default_words, _get_pred(context))
bot_1      |   File "/app/bot/skills/prism.py", line 78, in _eval_filter
bot_1      |     return list(filter(inner_pred, words.copy()))
bot_1      | AttributeError: 'Cursor' object has no attribute 'copy'

Normal'no zhe bylo! With list comprehension

@C-Pro Bylo normalno for /top with a predicats. But for a call /top without agrs words becomes a Curson instead of List

@C-Pro Bylo normalno for /top with a predicats. But for a call /top without agrs words becomes a Curson instead of List

Nu cho ty nachinaesh? Normalno zhe obshalis'


example cross complie

Feature Description

I'd like to compile for a variety of other platforms, on either osx or linux.
I see this in the enviroment variable documentation:
CARGO_MAKE_RUST_TARGET_ARCH - x86, x86_64, arm, etc ... (see rust cfg feature)
CARGO_MAKE_RUST_TARGET_ENV - gnu, msvc, etc ... (see rust cfg feature)
CARGO_MAKE_RUST_TARGET_OS - windows, macos, ios, linux, android, etc ... (see rust cfg feature)

but I'm not sure where the rust cfg feature is... with cargo i can just specify cargo build --target aarch64-apple-darwin for instance, and it looks like I have full control over the cargo build arguments in cargo-make but I'm wondering what the best approach is for supplying the information on the cargo make command line and getting the appropriate environment variables filled in.
Describe The Solution You'd Like
A nice example of how to do this.
Code Sample
cargo make --target aarch64-apple-darwin

i think you got those backwards :)
those env vars are to be read, not to be set by you. cargo-make sets them based on your rust setup.
if you want to target a different arch, you need to update the build task to include the needed args, or use a task that calls cross for example.
for example:
[tasks.build]
args = [
  "build",
  "--release",
  "--all-features",
  "--target",
  "${@}",
]

than you can call it like this:
cargo make build aarch64-apple-darwin

Okay, that makes sense. I have a number of settings that I base on environment variables to determine, for instance, a dylib suffix or replacement.. I guess I can just default based on the default os and architecture but then override if I want to cross compile.
Jul 17, 2021 8:06:43 AM Sagie Gur-Ari @.***>:

i think you got those backwards :)
those env vars are to be read, not to be set by you. cargo-make sets them based on your rust setup.
if you want to target a different arch, you need to update the build task to include the needed args, or use a task that calls cross for example.
for example:
[tasks.build]
args = [
  "build",
  "--release",
  "--all-features",
  "--target",
  "${@}",
]
than you can call it like this:
cargo make build aarch64-apple-darwin
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub[https://github.com/sagiegurari/cargo-make/issues/570#issuecomment-881911491], or unsubscribe[https://github.com/notifications/unsubscribe-auth/AAAQCBH5KTJJDMWNPEADLO3TYGLNLANCNFSM5AMG6QBA]. [data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAD8AAAA/CAYAAABXXxDfAAAAAXNSR0IArs4c6QAAAARzQklUCAgICHwIZIgAAAAmSURBVGiB7cEBDQAAAMKg909tDwcUAAAAAAAAAAAAAAAAAAAAJwY+QwABivJx1AAAAABJRU5ErkJggg==###24x24:true###][Tracking image][https://github.com/notifications/beacon/AAAQCBGPHWHJ5EXBRBNO463TYGLNLA5CNFSM5AMG6QBKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOGSIONQY.gif]


Generate Token on Docker

I'm having issues using node generate_token_v2.js and node generate_token.js using MM running on Docker. When I run node generate_token.js, I get the URL that I can paste on the browser to authorize, but when I paste the code in "Return here with your code." I get:
/opt/magic_mirror/modules/MMM-GooglePhotos/GPhotos.js:104
throw new Error("Error getting tokens:", error);
^
Error: Error getting tokens:
at processTokens (/opt/magic_mirror/modules/MMM-GooglePhotos/GPhotos.js:104:15)
at processTicksAndRejections (node:internal/process/task_queues:96:5)
and when I run node generate_token_v2.js, it just hangs forever and nothing is return.

Any ideas?

I have the same as well

I have the same as well

Any luck? I've tried reinstalling and nothing. Same issue.

As a workaround, I installed PiOS on virtualbox, installed MagicMirror>Google Photo Module and generated a token then copied over to the Docker container. Thus far, working on Docker with token generated on VM.

I've had luck using Windows:

Install Node.js
Open File Explorer and navigate to the MMM-GooglePhotos directory
Right-click in the white area where the files are sitting (don't select anything) and select "open in terminal"
Run "node generate_token_v2.js"
Follow the prompts in the web browser window (might have to give it a few seconds)

i get this error here and on docker :/
Error: Cannot find module 'mkdirp'
Require stack:

C:\Users\Marcus\Desktop\MMM-GooglePhotos\generate_token_v2.js
at Module._resolveFilename (node:internal/modules/cjs/loader:1075:15)
at Module._load (node:internal/modules/cjs/loader:920:27)
at Module.require (node:internal/modules/cjs/loader:1141:19)
at require (node:internal/modules/cjs/helpers:110:18)
at Object. (C:\Users\Marcus\Desktop\MMM-GooglePhotos\generate_token_v2.js:4:20)
at Module._compile (node:internal/modules/cjs/loader:1254:14)
at Module._extensions..js (node:internal/modules/cjs/loader:1308:10)
at Module.load (node:internal/modules/cjs/loader:1117:32)
at Module._load (node:internal/modules/cjs/loader:958:12)
at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12) {
code: 'MODULE_NOT_FOUND',
requireStack: [
'C:\Users\Marcus\Desktop\MMM-GooglePhotos\generate_token_v2.js'
]
}

Node.js v18.16.0

I struggled with this for a while before realizing it doesn't matter where the token is generated. I cloned the repo to my local workstation and ran the token generator there, where it opened the browser and I clicked through. Afterwards I put the resulting token.json into my host-mounted config directory.

i get this error on windows and on docker :/
Error: Cannot find module 'mkdirp' Require stack:

C:\Users\Marcus\Desktop\MMM-GooglePhotos\generate_token_v2.js
at Module._resolveFilename (node:internal/modules/cjs/loader:1075:15)
at Module._load (node:internal/modules/cjs/loader:920:27)
at Module.require (node:internal/modules/cjs/loader:1141:19)
at require (node:internal/modules/cjs/helpers:110:18)
at Object. (C:\Users\Marcus\Desktop\MMM-GooglePhotos\generate_token_v2.js:4:20)
at Module._compile (node:internal/modules/cjs/loader:1254:14)
at Module._extensions..js (node:internal/modules/cjs/loader:1308:10)
at Module.load (node:internal/modules/cjs/loader:1117:32)
at Module._load (node:internal/modules/cjs/loader:958:12)
at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12) {
code: 'MODULE_NOT_FOUND',
requireStack: [
'C:\Users\Marcus\Desktop\MMM-GooglePhotos\generate_token_v2.js'
]
}

Node.js v18.16.0

You need to install dependencies with npm (npm i) or another package manager (yarn, pnpm, bun, etc).

I struggled with this for a while before realizing it doesn't matter where the token is generated. I cloned the repo to my local workstation and ran the token generator there, where it opened the browser and I clicked through. Afterwards I put the resulting token.json into my host-mounted config directory.

Are you doing this every week? Hoping to figure out an automated way to generate new tokens because "publishing" the app in google (as recommended by install.md) seems wildly unrealistic given the requirements to do that in 2024.

I struggled with this for a while before realizing it doesn't matter where the token is generated. I cloned the repo to my local workstation and ran the token generator there, where it opened the browser and I clicked through. Afterwards I put the resulting token.json into my host-mounted config directory.

Are you doing this every week? Hoping to figure out an automated way to generate new tokens because "publishing" the app in google (as recommended by install.md) seems wildly unrealistic given the requirements to do that in 2024.

Is there any reason to keep the OAuth app in testing? For my own version, I have published it.

I struggled with this for a while before realizing it doesn't matter where the token is generated. I cloned the repo to my local workstation and ran the token generator there, where it opened the browser and I clicked through. Afterwards I put the resulting token.json into my host-mounted config directory.

Are you doing this every week? Hoping to figure out an automated way to generate new tokens because "publishing" the app in google (as recommended by install.md) seems wildly unrealistic given the requirements to do that in 2024.

Is there any reason to keep the OAuth app in testing? For my own version, I have published it.

In order to publish it, google says I need to complete all these steps for verification:

You've configured your app in a way that requires verification . To complete verification, you will need to provide:
An official link to your app's Privacy Policy
A YouTube video showing how you plan to use the Google user data you get from scopes
A written explanation telling Google why you need access to sensitive and/or restricted user data
All your domains verified in Google Search Console

Not sure if the issue is with how I configured the application when creating it, but none of these requirements seem reasonable to me given the purpose of this application.

I struggled with this for a while before realizing it doesn't matter where the token is generated. I cloned the repo to my local workstation and ran the token generator there, where it opened the browser and I clicked through. Afterwards I put the resulting token.json into my host-mounted config directory.

Are you doing this every week? Hoping to figure out an automated way to generate new tokens because "publishing" the app in google (as recommended by install.md) seems wildly unrealistic given the requirements to do that in 2024.

Is there any reason to keep the OAuth app in testing? For my own version, I have published it.

In order to publish it, google says I need to complete all these steps for verification:

You've configured your app in a way that requires verification . To complete verification, you will need to provide:
An official link to your app's Privacy Policy
A YouTube video showing how you plan to use the Google user data you get from scopes
A written explanation telling Google why you need access to sensitive and/or restricted user data
All your domains verified in Google Search Console

Not sure if the issue is with how I configured the application when creating it, but none of these requirements seem reasonable to me given the purpose of this application.

It should not require the verification when publishing.

I struggled with this for a while before realizing it doesn't matter where the token is generated. I cloned the repo to my local workstation and ran the token generator there, where it opened the browser and I clicked through. Afterwards I put the resulting token.json into my host-mounted config directory.

Are you doing this every week? Hoping to figure out an automated way to generate new tokens because "publishing" the app in google (as recommended by install.md) seems wildly unrealistic given the requirements to do that in 2024.

Is there any reason to keep the OAuth app in testing? For my own version, I have published it.

In order to publish it, google says I need to complete all these steps for verification:

You've configured your app in a way that requires verification . To complete verification, you will need to provide:
An official link to your app's Privacy Policy
A YouTube video showing how you plan to use the Google user data you get from scopes
A written explanation telling Google why you need access to sensitive and/or restricted user data
All your domains verified in Google Search Console

Not sure if the issue is with how I configured the application when creating it, but none of these requirements seem reasonable to me given the purpose of this application.

It should not require the verification when publishing.

Sounds like I made a mistake when setting up the app in google. What, if any, scopes are required? I selected the relevant Google Photos Library scopes but perhaps that was not necessary and is likely the reason why publishing my app requires verification.

I struggled with this for a while before realizing it doesn't matter where the token is generated. I cloned the repo to my local workstation and ran the token generator there, where it opened the browser and I clicked through. Afterwards I put the resulting token.json into my host-mounted config directory.

Are you doing this every week? Hoping to figure out an automated way to generate new tokens because "publishing" the app in google (as recommended by install.md) seems wildly unrealistic given the requirements to do that in 2024.

Is there any reason to keep the OAuth app in testing? For my own version, I have published it.

In order to publish it, google says I need to complete all these steps for verification:

You've configured your app in a way that requires verification . To complete verification, you will need to provide:
An official link to your app's Privacy Policy
A YouTube video showing how you plan to use the Google user data you get from scopes
A written explanation telling Google why you need access to sensitive and/or restricted user data
All your domains verified in Google Search Console

Not sure if the issue is with how I configured the application when creating it, but none of these requirements seem reasonable to me given the purpose of this application.

It should not require the verification when publishing.

Sounds like I made a mistake when setting up the app in google. What, if any, scopes are required? I selected the relevant Google Photos Library scopes but perhaps that was not necessary and is likely the reason why publishing my app requires verification.

No logo, no app domain, no any scopes

This is likely my issue. I'll try removing these scopes and report back. Thanks for the info!


More Info: To add action details, how to test and warning of overwriting

I am new to Blockchain, and am stuck wondering how to complete the steps.
There are a few issues and questions on this article.
https://docs.microsoft.com/en-gb/azure/blockchain/service/ethereum-logic-app

"From the Actions tab, select one of the available actions."
Please could this be specific on exactly which action? As a beginner, I have no idea which action and how to setup, which affects steps after it.

How to test the section below.
https://docs.microsoft.com/en-gb/azure/blockchain/service/ethereum-logic-app#use-actions
"You can now use your logic app. When the logic app event is triggered, the Ethereum Blockchain action runs. For example, an HTTP request trigger runs an Ethereum blockchain action to query a smart contract state value. This query results in an HTTP response that returns the value."

I even created a separate Logic App for generate-a-workflow and Use Action sections, but the Logic App in Use Action section is not triggered.

Please can you add a warning that the step below will overwrite the existing steps.

https://docs.microsoft.com/en-gb/azure/blockchain/service/ethereum-logic-app#generate-a-workflow
"In your logic app, select Logic app code view. Replace the existing JSON with the generated logic app JSON."
Without a warning, I was trusting the steps in the doc resulting in all previous steps lost.
Sadly, I had wasted un-necessary time because of the missing information on the doc.
[Enter feedback here]

Document details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 7f996344-41b2-7343-b4b3-4cdb0303dce8
Version Independent ID: 5a578d3a-52ad-574c-6a13-8f2db1c9d547
Content: Use Ethereum Blockchain connector with Azure Logic Apps - Azure Blockchain Service - Azure Blockchain
Content Source: articles/blockchain/service/ethereum-logic-app.md
Service: azure-blockchain
GitHub Login: @PatAltimore
Microsoft Alias: patricka

@PingPongSet Thanks for the feedback! We will review and update accordingly.

Hi @pingpongset
Thanks for the feedback. When we write articles, we have to make a decision on if the intent is for a specific scenario (Quickstarts and Tutorials) or more generic (How to). We decided for this article to make it more generic so we could talk about lots of different scenarios. That's why it doesn't always give specifics. We were initially going to write a tutorial but it would have only walked through one scenario. I'm sorry that wasn't clear. I'm adding some text to clarify that this article covers several scenarios and applies to your situation.
We also make an assumption that you're familiar with Azure Logic Apps. I'm adding the following to the prereq section.
If you are new to Azure Logic Apps, consider reviewing the Microsoft Learn modules Introduction to Azure Logic Apps and Call an API from a Logic Apps workflow using a custom connector.
The article section that is most like a walkthrough would be the Generate a workflow section.
Hope that helps,
Pat

Hi @PatAltimore
Thank you for your reply. However, I cannot see the answers to all my three questions on the OP, espcially questions 2 and 3.

Hi @PingPongSet
Question 2:
It depends on your scenario. We don't go into a lot of detail so we can talk about each scenario. It looks like the Ethereum connector is being affected currently by this issue: https://github.com/microsoft/vscode-azure-blockchain-ethereum/issues/57. It looks like the Execute smart contract function won't work until the issue is fixed.
Here are some quick instructions how you could query a blockchain. This is an example using the HelloBlockchain contract.
This is what the logic app would look like in the designer:

Add HTTP request trigger
In Choose an operation, search for Request.
In Triggers, choose When a HTTP request is received.
Add new parameter. Choose Method Select GET.
Select Save.

Query smart contract function
Add a new step.
Select New step.
Choose Ethereum Blockchain.
Choose Query smart contract function.
Follow https://docs.microsoft.com/en-gb/azure/blockchain/service/ethereum-logic-app#create-an-api-connection.
You only need the private key for query. You don't need account address or account password.
Set the function name to RequestMessage

Add response action
Add a new step.
In Choose an operation, search for Response.
Choose Action Response.
Set:

Property
Value
Description

Status code
200
HTTP status code returned.

Body

Add dynamic content. Select Function output

Trigger the Logic App
In a browser, make a request using the HTTP POST URL from the first step. For example, https://prod-26.westus2.logic.azure.com/workflows/c93f03708a6f4ba3ae859dc9617f1814/triggers/manual/paths/invoke?api-version=2016-10-01&sp=%2Ftriggers%2Fmanual%2Frun&sv=1.0&sig=zb1gCRoN8UboZbShmvPCwC6hXo3gK3VbySEN-h5tyWo
If you set the RequestMessage by sending a transaction, you should get a response in the browser.

Question 3
Logic apps are defined using JSON. The Logic Apps Designer represents the JSON graphically. You can switch between the designer view and the code view and make changes in either place. Making a change in the designer view affect the JSON and vice versa. This is by design. The Ethereum dev kit generates the Logic App JSON for you. So when you paste it in, it affects what's in the designer. It's intended to be a starting point for a blockchain logic app.
I hope that helps,
Pat

Hi @PatAltimore,
Thank you for providing the detailed steps.
However, my question 2 is about the step on section User Actions on the link below:
https://docs.microsoft.com/en-gb/azure/blockchain/service/ethereum-logic-app#use-actions
The answer you provided seems about section "Generate a workflow" below, for which I am able to setup.
https://docs.microsoft.com/en-gb/azure/blockchain/service/ethereum-logic-app#generate-a-workflow

Hi @PingPongSet,
I'm sorry. I misunderstood. I thought you were asking for a trigger and action scenario you could try for question 2. The microservices example is easy to explain, show, and grasp. But, it's just one example. Your scenario could be anything. Logic Apps allows you to connect two services together and perform some logic. The trigger could be anything you find in the trigger list. For example, when an email arrives, when a form is submitted, or when you are assigned a GitHub issue, etc. The action can be you calling a blockchain function. So, say you want to store something in the blockchain ledger when a new GitHub issue is created and assigned to you. You could use a logic app to enable the functionality. So, the Use actions section is saying when an event triggers the logic app, the action is performed on the blockchain. In the GitHub example, when a GitHub issue is created and assigned to you, information from the issue is stored on the blockchain ledger.
Pat

Hi @PatAltimore,
FYI, please see my original question 2 below. The problem that I have is that the trigger or the action in Logic App in User Action section are not called.

How to test the section below.
https://docs.microsoft.com/en-gb/azure/blockchain/service/ethereum-logic-app#use-actions
"You can now use your logic app. When the logic app event is triggered, the Ethereum Blockchain action runs. For example, an HTTP request trigger runs an Ethereum blockchain action to query a smart contract state value. This query results in an HTTP response that returns the value."

I even created a separate Logic App for generate-a-workflow and Use Action sections, but the Logic App in Use Action section is NOT triggered.

Please could you tell me why? Is it because of below

It looks like the Ethereum connector is being affected currently by this issue: microsoft/vscode-azure-blockchain-ethereum#57. It looks like the Execute smart contract function won't work until the issue is fixed.

Hi @pingpongset,
It is possible you experiencing the bug. Probably the best way to find out is to troubleshoot your logic app. This article explains how you can diagnose workflow failures: https://docs.microsoft.com/azure/logic-apps/logic-apps-diagnosing-failures.
I set up a simple test using a logic app and the Ethereum connector. Mine failed with the following error:
{
  "error": {
    "code": 500,
    "source": "logic-apis-westus2.azure-apim.net",
    "clientRequestId": "57bc72d3-2438-46ab-be80-1e4ec591f651",
    "message": "BadGateway",
    "innerError": {
      "status": 500,
      "message": "Object reference not set to an instance of an object.\r\nclientRequestId: 57bc72d3-2438-46ab-be80-1e4ec591f651",
      "error": {
        "message": "Object reference not set to an instance of an object."
      },
      "source": "blockchainethereum-wus2.azconn-wus2.p.azurewebsites.net"
    }
  }
}

I'll follow up to see if there's a workaround for the connector until there's a fix.
Thanks,
Pat

@PatAltimore Do we have any update on this issue? Can this issue be closed?

Hi @AshokPeddakotla-MSFT,
I opened and incident for the blockchain connector. It's still active. We can close this issue since it's unlikely the documentation will need to change. I'll add an item to the documentation backlog to verify the docs after the incident is resolved.
Thanks,
Pat
#please-close


Error handling improvement: default error content type to  application/json

Any chance you can add a circleCI build that runs tests on the python FDK?

@zootalures certain amount of tests already in CI job. The only tests we can add here is similar to those what we have in CLI but with an FDK PR branch.


Adding CMD/CTRL+K

What version of starlight are you using?
0.0.0
What is your idea?
Hello,
Any plans to add CMD/CTRL+K support to the searchbar?  As its becoming more common to see on doc sites, would be a nice feature to see supported on Starlight.
Why is this feature necessary?
Allows the user to quickly access the search modal without using the mouse.
Do you have examples of this feature in other projects?
https://docusaurus.io/
https://nextra.site/
Participation

[ ] I am willing to submit a pull request for this issue.

It is already implemented, you can use both of them.

Hello,
My apologies, must have been something on my end as I just tried again and its working.


Proof of concept for Terraform-backed node provider

https://github.com/rancher/rancher/pull/21229


Neoxa balance showing in Watch only mode.

Hello,
I have been mining Neoxa since like 6 months. I have recently lost my laptop and I could not recover the data on my PC. So I bought new PC and I reconnected my Neoxa core wallet through my passphrase and mnemonic, after I can not find my balance in my wallet so I opened debug window from tools section and added my receiving address in which I have been mining. After that I can see my balance but it's not spendable its only watch only mode. Can anyone here help please?

Go to tools>repair wallet>rebuild index.
When still watch only, you have entered incorrect recoverypassphrase.
I would recommend to join discord and ask for help there.
https://discord.gg/hwCyssqC


Refresh connection on connection reset

Hello! Thank you for this library and happy hacktoberfest I guess 🎃
I have encountered an issue where if the connection is reset while you have a long running connection open, unexpected behaviour can start to occur.
import asyncio
from pyle38 import Tile38

client = Tile38("redis://localhost:9851")

async def main():
    while True:
        try:
            print(await client.command("SET", ["fleet", "truck", "POINT", 1, 1]))
        except Exception as e:
            print(e)
        await asyncio.sleep(1)

asyncio.run(main())

Running the above and restarting the Docker container running Tile38 while it's running produces output as such
{'ok': True, 'elapsed': '202.166µs'}
{'ok': True, 'elapsed': '22.083µs'}
{'ok': True, 'elapsed': '33.25µs'}
Connection closed by server.
Error UNKNOWN while writing to socket. Connection lost.
Expecting value: line 1 column 1 (char 0)
Expecting value: line 1 column 1 (char 0)
Expecting value: line 1 column 1 (char 0)

The Redis object will throw a redis.ConnectionError if it attempts to use a closed connection the next time it is used. On the subsequent call everything will be fine. However, the OUTPUT JSON command has been lost!
From the output above you can see the errors are thrown by redis-py - then when the connection is available again the response is unable to be parsed as it's just "OK".
To illustrate in a different way:
await client.command("SET", ["fleet", "truck", "POINT", 1, 1])  # ok
# Restart Tile38 and wait for it to be available again
await client.command("SET", ["fleet", "truck", "POINT", 1, 1])  # ConnectionError
await client.command("SET", ["fleet", "truck", "POINT", 1, 1])  # JSON parse error

The time between the first command and the second command doesn't matter. If you restart the Tile38 server and wait for 1 minute the second call will still be a connection error - at which point the connection internal to Redis is reset.

This provides a couple of potential fixes.
First, it enables retry_on_error for the Redis client. This should address the ConnectionError when making a request after a dropped connection. I've set this to Retry(ExponentialBackoff(), 10) which is about 3 seconds.
Second, if there's a parse error in parse_response retry the call and reset the format. For this I added some basic retry logic as I didn't want to add a new dependency for this one use case.
I've added a test that addresses the parse issue after a connection reset, but the ConnectionError is hard to test as it relies on dropping the connection outside of the client.

Hi Alex! Thank you for the PR and the detailed description. I’ll try to take a look as soon as I can.

I think I was able to identify the initial issue and it does not come down to adding a retry strategy (just yet).
The singleton makes sure to retry to the connection if it is dropped, and the __on_connect method is supposed to reset the format. However the latter has a pesky typo resetting self.format instead of self.__format which leads to the behaviour you describe above. Fixing this typo, fixes the issue you're outlining.
I will address this in a separate commit, but will add you as co author for raising it, if that sits ok with you. That said I like the idea of configuring custom retry behaviour but would prefer this to happen in a separate PR/commit. We could add a subset of the connection options as a param on the init of the client that would allow a user to config this to some degree. Wdyt? @alxwrd

Ah, I see that now! I thought the __on_connect method wasn't being called on disconnect hence I added the retry.
Yes that works for me. I'm also happy to update this branch to remove the retry mechanism and add in the .__format fix.
In fact, I'll do that - and you can either merge this or make a separate commit 🙂

v0.13.1 is released, thank you very much for your effort, means a lot to me, not gonna lie. :)
Make sure to join Tile38 Discord as we're trying to move away from Slack. :)

@iwpnd unfortunately the test I wrote still fails after the .format → .__format fix.
With format fix
await client.command("SET", ["fleet", "truck", "POINT", 1, 1])  # ok
# Restart Tile38 and wait for it to be available again
await client.command("SET", ["fleet", "truck", "POINT", 1, 1])  # ConnectionError
await client.command("SET", ["fleet", "truck", "POINT", 1, 1])  # JSON parse error
await client.command("SET", ["fleet", "truck", "POINT", 1, 1])  # ok

With retry and no format fix
await client.command("SET", ["fleet", "truck", "POINT", 1, 1])  # ok
# Restart Tile38 and wait for it to be available again
await client.command("SET", ["fleet", "truck", "POINT", 1, 1])  # JSON parse error
await client.command("SET", ["fleet", "truck", "POINT", 1, 1])  # JSON parse error
await client.command("SET", ["fleet", "truck", "POINT", 1, 1])  # JSON parse error

With retry and format fix
await client.command("SET", ["fleet", "truck", "POINT", 1, 1])  # ok
# Restart Tile38 and wait for it to be available again
await client.command("SET", ["fleet", "truck", "POINT", 1, 1])  # JSON parse error
await client.command("SET", ["fleet", "truck", "POINT", 1, 1])  # ok

I won't raise the PR, but you can see a diff for a potential fix here: https://github.com/iwpnd/pyle38/compare/main...alxwrd:pyle38:reset-output

This is two issues:

is reconnecting if there's a connection loss

is reusing an instance of client when we call quit or aclose on the underlying redis client.

is fixed with v0.13.1

is a matter of resetting self.__format to Format.RESP on .quit()

What I will not address is the user calling .aclose() (like in your test) on the embedded redis client directly. Doing so is leaving the scope of this library and moves control of the connection elsewhere.

Sorry, I should have been clear, I do not mean a user should or would call .aclose() on the internal redis client, but the connection might be dropped outside of the code (network issues, database restart). I'm unsure how else to demonstate this in a test.
In such case the first call to .command() throws a ConnectionError from redis-py and the internally redis-py calls .disconnect() on it's connection.
The second .command() calls .force_json() first, which is a no-op because the connection hasn't been reset yet - meaning the response from Tile38 is not JSON. However, because Redis.execute_command() was called, the connection was recreated and .__on_connect is called.

So your concern is not that it eventually recovers but that the first command is a RESP response and you'd loose it?

Eventually recovering is nice, but I believe it could recover on the first call.
In this scenario nothing is wrong - the connection was just dropped briefly.

I believe setting the OUTPUT in .__on_connect() would solve this problem: see this diff.
Of course this is working with the raw connection - and it might be desirable to check the value of connection.read_response().

Hm, now I see what you mean. :thinking:
There's a reason I'm tracking the state of the Format in the client. But I cannot remember whether I do so because the on_connect was added later on and could replace tracking the Format OR whether there's another reason for having it this way. I give it some more thought :)

At first I tried calling Redis.execute_command() in the on connect handler, but it just hangs (it may even eventually fail with a recussion error, I didn't wait long enough!) - so maybe this was the reason.
Using the connection directly works, but the response should probably be checked! I would add this, but I'm not sure what the expected action would be? Raise an error?


Doesnt work with 4.2

sys/db/RecordMacros.hx:802: characters 12-14 : Unmatched patterns: OpSpread

Should be fixed now (#59).


Symlink to mainnet-genesis.json

Currently we have to create a symlink to mainnet-genesis.json in the root of cardano-explorer repository. Without it we stuck with an error:
$ ./launch_mainnet_exporter
...
cardano-explorer-node: mainnet-genesis.json: openBinaryFile: does not exist (No such file or directory)

To clarify, ouroboros-network should not expect a mainnet-genesis.json in dependency projects. Those dependency projects should either provide a genesis block (such as cardano-node) or just the genesis-hash (in the case of cardano-explorer).

Is this issue even related to ouroboros-network? Did you mean to create this issue for cardano-explorer?


Added incremental build output

We pass converter state to client side
Client side when next time requests build output sends last received converter state
Converter state is restored and generation is resumed from that place
All span's are properly restored
Currently there is simple check if build changed: offset present in state is within currently saved build output. This can be enhanced to introduce some simple checksum: CRC16 or CRC32 should be sufficient.

It's made to greatly reduce network traffic and increase performance of html generation.
THIS IS PROOF OF CONCEPT HOW IT CAN BE DONE

it is too oudated.

@vsizov but what do you think about the concept?


UICollectionView

Awfull gap at the top of the CollectionView It does work correctly on the tableview.

Hope that will not have this issue anymore. Thanks


Optimize "Shared Functions"

For objects (i.e. PIXI.Audio sprites) that all share a similar function, Zephyr should a provide only provide a link to a common function, not an entirely new function. This should save on memory and speed.

Solved


All projects: Add "all tests must extend base test" check

This PR added 'must extend base test class' checks to nd4j-tests and deeplearning4j-core.
https://github.com/KonduitAI/deeplearning4j/pull/290
We need the same thing in all other modules.
Why we need all tests extending the base test:

Test logging (helps debug when crashes occur on CI - can be difficult or impossible to know exactly which class, without extended logging)
Memory logging (helps debug memory issues such as leaks)
Timeouts by default
Reset environment (ensure any profiling config, default datatypes etc changes don't impact any other tests)

Important: The existing AssertTestsExtendBaseClass should be refactored so the core code is in deeplearning4j-common-tests and nd4j-common-tests (instead of copy/paste everything); then the AssertTestsExtendBaseClass that's copy/pasted into every module should simply look like:
    @Test
    public void checkTestClasses(){
        int count = SomeUtilMethod.checkTestClasses();
        assertEquals("Number of tests not extending BaseDL4JTest", 0, count);
    }

https://github.com/KonduitAI/deeplearning4j/pull/367


[JOB-STORY] Make 4 Security Level Configuration available (0, 1, 2, 3) complying with IEC 62443.

Related to #120
Job Story
When configuring the Trustpoint, I want to to be able to select a Target Security Level, so I can make sure, I'm not using any features of the Trustpoint that could or would undermine that Target Security Level.
Acceptance Criteria

[x] Add the Security Level Configuration to the Trustpoint
[x] Add Target Security Level 0 (Development / Debug)
[x] Add Target Security Level 1 (Low)
[x] Add Target Security Level 2 (Medium)
[x] Add Target Security Level 3 (High)
[x] Add Target Security Level 3 (Very High)
[x] Following the secure by default methodology, the default setting shall be Target Security Level 1

Note
The Acceptance Criteria still need refinement / discussion.

Note: Target security level 1 is called "Basic", otherwise this is fully implemented.


Add instruction for setting the CI password via the command line

The command would be something along the lines of
sudo qm set 8001 --cipassword HASH_PASSWORD

But I don't know how to generate a hashed password.

this should work:
qm set 8001 --cipassword $( openssl passwd -6 $CLUSTER_PASSWORD )


Problem  prover tx

Describe the bug
After sync node.
I tried sending me 500 tokens at a time.
1 time he proposed and proved both tx.
While 2 or 3 times it only provided 1 tx and started giving this error: ERROR[05-03|20:28:21.708] Failed to request proof height=82196 error="Post "localhost:8080": unsupported protocol scheme "localhost"" endpoint=localhost:8080
I have raiko on the same machine. https://holesky.etherscan.io/address/0x8fC62f005CCbb73Eec39a7F23891cEAd2988A284
Steps to reproduce
No response
Spam policy

[X] I verify that this issue is NOT SPAM and understand SPAM issues will be closed and reported to GitHub, resulting in ACCOUNT TERMINATION.

@lucajj3 are you using the most updated version of simple-taiko-node? You might have misconfigured the SGX_RAIKO_HOST in your .env

is the last version.
SGX_RAIKO_HOST=localhost:8080
I also had these errors:
taiko_client_prover_relayer-1  | WARN [05-03|10:32:02.679] nonce too low                            service=prover tx=795f6f..1b09e5 nonce=11 gasTipCap=1,000,000,000 gasFeeCap=5,625,646,650 gasLimit=3,000,000 err="nonce too low: next nonce 12, tx nonce 11"
WARN [05-03|10:32:50.844] nonce too low                            service=prover tx=512046..0f72f5 nonce=11 gasTipCap=1,100,000,000 gasFeeCap=6,188,211,315 gasLimit=258,096 err="nonce too low: next nonce 13, tx nonce 11"
today however with the arrival of the 1000 tokens, everything went perfectly

Hello @lucajj3 you can't use localhost or <IP_ADDRESS>. The prover container will know localhost or <IP_ADDRESS> is itself IP address not raiko
Try with your http://internal IP:8080 or http://public_ip:8080
Make sure you can connect that IP from prover container .
You can also try using http://host.docker.internal:8080

Hello @lucajj3 you can't use localhost or <IP_ADDRESS>. The prover container will know localhost or <IP_ADDRESS> is itself IP address not raiko
Try with your http://internal IP:8080 or http://public_ip:8080
Make sure you can connect that IP from prover container . You can also try using http://host.docker.internal:8080

now i have edited my .env with SGX_RAIKO_HOST=http://XXX.XX.0.1:8080
now i have this error:
ERROR[05-07|17:46:57.024] Failed to request proof   height=101,437 error="task 1295 panicked" endpoint=http://XXX.XX.0.1:8080

Hello @lucajj3 you can't use localhost or <IP_ADDRESS>. The prover container will know localhost or <IP_ADDRESS> is itself IP address not raiko
Try with your http://internal IP:8080 or http://public_ip:8080
Make sure you can connect that IP from prover container . You can also try using http://host.docker.internal:8080

now i have edited my .env with SGX_RAIKO_HOST=http://XXX.XX.0.1:8080
now i have this error: ERROR[05-07|17:46:57.024] Failed to request proof height=101,437 error="task 1295 panicked" endpoint=http://XXX.XX.0.1:8080
ERROR[05-07|17:50:38.025] Failed to request proof height=101,437 error="task 1339 panicked" endpoint=http://XXX.XX.0.1:8080

this looks like a raiko error, @smtmfft might be able to help; are there any other logs?

Hello @lucajj3 you can't use localhost or <IP_ADDRESS>. The prover container will know localhost or <IP_ADDRESS> is itself IP address not raiko
Try with your http://internal IP:8080 or http://public_ip:8080
Make sure you can connect that IP from prover container . You can also try using http://host.docker.internal:8080

now i have edited my .env with SGX_RAIKO_HOST=http://XXX.XX.0.1:8080
now i have this error: ERROR[05-07|17:46:57.024] Failed to request proof height=101,437 error="task 1295 panicked" endpoint=http://XXX.XX.0.1:8080
ERROR[05-07|17:50:38.025] Failed to request proof height=101,437 error="task 1339 panicked" endpoint=http://XXX.XX.0.1:8080

this looks like a raiko error, @smtmfft might be able to help; are there any other logs?

@RogerLamTd just these, tell me if you have other commands or files to look for them

Hello @lucajj3 you can't use localhost or <IP_ADDRESS>. The prover container will know localhost or <IP_ADDRESS> is itself IP address not raiko
Try with your http://internal IP:8080 or http://public_ip:8080
Make sure you can connect that IP from prover container . You can also try using http://host.docker.internal:8080

now i have edited my .env with SGX_RAIKO_HOST=http://XXX.XX.0.1:8080
now i have this error: ERROR[05-07|17:46:57.024] Failed to request proof height=101,437 error="task 1295 panicked" endpoint=http://XXX.XX.0.1:8080
ERROR[05-07|17:50:38.025] Failed to request proof height=101,437 error="task 1339 panicked" endpoint=http://XXX.XX.0.1:8080

this looks like a raiko error, @smtmfft might be able to help; are there any other logs?

@RogerLamTd just these, tell me if you have other commands or files to look for them

could you check the logs of your raiko container? docker compose logs -f raiko or something like that

Is your docker built from latest alpha7? if not, you could update and try again.
The prover panic has lots of reasons, and we are refining it. however, some high likely ones are: 1. bootstrap failure due to unsuccessful file loading, 2. unstable network makes the data fetching problematic. 3. a critical bug.
Also you can run prove_block.sh with like prove_block taiko_a7 sgx 66666 to see if prover itself works. but remember to re-register the bootstrap as the script will refresh it. or you can edit the script to disable the refresh by set
"setup": false,
"bootstrap": false,

Enjoy the play :)

Is your docker built from latest alpha7? if not, you could update and try again. The prover panic has lots of reasons, and we are refining it. however, some high likely ones are: 1. bootstrap failure due to unsuccessful file loading, 2. unstable network makes the data fetching problematic. 3. a critical bug. Also you can run prove_block.sh with like prove_block taiko_a7 sgx 66666 to see if prover itself works. but remember to re-register the bootstrap as the script will refresh it. or you can edit the script to disable the refresh by set
"setup": false,
"bootstrap": false,

Enjoy the play :)

I reinstalled the entire vps from 0. everything updated. Holensky knot mine. all on the same machine. first proposal and proved is ok without  errors. the second proved tx still gives me this error
ERROR[05-11|17:25:05.430] Failed to request proof                  height=125,808 error="task 64 panicked" endpoint=http://myexternalip:8080

height=125,808 error="task 64 panicked

Any other info besides panic? we just updated the error report, now it should show the error log together with the panic,

height=125,808 error="task 64 panicked

Any other info besides panic? we just updated the error report, now it should show the error log together with the panic,

these are the errors it gives me now. I had everything installed 7 days ago.
raikolog.txt

Closed as stale


Footer

I need a brand new footer.
I want you to implement your amazing UI skills and make a brand new responsive footer !!
The footer must be of dark color (not too dark)
Add the footer ONLY to the homepage

I want to work on this issue under JWOC

Assigned to you @KushalGhosh9899

I want to work on this issue under JWOC


fix: 谷歌新加region翻译

这个 PR 实现什么功能/修复什么问题:
谷歌新加region翻译
是否需要 backport 到之前的 release 分支:

release/2.13

/area region
/cc @swordqiu

Codecov Report

Merging #5117 into master will decrease coverage by <.01%.
The diff coverage is n/a.

@@            Coverage Diff            @@
##           master   #5117      +/-   ##
=========================================
- Coverage    6.75%   6.74%   -0.01%
=========================================
  Files         546     546
  Lines       95275   95275
=========================================
- Hits         6433    6431       -2
- Misses      88304   88305       +1
- Partials      538     539       +1

Flag
Coverage Δ

#aFlag
6.74% <ø> (-0.01%)
:arrow_down:

Impacted Files
Coverage Δ

pkg/appsrv/response.go
66.15% <0%> (-3.08%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e8c445f...e1e2160. Read the comment docs.

/lgtm

/approve


Update animated-list-demo.tsx

The Items Are installed in components/ui folder not components/magicui Folder

This is related with #430


Reorder CMake architecture detection

Fix bug to allow cross compiling using clang for targeting arm64. Previously when cross compiling using clang with the aarch64-linux-gnu triple, the build would fail due to the -march=armv7-a option being unsupported with the aarch64-linux-gnu triple.
Change-Id: Ib854997886c257efd7a5ad308533782081ca5900

Hi @cmacdonald-arm, thank you for your change, it looks good to me. I saw there was a recent change from @harishdm and it may be targeting the same issue and it's already merged. Could you try if the latest library still has the same problem? Thanks!

@DichenZhang1 this change is still needed as arm was getting preferred over aarch64.
@cmacdonald-arm can you please rebase this change on the latest commit (as these lines changed in a recent commit)?
Thank you.


Concerns about the removal of batch manager C++ API

System Info
Hello, in the latest release you say:

The C++ batch manager API is deprecated in favor of the C++ executor API, and it will be removed in a future release of TensorRT-LLM.

However, the C++ executor API is not as flexible -- for example, it's not possible to implement custom token sampling algorithms such as MinP or structured decoding parametrized by user provided schema or grammar. See here for details: https://github.com/NVIDIA/TensorRT-LLM/issues/2044
Can you please share your timeline for removal and whether you plan to bring the C++ executor API on par with the batch manager?
Who can help?
@bys
Information

[X] The official example scripts
[ ] My own modified scripts

Tasks

[X] An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
[ ] My own task or dataset (give details below)

Reproduction
N/A
Expected behavior
N/A
actual behavior
N/A
additional notes
N/A

I also am very concerned about this change, which would be easier to deal with if the executor API allowed you to set the requestId, which used to be possible in the batchmanager API
The fact that you cannot set this means it is hard to keep track of which requests match which responses, without keeping a distributed map across MPI if you are using multiple processes ... which is a huge amount of work.
We would then also have to additionally set up this mapping across all of MPI nodes for the logits post-processing etc, so we can look up request details.

It looks like the executor API now lets you specify clientId that is then passed to the logit post processor as well.

Logits processor runs only on one MPI node in latest version.
you can use LogitsPostProcessorConfig(replicate=True) if you want processor to run all tensor parallel ranks.

If I change logits at rank 0 in the logit processor, do I need to do the same logit transformation at other ranks? https://github.com/NVIDIA/TensorRT-LLM/issues/1415

there is no need for logits transformation on other ranks.
logits callback will be invoked only in first tensor parallel rank


Fullscreen Application Examples

Hi jonathan,
Thanks for the wonderful library! Currently I'm trying to build some console-based UI which should goes like this
┌──────────────────────────────────────────────────────┐
│                                                      │
│           ( displaying some text and                 │
│               maybe some ascii graph  )              │
│                                                      │
├────────────────────────────┬─────────────────────────┤
│                            │                         │
│  (a real terminal like     │    ( a custom shell     │
│      you did in pymux)     │        like ptpython )  │
│                            │                         │
└────────────────────────────┴─────────────────────────┘

But right now I have got no idea on how to:

embed a real termial to a fullscreen application
embed a custom prompt to a fullscreen application
in to this full screen prompt_tool kit app.

I've play around with the example/full-screen-layout.py and other examples, yet finding no clue on archiving my goals. Can you add some examples on fullscreen application with prompt()?
BTW, I've read the pymux for couple of hours but getting lost in Pane, Window, Clients... Is there any early code in pymux's development, which is only about launching a shell under the control of prompt_toolkit?

Hi @zhfuzzy,
Thanks for the interesting question!
This is possible, however not trivial. In order to embed a terminal, there's a lot to do, and I don't want to suggest that the way to do it, is to reproduce code from pymux, because that will be difficult to maintain, especially without a deep understanding of how pseudoterminals and many posix concepts work. (I made it too complex, unfortunately.)
The problem is mostly that prompt-toolkit was in the first place designed for creating simple input lines, like GNU Readline. Later, prompt-toolkit became much more powerful; however some parts of the internal API reflects that it it was not meant to create full screen applications. One example is that key bindings are defined globally, and not per UI widget. Further, the focus is defined by a "Buffer" data structure that has the focus, again instead of a UI widget. This is very inflexible for full screen applications as they become bigger. It also makes it impossible to create a reusable "terminal" widget that can be embedded in other applications.
I'm very aware of this and started working on a branch that refactors the API for full screen applications to make this as easy as it is for creating simple prompts. There's still a lot of work to be done. For me, this is a priority, but it can easily take 6 more months.
So, unfortunately, the answer is that right now, it is possible, but very difficult. And I don't have the time to support custom full screen applications built on the current API. But I very much believe that in about a year, this should be super easy. I hope you understand.
[1] https://github.com/jonathanslenders/python-prompt-toolkit/tree/full-screen-widgets

I understand that prompt_toolkit, by it's name, is not meant to be used as a fullscreen terminal framework, so thanks for the effort you made in extending it.
I'm trying to understand pseudo-terminals and some prompt_toolkit's internal by cutting pymux codebase into small pieces and stepping through the code base, but still, if there are any articles and experiment code you used in the early stage of pymux's development, you may put them in the documentation of pymux for new comers to get started.


Bluetooth connection - which device ?

I'm trying to set up a connection with the Daly BMS over Bluetooth. But I can't find any documentation regarding which -d parameter to use to do this.
Does anyone know the simple steps ?
I've seen some complex scripts from a different library, but those all require additional packages that pip can't seem to find...

Hello! I am facing the same issue here. Does anyone have any relevant instruction to provide?

Hello, i have the same question, the BL connection is not possible. I tried to understand how the script is going from the "daly-bms-cli"  to "daly_bms_bluetooth.py" but i haven't found any connection. The cli command doesnt accept the mac_address as device and also it doesn't accept if the device is binded via "sudo rfcomm bind 0 XX:XX:XX:XX:XX:XX" as "/dev/rfcomm0". So it's not possible to connect via bluetooth by typing:

daly-bms-cli -d /dev/rfcomm0 --uart --soc
daly-bms-cli -d XX:XX:XX:XX:XX:XX --uart --soc

The CLI tool doesn't have the functionality to connect via Bluetooth implemented yet, only the Python module does. If you're interested in using the Python module to create a small tool for your purpose, check out the other Bluetooth related issues.


Add timeout

Default timeout is set to 30 seconds.

Codecov Report
Base: 92.05% // Head: 84.13% // Decreases project coverage by -7.91% :warning:

Coverage data is based on head (771d5d9) compared to base (73fad7d).
Patch coverage: 10.52% of modified lines in pull request are covered.

:exclamation: Current head 771d5d9 differs from pull request most recent head 4e62685. Consider uploading reports for the commit 4e62685 to get more accurate results

Additional details and impacted files
@@            Coverage Diff             @@
##           master       #4      +/-   ##
==========================================
- Coverage   92.05%   84.13%   -7.92%
==========================================
  Files           2        2
  Lines         667      353     -314
==========================================
- Hits          614      297     -317
- Misses         53       56       +3

Impacted Files
Coverage Δ

src/error/mod.rs
0.00% <0.00%> (ø)

src/lib.rs
85.10% <13.33%> (-1.16%)
:arrow_down:

tests/unit_tests.rs

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

TODO: bump project version


Qualification Tool to output only the 'k' highest-ranked or 'k' lowest-ranked applications

This is adding support to the Qualification tool to output only the 'k' highest-ranked or 'k' lowest-ranked applications.
This is done with 2 different configs, 1 existing one was for numOutputRows, which was changed slightly to only affect number of output rows in the report text file and the other 1 is a new config called --order <asc|desc> which affects the sort order of the report.
with those 2 configs you can accomplish the  'k' highest-ranked or 'k' lowest-ranked applications.

build


[Filestore] Invalid CollectGarbage requests to blobstorage.

Errors like following stared causing IndexTablet to restart
NFS_SERVER[578573]: 2024-03-05T15:06:36.674043Z :NFS_TABLET ERROR: [f:***][t:***] CollectGarbage failed: SEVERITY_ERROR | FACILITY_KIKIMR | 1 Processed status# ERROR from VDisk# [8200021d:2:0:0:0] incarnationGuid# empty QuorumTracker status# ERROR

Looks like CollectGarbage requests sent by TIndexTablet does not guarantee the increasing order of (gen, step)
Started seeing this error much more often after enabling vhost-side reads on the whole cluster

CollectGarbage is executed with commitId = GetCurrentCommitId() = 42
Cleanup is started. It acquires a collect barrier with commitId = 42:

https://github.com/ydb-platform/nbs/blob/0555c7db24e936c1d359ead9087d2e24b9229d84/cloud/filestore/libs/storage/tablet/tablet_actor_cleanup.cpp#L68-L72
https://github.com/ydb-platform/nbs/blob/0555c7db24e936c1d359ead9087d2e24b9229d84/cloud/filestore/libs/storage/tablet/tablet_actor_cleanup.cpp#L100

Before the collect barrier is released on completing the Cleanup transaction, another CollectGarbage is exeсuted. CollectCommitId is selected as follows:

https://github.com/ydb-platform/nbs/blob/0555c7db24e936c1d359ead9087d2e24b9229d84/cloud/filestore/libs/storage/tablet/tablet_state_data.cpp#L935-L939

GarbageQueue.GetCollectCommitId():

https://github.com/ydb-platform/nbs/blob/0555c7db24e936c1d359ead9087d2e24b9229d84/cloud/filestore/libs/storage/tablet/model/garbage_queue.cpp#L224-L232

There is an unreleased collect barrier with commitId = 42, thus CollectCommitId will be equal to 41, which is less than LastCollectCommitId

Generating a new CommitId on the Cleanup execution will solve the issue

The main problem is that FlushBytes acquires collect barrier, which is less than the last collect commit id:

Consider that there were the following sequence of writes:

Write(0,       256 KiB, 'a') -> Blob(commitId = 42)
Write(256 KiB, 256 KiB, 'b') -> Blob(commitId = 43)
Write(512 KiB, 1,       'f') -> FreshBytes(commitId = 44)
Write(0,       256 KiB, 'c') -> Blob(commitId = 45)

This will lead to the following file layout: [ccccccc][bbbbbbb][f]

After execution of the CollectGarbage, all three new blobs will get a KeepFlag and the last collect commit id will be equal to 44

CommitId:     41      42        43       44
            Blob(a) Blob(b) FreshBytes Blob(c)
                                          |
                                 LastCollectCommitId

After execution of the Cleanup operation, the first blob will be marked as garbage

Let us execute FlushBytes operation, It will acquire collect barrier, equal to the minimal commitId, associated with FreshBlobs:
https://github.com/ydb-platform/nbs/blob/836a5162f99b998582dc4b476a212619954bfa22/cloud/filestore/libs/storage/tablet/tablet_actor_flush_bytes.cpp#L669-L671

After this acquisition there will be one barrier, equal to 43:
CommitId:     41      42        43       44
            Blob(a) Blob(b) FreshBytes Blob(c)
                                |         |
                             Barrier  LastCollectCommitId

When the next CollectGarbage operation is to be executed, it will choose 42 as a collectCommitId:

https://github.com/ydb-platform/nbs/blob/836a5162f99b998582dc4b476a212619954bfa22/cloud/filestore/libs/storage/tablet/model/garbage_queue.cpp#L227-L228
After it the CollectGarbage request with one new grabage will be sent, leading to a decrease in collectCommitIds sequence: 42 after 44

To reproduce the issue, one can use fio:
fio --name=random-write-test \
    --ioengine=libaio \
    --rw=randwrite \
    --bs=512-4k \
    --size=1G \
    --direct=1 \
    --iodepth=16 \
    --numjobs=4 \
    --offset_increment=512 \
    --do_verify=0 \
    --time_based \
    --runtime=$[120*60*60]

AppCriticalEvents/CollectGarbageError errors after starting afformentioned fio:

AppCriticalEvents/CollectGarbageError errors after deploying fix #1919:

AppCriticalEvents/CollectGarbageError after release to our production cluster:


On work edit page, display the pre-assigned collection-level keywords.

The user should see a list of keywords assigned at the collection level just above the keyword entry box. The collection-level keywords should not be editable or removable.
https://projects.invisionapp.com/share/EQXC9CLKCR2#/screens/454558560

Originally part of #1738.

This is no longer a priority, so I'm closing this ticket.


Change credentials directory to /app from /root

the matter server requires the credentials folder to be located in the working directory of the server

Did you successfully test commissioning with these changes?

Yeah before it gave an error when trying to commission any device cuz it was unable to find the certs