get_note from a string (or an example in general)

I'm sorry if this is an obvious question and not even really a bug report but there must be something I'm missing (especially because I don't use misskey).
I'm making a site that generates embeds for Mastodon, Pleroma, and Misskey, for Discord, and Misskey could very much benefit from it seeing as its embeds don't even show the post contents at all. The library I chose doesn't seem to support misskey and it would be a large undertaking for it to do so, and an equally large undertaking would be dealing with ActivityPub which would seemingly be just as hard as just supporting the sites individually, so i'd like to use this in conjunction with it.
What I'm missing is:
If I have https://misskey.bubbletea.dev/notes/8s27fg0gok, 8s27fg0gok being the id, and the usage for get_note is get_note(&self, Id<Post>), and all the implementations for Id can only have numbers fields...how am I supposed to get the note from that id?
If nothing else it would be nice to see an example for get_note in the docs.

Thank you for the feedback. You can use FromStr implementation of Id<T> to get Id<T> from string. In other words, you can use parse:
let note = client.get_note("8s27fg0gok".parse()?).await?;
println!("{}", note.text.unwrap());

An example of get_note would be added in #51.


missing shortcuts

When I chat, i like to stay on the keyboard and do not have to switch between mouse and keyboard.
Are there some shortcuts that can make live easier? f.e. Ctrl+F to search in the user list to switch to a option private chat or create a new chat?
Your Rocket.Chat version: 1.3.1

---
Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/34973111-missing-shortcuts?utm_campaign=plugin&utm_content=tracker%2F19476668&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F19476668&utm_medium=issues&utm_source=github).

ok. i got at least one shortcut: with Ctrl+P i can search for users.
Would be nice to have a list of all shortcuts somewhere.... can someone help me?

ok, got it: https://rocket.chat/docs/user-guides/keyboard-shortcuts/
i'll close this issue. more shortcuts whould be nice...


ci: Added doc linting rules to ensure google style docstrings.

We have specified google style docstring in the pyproject.toml, but the rule is not enforced. This PR enforces that.
Many of the changes are just autofixes by ruff.

We saw the tmp/results folder before in https://github.com/embeddings-benchmark/mteb/pull/442. Is this intended?

No I have removed it. I have also removed it locally so it does not happen again


additional documentation fixes (follow-up to #322)

Here are some documentation improvements that I failed to get ready in time to piggyback #322.
Align options with descriptions if there is space for it.
Add some missing section numbers.
Add mention that sudo wildcarding aur might run aur- prefixed scripts from $PATH
Add some missed escapes for -
Fix inconsistency between manpage formatting of arguments inside square brackets

Thanks!


Is it possible to get a complete go implementation by using golang call rust?

Hello, I know there is an incomplete implementation of blakes by golang, well it doesn't support mutli-thread, nor full interfaces. I' am wondering if I can implement blake3 of golang by using golang to call rust, with FFI or CGO. I am going to use BLAKE3 in a golang project receiving file stream. I am not familiar with rust, Is there any difficulty to do that?

There are excellent, pure Go implementations:

https://github.com/lukechampine/blake3
https://github.com/zeebo/blake3

Agreed with Frank, both of those implementations are mature and full-featured, and I'd encourage any Go application to use them. That said, as far as I know, they don't support multithreaded/multi-goroutine hashing. (Neither does the official C implementation, for what it's worth.) If that was a requirement for you, and you didn't want to just shell out to b3sum, it is possible to call the Rust implementation from Go. This requires both unsafe Go and unsafe Rust, and I wouldn't suggest doing it in important code unless you were comfortable with both languages.
Here's a demo of how you stitch it together: https://github.com/oconnor663/go_calling_rust_example. The high level steps are:

expose one or more extern "C" functions from Rust that wrap the parts of the blake3 crate you want to use
add declarations for those functions in a cgo block
configure the Rust code to build a staticlib
use #cgo LDFLAGS to make Go link against that staticlib

Thanks frank! I know one of the go implemention like zeebo.Well both implementions can't support multithreaded. But I need to use blake3 to process relatively big data file and I have enough cpu cores in my server. I will make a test for the performance difference between rust multi-threaded and the two go implementions.

Agreed with Frank, both of those implementations are mature and full-featured, and I'd encourage any Go application to use them. That said, as far as I know, they don't support multithreaded/multi-goroutine hashing. (Neither does our official C implementation, for what it's worth.) If that's a requirement for you, and you can't just shell out to b3sum, it is possible to call the Rust implementation from Go. This requires both unsafe Go and unsafe Rust, and I wouldn't suggest doing it in important code unless you were comfortable with both languages.
Here's a demo of how you stitch it together: https://github.com/oconnor663/go_calling_rust_example. The high level steps are:

expose one or more extern "C" functions from Rust that wrap the parts of the blake3 crate you want to use
add C declarations for those functions in a cgo block
configure the Rust code to build a staticlib
use #cgo LDFLAGS to make Go link against that staticlib

So yes, this is quite involved, and I'm not entirely sure I've even done it right. Please be careful with it :)

Thank you Jack for your patient detail reply and calling demo. I just tried but met some errors, "undefined reference to `dlsym'", I need some time to trick it. I also prefer to use pure go implemention after got your advices. So firstly I will make a test for the performance difference between go single-threaded and rust multi-threaded implementions for big files. If not significant difference, I will use go implemention

Note that to get the benefit of multithreaded hashing, you usually need to mmap the files your reading, which adds some more complexity and unsafety. One advantage of shelling out to b3sum is that it takes care of those details for you, and it doesn't require any tricky FFI hacks. If your files are big enough, the process spawning overhead might be negligible. That could be worth benchmarking.


Changed component module exports?

been using the DropdownModule this far.
Did a clean build for testing (clean npm install) and the index exports now:
export { ModalDirective, ModalBackdropOptions, ModalBackdropComponent, ModalModule } from './modal';
export { BsDropdownModule, BsDropdownConfig, BsDropdownState, BsDropdownContainerComponent, BsDropdownDirective, BsDropdownMenuDirective, BsDropdownToggleDirective } from './dropdown';

Why the sudden name change?

Also got my build status red :D

This is indeed very weird :(

@valorkin mentioned this in the release notes -> close...


Missing message profile identifier from FHIR and HL7 data

User Story:
As a receiver, I want to have the message profile identifier in the HL7 data I receiver, so I can have all data.
Description/Use Case
The message profile identifier was not converted to FHIR thinking that the HL7 message we would generate was going to be RS specific and hence we would not have a message profile.  That is not the case with HCA as we are providing the data as is, and may happen with other data as well.  Add this data piece.  See MSH-21
Risks/Impacts/Considerations

Dev Notes:

Acceptance Criteria

MSH-21 is populated in the the resulting FHIR to HL7 conversion when using the HCA sample sample_co_1_20220518-0001.hl7

OBE.  We do not want the message profile since we are generating our own HL7


Access to Component's props from NavBar.

Version
Tell us which versions you are using:

react-native-router-flux v4.0.0-beta.15 (v3 is not supported)
react-native v0.46.4?

Expected behaviour
First, great library. it's been a ton of help so far. There is some important functionality though, that I'd like some clarification on.
In v4, I expect to be able to call a static method that has access to the component's props.
Being able to save and interact with the store, seems like pretty necessary functionality....maybe that's just me, though. I've seen you discuss this in a couple of threads, but there hasn't been any clear solution. If it's the custom navigation that's the answer, can you please provide an example of how to use it with the rnrf? The documentation isn't very clear.
Thanks.
Actual behaviour
I can't access the component's props.

Check 4.0.0-beta.18 - now you can access your component instance via Actions.refs.SCENE_NAME


How to add a keyword before another one

Documentation

[X] I have consulted the documentation, and my question isn't answered there.

Nuget Package
JsonSchema.Net
Package Version
7.1.2
How can I help?  Please provide as much context as possible.
We have a usecase where we first need to sanitize (clean) user input before evaluating the regex patterns and minlength / maxlength keywords
So we created a custom keyword and implemented the Evaluator method to replace the value:
   private static void Evaluator(KeywordEvaluation evaluation, EvaluationContext context,
           SanitizerDelegate sanitizer)
   {
       // this is not really an evaluator but actually it's changes (sanitizes) the value of the incoming node
       var schemaValueType = evaluation.LocalInstance.GetSchemaValueType();
       if (schemaValueType is not SchemaValueType.String)
       {
           evaluation.MarkAsSkipped();
           return;
       }

       var str = evaluation.LocalInstance!.GetValue<string>();
       var sanitized = sanitizer(str);
       evaluation.LocalInstance.ReplaceWith(sanitized);
   }

Initially I thought I could use the SchemaPriorityAttribute and ensure this keyword is evaluated before the PatternKeyword, MinLengthKeyword and MaxLengthKeyword. But theSchemaPriorityAttribute was removed in version 5 in favor of the DependsOnAnnotationsFromAttribute.
Is there a way to insert a keyword before other keywords? Or is another solution preferred?
Code of Conduct

[X] I agree to follow this project's Code of Conduct

The santization you're trying to do needs to be performed before schema evaluation.  JSON Schema absolutely should not modify the instance.

As far as getting a keyword to run before another.... I don't think I have a solution for that.  [DependsOnAnnotationsFrom] instructs a keyword to run after another.
What you could do is subclass MinKeyword and MaxKeyword and apply the attribute on the subclasses.  When you register the keywords, they'll override the default ones.  (You'll need to provide converters as well.)

I agree, the santizing has to be done before schema evaluation. However, in our case we're using json schema to first detect the used schema, to know what properties we need to sanitize.
We've implemented it in another way, only annotating the fields with the sanitizers that need to run for the field. Our system now first evaluates the jsonschema to detect sanitizers, apply the sanitizers and afterwards evaluate the schema again to see if sanitizing the values did not break the patterns or min/max length constraints.
Thanks for your input!


Updated zocalo message format

See https://github.com/DiamondLightSource/python-dlstbx/pull/215
To test:

Do grid scan
Confirm artemis correctly handles the new format of XRC result returned by zocalo

Dials side merged, we should find a time to test this

Can't test on beamline at the moment, hopefully in a couple of weeks.


Added new images on documentation and examples page

Issue: #1106

I have made a new PR on same problem because I was having some problem removing Thumbs.db


Plugins are not applied to the editor

Informations
Browser + version: Firefox 46
OS: OS X
Resolution: 1920x1080
How to reproduce the bug?
I'm trying to add two plugins to my editor. Unless I'm missing something, this is not covered in the documentation. I am including both trumbowyg.upload.js and trumbowyg.colors.js on my page.
Here is my initialization code.
<script type="text/javascript">
  $(document).ready(function(){
    $('#trumbowyg-demo').trumbowyg({
      svgPath:'/assets/trumbowyg/images/icons.svg',
      plugins: {
        colors: {},
        upload: {}
      }
    });
  });
</script>

I would expect to see the upload and color buttons on the editor, but they do not appear.

You need to add plugins buttons to your editor: https://github.com/Alex-D/Trumbowyg/blob/gh-pages/js/main.js#L17

Thanks! You should add this to the documentation.


⚠ Bofrost.lu has degraded performance

In 308642c, Bofrost.lu (https://www.bofrost.lu) experienced degraded performance:

HTTP code: 200
Response time: 3392 ms

Resolved: Bofrost.lu performance has improved in b6c771b after 27 minutes.


GROOVY-8426 2.5.X Fix line and column numbers assigned to BlockStatements generated out of method definitions

…enerated out of method definitions

Merged. Thanks!
https://github.com/apache/groovy/commit/7e651110712a04fa3cdef0c89aaf257e155e8b74


EDNS information logging in log

I have implemented EDNS on two unbound.
The first unbound correctly sends the subnet information to the front-end unbound.

I can't figure out how?
The DNS service still log the IP address of the first Unbound server

The unbound that wants to log queries, there the edns subnet mod prints query has edns subnet ... and this is the subnet information for the incoming query.
The set up sounds similar to the use of proxy-protocol. With that the downstream IP address would be logged with log-queries: yes, due to the proxy protocol carrying that to the server. Proxy protocol processing, if that sort of load balancing towards some servers is considered as useful, is enabled with proxy-protocol-port: <port> for the server that performs recursion.

hi
Make sense...
This is one of the methods I wanted to implement, but it doesn't work in a purely "unbound" ecosystem.
In fact, I didn't see the possibility of enabling the proxy protocol in forward-addr/stub-addr
When you have Unboud backends servers  to Unboud front-ends, this architecture using Proxy-Protocol doesn't currently seem to be supported.
For PowerDNS/DnsDist to Unboud, no problem.
However, I was able to answer this question using DNSTAP where EDNS can be extracted.

Hello ,
You can do that easily with dnstap !
You may use a dnstap collector on the same server and configure unbound to forward queries to the collector. You will be able to see all details including EDNS/ECS. Plus, this is much less resource expensive in contrast with normal query logging.


[MM-34306] Migrate 'components/admin_console/brand_image_setting' and associated tests to TypeScript

Summary
migrated brand_image_setting to typescript.
Ticket Link
Fixes https://github.com/mattermost/mattermost-server/issues/17244
Release Note
NONE

Hello @majidsajadi,
Thanks for your pull request! A Core Committer will review your pull request soon. For code contributions, you can learn more about the review process here.

@majidsajadi Would you be able to fix the merge conflict as well as the test? It looks like there's a couple test cases still using the old import path for the file.

This PR has been automatically labelled "stale" because it hasn't had recent activity.
A core team member will check in on the status of the PR to help with questions.
Thank you for your contribution!
/cc @jasonblais @jfrerich @emilyacook

Hey! Could you fix the test and merge conflicts so we can pass onto QA for testing? Thanks

@hahmadia merged and resolved.


[SPARK-40130][PYTHON] Support NumPy scalars in built-in functions

What changes were proposed in this pull request?
Support NumPy scalars in built-in functions by introducing Py4J input converter NumpyScalarConverter.
Specifically,

np.int8, np.int16, np.int32, np.int64 are mapped to Spark int/bigint.
np.float32, np.float64 are mapped to Spark double.

Note that<PHONE_NUMBER> is the boundary between Spark int and bigint:
>>> df.select(lit(np.int64(max_int + 1))).dtypes
[('2147483648', 'bigint')]
>>> df.select(lit(np.int64(max_int))).dtypes
[('2147483647', 'int')]

Why are the changes needed?
As part of SPARK-39405 for NumPy support in SQL.
Does this PR introduce any user-facing change?
Yes. NumPy scalars are supported in built-in functions when input parameter accepts scalars;
Influenced functions include lit, when, array_contains, array_position, element_at, array_remove.
Take lit for example,
>>> df.select(lit(np.int8(1))).dtypes
[('1', 'int')]
>>> df.select(lit(np.float32(1))).dtypes
[('1.0', 'double')]

How was this patch tested?
Unit tests.

Can one of the admins verify this patch?

CC @ueshin @zhengruifeng

Rebased to the latest master to resolve code conflicts.

Merged to master, thank you all!


Attach stdout and stderr as streams to LogStream

Looking for a way to get the stderr and stdout streams from a running (attach) or stopped (logs) container as InputStream, I eventually came up with the following solution. I added a method attach(outputstream, outputstream) to LogStream, which then iterates over all incoming LogMessages and sends them to the correct stream. I'd like to hear your thoughts on this approach.
This can be used in the following ways:
Binding to System.err and System.out
dockerClient
  .logs(containerId, LogsParameter.FOLLOW, LogsParameter.STDERR, LogsParameter.STDOUT)
  .attach(System.out, System.err);

Binding to piped inputstreams
final PipedInputStream stdout = new PipedInputStream();
final PipedInputStream stderr = new PipedInputStream();
final PipedOutputStream stdout_pipe = new PipedOutputStream(stdout);
final PipedOutputStream stderr_pipe = new PipedOutputStream(stderr);

executor.submit(new Callable<Void>() {

  @Override
  public Void call() throws Exception {
    dockerClient
      .logs(containerId, LogsParameter.FOLLOW, LogsParameter.STDERR, LogsParameter.STDOUT)
      .attach(stdout_pipe, stderr_pipe);
    return null;
  }

});

try (Scanner sc_stdout = new Scanner(stdout);
    Scanner sc_stderr = new Scanner(stderr)) {
  assertThat(sc_stdout.next(), equalTo("stdout"));
  assertThat(sc_stderr.next(), equalTo("stderr"));
}

@JWGmeligMeyling Apologies for the delay on this. Reviewing it now.

Hi,
any chance to get this merged, i really like this feature.
Greetings

Merged now!
@JWGmeligMeyling Thanks for the great contribution with detailed comments!

:smile:

:+1:

Hi,
Does this code still function? I can't find references to LogsParameter anywhere else.
I would like to get the StdOut from a container as a stream. Thanks!
dockerClient
.logs(containerId, LogsParameter.FOLLOW, LogsParameter.STDERR, LogsParameter.STDOUT)
.attach(System.out, System.err);


Xcode 9 - dyld: Symbol not found: OBJC_CLASS$_NSURLSessionStreamTask

Hi,
I have a problem when an application built using Xcode 9. Runtime error right after when application is going to start:
dyld: Symbol not found: _OBJC_CLASS_$_NSURLSessionStreamTask
  Referenced from: /Users/.../Library/Developer/Xcode/DerivedData/###-bcrzoulrsfdubrbahknmpsvtakka/Build/Products/Debug/###.app/Contents/MacOS/../Frameworks/Alamofire.framework/Versions/A/Alamofire
  Expected in: /System/Library/Frameworks/Foundation.framework/Versions/C/Foundation
 in /Users/.../Library/Developer/Xcode/DerivedData/###-bcrzoulrsfdubrbahknmpsvtakka/Build/Products/Debug/###.app/Contents/MacOS/../Frameworks/Alamofire.framework/Versions/A/Alamofire

Cocoapods configuration:
platform :osx, '10.11'

use_frameworks!

target 'TranslateEverywhere' do

    pod 'MASShortcut'
    pod 'PySwiftyRegex',    '~> 1.0'
    pod 'Alamofire',        '~> 4.0'
    pod 'SwiftyJSON'
    pod 'RxSwift',          '~> 3.0'
    pod 'RxCocoa',          '~> 3.0'
    pod 'RxAlamofire',      '~> 3.0'
    pod 'SwiftyXML', 		'~> 1.4.0'
    pod 'SwiftString3',     '~> 1.0'

end

post_install do |installer|
  installer.pods_project.targets.each do |target|
    target.build_configurations.each do |config|
      config.build_settings['MACOSX_DEPLOYMENT_TARGET'] = '10.11'
    end
  end
end

pod --version
1.2.1

Most likely you just need to clear your derived data. Otherwise, are you actually use the stream APIs through Alamofire or not?

@jshier  Tried to clean up the project, removed delivered data. Also tried to build with a new builder. Nothing helps. Also found somewhere that adding to other linker flags -Objc and -framework Foundation could help but without success.

After clearing Delivered Data, removing pods configuration:
post_install do |installer|
  installer.pods_project.targets.each do |target|
    target.build_configurations.each do |config|
      config.build_settings['MACOSX_DEPLOYMENT_TARGET'] = '10.11'
    end
  end
end

and pod install problem seems to be solved.

In the linked frameworks and libraries section changing the Foundation.framework status to optional solved the problem for me.


Proposal: filter by model-scenario combinations

Through our discussion on slack, we identified a desire for a feature to allow for filtering unique model-scenario combinations. Here is one proposed implementation:
# data is a pyam.IamDataFrame
# ms_df if a dataframe with columns model and scenario

def subset_model_scenario(data, ms_df):
    idxs = [data.slice(model=m, scenario=s) for m, s in list(ms_df.itertuples(index=False, name=None))]
    idx = functools.reduce(operator.or_, idxs, False)
    return data[idx]

This utilizes the new IamSlice feature, constructing a list of slices of size N where N is the number of unique combinations and then using logical_or across all entries.
Thoughts @coroa @znicholls @danielhuppmann (and others)?

Argument name for the slice method is the same as for the filter method on IamDataFrame. And i agree it makes sense to call it index.


feat(contracts): Implement a default withdraw function

What does this PR do?
This PR implements default behaviour of withdraw function at Abstract Portal level
Default portal will also use the same behaviour for withdrawal function
Related ticket
Fixes #790
Type of change

[ ] Chore
[ ] Bug fix
[x] New feature
[ ] Documentation update

Check list

[x] Unit tests for any smart contract change
[ ] Contracts and functions are documented

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 46.80%. Comparing base (e56a25f) to head (3321d59).

Additional details and impacted files
@@           Coverage Diff           @@
##              dev     #801   +/-   ##
=======================================
  Coverage   46.80%   46.80%
=======================================
  Files          18       18
  Lines         391      391
  Branches       59       59
=======================================
  Hits          183      183
  Misses        205      205
  Partials        3        3

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


feat: make public key state validator to match the JS implementation

Issue being fixed or feature implemented

What was done?

How Has This Been Tested?

Breaking Changes

Checklist:

[ ] I have performed a self-review of my own code
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have added or updated relevant unit/integration/functional/e2e tests
[ ] I have made corresponding changes to the documentation

For repository code-owners and collaborators only

[ ] I have assigned this pull request to a milestone

Already done in another PR


Choose a create/library for interacting with the Docker daemon

Prepare two different pull request to show the difference between:

https://crates.io/crates/bollard
https://crates.io/crates/shiplift

Focusing on :

coverage of Docker Engine API
simplicity of API to use
create/repo maintenance ( Actively Developed / Maintained )
....

The two crates have the same in scope and features, but I think the one we should use is bollard since the interface seems nicer to work with.
They have a similar structure and code, since they are based on the same API, so it's hard to find points against one or the other.
Although  shiplift has a simpler interface, the biggest hurdle I had trying it was that the Container struct keeps a reference to the Docker struct for the Transport object. In the other one, the transport is wrapped in an Arc, so it's easier to pass around.
 But I think it can make things easier if we start to pass those objects around.
You can see an example of creating and running a container in the following PR for both libraries:

#6
#7

Bollard  is also actively developed / maintained , last release  v.0.14.0 - Jan 21, 2023.
Shiplift last release  v0.7.0 - Feb 21, 2021

I vote for Bollard, What do you think @FrancescoVaiani @sorru94 ?


Create a new version generator for Hot Rod writes

Describe the bug
The current version generator for the Hot Rod writes has the following format:
| view id (2 bytes) | rank (2 bytes) | version counter (4 bytes) |
It works fine for single cluster deployments but not for cross-site deployments. When cross-site is enabled, the view id and rank may match and generate the same version on both sites.
For example, a single node on 2 sites will have the same view id (it will be 1) and rank (once again 1 on both sites since they are the only member).
Expected behavior
No response
Actual behavior
No response
How to Reproduce?
No response
Output of uname -a or ver
No response
Output of java -version
No response
Quarkus version or git rev
No response
Build tool (ie. output of mvnw --version or gradlew --version)
No response
Additional information
Causes https://github.com/keycloak/keycloak/issues/35015

Requirements

The version must be unique for each stream of modification for each key.


HTML

HTML
User Story
As a bootcamp student
I want the prework notes to be structured on a webpage
so that I can easily find and read information
Acceptance Criteria
GIVEN a Prework Study Guide website
WHEN I visit the website in my browser
THEN I see four boxes titled HTML, CSS, Git, and JavaScript with associated notes listed

Just adding the basic HTML, CSS, Git, and JS layout


Add support for maps in HivePartitionFunction

Summary:
As the title suggests, this adds support for computing the hash of maps in HivePartitionFunction.
This depends on the refactoring done here https://github.com/facebookincubator/velox/pull/6876
and the addition of support for arrays here https://github.com/facebookincubator/velox/pull/6877
It's fairly straightforward, the flow is:

decode the keys and values Vectors
create a SelectivityVector selecting only values in the keys/values Vectors referred to by maps in the parent Vector
hash the keys/values
combine the hashes per map (handling gaps and overlaps in the values pointed to by maps)
combine the map hashes into the hashes per row

Added a benchmark:
============================================================================
[...]ks/HivePartitionFunctionBenchmark.cpp     relative  time/iter   iters/s
============================================================================
mapFewRowsFewBuckets                                       56.20us    17.79K
mapFewRowsManyBuckets                           97.625%    57.57us    17.37K
mapManyRowsFewBuckets                                     188.97us     5.29K
mapManyRowsManyBuckets                          99.891%   189.18us     5.29K

Validated all values in the tests with those produced by Presto Java.
Differential Revision: D49886111

linux-adapters is failing due to a known flaky test https://github.com/facebookincubator/velox/issues/6819
Aggregation fuzzer is failing due to a known issue https://github.com/facebookincubator/velox/issues/6833


Move shoutbox Quote button to Dot menu / hover

Description:
Stuff like #24 and #76, so as to not clutter the shoutbox. With the Social Milestone, a lot of features might be added to shoutbox and this can pave the way for future features.
Implementation:

[ ] Create an "Add to dot menu" helper function, based off existing logic for SB Gift
[ ] When dot menu is clicked, ensure all collected functions are displayed
[ ] Move Reply With Quote to dot menu

#76 will be pretty easy to accomplish within the Dot Menu as a standalone item. However, once we intend to migrate Reply With Quote and future features into the same menu, it will likely have to be restructured to have the Dot Menu click its own core event which does checks against the other features being enabled and processes everything at that time.  At least with the way I am approaching it.
When I have more complete on the standalone #76 you can see my approach and we can discuss if ideal or not.

Would likely end up being a Core feature as it wouldn't necessarily need any Settings to be displayed and would have multiple features relying on it.
It'll be easier to say once an implementation exists. As always, all these Issues are just reminders for myself to not forget any possibilities, so I haven't explored any actual implementations yet.

I am still torn on this enhancement in regards to #24 ... I really like the ease of the Quick reply button being in-line with the text. I feel like adding it to the three-dot menu would make it easier to just drag the name down from chat than click reply. Personally, I do not even mind having both the quote and quick next to each other, but I know some find it takes too much space. Maybe I have my text smaller on the screen and so its less intrusive. I like having Quote within three-dot menu, and quick in-line. Perhaps we can have a toggle selection for people to choose where to put those two?

Quick Reply should stay inline. I'm still undecided on whether or not Reply With Quote stays with it or is moved.

After using the SB quick action buttons for many months now... Reply With Quote is not being overused in the SB and I don't think it's taking up too much extra space, so I'm going to close this. Additional features should likely be added to the dot menu in the future. Tidying up the SB will happen at a later date if needed.


adapt the Bazel output directory for the incubator-heron project name

this fixes the issue #3213

@nwangtw

👍


Fix key addition for osd_directory scenario

Fixes: #147
Signed-off-by: Sébastien Han<EMAIL_ADDRESS>

That makes more sense.  Thanks!


snakemake: edge cases and extensions

what happens if we run out of sequence at some point?
e.g. if the whole sequence hits a PDB record and there is nothing left for subsequent steps?

create a function to (possibly) make subdirectories for outputs
e.g. input:
'results/subdir({seq})/01-search_pdb/{seq}.out'
def subdir(dir_name):
return dir_name

first point addressed by PR #46


lang: Allow CPI return values

If an instruction returns a type then write the result with the Solana set_return_data syscall. If the CPI client determines an instruction returns something it'll read it with get_return_data.
This shouldn't change anything for standard Result<()> returns from instructions, but specifying a Result with a type will result in a set_return_data call regardless of whether the call is made via CPI. I haven't figured out whether that is a good thing or a bad thing.
Closes #20

the generated idl should also include the return type of the instruction

Added this.

seems like a general problem with set_return_data though, not a problem with your implementation. Or am I misunderstanding you?

Yea I think this is OK.

LGTM! Is there anything left you want to add or can I merge?

thank you for the contribution!


Add Java code for LOJ - 1082

Implementation is a to z consistent with the editorial explained by the previous author, just the Java code needs to be approved.


Many timeouts when changing column breakpoints while debugging session is running

Testing #20192

VSCode Version: Code - Insiders 1.10.0-insider (256d76fed7947e7289f921df10658d7a69233b65, 2017-02-22T07:10:23.895Z)
OS Version: Windows_NT ia32 10.0.14393
Extensions: None

Steps to Reproduce:

Start debugging session
While debugging session is running: Add/remove column breakpoints
-> Quite often a warning about node being unresponsive is shown

Using node 7.6.0 here, but have seen it with 6.5.0 too.

Please share your test program or project.

Can't reproduce at the moment. I also can't set the column breakpoint before the second console anymore. It worked yesterday, as shown in the screenshot above. Double-checked the launch.json uses node.

Not sure why I couldn't set that column breakpoint, but it works again now.
Also got steps that reproduce it:

File:

for (let i = 2; i; i--) {
    console.log('i: ' + i); console.log('i: ' + i);
}

Start debugging without any breakpoints
Code runs to end, debugger stays on
Place column breakpoint in front of the second console
Wait like 10 seconds
-> Above warning message about timeout in shown

Reproduce on Windows and Mac. (Node 7.6.0 and 6.5.0 on Windows. Node 6.9.4 on Mac.)

no longer a problem in node.js >= 8


[BD-46] docs: fix links to sections in default mdx page template

Description
Fixes how anchor links are displayed when using default mdx page and updates scroll behaviour to those headings to include sticky header size
Deploy Preview
See Installation and Usage page for example, and its version before this PR
Merge Checklist

[x] If your update includes visual changes, have they been reviewed by a designer? Send them a link to the Netlify deploy preview, if applicable.
[x] Does your change adhere to the documented style conventions?
[x] Do any prop types have missing descriptions in the Props API tables in the documentation site (check deploy preview)?
[x] Were your changes tested using all available themes (see theme switcher in the header of the deploy preview, under the "Settings" icon)?
[ ] Were your changes tested in the example app?
[x] Is there adequate test coverage for your changes?
[x] Consider whether this change needs to reviewed/QA'ed for accessibility (a11y). If so, please add wittjeff and adamstankiewicz as reviewers on this PR.

Post-merge Checklist

[ ] Verify your changes were released to NPM at the expected version.
[ ] If you'd like, share your contribution in #show-and-tell.
[ ] 🎉 🙌 Celebrate! Thanks for your contribution.

Thanks for the pull request, @viktorrusakov! I've created BLENDED-1227 to keep track of it in Jira.   More details are on the BD-46 project page.
When this pull request is ready, tag your edX technical lead.

@viktorrusakov 🎉 Your pull request was merged!  Please take a moment  to answer a two question survey so we can improve your experience in the future.

:tada: This PR is included in version 19.22.2 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Use latest version of cppcheck (2.15.0) to analyze codebase

what

Update the cppcheck GitHub workflow to run on the latest version of that tool (2.15.0) for better codebase analysis (more aligned with that of Sonarcloud).

NOTE: This PR references cppcheck version 2.14.2 because that was the last version at the time it was created. However, the GitHub macOS runner will now install 2.15.0 (the latest version), which has been released since then (Aug 31, 2024).

Update the cppcheck configuration to have the analysis be performed using C++17, which is aligned with the build configuration for the project.
Address warnings identified by the newer version of the tool.

This should also address some similar Sonarcloud issues as well.

why
The latest version of cppcheck includes improved analysis and address some limitations of the version currently used (2.7.1, from Feb, 2022).
For example, std::string_view instances are expected to be passed by value (see https://quuxplusone.github.io/blog/2021/11/09/pass-string-view-by-value/), but the older version of cppcheck would report a warning incorrectly stating that the argument should be passed by reference instead. This was addressed in danmar/cppcheck#3817
notes
The cppcheck GitHub workflow now runs on the macOS image (instead of the Ubuntu/Linux image previously used) for two reasons:

The Ubuntu/Linux image didn't have the latest version of cppcheck available, so we needed to manually checkout the cppcheck repository and compile the tool, which would make the workflow more complex and take more time.
The cppcheck build in Ubuntu/Linux would run much slower than the package available on macOS (probably due to some additional build instructions being required to improve performance which I couldn't determine/find).

Failed conditions  B Maintainability Rating on New Code (required ≥ A)

I added two commits (unrelated to this PR) to address Sonarcloud reports. The first blocked analysis of the commits due to Sonarcloud finding (existing) duplicate code in ValidateSchema & ValidateDTD where initially there was a single line change to include the override keyword in those classes' destructors (and then the destructor was removed altogether).
The set of "new" (between quotes) issues reported by Sonarcloud after these changes are:

src/actions/exec.h
Add a using-declaration to this derived class to inherit the constructors of "Action", and remove the ones you manually duplicated. Note that this may add other constructors to your derived class.
* Addressing this issue would include additional constructors to Exec, which are not currently supported, so it's not an equivalent change.
src/operators/validate_dtd.h & src/operators/validate_schema.h

Remove use of ellipsis notation (2)
Replace this use of "void *" with a more meaningful type. (6)
Make the type of this parameter a pointer-to-const. The current type of this unnamed parameter is "void *" (3)
Replace "reinterpret_cast" with a "static_cast" (2)

All these are required by the signature of libxml2 callback functions.

test/regression/regression.cc

Refactor this function to reduce its Cognitive Complexity from ... (5)

Existing code, unrelated to these changes.

Hi @eduar-hte,
many thanks for this great PR. Also thanks for adding false positive labels to cppcheck  suppression's.
I have only one question regarding the new inline suppression's: are those there because of the new cppcheck? Why those haven't appeared until now? (I marked the first two items and later I saw that there are many others there)
About SonarCloud: I already sent you an invitation to have an admin role on SonarCloud - haven't you received that? With that you could mark those issues (that you explain every time really accurately) as FP and suppress them.

I have only one question regarding the new inline suppression's: are those there because of the new cppcheck? Why those haven't appeared until now? (I marked the first two items and later I saw that there are many others there)

Yes, the new version reported some new issues, some of which were addressed with changes to the code as they were correct (see commits related to using the override keyword or adding const for example).
At the same time, the new version also introduced a (limited) number of inaccurate issues, which I had to address by adding the inline suppressions you noticed (those are grouped in commit d053ec6).
I understand that while they may add new features and address bugs in newer versions, some of these changes may have introduced inaccurate reports as well. 🤷♂

Yes, the new version reported some new issues, some of which were addressed with changes to the code as they were correct (see commits related to using the override keyword or adding const for example).

I saw and I though you aligned those because of this reason.

At the same time, the new version also introduced a (limited) number of inaccurate issues, which I had to address by adding the inline suppressions you noticed (those are grouped in commit d053ec6).

Thanks,

I understand that while they may add new features and address bugs in newer versions, some of these changes may have introduced inaccurate reports as well. 🤷♂

Thanks again.
I saw you added some more stuff (removed some inline suppression). Let me know if you finish this PR.

I saw you added some more stuff (removed some inline suppression). Let me know if you finish this PR.

That's right, I did away with a few of the cppcheck inline suppressions, as I ended up explaining in an edit of my previous reply (which I was in the process of writing when the build with the new changes completed so I decided to merge them into the PR and then resume the lengthy reply 😁).
PS: I ended up even addressing the new issue Sonarcloud reported and that I mentioned here:

Then there are some that are opinionated, like this one that just came up after commit https://github.com/owasp-modsecurity/ModSecurity/commit/651536fa0eacd60ca5b99ca0cbe675f0674e327b. I think that c++11 default member initializers are nice, but a matter of style whether to use them or not.

just not to add up to the number of outstanding Sonarcloud issues.
I'm now done with this PR.

Thanks for the PR again and for the detailed explanations.
I'm merging this now.


[go_router] Special character (+) is encode in url

Hello there !
I'm using go_router to handle url navigation for my Flutter web application.
It's all working fine, but my params contains a special character: +. And, when writing the url, the character is encode in %2B.
Here my params is 'company+2'
GoRouter.of(context).goNamed(RoutesName.homeRoute,
          params: {'company': company});

result  http://localhost:9999/company%2B2
Is there a way to not encode it ?
I'm using last version (3.0.7) of go_router.
Thanks in advance for your help.
Steps to Reproduce
Here a runnable code sample
The result of the url is   http://localhost:9999/company%2B1 but I want it to be
 http://localhost:9999/company+1
go_router: ^3.0.7

void main() {
  runApp(MyApp());
}

class MyApp extends StatelessWidget {
  MyApp({Key? key}) : super(key: key);

  final _router = GoRouter(
      initialLocation: '/',
      routes: [
    GoRoute(
      name: 'home',
      path: '/',
      pageBuilder: (context, state) => MaterialPage(
        key: state.pageKey,
        child: const Home(),
      ),
      routes: [
        GoRoute(
          name: 'company',
          path: ':companyId',
          pageBuilder: (context, state) => MaterialPage(
            key: state.pageKey,
            child: Company(
              companyId: state.params['companyId']!,
            ),
          ),
        ),
      ]
    ),
  ]);

  @override
  Widget build(BuildContext context) {
    return MaterialApp.router(
            theme: ThemeData(
                textSelectionTheme:
                    const TextSelectionThemeData(selectionColor: Constants.sideColor)),
            routeInformationParser: _router.routeInformationParser,
            routerDelegate: _router.routerDelegate,
            debugShowCheckedModeBanner: false,
            scrollBehavior: AppScrollBehavior(),
            title: 'Showroom',
           )
      }
}

class Home extends StatelessWidget {
  const Home({Key? key}) : super(key: key);

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      body: Center(
        child: TextButton(
            onPressed: () {
              GoRouter.of(context).goNamed('company', params: {'companyId': 'company+1'});
            },
            child: Text("Next page")),
      ),
    );
  }
}

class Company extends StatelessWidget {
  const Company({Key? key, required this.companyId}) : super(key: key);

  final String companyId;

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      body: Center(
        child: Text(companyId),
      ),
    );
  }
}

[✓] Flutter (Channel stable, 2.10.4, on macOS 12.3 21E230 darwin-arm, locale fr-FR)
    • Flutter version 2.10.4 at /Users/appspanel/Documents/development/utils/flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision c860cba910 (3 weeks ago), 2022-03-25 00:23:12 -0500
    • Engine revision 57d3bac3dd
    • Dart version 2.16.2
    • DevTools version 2.9.2

[✓] Android toolchain - develop for Android devices (Android SDK version 32.0.0)
    • Android SDK at /Users/appspanel/Library/Android/sdk
    • Platform android-32, build-tools 32.0.0
    • Java binary at: /Applications/Android Studio.app/Contents/jre/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 11.0.11+0-b60-7772763)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 13.3)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • CocoaPods version 1.11.2

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2021.1)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 11.0.11+0-b60-7772763)

[✓] Connected device (2 available)
    • Android SDK built for arm64 (mobile) • emulator-5554 • android-arm64  • Android 10 (API 29) (emulator)
    • Chrome (web)                         • chrome        • web-javascript • Google Chrome 100.0.4896.75

[✓] HTTP Host Availability
    • All required HTTP hosts are available

@Velliane
Can you provide complete runnable code sample along with flutter doctor -v that we can use that shows the behavior you are reporting ?
Also, what's the expected vs actual result you are getting ?

Hello @darshankawar
I edit my first post

Thanks for the update. Using it and navigating to next page on web, it gives me http://localhost:51595/#/company%2B1 url.
I think + is rendered as %2B.
Keeping it open for further insights from the team on current behavior.
/cc @johnpryan @chunhtai

 stable, master flutter doctor -v

[✓] Flutter (Channel stable, 2.10.4, on macOS 12.2.1 21D62 darwin-x64, locale
    en-GB)
    • Flutter version 2.10.4 at /Users/dhs/documents/fluttersdk/flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision c860cba910 (3 days ago), 2022-03-25 00:23:12 -0500
    • Engine revision 57d3bac3dd
    • Dart version 2.16.2
    • DevTools version 2.9.2

[!] Xcode - develop for iOS and macOS (Xcode 12.3)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    ! Flutter recommends a minimum Xcode version of 13.
      Download the latest version or update via the Mac App Store.
    • CocoaPods version 1.11.2

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] VS Code (version 1.62.0)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.21.0

[✓] Connected device (5 available)
    • SM G975F (mobile)       • RZ8M802WY0X • android-arm64   • Android 11 (API 30)
    • Darshan's iphone (mobile)  • 21150b119064aecc249dfcfe05e259197461ce23 •
      ios            • iOS 14.4.1 18D61
    • iPhone 12 Pro Max (mobile) • A5473606-0213-4FD8-BA16-553433949729     •
      ios            • com.apple.CoreSimulator.SimRuntime.iOS-14-3 (simulator)
    • macOS (desktop)            • macos                                    •
      darwin-x64     • Mac OS X 10.15.4 19E2269 darwin-x64
    • Chrome (web)               • chrome                                   •
      web-javascript • Google Chrome 98.0.4758.80

[✓] HTTP Host Availability
    • All required HTTP hosts are available

! Doctor found issues in 1 category.

[✓] Flutter (Channel master, 2.13.0-0.0.pre.523, on macOS 12.2.1 21D62
    darwin-x64, locale en-GB)
    • Flutter version 2.13.0-0.0.pre.523 at
      /Users/dhs/documents/fluttersdk/flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 009688a99e (76 minutes ago), 2022-04-12 23:29:10 -0400
    • Engine revision e33c024d94
    • Dart version 2.18.0 (build 2.18.0-10.0.dev)
    • DevTools version 2.12.2

[!] Xcode - develop for iOS and macOS (Xcode 12.3)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    ! Flutter recommends a minimum Xcode version of 13.
      Download the latest version or update via the Mac App Store.
    • CocoaPods version 1.11.2

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] VS Code (version 1.62.0)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.21.0

[✓] Connected device (5 available)
    • SM G975F (mobile)       • RZ8M802WY0X • android-arm64   • Android 11 (API 30)
    • Darshan's iphone (mobile)  • 21150b119064aecc249dfcfe05e259197461ce23 •
      ios            • iOS 14.4.1 18D61
    • iPhone 12 Pro Max (mobile) • A5473606-0213-4FD8-BA16-553433949729     •
      ios            • com.apple.CoreSimulator.SimRuntime.iOS-14-3 (simulator)
    • macOS (desktop)            • macos                                    •
      darwin-x64     • Mac OS X 10.15.4 19E2269 darwin-x64
    • Chrome (web)               • chrome                                   •
      web-javascript • Google Chrome 98.0.4758.80

[✓] HTTP Host Availability
    • All required HTTP hosts are available

! Doctor found issues in 1 category.

If you literally put + in the URL it will be interpreted as the space character. For example, "special character" would be encoded as "special+character" in the URL. So the URL encoder does the right thing by escaping the "+" character as %2B so that it is interpreted as the + character.


Fix compile python bindings

This fixes the python build error

Thanks for the PR @nikulin-94!
I still prefer to have the bindings compile from source in the Dockerfile. I have created a new PR where I've updated the entire boilerplate to Deepstream 6.1. I have updated the documentation and verified everything works on a new VM. Feel free to test out that version and let me know if it works for you.
https://github.com/ml6team/deepstream-python/pull/7


Width and height have unexpected results

I'm using this simply for SVG icons using Path. When specifying the width and height on a <Svg> element, there seems to be unexpected results. For example setting an element's width to 40 results in a very large svg element, while setting it 500 results in a very small svg element.

try to use 'scale' parameter to define what scale should be used.

:+1: This helped thanks.


Navigate to specific date from custom toolbar

hello there and thx for the fine, really fine lib.
what I'm trying to achieve is to navigate to specific date from custom toolbar by clicking on a date inside react-day-picker.
some code samples:
CustomToolbar:
class CustomToolbar extends React.PureComponent {
     ...
     render() {
         const goToSelectedDate = date => {
                console.log(new Date(date)); // here I get the date I need, the date I actually clicked in the react-day-picker
                onNavigate(new Date(date));
         };

          return (

                 goToBack()} className='rbc-custom-back'>
                    <Icon glyph={ChevronLeft} className='icon_chevron-left'/>

                 { this.dayPickerToggle = dayPicker; }}
                    onClick={e => {
                        this.toggleDayPicker(e);
                        this.getDayPickerSelectedRange();
                    }}>
                    {label}

                {isDayPickerShown &&
                     <DayPicker
                         onDayClick={goToSelectedDate}
                         month={new Date(date)}
                        selectedDays={view === 'week' ? dayPickersSelectedDays : [date]}
                        enableOutsideDays/>
                }
                 goToNext()} className='rbc-custom-next'>
                    <Icon glyph={ChevronRight} className='icon_chevron-right'/>

                 {
                        goToDayView();
                        goToCurrent();
                    }}
                    className={todayClassNames}>
                    Go to Today

                    <button onClick={() => goToDayView()} className={dayClassNames} type='button'>day</button>
                    <button onClick={() => goToWeekView()} className={weekClassNames} type='button'>week</button>
                    <button onClick={() => goToMonthView()} className={monthClassNames} type='button'>month</button>

          );
     }
}

CalendarComponent:
class CalendarComponent extends React.PureComponent {
     render() {
          return (
               <DragAndDropCalendar
                            {...this.props}
                            selectable
                            popup
                            defaultView='day'
                            date={selectedDay}
                            onNavigate={date => {
                                console.log(date); // but here I still get the current date
                                this.setState({ selectedDay: new Date(date) });
                            }}
          );
     }
}

I don't get what I'm doing wrong since the date is being passed to the calendars onNavigate prop but it seems like it loses context or something like this.
any help or advice is appreciated. I may provide more code samples if anyone would like to debug .

Its a bit hard to tell from your example but the approach looks correct, the calendar will "select" whatever date you pass to date and onNavigate is called when you click or navigate from within the calendar (e.g. a date header), you have to manage the state coming from the calendar and your datepicker.

 But I get the correct date. Thank @maxefi


Create basic Electron package

No UI, it should just run the Flask backend and access localhost:13000 inside Electron.
Add the proper packaging flow, so we have an Electron based installation

Version 2.3 has a working Electron setup, with a proper packaging flow.

There is an electron package. Unless there is something else pending related to this particular issue, @zmbq feel free to close


Unable to export loss values from TD3

First, thank you very much for this wonderful package.
Visualization of loss trends gives us an indication of where the training process is bound for.
Where TD3 is concerned, i think that there is a potential bug in line 113 'return {}'.
With this, it is very difficult to monitor the loss values on tensorboard.

Thanks, and thank you for letting us know! I have submitted a fix. It needs to be approved and then reflected to GitHub, which should happen this Thursday.

If you're in a hurry, all you have to do is replace the learn_batch function in policy_learners/sequential_decision_making/td3.py by
    def learn_batch(self, batch: TransitionBatch) -> Dict[str, Any]:
        # TODO: this method is very similar to that of ActorCriticBase.
        # Can we refactor?

        critic_loss = self._critic_loss(batch)  # critic update
        self._critic_update_count += 1

        # delayed actor update
        self._critic_optimizer.zero_grad()
        if self._critic_update_count % self._actor_update_freq == 0:
            # see ddpg base class for actor update details
            actor_loss = self._actor_loss(batch)
            self._actor_optimizer.zero_grad()
            (actor_loss + critic_loss).backward()
            self._actor_optimizer.step()
            self._critic_optimizer.step()
            report = {
                "actor_loss": actor_loss.item(),
                "critic_loss": critic_loss.item(),
            }
        else:
            critic_loss.backward()
            self._critic_optimizer.step()
            report = {"critic_loss": critic_loss.item()}

        if self._critic_update_count % self._actor_update_freq == 0:
            # update targets of critics using soft updates
            update_critic_target_network(
                self._critic_target,
                self._critic,
                self._use_twin_critic,
                self._critic_soft_update_tau,
            )
            # update target of actor network using soft updates
            update_target_network(
                self._actor_target, self._actor, self._actor_soft_update_tau
            )

        return report

Thank you very much!

The code provided above is buggy. I think that I have fixed it in the following. Would be nice to have a 2nd pair of eyes to verify:
def learn_batch(self, batch: TransitionBatch) -> Dict[str, Any]:
    # TODO: this method is very similar to that of ActorCriticBase.
    # Can we refactor?

    critic_loss = self._critic_loss(batch)  # critic update
    self._critic_update_count += 1
report = {"critic_loss": critic_loss.items()}

    # delayed actor update
    self._critic_optimizer.zero_grad()
    if self._critic_update_count % self._actor_update_freq == 0:
        # see ddpg base class for actor update details
        actor_loss = self._actor_loss(batch)
        ## self._actor_optimizer.zero_grad()
        ## (actor_loss + critic_loss).backward()
        ## self._actor_optimizer.step()
        ## self._critic_optimizer.step()
        report = {
            "actor_loss": actor_loss.items(),
            "critic_loss": critic_loss.items(),
        }
    ## else:
    ##     critic_loss.backward()
    ##     self._critic_optimizer.step()
    ##     report = {"critic_loss": critic_loss.item()}

    if self._critic_update_count % self._actor_update_freq == 0:
        # update targets of critics using soft updates
        update_critic_target_network(
            self._critic_target,
            self._critic,
            self._use_twin_critic,
            self._critic_soft_update_tau,
        )
        # update target of actor network using soft updates
        update_target_network(
            self._actor_target, self._actor, self._actor_soft_update_tau
        )

    return report

Why do you think those lines need to be commented out? You've commented out all calls to optimizers' step() methods, which would mean no learning taking place.

Because they are already done in the self._critic_loss(batch) and self._actor_loss(batch) methods.
we cannot backprop a float.

Because they are already done in the self._critic_loss(batch) and self._actor_loss(batch) methods.

Sorry, I am not seeing that. td3.py defines _critic_loss(batch) and that methods does not include a .step() call. It does not define _actor_loss, instead reusing that of its base class in ddpg.py, and I don't see a step() call in the _actor_loss(batch) there either. Am I missing something?

we cannot backprop a float.

Ok, here is my understanding of the flow...

critic_loss = self._critic_learn_batch(batch)  # critic update
self._critic_learn_batch(batch) calls

loss_critic_update = twin_critic_action_value_update(
state_batch=batch.state,
action_batch=batch.action,
expected_target_batch=expected_state_action_values,
optimizer=self._critic_optimizer,
critic=self._critic,
)
3. It seems that the critic networks have already been updated inside self._critic_learn_batch(batch).
4. There for we no longer seem to have to do:
        critic_loss.backward()
        self._critic_optimizer.step()
        report = {"critic_loss": critic_loss.item()}

If you're in a hurry, all you have to do is replace the learn_batch function in policy_learners/sequential_decision_making/td3.py by
    def learn_batch(self, batch: TransitionBatch) -> Dict[str, Any]:
        # TODO: this method is very similar to that of ActorCriticBase.
        # Can we refactor?

        critic_loss = self._critic_loss(batch)  # critic update
        self._critic_update_count += 1

        # delayed actor update
        self._critic_optimizer.zero_grad()
        if self._critic_update_count % self._actor_update_freq == 0:
            # see ddpg base class for actor update details
            actor_loss = self._actor_loss(batch)
            self._actor_optimizer.zero_grad()
            (actor_loss + critic_loss).backward()
            self._actor_optimizer.step()
            self._critic_optimizer.step()
            report = {
                "actor_loss": actor_loss.item(),
                "critic_loss": critic_loss.item(),
            }
        else:
            critic_loss.backward()
            self._critic_optimizer.step()
            report = {"critic_loss": critic_loss.item()}

        if self._critic_update_count % self._actor_update_freq == 0:
            # update targets of critics using soft updates
            update_critic_target_network(
                self._critic_target,
                self._critic,
                self._use_twin_critic,
                self._critic_soft_update_tau,
            )
            # update target of actor network using soft updates
            update_target_network(
                self._actor_target, self._actor, self._actor_soft_update_tau
            )

        return report

Specifically if the above were to be used:

That led me to investigate more deeply in to what the following actually does.
def _critic_learn_batch(self, batch: TransitionBatch) -> Dict[str, Any]:
    with torch.no_grad():
        # sample next_action from actor's target network; shape (batch_size, action_dim)
        next_action = self._actor_target.sample_action(batch.next_state)

        # sample clipped gaussian noise
        noise = torch.normal(
            mean=0,
            std=self._actor_update_noise,
            size=next_action.size(),
            device=batch.device,
        )

        noise = torch.clamp(
            noise,
            -self._actor_update_noise_clip,
            self._actor_update_noise_clip,
        )  # shape (batch_size, action_dim)

        # add clipped noise to next_action
        low = torch.tensor(self._action_space.low, device=batch.device)
        high = torch.tensor(self._action_space.high, device=batch.device)

        next_action = torch.clamp(
            next_action + noise, low, high
        )  # shape (batch_size, action_dim)

        # sample q values of (next_state, next_action) from targets of critics
        next_q1, next_q2 = self._critic_target.get_q_values(
            state_batch=batch.next_state,
            action_batch=next_action,
        )  # shape (batch_size)

        # clipped double q learning (reduce overestimation bias)
        next_q = torch.minimum(next_q1, next_q2)

        # compute bellman target:
        # r + gamma * (min{Qtarget_1(s', a from target actor network),
        #                  Qtarget_2(s', a from target actor network)})
        expected_state_action_values = (
            next_q * self._discount_factor * (1 - batch.done.float())
        ) + batch.reward  # (batch_size)

    # update twin critics towards bellman target
    assert isinstance(self._critic, TwinCritic)
    loss_critic_update = twin_critic_action_value_update(
        state_batch=batch.state,
        action_batch=batch.action,
        expected_target_batch=expected_state_action_values,
        optimizer=self._critic_optimizer,
        critic=self._critic,
    )
    return loss_critic_update

It seems that the critic network has already been updated.

It looks like you renamed _critic_loss to _critic_learn_batch and this is causing the first error. We don't have a _critic_learn_batch function in the code base (I did a full search).
Then critic_loss, which you have as a dict, should have been the tensor returned by _critic_loss(batch), but since that generated an error, then I am not sure how critic_loss has a value at all.
You are saying that the "critic network has already been updated", but what actually updates the parameters of the networks is the optimizer.step() method (do you agree with that?), whose calls you commented out.
If you could please go back to the code base as we have it here, check if you still have a bug in that version, and let us know, I think that would be the most effective way of knowing what is going on.

I know what is going on now - I am on an older version of Pearl. It's my bad. Sorry and thank you for your time!

Sorry, searching some more I see we did use to have _critic_learn_batch, but that is in older versions. So the errors you are seeing come from mixing the code I pasted here, which is for the current code, and your code, which seems to be at an earlier point.
So it seems like the best thing to do is for you to update your local repository, or if that is hard right now, to manually insert the setting of report in the version you have.


Web Is Not Working

My mongo db stuff works, and when I run the gosint file itself, it has the standard running thing. But, I am having problems with nginx. My nginx file looks like:
`# For more information on configuration, see:
* Official English Documentation: http://nginx.org/en/docs/
* Official Russian Documentation: http://nginx.org/ru/docs/
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log;
pid /run/nginx.pid;
Load dynamic modules. See /usr/share/nginx/README.dynamic.
include /usr/share/nginx/modules/*.conf;
events {
worker_connections 1024;
}
http {
log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log  /var/log/nginx/access.log  main;

sendfile            on;
tcp_nopush          on;
tcp_nodelay         on;
keepalive_timeout   65;
types_hash_max_size 2048;

include             /etc/nginx/mime.types;
default_type        application/octet-stream;

# Load modular configuration files from the /etc/nginx/conf.d directory.
# See http://nginx.org/en/docs/ngx_core_module.html#include
# for more information.
include /etc/nginx/conf.d/*.conf;

server {

listen 80;
root /home/gosint/projects/src/GOSINT/website;
index index.php index.html index.htm;
try_files $uri $uri/ @apachesite;
server_name someserver.yourcompany.com;
gzip on;
gzip_proxied any;
gzip_types
text/css
text/javascript
text/xml
text/plain
application/javascript
application/x-javascript
application/json;
#location / {
try_files $uri $uri/ =404;
#}
error_page 404 /404.html;
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root /usr/share/nginx/html;
}
location @apachesite {
  proxy_pass http://localhost:8000;

}
location ~ .php$ {
try_files $uri =404;
fastcgi_split_path_info ^(.+.php)(/.+)$;
  # PHP 7
  fastcgi_pass unix:/var/run/php/php7.0-fpm.sock;

  # PHP 5
  # fastcgi_pass unix:/run/php5-fpm.sock;
  fastcgi_index index.php;
  fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
  include fastcgi_params;

}
}
Settings for a TLS enabled server.

server {
listen       443 ssl http2 default_server;
listen       [::]:443 ssl http2 default_server;
server_name  _;
root         /usr/share/nginx/html;

ssl_certificate "/etc/pki/nginx/server.crt";
ssl_certificate_key "/etc/pki/nginx/private/server.key";
ssl_session_cache shared:SSL:1m;
ssl_session_timeout  10m;
ssl_ciphers HIGH:!aNULL:!MD5;
ssl_prefer_server_ciphers on;

# Load configuration files for the default server block.
include /etc/nginx/default.d/*.conf;

location / {
}

error_page 404 /404.html;
location = /40x.html {
}

error_page 500 502 503 504 /50x.html;
location = /50x.html {
}
}
}
`
What do I need to do to get this to work properly?

Wow, that did not turn out right.


OPC Publisher continuously restarts after an error

Describe the bug
We see the OPC Publisher module continuously restarting with the following message in the logs:
2020-06-01 04:57:45.412 +00:00 [VRB] [Exit] "Informational": MqttIotHubAdapter#35236192 ProcessMessage MqttIotHubAdapter#0, PUBLISH
2020-06-01 04:57:45.412 +00:00 [VRB] [Exit] "Informational": MqttIotHubAdapter#35236192 ChannelRead MqttIotHubAdapter#0
2020-06-01 04:57:45.413 +00:00 [VRB] [Enter] "Informational": MqttIotHubAdapter#35236192 ChannelReadComplete (MqttIotHubAdapter#0)
2020-06-01 04:57:45.413 +00:00 [VRB] [Exit] "Informational": MqttIotHubAdapter#35236192 ChannelReadComplete MqttIotHubAdapter#0
2020-06-01 04:57:45.482 +00:00 [VRB] [Exit] "Informational": ErrorDelegatingHandler#63993496 ExecuteWithErrorHandlingAsync ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync
2020-06-01 04:57:45.486 +00:00 [VRB] [Exit] "Informational": RetryDelegatingHandler#39070558 SendTwinGetAsync CancellationToken#51484875
2020-06-01 04:57:45.490 +00:00 [INF] Initialize device twin for at-iot-gw-home-bsroom-01 - at-iot-gw-home-bsroom-01-opc-publisher
2020-06-01 04:57:45.754 +00:00 [DBG] [Info] "Informational": MqttTransportHandler#46964992 OnTransportClosedGracefully OnTransportClosedGracefully
2020-06-01 04:57:45.759 +00:00 [DBG] [Info] "Informational": RetryDelegatingHandler#39070558 HandleDisconnect Transport disconnected: closed by application.
2020-06-01 04:57:45.760 +00:00 [VRB] [Enter] "Informational": InternalClient#45596481 OnConnectionStatusChanged (Disabled, Client_Close)
2020-06-01 04:57:45.761 +00:00 [INF] 1: Module at-iot-gw-home-bsroom-01_at-iot-gw-home-bsroom-01-opc-publisher disconnected due to "Client_Close" - now "Disabled"...
2020-06-01 04:57:45.761 +00:00 [VRB] [Exit] "Informational": InternalClient#45596481 OnConnectionStatusChanged Disabled, Client_Close
2020-06-01 04:57:45.770 +00:00 [VRB] [Enter] "Informational": MqttIotHubAdapter#35236192 WriteAsync (MqttIotHubAdapter#0, DisconnectPacket[Type=DISCONNECT, QualityOfService=AtMostOnce, Duplicate=False,
Retain=False])
2020-06-01 04:57:45.771 +00:00 [VRB] [Exit] "Informational": MqttIotHubAdapter#35236192 WriteAsync MqttIotHubAdapter#0
2020-06-01 04:57:45.782 +00:00 [ERR] Error during module execution - restarting!
System.InvalidCastException: Unable to cast object of type 'Newtonsoft.Json.Linq.JValue' to type 'System.String'.
   at Microsoft.Azure.IIoT.Module.Framework.Hosting.ModuleHost.StartAsync(String type, String siteId, String serviceInfo, IProcessControl reset) in D:\a\1\s\common\src\Microsoft.Azure.IIoT.Hub.Module.Fr
amework\src\Hosting\ModuleHost.cs:line 152
   at Microsoft.Azure.IIoT.Modules.OpcUa.Publisher.ModuleProcess.RunAsync() in D:\a\1\s\modules\src\Microsoft.Azure.IIoT.Modules.OpcUa.Publisher\src\ModuleProcess.cs:line 89
2020-06-01 04:57:45.801 +00:00 [VRB] [Enter] "Informational": MqttIotHubAdapter#35236192 ChannelInactive (MqttIotHubAdapter#0)
2020-06-01 04:57:45.829 +00:00 [VRB] [Enter] "Informational": MqttIotHubAdapter#35236192 Shutdown (MqttIotHubAdapter#0)
2020-06-01 04:57:45.832 +00:00 [VRB] [Enter] "Informational": MqttIotHubAdapter#35236192 CloseIotHubConnection (CloseIotHubConnection)
2020-06-01 04:57:45.836 +00:00 [VRB] [Exit] "Informational": MqttIotHubAdapter#35236192 CloseIotHubConnection CloseIotHubConnection
2020-06-01 04:57:45.837 +00:00 [VRB] [Exit] "Informational": MqttIotHubAdapter#35236192 Shutdown MqttIotHubAdapter#0
2020-06-01 04:57:45.838 +00:00 [VRB] [Exit] "Informational": MqttIotHubAdapter#35236192 ChannelInactive MqttIotHubAdapter#0
2020-06-01 04:57:45.868 +00:00 [INF] Connecting all clients to at-iot-gw-home-bsroom-01 using "MqttOverTcp".
2020-06-01 04:57:45.874 +00:00 [DBG] Starting Module Host...
2020-06-01 04:57:45.875 +00:00 [WRN] Bypassing certificate validation for client.
2020-06-01 04:57:45.875 +00:00 [INF] Running in iotedge context.
2020-06-01 04:57:45.906 +00:00 [DBG] [Info] "Informational": IotHubConnectionString#54135081 .ctor IAuthenticationMethod is AuthenticationWithTokenRefresh: ModuleAuthenticationWithHsm#5773521
2020-06-01 04:57:45.907 +00:00 [VRB] [Associate] "Informational": IotHubConnectionString#54135081 TokenRefresher IotHubConnectionString#54135081 ModuleAuthenticationWithHsm#5773521 [{2}]<-->[{3}]
2020-06-01 04:57:45.907 +00:00 [VRB] [Enter] "Informational": InternalClient#63094882 InternalClient_ctor (Microsoft.Azure.Devices.Client.ITransportSettings[1], DeviceClientPipelineBuilder#49924125)

Particularly the issue seems to be with the following cast error:
System.InvalidCastException: Unable to cast object of type 'Newtonsoft.Json.Linq.JValue' to type 'System.String'.
   at Microsoft.Azure.IIoT.Module.Framework.Hosting.ModuleHost.StartAsync(String type, String siteId, String serviceInfo, IProcessControl reset) in D:\a\1\s\common\src\Microsoft.Azure.IIoT.Hub.Module.Fr
amework\src\Hosting\ModuleHost.cs:line 152
   at Microsoft.Azure.IIoT.Modules.OpcUa.Publisher.ModuleProcess.RunAsync() in D:\a\1\s\modules\src\Microsoft.Azure.IIoT.Modules.OpcUa.Publisher\src\ModuleProcess.cs:line 89

This is related to the container configuration, however looking at the code only it is not clear that the issue is. Here is the Container Create Options that are set for the module in Azure Portal:
{
  "Hostname": "publisher",
  "Cmd": [
    "--ap=/appdata",
    "--pf=/appdata/pn.json",
    "--site=bsroom-01",
    "--di=60",
    "--logfile=/appdata/publisher.log",
    "--loglevel=verbose",
    "--fetchname=True",
    "--appcertstoretype=X509Store",
    "--aa",
    "--to",
    "--si=0",
    "--ms=0",
    "--ih=Amqp"
  ],
  "HostConfig": {
    "Binds": [
      "/appdata:/appdata"
    ]
  }
}

Desktop (please complete the following information):

OS: Raspbian Buster
IoT Edge Runtime Version: iotedge 1.0.9

@marcschier Thank you for your offline help. I did two things:

Tried to remove the site setting as you suggested but the latest/2.6.146 continued to fail with the same error
Getting the latest version 2.7.107 solved the above issue but it seems it is not pulling any  data from the OPC server

Here are the logs when the module starts:
2020-06-02 20:15:50.810 +00:00 [INF] Connecting all clients to at-iot-gw-home-bsroom-01 using "MqttOverTcp".
2020-06-02 20:15:50.949 +00:00 [INF] Initiating prometheus at port 9702/metrics
2020-06-02 20:15:51.161 +00:00 [INF] Prometheus metric server started.
2020-06-02 20:15:51.209 +00:00 [INF] Starting module OpcPublisher version <IP_ADDRESS>64.
2020-06-02 20:15:51.223 +00:00 [WRN] Bypassing certificate validation for client.
2020-06-02 20:15:51.239 +00:00 [INF] Running in iotedge context.
2020-06-02 20:15:52.977 +00:00 [INF] 0: Module at-iot-gw-home-bsroom-01_at-iot-gw-home-bsroom-01-opc-publisher reconnected due to "Connection_Ok".
2020-06-02 20:15:53.266 +00:00 [INF] Initialize device twin for at-iot-gw-home-bsroom-01 - at-iot-gw-home-bsroom-01-opc-publisher
2020-06-02 20:15:53.566 +00:00 [INF] Applying initial desired state.
2020-06-02 20:15:53.579 +00:00 [INF] Reporting currently initial state.
2020-06-02 20:15:53.675 +00:00 [INF] Module Host started.
2020-06-02 20:15:53.688 +00:00 [INF] Creating new worker...
2020-06-02 20:15:53.761 +00:00 [INF] File /appdata/pn.json has changed, reloading...
2020-06-02 20:15:53.853 +00:00 [INF] Read 1 items from published nodes file in "00:00:00.0893189"
2020-06-02 20:15:53.955 +00:00 [INF] Converted items to jobs in "00:00:00.1010797"
2020-06-02 20:15:54.224 +00:00 [INF] Starting worker 'StandalonePublisher_0'...
2020-06-02 20:15:54.236 +00:00 [INF] Worker WorkerId: StandalonePublisher_0 {}
2020-06-02 20:15:54.275 +00:00 [INF] Worker: StandalonePublisher_0, start processing new job: Standalone_at-iot-gw-home-bsroom-01_at-iot-gw-home-bsroom-01-opc-publisher, mode: "Active"
2020-06-02 20:15:54.680 +00:00 [WRN] Subscription not registered
2020-06-02 20:16:54.492 +00:00 [INF]
   DIAGNOSTICS INFORMATION for Engine : opc.tcp://<IP_ADDRESS>:4840_92e098a7bcc7d553d6fdc2cf5fe5c0836e11da52
   # Ingestion StartTime - Duration   : "0001-01-01T00:00:00.0000000" - 0 seconds
   # Ingress DataChanges (from OPC)   : 0 - 0/second
   # Ingress ValueChanges (from OPC)  : 0 - 0/second
   # Ingress BatchBlock buffer size   : 0
   # Encoding Block input/output size : 0/0
   # Encoder Notifications processed  : 0
   # Encoder Notifications dropped    : 0
   # Encoder IoT Messages processed   : 0
   # Encoder avg Notifications/Message: 0
   # Encoder avg IoT Message body size: 0
   # Outgress Batch Block buffer size : 0
   # Outgress input buffer count      : 0
   # Outgress IoT message count       : 0 - 0/second
   # Connection retries               : 0

It seems that the module starts and reads the pn.json but it doesn't collect any information from it. The server is working because we are able to connect using UA Expert.
Occasionally, we see the following message in the logs:
2020-06-02 20:19:24.875 +00:00 [WRN] Failed to get or create session 5658e347c46f10c69d1d8a0137366f3156799b86 due to One or more errors occurred. (OPC UA configuration not valid).

How can we troubleshoot this new issue?
Also, why is the timestamp always the beginning of time?

Can you try removing the '--appcertstoretype=X509Store' from your configuration. I think this setting is only working on a windows based host system. The default is for the AppCertStoreType is Directory

@cristipogacean I tried that but it doesn't seem to work. I increased the debug level and here is the output:
2020-06-03 15:30:43.110 +00:00 [INF] Module Host started.
2020-06-03 15:30:43.122 +00:00 [INF] Creating new worker...
2020-06-03 15:30:43.197 +00:00 [INF] File /appdata/pn.json has changed, reloading...
2020-06-03 15:30:43.200 +00:00 [DBG] Reading published nodes file ("00:00:00.0000045"
2020-06-03 15:30:43.288 +00:00 [INF] Read 1 items from published nodes file in "00:00:00.0882693"
2020-06-03 15:30:43.391 +00:00 [INF] Converted items to jobs in "00:00:00.1013629"
2020-06-03 15:30:43.675 +00:00 [INF] Starting worker 'StandalonePublisher_0'...
2020-06-03 15:30:43.682 +00:00 [DBG] Sending heartbeat...
2020-06-03 15:30:43.686 +00:00 [INF] Worker WorkerId: StandalonePublisher_0 {}
2020-06-03 15:30:43.707 +00:00 [DBG] Worker starting...
2020-06-03 15:30:43.707 +00:00 [DBG] Try querying available job...
2020-06-03 15:30:43.726 +00:00 [INF] Worker: StandalonePublisher_0, start processing new job: Standalone_at-iot-gw-home-bsroom-01_at-iot-gw-home-bsroom-01-opc-publisher, mode: "Active"
2020-06-03 15:30:44.919 +00:00 [DBG] Sending job processor heartbeat...
2020-06-03 15:30:51.826 +00:00 [VRB] [Exit] "Informational": MqttIotHubAdapter#14347911 ScheduleCheckConnectTimeout MqttIotHubAdapter#0
2020-06-03 15:30:58.932 +00:00 [DBG] Not found: Session not available - endpointUrl: opc.tcp://<IP_ADDRESS>:4840
2020-06-03 15:30:58.935 +00:00 [DBG] CheckAsync succeeded.
2020-06-03 15:31:03.933 +00:00 [DBG] Not found: Session not available - endpointUrl: opc.tcp://<IP_ADDRESS>:4840
2020-06-03 15:31:03.940 +00:00 [DBG] CheckAsync succeeded.
2020-06-03 15:31:07.589 +00:00 [DBG] Failed to connect on opc.tcp://<IP_ADDRESS>:4840: BadRequestTimeout - try again...
2020-06-03 15:31:07.592 +00:00 [WRN] Failed to get or create session 5658e347c46f10c69d1d8a0137366f3156799b86 due to One or more errors occurred. (BadRequestTimeout).
2020-06-03 15:31:07.594 +00:00 [DBG] Not found: Session/Subscription not available - endpointUrl: opc.tcp://<IP_ADDRESS>:4840
2020-06-03 15:31:08.935 +00:00 [DBG] Not found: Session not available - endpointUrl: opc.tcp://<IP_ADDRESS>:4840
2020-06-03 15:31:08.939 +00:00 [DBG] CheckAsync succeeded.
2020-06-03 15:31:13.938 +00:00 [DBG] Not found: Session not available - endpointUrl: opc.tcp://<IP_ADDRESS>:4840
2020-06-03 15:31:13.943 +00:00 [DBG] CheckAsync succeeded.
2020-06-03 15:31:14.903 +00:00 [DBG] Sending job processor heartbeat...
2020-06-03 15:31:18.957 +00:00 [DBG] Not found: Session not available - endpointUrl: opc.tcp://<IP_ADDRESS>:4840
2020-06-03 15:31:18.961 +00:00 [DBG] CheckAsync succeeded.
2020-06-03 15:31:22.777 +00:00 [DBG] Failed to connect on opc.tcp://<IP_ADDRESS>:4840: BadRequestTimeout - try again...
2020-06-03 15:31:22.783 +00:00 [WRN] Failed to get or create session 5658e347c46f10c69d1d8a0137366f3156799b86 due to One or more errors occurred. (BadRequestTimeout).
2020-06-03 15:31:22.788 +00:00 [DBG] Not found: Session/Subscription not available - endpointUrl: opc.tcp://<IP_ADDRESS>:4840
2020-06-03 15:31:39.092 +00:00 [DBG] Failed to connect on opc.tcp://<IP_ADDRESS>:4840: BadRequestTimeout - try again...
2020-06-03 15:31:39.098 +00:00 [WRN] Failed to get or create session 5658e347c46f10c69d1d8a0137366f3156799b86 due to One or more errors occurred. (BadRequestTimeout).
2020-06-03 15:31:39.102 +00:00 [DBG] Not found: Session not available - endpointUrl: opc.tcp://<IP_ADDRESS>:4840
2020-06-03 15:31:39.104 +00:00 [DBG] CheckAsync succeeded.
2020-06-03 15:31:43.965 +00:00 [DBG] Identity at-iot-gw-home-bsroom-01; at-iot-gw-home-bsroom-01-opc-publisher
2020-06-03 15:31:43.979 +00:00 [INF]
   DIAGNOSTICS INFORMATION for Engine : opc.tcp://<IP_ADDRESS>:4840_bd8ffeb0828845b9cf378c19e3a3e04c013f1a25
   # Ingestion StartTime - Duration   : "0001-01-01T00:00:00.0000000" - 0 seconds
   # Ingress DataChanges (from OPC)   : 0 - 0/second
   # Ingress ValueChanges (from OPC)  : 0 - 0/second
   # Ingress BatchBlock buffer size   : 0
   # Encoding Block input/output size : 0/0
   # Encoder Notifications processed  : 0
   # Encoder Notifications dropped    : 0
   # Encoder IoT Messages processed   : 0
   # Encoder avg Notifications/Message: 0
   # Encoder avg IoT Message body size: 0
   # Outgress Batch Block buffer size : 0
   # Outgress input buffer count      : 0
   # Outgress IoT message count       : 0 - 0/second
   # Connection retries               : 0

2020-06-03 15:32:44.919 +00:00 [DBG] Sending job processor heartbeat...
2020-06-03 15:32:59.511 +00:00 [DBG] Failed to connect on opc.tcp://<IP_ADDRESS>:4840: BadRequestTimeout - try again...
2020-06-03 15:32:59.515 +00:00 [WRN] Failed to get or create session 5658e347c46f10c69d1d8a0137366f3156799b86 due to One or more errors occurred. (BadRequestTimeout).
2020-06-03 15:32:59.518 +00:00 [DBG] Not found: Session not available - endpointUrl: opc.tcp://<IP_ADDRESS>:4840

So, it seems that the publisher cannot connect to the OPC server. We checked the server and we are also able to connect to it using UA Expert. Also, the Publisher container is able to ping the IP address. So, right now we are stuck with figuring out this connectivity issue and why it is failing.
We also tried older versions like 2.5.1, 2.5.2, and 2.5.3 - the errors we see there are from the OPC-UA library:
2020-06-03 15:46:36.064 +00:00 [INF] Publisher is running. Press CTRL-C to quit.
2020-06-03 15:46:36.105 +00:00 [INF] Connect and monitor session and nodes on endpoint 'opc.tcp://<IP_ADDRESS>:4840'.
2020-06-03 15:46:36.133 +00:00 [VRB] OPC: GetEndpoints Called. RequestHandle=1, PendingRequestCount=1
2020-06-03 15:46:36.158 +00:00 [VRB] OPC: Channel 0 in Connecting state.
2020-06-03 15:47:09.297 +00:00 [VRB] OPC: ***EXCEPTION*** BadNotConnected
2020-06-03 15:47:09.387 +00:00 [VRB] OPC: ***EXCEPTION*** BadNotConnected 'Error establishing a connection: BadNotConnected'
2020-06-03 15:47:09.393 +00:00 [VRB] OPC: GetEndpoints Completed. RequestHandle=1, PendingRequestCount=0, StatusCode=Bad
2020-06-03 15:47:09.398 +00:00 [VRB] OPC: Channel 0: Close
2020-06-03 15:47:09.406 +00:00 [VRB] OPC: Channel 0 in Closed state.
2020-06-03 15:47:09.408 +00:00 [VRB] OPC: CLIENTCHANNEL SOCKET CLOSED: FFFFFFFF, ChannelId=0
2020-06-03 15:47:09.412 +00:00 [ERR] Session creation to endpoint 'opc.tcp://<IP_ADDRESS>:4840' failed 1 time(s). Please verify if server is up and Publisher configuration is correct.
Opc.Ua.ServiceResultException: Error establishing a connection: BadNotConnected
   at Opc.Ua.Bindings.ChannelAsyncOperation`1.End(Int32 timeout, Boolean throwOnError)
   at Opc.Ua.Bindings.UaSCUaBinaryClientChannel.EndSendRequest(IAsyncResult result)
   at Opc.Ua.Bindings.UaSCUaBinaryTransportChannel.EndSendRequest(IAsyncResult result)
   at Opc.Ua.Bindings.UaSCUaBinaryTransportChannel.SendRequest(IServiceRequest request)
   at Opc.Ua.DiscoveryClient.GetEndpoints(RequestHeader requestHeader, String endpointUrl, StringCollection localeIds, StringCollection profileUris, EndpointDescriptionCollection& endpoints)
   at Opc.Ua.DiscoveryClient.GetEndpoints(StringCollection profileUris)
   at Opc.Ua.Client.CoreClientUtils.SelectEndpoint(String discoveryUrl, Boolean useSecurity, Int32 operationTimeout)
   at OpcPublisher.OpcSession.ConnectSessionAsync(CancellationToken ct) in D:\a\1\s\modules\src\Microsoft.Azure.IIoT.Modules.OpcUa.Publisher\src\OpcSession.cs:line 431
2020-06-03 15:47:09.541 +00:00 [DBG] Update node configuration file, version: 000000B2

It seems to be a connectivity issue between the Publisher and the OPC server but the logs are not enough to understand how to fix that. Any ideas?

@cristipogacean We tried all of those. There are no firewalls between the host and the OPC server. We even did a telnet on the port 4840 from within the container and telnet can connect to the IP address on that port. We also restarted the server, which should disconnect the sessions.
One of the issues we used to have in the past is the security because the server uses self signed cert. Hence, we use the -aa switch to accept all certs. Any other suggestions?

Does the server have an endpoint that allows security policy none? using security policy none, the app certificates are disregarded.
What kind of server is this? any chance to look into the logs of the server?

@cristipogacean Would you mind explaining what this means: using security policy none, the app certificates are disregarded?
The OPC server is a proprietary device from a German manufacturer Rittal. When we connect to the server via UA Expert, there are several messages that show up that are related to (we assume) how the protocol is implemented and how the device is configured.
Here the sequence:
 (we can change the host name to the IP address or add a DNS entry for the server if needed.
Yes, there is an endpoint with security None:

Also, the certificate is untrusted because it is self signed:

For the latter two, we don't have a lot of wiggle room to change because this is done by the manufacturer and we have no way to update the endpoint or the cert. So, we are stuck with them.
Any suggestions?

Just FYI, the connectivity to the server works from within the container. Here are all the tests we did:
# env
HOSTNAME=at-iot-gw-home-bsroom-01-pub
HOME=/root
OLDPWD=/app
DOTNET_RUNNING_IN_CONTAINER=true
IOTEDGE_DEVICEID=at-iot-gw-home-bsroom-01
IOTEDGE_APIVERSION=2019-11-05
IOTEDGE_WORKLOADURI=unix:///var/run/iotedge/workload.sock
TERM=xterm
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
IOTEDGE_AUTHSCHEME=sasToken
IOTEDGE_GATEWAYHOSTNAME=at-iot-gw-home-bsroom-01
IOTEDGE_MODULEID=at-iot-gw-home-bsroom-01-opc-publisher
ASPNETCORE_URLS=http://+:80
PWD=/
IOTEDGE_IOTHUBHOSTNAME=iotgw-ioth-dev-usw2-sandbox.azure-devices.net
RuntimeLogLevel=Information
IOTEDGE_MODULEGENERATIONID=637263279964864624
# ping <IP_ADDRESS>
PING <IP_ADDRESS> (<IP_ADDRESS>) 56(84) bytes of data.
64 bytes from <IP_ADDRESS>: icmp_seq=1 ttl=63 time=2.52 ms
64 bytes from <IP_ADDRESS>: icmp_seq=2 ttl=63 time=4.11 ms
64 bytes from <IP_ADDRESS>: icmp_seq=3 ttl=63 time=6.37 ms
^C
--- <IP_ADDRESS> ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 2.526/4.339/6.379/1.582 ms
# telnet <IP_ADDRESS> 4840
Trying <IP_ADDRESS>...
Connected to <IP_ADDRESS>.
Escape character is '^]'.

We don't think this is a connectivity issue but an issue how the Publisher tries to connect to the OPC server.
To answer your question about the server logs, the manufacturer doesn't expose those and we have no way to look at the device logs.

@toddysm thanks for the details! I might now have an idea what's going wrong.
Apparently the server uses it's hostname when responding to the GetEndpoints service call from the publisher. then the publisher is trying to connect using the hostname instead of the ip address. This is probably not properly resolved by the DNS in the container's context and we have the network timeout ... The UA expert is either capable to resolve the hostname since is not running in a container context (I assume) or acts smarter and eventually forces the connection using the provided IP address after failing to connect using the server's retrieved endpoint url. if that's the case, the server's behavior is not exactly compliant, but we should be smarter in the publisher as well and handle the issue - I'll add a work item on this.
you can try pinging the funny hostname "CMCIII-PU-..." and see what happens. You can maybe add  force host's dns to resolve it properly somehow (e.g. by adding it in the C:\Windows\System32\drivers\etc\hosts on a windows box)
btw, on the point: using security policy none, the app certificates are disregarded? I meant that when a client connects to a server using security policy none, both client and the server will allow the connection even if the pair's certificate is not trusted - typically the apps will log a warning, but will continue the communication.
hope this helps you moving further.

hi @toddysm, I'm running out of ideas. Can you share more logs, or sync for a remote debugging session?

@cristipogacean We don't know what the problem was but after several (in the number of 10s if not 100s) restarts of the container and the OPC server, the things connected. We are totally puzzled for the reason why is this happening. I think we can close this but the root cause is unclear.
One suggestion if you have any influence into the Opc.Ua libraries - the logging there seems to be not very detailed. It will be good to have more detailed logging in that library with detailed messages that will help with troubleshooting. Also, if there is any caching in the libraries, it will be good to know how to clean it because my assumption was that this was a cache problem from the initially wrong configured hostname.
As I mentioned, everything else worked more reliably and responded with understandable errors. We even tried Python asyncua and it worked out of the box.

@cristipogacean I did speak too soon though. Here is the situation:
We managed to get this working with v 2.5.3 but switching to 2.7.107 breaks the container and it start cycling continuously and we cannot get logs out of it. The configuration is exactly the same, the only thing we changed was the container tag from 2.5.3 -> 2.7.107.

Good to see you're unlocked! and thanks for the feedback. There is no caching or something similar in the modules.
Can you please share the configuration? there are some changes in the arguments and some were obsoleted in 2.7.
If the config is same as above can you try removing the following:
"--appcertstoretype=X509Store"
maybe you can also add:
"--PkiRootPath=/appdata/pki"  this will set the certificates root path to your mounted volume

Hi @cristipogacean, we already tried removing the
--appcertstoretype=X509Store

but it didn't help. Adding
--PkiRootPath=/appdata/pki

didn't help either - the container fails at start and continues restarting.


Implement secure boost scheme phase 2 - enable secure xgboost via processor interface

Add extensions to the vertical pipeline implemented in phase 1 to call Processor interface at proper locations to enable secure features:

Before broadcasting g/h pairs, call interface to prepare buffer
Communicate GHistIndexMatrix and RowSetCollection to interface for local gRPC handler in response to AllGather call

This PR is dependent on https://github.com/dmlc/xgboost/pull/10073, which should be merged first

Hi @trivialfis, this PR is ready for review, we have some remaining failed checks due to two issues:
3 windows checks: Cannot open include file: 'dlfcn.h' -> @nvidianz is looking into this
2 mac_os: Error: The version '3.8' with architecture 'arm64' was not found for macOS 14.4.1. -> this seems to be a recent issue impacting everyone.

Hi @trivialfis, this PR is ready for review, currently we have 6 failed checks, but I think they are out of the scope of this PR, we could have more issues when triggering buildkite:

3 for windows, as we do not support windows at this time
2 for R, due to Error: Failed to get R release: Failed to get Ghostscript
1 for pylint, related to spark.core (is this a new thing?)

Run python tests/ci_build/lint_python.py --format=0 --type-check=0 --pylint=1
************* Module xgboost.spark.core
xgboost/spark/core.py:1450:5[7](https://github.com/dmlc/xgboost/actions/runs/9134298149/job/25119550353?pr=10124#step:7:8): E0606: Possibly using variable 'base_margin_col' before assignment (possibly-used-before-assignment)

combined with https://github.com/dmlc/xgboost/pull/10231, close this one


As a user I want to be able to manage what categories I am subscribed to

Allow users to manage their current category subscriptions within the profile modal. Would like to implementation to act the same as Create/Edit Listing->Categories (Have a 'bar' which will hold all the categories currently subscribed and have an option to remove categories.

merged


ReferenceError: JSONEditor is not defined In production

django-json-widget version: 2.0.1
Django version: 2.2.28
Python version:3.7
Operating System:Ubuntu

Description
It's working as expected in local but when I am trying to run it on production for postgres json fields, it's showing blank and in the console it's throwing the error  ReferenceError: JSONEditor is not defined

It was due to cross origin request


Getting New Error

Hi, just updated to latest version. Was working great before and has really nice configuration, functionality.
Now I'm getting this when I try an action.
4/26/2020, 13:40:34] TypeError: Cannot read property 'replace' of undefined
at new PushOverMessenger (/usr/local/lib/node_modules/homebridge-messenger/lib/pushover.js:16:45)
at HomebridgeMessenger. (/usr/local/lib/node_modules/homebridge-messenger/index.js:78:27)
at Characteristic.On.emit (events.js:310:20)
at Characteristic.On.Characteristic.setValue (/usr/local/lib/node_modules/homebridge/node_modules/hap-nodejs/lib/Characteristic.js:321:10)
at Bridge. (/usr/local/lib/node_modules/homebridge/node_modules/hap-nodejs/lib/Accessory.js:882:22)
at Array.forEach ()
at Bridge.Accessory._handleSetCharacteristics (/usr/local/lib/node_modules/homebridge/node_modules/hap-nodejs/lib/Accessory.js:822:8)
at HAPServer.emit (events.js:310:20)
at HAPServer._handleCharacteristics (/usr/local/lib/node_modules/homebridge/node_modules/hap-nodejs/lib/HAPServer.js:974:10)
at HAPServer. (/usr/local/lib/node_modules/homebridge/node_modules/hap-nodejs/lib/HAPServer.js:209:39)
[4/26/2020, 13:40:34] Got SIGTERM, shutting down Homebridge...
[4/26/2020, 13:40:39] [Config] [HapClient] [<IP_ADDRESS>:51748 (0E:5C:34:12:B7:CD)] Failed to set value for Alarm - Downstairs.
[4/26/2020, 13:40:39] [HB Supervisor] Homebridge Process Ended. Code: 143, Signal: null
Any ideas what I did wrong?

@txcvedvr  : Solved in new version 0.0.6. It's published and available now.

Confirmed and working great. Thanks.


Several features mentionning 2020 when we are already end of 2021

Some feature availability mention 2020 as potential availability date.
As we are already end of 2021, these dates should be updated to reflect our current roadmap.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 19a75e1f-470d-2dda-a269-80d52b2a6314
Version Independent ID: 634dbb4a-815f-3e65-c99f-1fa75a5752d7
Content: Power Apps operated by 21Vianet and Power Automate operated by 21Vianet - Power Platform
Content Source: power-platform/admin/business-applications-availability-china.md
Service: power-platform
Sub-service: admin
GitHub Login: @jimholtz
Microsoft Alias: jimholtz

Taking to internal thread.