Thoughts on reducing some abstraction around the wallet interface

ACA-Py started off built on the Indy SDK and then later transitioned to Aries Askar for the wallet. An in-memory wallet has also been supported for some time.
We recently dropped the Indy SDK, leaving just the in-memory wallet and Askar.
What do we think of dropping the in-memory wallet and leaning into Askar? Do we find value in continuing to leave the interface abstract?
It would take some effort but I think we would see a fairly significant simplification if we just committed to Askar.
@swcurran @ianco @PatStLouis @jamshale @TelegramSam

I'm all for simplification.  I like the idea of dropping the in-memory wallet, I've actually seen this cause trouble for newcomers.
Askar itself is an interface/abstraction to secure storage.  It makes sense to me to view and rely on it as such.  Allowing the abstractions to be removed from ACA-Py to simplify the interface and integration.

I don't use, or really understand the need for the in-memory wallet so I'm all for removing the complexity.

I think some tests use an in-memory wallet (where a wallet is needed) however if so they should be able to use sqlite, so I agree let's get rid of the in-memory wallet.

Sounds good.  Do we need to deprecate or just remove?  Is it worth replacing it with SQLite In Memory?  I had thought that was what the in-memory used — didn’t realize it was a separate implementation.  Would Ask ar need to be updated for that?

Most likely follow a deprecation path as we've seen with some of the routes / protocols.
What is the difference between askar and askar-anoncreds? could we see some alignment eventually? Ideally we can get rid of the wallet-type entirely and just use askar?

askar-anoncreds is used to load different contexts. It decides whether to load the anoncreds api and modules. There's also some logic in credential and presentation handling that checks the wallet type. It still uses askar but some of the models stored in the wallet are different.
Eventually we'd want to only have the askar-anoncreds wallet type but that seems a long way off if it ever happens. It probably could have been done a different way than using wallet type but it works fine, other than being a bit confusing.

I don't see why we can't just remove it?

What is the difference between askar and askar-anoncreds? could we see some alignment eventually? Ideally we can get rid of the wallet-type entirely and just use askar?

askar-anoncreds is used to load different contexts. It decides whether to load the anoncreds api and modules. There's also some logic in credential and presentation handling that checks the wallet type. It still uses askar but some of the models stored in the wallet are different.

We can persist the idea of selecting a wallet-type even while collapsing some of the abstractions in the background. The askar-anoncreds type is still an Askar wallet, of course, so it will behave the same the vast majority of the time.

Okay, I think we have a consensus. I'll reword the issue title to reflect that. I'll see where this fits into other planned and ongoing refactors.

I'm going to work on this when I have time. Anything that removes unneeded complexity is worth the effort imo.

Going to close this. Wallet types aren't flattened yet but the in_memory wallet has been removed.


Handle concurrent connections

Hello,
I have a simple Vue.js app, directly connected to Cockpit. One of my clients want to prevent a user to login multiple times on different devices.
Is there a way to do this ?

please visit https://discourse.getcockpit.com/ for support questions. Thank you!


Add timestamps to stdout, file loggers and editor Output panel

Timestamps use ISO 8601 format, prefixing every printed line with YYYY-MM-DDTHH:MM:SS+HH:MM |. +HH:MM at the end denotes the timezone offset.
Timestamp display can be toggled separately for stdout/stderr and file loggers in the Project Settings. By default, timestamps are logged to files, are not printed in stdout/stderr and are not shown in the editor Output panel. The "compact identical lines" option in the editor Output panel works correctly when timestamps are shown, with the timestamp becoming the time of the latest print.

This closes https://github.com/godotengine/godot-proposals/issues/1786.

Example log file
2023-07-16T00:01:15+02:00 | Godot Engine v4.2.dev.custom_build.6b39157b9 - https://godotengine.org
2023-07-16T00:01:15+02:00 | Vulkan API 1.3.242 - Forward+ - Using Vulkan Device #0: NVIDIA - NVIDIA GeForce RTX 4090
2023-07-16T00:01:16+02:00 |
2023-07-16T00:01:16+02:00 | Test message.
2023-07-16T00:01:16+02:00 |
2023-07-16T00:01:16+02:00 | Test message.
2023-07-16T00:01:16+02:00 | Test message 2.
2023-07-16T00:01:16+02:00 | Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nullam est sapien, hendrerit a leo a, accumsan condimentum magna. Aliquam et nisl et nisi aliquet elementum ac id sapien. Cras nec rutrum risus. Suspendisse vel urna vel ante posuere varius ac sit amet nibh. Pellentesque ante est, venenatis a laoreet ut, aliquam quis nulla. Donec interdum malesuada lorem, in egestas augue ultricies quis. Integer sagittis dolor nunc, in ullamcorper felis scelerisque cursus. Nulla suscipit ut velit at vehicula. Nulla tempor dui nec fermentum porttitor. Sed ullamcorper hendrerit dictum. Curabitur non accumsan velit. Praesent eleifend dolor magna, tristique vehicula lacus dignissim at. Cras vel ultrices ipsum, quis condimentum sapien. Proin ut tortor varius, egestas nulla ut, aliquam nisi. Maecenas eu mi nec tellus pulvinar posuere ac a risus. Suspendisse nisl metus, fermentum vel tortor sit amet, mollis dapibus tortor. Etiam luctus ante elit, nec ultrices est dignissim a. Aenean vel facilisis nibh. Praesent eros leo, luctus non convallis quis, finibus et arcu. In ac elementum augue. Maecenas efficitur tortor eros, nec cursus est sagittis posuere. Curabitur bibendum posuere risus, vel rutrum lacus fermentum non. Sed quis quam eget justo posuere dictum. Donec nec feugiat libero. Donec suscipit dui at leo vestibulum, eget efficitur sapien porta. Maecenas et nunc vel nibh egestas sagittis eget nec nibh. Mauris fermentum est at aliquet auctor. Morbi a erat eget lectus porttitor maximus id vel diam. Proin et sodales leo. Fusce suscipit tincidunt porta. Donec sed turpis vel nulla lobortis rhoncus. Sed sollicitudin eget nulla quis blandit. Aliquam cursus porttitor neque et eleifend. Pellentesque efficitur finibus erat. Aliquam hendrerit arcu at elit feugiat molestie. Donec consectetur, nisi id tempus luctus, velit tellus auctor lorem, vitae facilisis urna ex eget felis. Morbi nec purus enim. Integer et quam tincidunt, interdum neque vel, ultricies sapien. Donec lobortis metus ipsum, id semper diam maximus ut. Mauris a felis et mauris ullamcorper viverra vitae in massa. Proin egestas felis euismod sapien sagittis ultricies. Quisque suscipit tristique ligula, a blandit lorem aliquet id. Quisque feugiat id est tristique cursus. Sed tincidunt eros ac ipsum consectetur tempus. Sed lacinia, sapien in dictum consectetur, tortor sapien malesuada orci, sit amet elementum lacus lorem a ante. Etiam ultricies, urna finibus maximus iaculis, nibh velit egestas nibh, et efficitur nibh nibh vitae sapien. Morbi quis suscipit est, nec rutrum elit. Pellentesque at mattis elit. Aliquam nulla urna, vehicula suscipit justo a, cursus dignissim ipsum. Curabitur finibus vestibulum aliquet. Nam scelerisque justo sed bibendum tempus. Aliquam erat volutpat. Aliquam eget malesuada ex. In hac habitasse platea dictumst. Suspendisse tempus dolor vitae dignissim aliquet. Sed magna dolor, euismod eget fringilla non, imperdiet et libero.

I've fixed timestamps not appearing in the written log files, so this PR is now ready for review.

Doesn't compile. How expensive are the added timestamp and timezone calculations on each message? Logging/printing is already fairly expensive, we should be careful about possibly increasingly its cost significantly.

Doesn't compile. How expensive are the added timestamp and timezone calculations on each message? Logging/printing is already fairly expensive, we should be careful about possibly increasingly its cost significantly.

I've fixed compilation and have benchmarked the new logging code. Unfortunately, it's about 4× slower than before, with each log taking on average 8 microseconds (where it previously required 1-2 microseconds).
This isn't due to using the higher-level String approach (I've tried replacing the timestamp string with a constant), but OS::get_singleton()->get_datetime_string_from_system() is relatively slow on its own. Maybe we could ensure it's called only once per second at most to reduce the overhead when spamming prints (we only display time up to the nearest second anyway).

As a way to minimize performance impact, would it be useful to only log ticks (formatted in MM:SS:msec) rather than system time? The log file itself I believe has the process launch time in the filename, so it would be possible to figure out the absolute time if needed.

As a way to minimize performance impact, would it be useful to only log ticks (formatted in MM:SS:msec) rather than system time? The log file itself I believe has the process launch time in the filename, so it would be possible to figure out the absolute time if needed.

Yeah I think using engine tick count would definitely be the way to go. It can still be formatted as absolute time if wanted, by calculating the diff to the original system time when the tick count started. It might drift a bit but for this purpose it's probably not critical.
Alternatively, a relative time since the start of the process also makes sense to me in a game log context (i.e. "how far into the game session did the user run into these bugs?").
We still need to make sure that whatever calculations are done for pretty formatting aren't too heavy - but avoiding system calls is definitely a must if we want to log timestamps.


Add Support for Pydantic Discriminated Union

Since Pydantic 1.9, pydantic has supported the use of discriminated unions as a way to force what union type Pydantic evaluates an object by. Pydantic models with discriminated unions raise an error, at least because the typing.Literal is not supported
Feature Request Type

[ ] Core functionality
[x] Alteration (enhancement/optimization) of existing feature(s)
[ ] New behavior

Description
The following code will raise the error when it's converted to a GraphQL model: TypeError: fields cannot be resolved. Unexpected type 'typing.Literal"
class Foo(BaseModel):
    class_type: Literal["foo"] = "foo"
    foo: str

class Bar(BaseModel):
    class_type: Literal["bar"] = "bar"
    bar: str

class Model(BaseModel):
    foobar: Union[Foo, Bar] = Field(discriminator="class_type")

The point of this, is to tell pydantic explicitly if the foobar variable should be parsed as a Bar object or a Foo object. The default behavior without discriminated unions is to try Foo first, which will always succeed in this case.

I'm also running into this when generating Pydantic models using https://github.com/jhnnsrs/turms


Status of shared data

[x] Code changes
[x] Unit testing
[ ] QA

https://github.com/atsign-company/server/issues/13

Have to merge PR from feature branch to trunk

Tested with sdk_refactoring branch. I could see the shared key status as local_updated, remote_updated, shared_with_notifier, Shared_with_llookup
local_updated:
Metadata{ttl: null, ttb: null, ttr: null,ccd: null, isPublic: false, isHidden: false, availableAt : null, expiresAt : null, refreshAt : null, createdAt : 2021-07-22 06:24:42.315807Z,updatedAt : 2021-07-22 06:24:42.315844Z,isBinary : null, isEncrypted : null, isCached : false, dataSignature: null, sharedKeyStatus: LOCAL_UPDATED}
remote_updated:
{"key":"@sitaram:pincode@murali","data":"634509","metaData":{"createdBy":null,"updatedBy":null,"createdAt":"2021-07-22 06:31:53.060984Z","updatedAt":"2021-07-22 06:31:53.060984Z","availableAt":null,"expiresAt":null,"refreshAt":null,"status":"active","version":0,"ttl":null,"ttb":null,"ttr":null,"ccd":null,"isBinary":false,"isEncrypted":false,"dataSignature":null,"sharedKeyStatus":"REMOTE_UPDATED"}}


feat(Analysis/PSeries): some summability results

Summability of n ↦ 1 / |n + a| ^ s and n ↦ n ^ k exp (-r * n)

[ ] depends on: #11069

The results that are not in #11069 (in Analysis.PSeries and Analysis.SpecialFunctions.Exp) look good.


Add the page in Japanese to the doc

What this PR does / why we need it:
Which issue(s) this PR fixes:
Fixes #
Does this PR introduce a user-facing change?:

NONE

@ono-max I appreciate you. I left a couple of comments.

@nakabonne
Hey, could you review this one?
Thanks!

Thank you!
/approve


Create scripts for the calculation of the indicator and add sample data:

[ ] SPSS
[ ] R
[ ] Stata
[ ] Sample data file

wfp_ici_syntaxe_1108.pdf


Examples more friendly to share on social networks

A feature that blogs usually get for free, allowing them to share links on social networks (chats, forums, etc.) that look pretty, with an image, description, etc.

https://github.com/wpilibsuite/sphinxext-opengraph
https://github.com/choldgraf/sphinx-social-previews
Sphinx meta directive https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html#html-metadata
To preview a specific page: https://socialsharepreview.com/

Good use case for https://github.com/holoviz-topics/examples/issues/91 ! :-)

Yep, although we'll need to make sure social platforms follow the redirects, to the page that has the opengraph/whatever metadata.


Unable to delete the first or (only) question

The first question contained in the set of questions can't be deleted.

Fixed in commit 18a462368d796d43977c2190c8498821d4e44a3c.


[EAGLE-1090] - update for compatibiliy to python3

used requests instead of urllib
updated print statements
used input instead of raw_input

why this was closed and not merged?


Unexpected roadnet rendering

I generated a raw roadnet.json with some roadLinks like below:
     {
          "type": "turn_left",
          "startRoad": "edge-1",
          "endRoad": "edge-10",
          "laneLinks": [
            {
              "startLaneIndex": 0,
              "endLaneIndex": 0,
              "points": [
                {
                  "x": 960,
                  "y": 390
                },
                {
                  "x": 978.6,
                  "y": 392.52
                },
                {
                  "x": 996.98,
                  "y": 399.63
                },
                {
                  "x": 1014.6,
                  "y": 410.62
                },
                {
                  "x": 1030.94,
                  "y": 424.8
                },
                {
                  "x": 1045.45,
                  "y": 441.46
                },
                {
                  "x": 1057.62,
                  "y": 459.91
                },
                {
                  "x": 1066.91,
                  "y": 479.44
                },
                {
                  "x": 1072.78,
                  "y": 499.36
                },
                {
                  "x": 1074.71,
                  "y": 518.97
                },
                {
                  "x": 1072.17,
                  "y": 537.57
                }
              ]
            }
          ]
        },

As posted, the lane link is organized by some points which should be performed like a real curve. But I got weird intersections without any internal lane link displaying:

Does the frontend tool support this feature?

Hi.
I guess the problem is caused by "width". Did you set the "width" property of the intersection? This represents the size of the intersection. If it is set to 0, the endpoints of the roads will overlap. It is recommended to set width to a large enough value so that the roads do not overlap.
And lanelinks won't be displayed explicitly. Vehicles which have entered an intersetion will run along the lanelink but it won't be displayed as a line or curve.

Thank you for your help. @soodoshll

I have some questions, How the road figure was rendered, I can not get the commands that can render the rodent.json. Could you answer this problem? Thank you very much.

I have some questions, How the road figure was rendered, I can not get the commands that can render the rodent.json. Could you answer this problem? Thank you very much.

If you already have roadnet.json, you can use this roadnet file to run a simulation and it will generate a replay file (maybe roadnet_replay.json) and you can use web browser to open frontend/index.html to replay it. For this part, please refer to docs about how to run a simulation. If you don't have the vehicle flow configuration, you can use a empty file.


px-data-table-column  : Documentation and explicit assignment

type="dropdown" dropdown-items='[{"key":"one","val":"Mike"},{"key":"two","val":"Maria"},{"key":"three","val":"Wei"}]'
Please provide list of various different 'type'
and also a way to have different drop down for different row.

I've updated the code comments / API documentation for px-data-table-column to reflect all three type options (string, html, dropdown).
We'll have to think about your enhancement request, I'm not sure how we would go about making the dropdown options dynamic by row, since everything except the table-data is column-based rather than row-based.

It's hard to imagine the use case with such a short description, please elaborate further what you are trying to achieve. I'm going to close this for now, if I'm guessing what you are trying to do, it would be pretty hard to achieve and would require us embedding meta data for each cell into the data itself. Which would be slow/painful/API breaking.


mime: please add 'text/x-markdown' for markdown file extensions (md, markdown) to the _defaultExtensionMap

This issue was originally filed by<EMAIL_ADDRESS>
0.6.21_r26639
As markdown is quite common these days, and this change does not conflict with anything currently in the _defaultExtensionMap, I think it is reasonable to add it?
'md':'text/x-markdown',
'markdown':'text/x-markdown',
The mime type 'text/x-markdown' seems to be what is most commonly used, for example:
node-mime: https://github.com/broofa/node-mime/pull/48/files
CodeMirror: http://codemirror.net/mode/markdown/
thanks!

This issue has been moved to dart-lang/mime#1.


Mounting host directories into container from Windows self-hosted agent

Agent Version and Platform
2.155.1
OS of the machine running the agent? OSX/Windows/Linux/...
Windows Server 2019 (1903)
Azure DevOps Type and Version
dev.azure.com
What's not working?
Running a build within a Windows container + mounting a host directory (this is on a self-hosted agent). The mount works fine (I'm guessing the mount string is just passed as is, with variable replacement). However, subsequent steps fail with the following error:
##[error]Value cannot be null. |
-- | --
  | Parameter name: value |
  | ##[debug]System.ArgumentNullException: Value cannot be null. |
  | Parameter name: value |
  | at System.String.StartsWith(String value, StringComparison comparisonType) |
  | at System.Collections.Generic.List1.FindIndex(Int32 startIndex, Int32 count, Predicate1 match) |
  | at Microsoft.VisualStudio.Services.Agent.Worker.Handlers.ContainerStepHost.ResolvePathForStepHost(String path) |
  | at Microsoft.VisualStudio.Services.Agent.Worker.TaskRunner.RunAsync() |
  | at Microsoft.VisualStudio.Services.Age
I believe the issue is in https://github.com/microsoft/azure-pipelines-agent/blob/13a2f4823d584526ae801b73fdf46c8e45e8364e/src/Agent.Worker/Container/ContainerInfo.cs#L262 in that it does not account for the possibility of drive letters in the volume mount, resulting in more colons than any of it's expected use cases.

@richardgavel  - are you able to try this with agent version 2.156 or later and verify this fixes your issue?

Closing after no response and issue is determined to be fixed from our side.


Jail created with -m src=/path might not have a consistent /usr/src after update.

It looks like after an update to a jail that was created with -m src=/path the jail's /usr/src could be in an inconsistent state if the updated sources had files deleted compared to what was in the jail /usr/src before the update. I suggest that install_from_src in jail.sh deletes the jail /usr/src completely before copying a new set of sources to avoid this problem.

8808b527c462834 should resolve this.

That is correct and the fix seems to be already in poudriere-devel. Issue solved for me.


Twitter friends of a specific app

Hi,
Using 'friends/list' , I am able to get list of friends of a particular user. But I want to get only those friends who have logged in using my particular app . Is there way to do this?

Twitter doesn't have a concept of online or offline so nope.
You can check their profile to see what twitter client they used to send each tweet and check if your app is in there which is probably the next closest thing?


forward_client_cert_details cannot be given the value SANITIZE

Bug Description
The following EnvoyFilter doesn't change Envoy's forward_client_cert_details:
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: forward-client-cert
  namespace: istio-system
spec:
  configPatches:
  - applyTo: NETWORK_FILTER
    match:
      listener:
        filterChain:
          filter:
            name: "envoy.http_connection_manager"
    patch:
      operation: MERGE
      value:
        typed_config:
          "@type": "type.googleapis.com/envoy.config.filter.network.http_connection_manager.v2.HttpConnectionManager"
          forward_client_cert_details: SANITIZE
          max_request_headers_kb: 86

See an inspection of Envoy's config dump:
$ curl -s <IP_ADDRESS>:15000/config_dump | grep -E "max_request_headers_kb|forward_client_cert_details"
            "max_request_headers_kb": 86,
            "max_request_headers_kb": 86,
            "max_request_headers_kb": 86,
            "max_request_headers_kb": 86,
            "forward_client_cert_details": "APPEND_FORWARD",
            "max_request_headers_kb": 86,
            "forward_client_cert_details": "APPEND_FORWARD",
            "max_request_headers_kb": 86,
            "forward_client_cert_details": "APPEND_FORWARD",
            "max_request_headers_kb": 86,
            "forward_client_cert_details": "APPEND_FORWARD",
            "max_request_headers_kb": 86,
            "forward_client_cert_details": "APPEND_FORWARD",
            "max_request_headers_kb": 86,
            "forward_client_cert_details": "APPEND_FORWARD",
            "max_request_headers_kb": 86,
            "max_request_headers_kb": 86,

Other values of forward_client_cert_details do though; for example the following EnvoyFilter:
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: forward-client-cert-details
  namespace: istio-system
spec:
  configPatches:
  - applyTo: NETWORK_FILTER # http connection manager is a filter in Envoy
    match:
      # context omitted so that this applies to both sidecars and gateways
      listener:
        filterChain:
          filter:
            name: "envoy.filters.network.http_connection_manager"
    patch:
      operation: MERGE
      value:
        name: "envoy.filters.network.http_connection_manager"
        typed_config:
          "@type": "type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager"
          forward_client_cert_details: FORWARD_ONLY
          max_request_headers_kb: 87

does:
$ curl -s <IP_ADDRESS>:15000/config_dump | grep -E "max_request_headers_kb|forward_client_cert_details"
            "forward_client_cert_details": "FORWARD_ONLY",
            "max_request_headers_kb": 87,
            "forward_client_cert_details": "FORWARD_ONLY",
            "max_request_headers_kb": 87,
            "forward_client_cert_details": "FORWARD_ONLY",
            "max_request_headers_kb": 87,
            "forward_client_cert_details": "FORWARD_ONLY",
            "max_request_headers_kb": 87,
            "forward_client_cert_details": "FORWARD_ONLY",
            "max_request_headers_kb": 87,
            "forward_client_cert_details": "FORWARD_ONLY",
            "max_request_headers_kb": 87,
            "forward_client_cert_details": "FORWARD_ONLY",
            "max_request_headers_kb": 87,
            "forward_client_cert_details": "FORWARD_ONLY",
            "max_request_headers_kb": 87,
            "forward_client_cert_details": "FORWARD_ONLY",
            "max_request_headers_kb": 87,
            "forward_client_cert_details": "FORWARD_ONLY",
            "max_request_headers_kb": 87,
            "forward_client_cert_details": "FORWARD_ONLY",
            "max_request_headers_kb": 87,

I would expect to be able to set the value SANITIZE for the forward_client_cert_details.
Version
$ helm version --short
v3.6.1+g61d8e8c
$ k0 version --short
Client Version: v1.23.0
Server Version: v1.20.9

istiod & base helm charts at versions `1.12.1`

Additional Information
No response

This is essentially the same as https://github.com/istio/istio/issues/18169 in terms of root cause

Thanks, does that mean you don't think it is fixable (given https://github.com/istio/istio/issues/18169#issuecomment-749593973)?

Perhaps this & a few other limitations w/ EnvoyFilter could be fixed by adding them to the Gateway object as an optional field instead?

another thought... what about adding a path argument for REPLACE?

Is there any way to achieve this? I need to disable forwarding cert:
"set_current_client_cert_details": {
             "subject": true,
             "cert": false,
             "dns": true,
             "uri": true
},


Unified pipeline requirements

Addresses #65, aiming at a smooth hand off to #39
Overview
This PR adds the following behavior to the initialization of pipeline/step objects:
steps now extract model-specific dependencies from any models referenced in a default argument, e.g. @step-decorated function like def fn(*arg, model=garden_ai.Model('...')): pass will have recorded the dependencies listed in the mlflow model's conda.yml in fn.model_conda_dependencies and/or fn.model_pip_dependencies, per the conda.yml
pipelines collect any of those step-level dependencies in pipeline.conda_dependencies, and collect any pipeline-level dependencies listed by pipeline.requirements_file in either pipeline.conda_dependencies or pipeline.pip_dependencies, depending on the extension of pipeline.requirements_file.
if a version of python or pip are specified by the mlflow files, they should appear first in the pipeline.conda_dependencies list. only pip packages should appear in pip_dependencies.
Discussion
for #39 :
ContainerSpec(..., pip=pipeline.pip_dependencies, conda=pipeline.conda_dependencies)
Testing
unit tests included, plus some manual testing (and debugging) with Logan's real model with a real non-pypi dependency

:books: Documentation preview :books:: https://garden-ai--84.org.readthedocs.build/en/84/

Codecov Report

Merging #84 (f0d5658) into main (6129dfe) will increase coverage by 0.27%.
The diff coverage is 90.41%.

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
@@            Coverage Diff             @@
##             main      #84      +/-   ##
==========================================
+ Coverage   86.30%   86.57%   +0.27%
==========================================
  Files          21       21
  Lines        1373     1505     +132
==========================================
+ Hits         1185     1303     +118
- Misses        188      202      +14

Impacted Files
Coverage Δ

garden_ai/pipelines.py
88.46% <84.44%> (-3.21%)
:arrow_down:

garden_ai/utils.py
80.95% <88.00%> (+1.92%)
:arrow_up:

tests/test_models.py
88.95% <91.66%> (-1.17%)
:arrow_down:

tests/conftest.py
97.56% <93.47%> (-2.44%)
:arrow_down:

garden_ai/steps.py
92.13% <94.73%> (+3.09%)
:arrow_up:

garden_ai/app/pipeline.py
75.34% <100.00%> (+0.69%)
:arrow_up:

garden_ai/mlmodel.py
78.12% <100.00%> (+2.26%)
:arrow_up:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


ARTEMIS-3198 Add concurrency option on core bridges

This commit adds a concurrency-option for the core bridges to handle high latency networks better

Nice contribution!!
Just curious, as we have made with ReplicationManager on https://github.com/apache/activemq-artemis/pull/3392 probably we are not correctly filling enough packets to maximize the TCP packet size and amortize network latencies: have you tried by disabling TCP no delay too?

Nice contribution!!
Just curious, as we have made with ReplicationManager on #3392 probably we are not correctly filling enough packets to maximize the TCP packet size and amortize network latencies: have you tried by disabling TCP no delay too?

Thanks,
No I haven't tried that and it looks like a real nice optimization, but for my use case I am looking to increase the throughput I am currently seeing by some 10-15 times at least. I will definitely try that out as well though!


WFLY-6513 Allow to configure the size of the global Weld thread pool

https://issues.jboss.org/browse/WFLY-6513

Oops, I forgot about WeldSubsystemTestCase. Let me fix the test.


Lab 5 - No Azure CLI should be used in this Lab

In the Section Azure CLI - incremental mode (https://github.com/MSDEVMTL/GlobalAzureBootcamp-2019/tree/master/Lab5#azure-cli---incremental-mode)
The explanation of "Complete" vs "Incremental" is good. However we should not be using Azure CLI to deploy but instead, commit and push our changes.

@FBoucher Should we remove this section then?

yes please.
We shouldn't be using Azure CLI at that time... the CI/CD is all set.

I removed the cli section, please review the section about the deployment


Fix encoding when creating oAuth token via Metadata Server

Hi @yoshidan , thank you for the crate.
I faced the problem that I could not generate oAuth token when I use it with google metadata server .
Looks like it was just a typo in implementation. The = sign in scopes= does not need to be encoded.
With the fix in this PR obtaining oAuth token from metadata server works as expected.

Thanks!


MakeCode randint() not random when Bluetooth enabled

From:

https://github.com/microsoft/pxt-microbit/issues/5409

Replicable example from:

https://github.com/microsoft/pxt-microbit/issues/5409#issuecomment-1946206199

#include "MicroBit.h"

MicroBit uBit;

int main()
{
    uBit.init();
     microbit_seed_random();
     int seed = microbit_random(0x7fffffff);
     DMESG("random seed: %d", seed);
     uBit.seedRandom(seed);
     int randomInt = uBit.random(9);
     uBit.display.print(randomInt);
}

{
    "target": {
        "name": "codal-microbit-v2",
        "url": "https://github.com/lancaster-university/codal-microbit-v2",
        "branch": "master",
        "type": "git"
    },
    "config":{
        "MICROBIT_BLE_ENABLED" : 1
    }
}

With this example the random numbers are always seeded with the same value. Changing MICROBIT_BLE_ENABLED to 1 works.
Possible solution:

https://github.com/microsoft/pxt-microbit/issues/5409#issuecomment-1946863177

@microbit-carlos
seedRandom() does not generate a random seed when BLE is running
https://github.com/lancaster-university/codal-microbit-v2/blob/96faa8526a4127f6738b2cd5d66595f25ee50b35/source/MicroBitDevice.cpp#L67
seedRandom is called before BLE is enabled
https://github.com/lancaster-university/codal-microbit-v2/blob/96faa8526a4127f6738b2cd5d66595f25ee50b35/model/MicroBit.cpp#L197
The seed for subsequent calls to seedRandom while BLE is running could be made pseudo random by using:
   r = random(INT_MAX);
The softdevice calls work - they don't seem to need a call to start the RNG peripheral
But I don't see an equivalent of NRF_RNG->TASKS_STOP = 1 to stop the RNG afterwards.
        for (int i = 0; i < 4; i++)
        {
            // Wait for a number to be generated.
            uint8_t byte;
            do
            {
              sd_rand_application_bytes_available_get( &byte);
            }
            while( byte == 0);

            sd_rand_application_vector_get( &byte, 1);

            r = (r << 8) | ((int) byte);
        }

Thanks Martin, that's really useful!
I was playing with this and noticed that waiting until sd_rand_application_bytes_available_get() returns 4 (so that it has 4 random bytes) can take over 12 milliseconds.
Looking at the datasheet it should be 128 us to start, and 120 per byte, so 128 + 120 * 4 = 608 microseconds. Not sure why it's taking 20 times longer. Anything similar you've seen or experienced @martinwork @finneyj ?

@microbit-carlos Good catch! I followed the non-BLE code without thinking, and didn't look at the DAL either!
https://github.com/lancaster-university/microbit-dal/blob/master/source/core/MicroBitDevice.cpp#L343
If we can guarantee the call will eventually work, maybe this is OK.
`while ( sd_rand_application_vector_get((uint8_t*)&r, sizeof(r))) /empty/;
It's quicker than the byte by byte version.
`
It seems it may take a while immediately after initialising softdevice
https://devzone.nordicsemi.com/f/nordic-q-a/47707/how-to-use-sd_rand_application_vector_get-correctly
I see up to 810 microseconds, but not 12ms. It seems a bit random.
#include "MicroBit.h"

MicroBit uBit;

void seedRandom()
{
    uint32_t r = 0xBBC5EED;

    if(!ble_running())
    {
        // Start the Random number generator. No need to leave it running... I hope. :-)
        NRF_RNG->TASKS_START = 1;

        for(int i = 0; i < 4; i++)
        {
            // Clear the VALRDY EVENT
            NRF_RNG->EVENTS_VALRDY = 0;

            // Wait for a number ot be generated.
            while(NRF_RNG->EVENTS_VALRDY == 0);

            r = (r << 8) | ((int) NRF_RNG->VALUE);
        }

        // Disable the generator to save power.
        NRF_RNG->TASKS_STOP = 1;
    }
    else
    {
      CODAL_TIMESTAMP t0, t1;

      t0 = system_timer_current_time_us();
      while ( sd_rand_application_vector_get((uint8_t*)&r, sizeof(r))) /*empty*/;
      t1 = system_timer_current_time_us();
      DMESGN("%d, ", (int)(t1-t0));

      //t0 = system_timer_current_time_us();
      //for (int i = 0; i < 4; i++)
      //{
      //    // Wait for a number to be generated.
      //    uint8_t byte;
      //    do
      //    {
      //      sd_rand_application_bytes_available_get( &byte);
      //    }
      //    while( byte == 0);

      //    sd_rand_application_vector_get( &byte, 1);

      //    r = (r << 8) | ((int) byte);
      //}
      //t1 = system_timer_current_time_us();
      //DMESGN("%d, ", (int)(t1-t0));

      DMESG("");
    }

    uBit.seedRandom(r);
}

int main()
{
    int randomInt = 0;

    uBit.init();

    seedRandom();

    while (true)
    {
      uBit.display.print('*');
      uBit.sleep(1000);

      uBit.seedRandom();
      randomInt = uBit.random(9);
      uBit.display.print( randomInt);
      uBit.sleep(1000);

      seedRandom();
      randomInt = uBit.random(9);
      uBit.display.print( randomInt);
      uBit.sleep(1000);
    }
}

The non-BLE code takes around 240us every time.
Waiting for 4 bytes with sd_rand_application_bytes_available_get takes a bit longer because of the extra code.
Quite a long wait (between 500 and 1000us) is needed to remove the randomness of the first call, and a wait inside the loop doesn’t seem to help.
Code variations
#include "MicroBit.h"

MicroBit uBit;

void seedRandom()
{
    uint32_t r = 0xBBC5EED;

    if(!ble_running())
    {
        // Start the Random number generator. No need to leave it running... I hope. :-)
        NRF_RNG->TASKS_START = 1;

        for(int i = 0; i < 4; i++)
        {
            // Clear the VALRDY EVENT
            NRF_RNG->EVENTS_VALRDY = 0;

            // Wait for a number ot be generated.
            while(NRF_RNG->EVENTS_VALRDY == 0);

            r = (r << 8) | ((int) NRF_RNG->VALUE);
        }

        // Disable the generator to save power.
        NRF_RNG->TASKS_STOP = 1;
    }
    else
    {
      while ( sd_rand_application_vector_get((uint8_t*)&r, sizeof(r)))
      {
        //target_wait_us(100);
      }

      //uint8_t byte;
      //do
      //{
      //  sd_rand_application_bytes_available_get( &byte);
      //}
      //while( byte < sizeof(r));
      //sd_rand_application_vector_get( (uint8_t*)&r, sizeof(r));

      //for (int i = 0; i < 4; i++)
      //{
      //    // Wait for a number to be generated.
      //    uint8_t byte;
      //    do
      //    {
      //      sd_rand_application_bytes_available_get( &byte);
      //    }
      //    while( byte == 0);
      //    sd_rand_application_vector_get( &byte, 1);
      //    r = (r << 8) | ((int) byte);
      //}
    }

    uBit.seedRandom(r);
}

int main()
{
    CODAL_TIMESTAMP t0, t1;
    int randomInt = 0;

    uBit.init();

    target_wait_us(1000);

    //t0 = system_timer_current_time_us();
    //uBit.seedRandom();
    //t1 = system_timer_current_time_us();
    //uBit.sleep(1000);
    //DMESG("uBit.seedRandom %d", (int)(t1-t0));

    t0 = system_timer_current_time_us();
    seedRandom();
    t1 = system_timer_current_time_us();
    uBit.sleep(1000);
    DMESG("     seedRandom %d", (int)(t1-t0));

    while (true)
    {
      uBit.display.print('*');
      uBit.sleep(1000);

      t0 = system_timer_current_time_us();
      uBit.seedRandom();
      t1 = system_timer_current_time_us();
      DMESG("uBit.seedRandom %d", (int)(t1-t0));

      randomInt = uBit.random(9);
      uBit.display.print( randomInt);
      uBit.sleep(1000);

      t0 = system_timer_current_time_us();
      seedRandom();
      t1 = system_timer_current_time_us();
      DMESG("     seedRandom %d", (int)(t1-t0));

      randomInt = uBit.random(9);
      uBit.display.print( randomInt);
      uBit.sleep(1000);
    }
}

I think maybe we are seeing differences in time because you are seeding and measuring after uBit.init(), which has a sleep(10) at the end, so there is already a 10ms wait since the softdevice is initialised.
I'm getting my measurements by moving seedRandom() from within uBit.init() from the top to the end, right before the sleep(10) and I'm measuring how long seedRandom() takes to return there:
    // Seed our random number generator
    uint32_t t = system_timer_current_time_us();
    seedRandom();
    uint32_t t2 = system_timer_current_time_us();
    serial.printf("seed t: %d\n", t2 - t);

    // Deschedule for a little while, just to allow for any components that finialise initialisation
    // as a background task, and to allow the power mamanger to repsonse to background events from the KL27
    // before any user code begins running.

    sleep(10);

    return DEVICE_OK;
}

In this case it does make sense to move it after the sleep, but I still wonder why it would take so long...

Oh, yes! I see the same - sometimes over 13ms. That explains why a ms pause is mentioned in
https://devzone.nordicsemi.com/f/nordic-q-a/47707/how-to-use-sd_rand_application_vector_get-correctly
For the initial seedRandom call in MicroBit:init(), I think it should stay before BLE is initialised. I have wondered why it wasn't further up, say before the component init() loop at least.
The BLE mode problem was revealed by MakeCode's extra call after uBit.init (which I don't understand).
https://github.com/microsoft/pxt-microbit/blob/7cc19bb881b1404071a521a3d290ef980eca063a/libs/core/codal.cpp#L52
It looks like MakeCode has a fair amount of code between uBit.init() and platform_init(), which hopefully would act like a long wait.

Yes, you are right, I was mostly playing around with a few different options initially when I encountered the odd timings, and 12ms felt excessive.
So we don't need to move seedRandom() within uBit.init(), as in its current position softdevice has not been initialised and ble_running() returns false, so the initial seed is successfully random.
The issue at hand in CODAL is (as you have mentioned) that because MakeCode re-seeds on its platform_init() function, at that point CODAL softdevice is running, and so MicroBitDevice::seedRandom() currently always uses 0xBBC5EED as the seed.
So, we should add the softdevice call in MicroBitDevice::seedRandom().
However, the weird thing is that in MakeCode, this is also a problem for V1, and there we already this code:
https://github.com/lancaster-university/microbit-dal/blob/318c6abbb3d952d3f49a595a726486230e82fb5b/source/core/MicroBitDevice.cpp#L340-L347
So not sure yet what might be the problem in V1, maybe sd_rand_application_vector_get() is not succeeding?

Oh, I hadn't realised the report included V1. The problem there is that the V1 code gives up after one try rather than looping until it succeeds. I noticed this but thought maybe it happened to be working in V1. It does seem to work first time after the start-up delay.

Thanks Martin for all the help here!
I've created this PR code codal-microbit-v2:

https://github.com/lancaster-university/microbit-dal/issues/495

This issue for DAL:

https://github.com/lancaster-university/microbit-dal/issues/495

And this workaround for MakeCode (as it'll take a while for the DAL fix to be released):

https://github.com/microsoft/pxt-microbit/pull/5945


Add support for configuring headscale

This includes an opinionated config file template. Common options can be configured via snap config (eg. snap set headscale <key>=<value>).
Pros of a config file template:

paths and such can be opinionated, chosen to work well for the strictly confined snap
potentially easier for the user to get started, easier for future charms to configure the service
don't need to have more docs for informing the user where the config file can go (eg. must be in $SNAP_DATA, etc.), no gotchas with file paths to databases, caches, etc.

Cons:

less flexibility (the user can't edit the config file as they wish)
it's now our responsibility to keep the config file template and options up to date with future headscale releases

NOTE: if this method is approved, then I'll polish and add docs (maybe in a follow up PR). :)

From my point of view is a right approach to only allow user to configure through the snap set command. Support both configuration file and command may let user confuse at this moment. So LGTM.


As user i want to move files to folders

File and Folder Management

Develop functionality to list the files and folders that a user has in their account.
Create a method for users to select one or more files to be moved.

User Interface

Design and implement a user interface that allows users to drag and drop files into folders, or use a selection tool for moving files.
Provide visual feedback that a file has been selected and is ready to be moved.

Backend Logic for Moving Files

Write server-side logic to change the file's path or update the database records to reflect the file's new location within a different folder.
Handle potential issues such as file name conflicts, permission changes, and data integrity during the move operation.

Error Handling and User Feedback

Implement error handling for scenarios where files cannot be moved (due to permissions, space limitations, etc.).
Communicate success and error messages to the user after an attempt to move files.

To switch to this branch use:
git fetch origin
git checkout 4-as-user-i-want-to-move-files-to-folders


V2Ray Linux/Windows client config

Hello! Thanks a lot for sharing the instructions, the V2Ray method works well on Android!
May I ask you to add a JSON client config for Linux/Windows as well? Thanks again <3

@Saya47:
I'm happy it worked for you! Sure, I'm gonna add that to the repo.


Fix max_packet_size

Max Packet size is being sent wrong on configuration request, thus emulated devices will fail under Windows and MAC. This fix sorts out that issue :)

Bug is different ... reviewing first ...


_ma_target_ref_db_details.gene_name item type change

When parsing AlphaFoldDB entries we found that some cases have a value for _ma_target_ref_db_details.gene_name that is incompatible with the item type as defined in ModelCIF (code). Examples are:
AF-A0A0M3KKW8-F1
AF-H6SHY4-F1
AF-O78126-F1
AF-Q6RH29-F1
AF-Q6RH28-F1
AF-Q6RH30-F1
AF-Q6XEC0-F1
We reported this issue to to the helpdesk of AlphaFoldDB who answered:
//
The pipeline for predictions of AlphaFold, retrieves metadata from UniProt.
Since, the mmCIF files were created by Google Deepmind, we can't modify the
content of the field ourselves.
We recommend you to please reach out to  https://pdb-dev.wwpdb.org/ to look
into the updating regex for this field as it does accept whitespaces.
//
Rather than changing the regex for the type "code", perhaps the type for _ma_target_ref_db_details.gene_name can be set to "text" or another type that is compatible with Uniprot entries that have a gene name (from the 'GN' record I suppose) with spaces in it.

e.g. https://www.alphafold.ebi.ac.uk/entry/H6SHY4 contains
_ma_target_ref_db_details.gene_name                    "celH or egH"

while https://www.alphafold.ebi.ac.uk/entry/O78126 contains
_ma_target_ref_db_details.gene_name                    "MHC class I HLA-A"