Defining a datapackage.json for Movebank data

I'm attempting to convert a published Movebank dataset into a Frictionless data package, by describing the files and their structure in a datapackage.json file. I created a use case for this here. This issue is to capture feedback.
Link to file: datapackage.json

[ ] Add rdfType to terms with link to term URL
[ ] Should term URL be the versioned one?
[ ] Should we include the definition?

Feedback by @sarahcd:

First thoughts: Could/should we include the URIs with the attribute info? Is it appropriate to add package-level info like license and citations, and if so how? Could we write script to create the files with R (easiest option for me unless that is a terrible idea for some reason)? Can this be written in a way that facilitates ingestion to GBIF? Feel free to start a new thread or move to github....

@sarahcd:

Yes, we should include the URIs with attribute info. I think that is done with "rdfType". I will add those.
I've purposely not included any package level metadata like license and citations, because 1) I didn't want to repeat information that is already available at the (Zenodo) repository (and then having to maintain both) and 2) I want to have the flexibility to update the dataset metadata without creating a new version (updates to files on Zenodo trigger a new version, while updates/corrections to metadata do not). So I just used the datapackage.json for the technical description of the data. It is possible however to include metadata: I tried in an earlier version of the file.
Yes, we could create the file with an R script. I'm not sure if datapackage-r would be of use here, but what we need is:

1. provide file names of csv files + their headers
2. get information about those headers from the movebank attribute dictionary (is it available as xml/json?)
3. write that information as a datapackage.json

This would not be ingested by GBIF directly. The datapackage.json would be used to understand the structure so it can be converted to Darwin Core.

@sarahcd why do tag id and tag local indentifier (sic) both exist as concept in the Movebank Attribute dictionary? They are the same concepts and have the same definition. It is unclear which one I should refer to (I think tag id). Same for animal id vs animal local identifier.

The two versions of tag/animal identifier labels are something created a long time ago, and I'm not sure what the rationale was. tag-id/animal-id are the names used in reference data downloads, and tag-local-identifier/animal-local-identifier are he names used in event data downloads. As you see in the vocabulary I use the alternative label to show they are the same thing. I could delete one set of the entries so that there is just tag-local-identifier/animal-local-identifier with the alt labels. When I make readme files I use tag/animal id, which is an arbitrary decision; fyi "local-identifier" indicates it is the user-defined ID rather than a numeric identifier assigned by the database.
Extra metadata in readmes: I see your point and certainly have cases where I update the DataCite metadata and of course cannot update the readme.txt. I hesitate to eliminate the text readme entirely from the repository, because they are human readable and easy to store locally, which reduces provenance loss especially if files get passed around after download and exists without the internet. Does Zenodo offer a human readable download option for the update-able metadata?

Re
2. get information about those headers from the movebank attribute dictionary (is it available as xml/json?)
Can the machine readable terms be harvested from the NERC vocabulary service?

@peggynewman this is something @sarahcd will inquire NERC about, as well as adding data type and format for terms. That way, a datapackage.json could be build from the NERC vocab.

Note: If you go to a NERC vocab, e.g. vocab.nerc.ac.uk/collection/MVB, see Alternate Profiles in the upper right for several JSON formats. I haven't explored them yet but they look potentially useful for harvesting.

A datapackage.json file as described above can now be generated automatically with the movepub R package (see tutorial).
This is done by making use of the general purpose frictionless R package and looking up the definition and URL for every field in the Movebank Attribute Dictionary (using the get_mvb_term() function). The resulting data in datapackage.json looks as follows:
{
  "name": "tag-id",
  "title": "tag ID",
  "description": "A unique identifier for the tag, provided by the data owner. If the data owner does not provide a tag ID, an internal Movebank tag identifier may sometimes be shown. Example: '2342'; Units: none; Entity described: tag",
  "type": "string",
  "format": "default",
  "skos:exactMatch": "http://vocab.nerc.ac.uk/collection/MVB/current/MVB000181/2/"
}

skos:exactMatch was chosen over rdfType. Full example at https://github.com/inbo/bird-tracking/blob/master/data/processed/O_ASSEN/datapackage.json
The issues regarding synonyms or improvements to be made to the attribute dictionary are discussed in the movepub repo.
Closing this issue.


change kernel_config for iotop with I/O accounting support

fixed for iotop: add CONFIG_TASK_XACCT=y and  CONFIG_TASK_IO_ACCOUNTING=y in kernel_config
Otherwise iotop fail with:
Could not run iotop as some of the requirements are not met:
- Linux >= 2.6.20 with
  - I/O accounting support (CONFIG_TASKSTATS, CONFIG_TASK_DELAY_ACCT, CONFIG_TASK_IO_ACCOUNTING)

Looks good, thanks! (and matches Debian's configuration for these flags)


Variable "i" not defined in function "public static int crypto_box_keypair_with_seed(byte[] pk, byte[] sk, byte[] seed)"

File: curve25519xsalsa20poly1305.java line 66. "i" isn't defined in the function or the class, causing the build to fail. I'm not sure how to fix it, but another library I'm using requires this one

Can you try reverting to commit 7da60ebb13c058b78af27b13f53f4ce3c909745b and see if the problem is resolved?

Yes, that does solve the problem

Thanks, the problem seemed to originate from a pull request. I have reverted it in commit 40c322e0a42637ab17cdf941138eeaf2494055f8 which will remove the offending function for now.


Method to drop 'padded' pv systems

Detailed Description
Would be useful to have a method to drop 'padded' pv systems. These are padding out zero so that the dataset can be save in an efficient way. However for plotting its useful to drop the 'padded' systems
Would be also good to so this for 'gsp' too
Possible Implementation
in from nowcasting_dataset.data_sources.pv.pv_data_source import PV have function that removes any zero values. This would be for 'data', 'pv_systems', 'x_coords' and 'y_coords'

Interesting!
I'm probably remembering wrong but I thought the code padded PV data with NaNs, not zeros?
Is it possible that the zeros are "legitimate"?  (e.g. zero power generation at night?)

Yea, got to be careful that the zeros are not true. Could be done by looking at xcoords as 0 is not relastic there.
another option is to change them back to nans, and then do some filling before passing to ML models


Use Kotlin 2.0.0, add wasmJs target

Hi! Thank you for your contribution! Have you tested your changes? Does it work properly?

Yes, I've tested this. You can see the succeeded build on Jitpack here.

Yeah, I see WASM binaries are built properly. Thank you again! We'll see how it's gonna be published on my repo.

https://maven.d1s.dev/#/snapshots/com/catppuccin
WASM binaries are published successfully. Thank you again. I'll publish a release as soon as possible.


DA-3055: Fix issue bugzillarest not exported to CSV

In case of DPDK project - bugzillarest ds (bugzilla) fields were not exported to CSV. However, topcontributors API response was showing the data correctly.
Below are few debug logs for reference -
actual=map[bugzilla:{} git:{}]
configured=[bugzillarest git]
selected=map[bugzillarest:{} git:{}]

/lgtm


User must Change password on First Login using OpenLdap

Hi,
Can someone point out to me any documentation on how to setup pgina fork to make user change the password on their first login (if there is any). I have configured OpenLdap on Ubuntu and enabled "Passwd Must Change" flag to True. So user can change their password on the first login. This feature is working in Ubuntu but when using pgina I am not seeing any option like that. I tried pgina fork by enabling change password but its not working.
Thanks,
Rajesh

I figured it out. I need to use ctrl+alt+delete to change the password in windows


feat(consolerole): Add support for namespaced token used for interactive console access to containers

This PR adds the Service Account token needed for deis ps:console (see feat(pkg/console): support interactive console PR on workflow-cli )
The feature itself consists of the current PR and the feat(pkg/console): support interactive console PR on workflow-cli counterpart. The idea is to mimic the mechanism that kubectl is using but with using the k8s API itself. Specifically we can get a websocket access to a console in a pod but we need a token. In order to be able to use the token we need to:

create a clusterrole named deis:deis-console that will actually have the pods/exec permission
add the needed permissions to deis:deis-controller in order to be able to attach the  deis:deis-console clusterrole to a Service Account that belongs to a single namespace
create a Service Account and a token (secret) when we create a hephy application
instruct hephy controller to send that token when asked (this means that the user has access to the requested application)

The above ensures that:

the token can be obtained only from the users that can access an application
no global token is used
the token will be cleared once the application is destroyed (since it belongs to the namespace of the application)

The requirements are to set the variables for the  K8S_API_ENDPOINT and CONTAINER_CONSOLE_ENABLED.
I tested it on my minikube with setting:
ip-10-0-0-119 ~ # minikube ip
<IP_ADDRESS>

ip-10-0-0-119 ~ # kubectl describe deployment deis-controller -n deis | grep "K8S_API_ENDPOINT\|CONTAINER_CONSOLE_ENABLED"
      CONTAINER_CONSOLE_ENABLED:             true
      K8S_API_ENDPOINT:                      https://<IP_ADDRESS>:8443

An application named testadminapp3 had a service account and token attached that was able to give me pod/exec access:
ip-10-0-0-119 ~ # kubectl auth can-i list pods --subresource=exec --as=system:serviceaccount:testadminapp3:deis-console-testadminapp3 --namespace=testadminapp3
yes

@giannoul , thank you for the contribution!
Is there a way to make the tokens ephemeral, requested on-demand, and expire. Right now, it means that the tokens will be created when the app is created and the SA/token remains there until app/namespace is deleted. Can we create a route that will generate console-token for let's say 20-30 min sessions.
https://github.com/teamhephy/controller/pull/148/files#diff-79aefc2bc0c74e445347a254ecfa621edf7e334f1c8a3efd8358008427d48fb3R236
Is it possible to set expiration on the JWT token using k8s tricks like:
expirationSeconds: 3600 #expires in 60 mins

https://kubernetes.io/docs/reference/access-authn-authz/service-accounts-admin/#bound-service-account-token-volume
This would mean hephy workflow cli requests a new token each time or controller uses a sort of cache for tokens. However, this may not be possible at this time.
The other way that this can be done is to store the service_account_name, service_account_creation_timestamp, service_account_expiration_timestamp in the django db model. Once timestamp reaches service_account_expiration_timestamp we can consider this SA token invalid and perform a conjob on in the controller to delete the tokens from the api. The on_delete method on the django model will also delete the SA from the app's namespace.

@Cryptophobia Investigating the token expiration option I found the following:
1. An existing session is independent of the token
As we may see from the implementation, all users will use the same underline Service Account in order to get a console access to the container within the pod. Since the console access is actually a back and forth using websockets, the Service Account token is just used in order to initialize the connection. Experimenting with that I saw that for an existing console session, even if I deleted the token, the connection was left intact. This means no disruptions for the existing console connections when/if we remove the Service Account token (secret).
2. Service Account tokens are automatically recreated
If we delete the token (basically it is a k8s secret) it will get regenerated automatically. This seems to be the default function of the token-controller. During my testing I found that deleting the token via kubectl will just lead to a new token creation with a different value and name suffix.
A way to implement the token expiration
Since:

the token deletion will not disrupt the existing console sessions
all users use the same token since they effectively utilize the single Service Account in the namespace
the token of the Service Account will be regenerated once deleted
the token is basically a k8s secret with the field creationTimestamp stating when it was created

we can regenerate the token upon a console session request and return the new token. This basically means that upon a new console request we check if the existing token is older than e.g. 20mins and delete it (the k8s token-controller will create a new one) and return the new token in order to be used. The connections using the old token will continue to operate and the new ones will use the new token.
The above approach is actually invalidation on demand and not cron job based action, but it has a lot less moving parts.

Okay @giannoul, the above seems like a good solution for ephemeral tokens. Does that mean that the session created by an old token can continue to be open forever? We should also create a mechanism to break the session every 20 mins to retry with same token as previously given when creating the session. This ensures that a long-running connection cannot use a very old token.
This is similar to how SSH long-running open connections are closed via timeouts.
If the above works then we should go with this new approach and add a timeout to make sure a long-running connection is not a security risk.

The sessions opened by any token may remain open (idle) for as long as the kubelet parameter --streaming-connection-idle-timeout says:

https://github.com/kubernetes/kubernetes/issues/66661

The above means that a session can be idle for a maximum of ~4 hours by default.
The other end that could terminate the connection would be the gorilla websocket
but it does not set such a timeout by default:

https://github.com/gorilla/websocket/issues/473

The above mean that the idle connections should be terminated after the time instructed by kubelet's --streaming-connection-idle-timeout.
In general the session itself is controlled via the Kubernetes and the workflow-cli via the gorilla websockets. This means that hephy controller is not able to actually terminate anything. It is just the intermediate that passes the token to the workflow-cli.
So, the only way to avoid the security risk you mentioned would be to set a hard timeout on the workflow-cli, but I am afraid that the first thing that will be requested afterwards would be "how to increase the timeout"  🤣.

So, the only way to avoid the security risk you mentioned would be to set a hard timeout on the workflow-cli, but I am afraid that the first thing that will be requested afterwards would be "how to increase the timeout"

Adding the hard timeout on the workflow-cli client is easily bypassed and is no security at all... so there would be no point to do it for security anyways.
Is there any other way we can terminate the websocket on a set timeout using controller's permissions? For example, a simple solution like set an ENV var for WEBSOCKET_TIMEOUT on controller. It would would set that value when the workflow-cli client sets the Keep-Alive for the websocket connection. Then immediately after the token is returned it will recreate it on the controller side. If a get for a token comes in, wait 20 seconds then recreate the token immediately. That way the session is guaranteed to timeout and the token is no longer valid after establishing this session, a token is only valid per single session.
This will ensure if a user is deleted, their access is gone as soon as session expires. Still not perfect, but the token will be recreated each time.

@giannoul , any update on this? I would like to get out a new minor release of hephy soon. If you are still working on this one we can get it out for next release.

I didn't get the chance to investigate your suggestion due to a very busy program these days. Please proceed to the minor release without this one.

Okay, thank you for getting back so soon. I can help push this forward for the subsequent release after next.

The Header Keep-Alive cannot be set for the websocket connection. In order to create the functionality we discussed I did the following:

on the controller I added the parameter (env variable) named CONTAINER_CONSOLE_WEBSOCKET_TIMEOUT which is the timeout in seconds. This parameter is sent to the workflow-cli along with the token, when a user requests access to a pod.
on the controller, I added a custom ResponseWithCallback class that sends the requested body and then executes a callback function. That function in our case is the re-creation of the token. This means that immediately after receiving the token, it gets re-created.
on the workflow-cli I added a context with timeout that closes the channel after the timeout

Awesome work @giannoul ! Thank you for working with me on a bit more secure design. Will get this in right after the patch release v2.23.1 coming up.


Unable to switch language

Sorry, this is not really an 'issue' I see there are language settings. I do not see any documentation on changing to English, but I can see lang info in vue devtools, but I am unsure how/where to change it. I tried translating to search easier.
I cannot seem to find this. Can you provide information on how to change?

It is a bug and has been fixed. You can try https://kms.ikxin.com

Closed. Please feel free to reopen if needed.


Add missing switches for Grafana manifests.

[x] Have you signed the Contributor License Agreement?
[x] Have you checked that there aren't other open pull requests for the same manifest update/change?

Microsoft Reviewers: Open in CodeFlow

@KevinLaMS Do I have to fix manfiests one by one or any MS reviewer can review it and force to merge? I will be happy if you answer that.

Closing pr in order to do fixes one by one: microsoft/winget-pkgs#125


mach-nix without entering into the shell after mach-nix shell.nix execution

I don't know what the best way of debugging is
For pynixify I can run the command in a loop for all pkgs and print out the broken ones.
but for mach-nix it enters automatically into the shell -> exit status is not acesseable

and I don't know how to auto exit (back to bash shell again [and getting the former exit-status of the mach-nix execution ])

is there a debug mode for one to one (out of the req.txt) pkg evaluation?

nix-shell shell.nix --show-trace &> "zz_mach-nix-shell-nix.log"

when redirecting the stdin/err works for that

(sorry)


New API Doc styles break in Safari at 600px width breakpoint

At 600px width or less the new API doc styles break, which is most prominent on iPhones. This issue, however, does not affect Chrome or Firefox.
Safari

Chrome

Firefox

closed by merge of #1100


EXSWHTEC-16 - Implement hipEventSynchronize tests

Add hipEventSynchronize.cc file with positive unit tests: Simple recorded event synchronization and without event recording
Add newly created file into CMakeLists.txt

@chrispaquot In these tests we just wanted to verify hipEventSynchronize API, as it is in the Event Management group. The point of the test was to see if further execution will be blocked until specified event is completed. If you think additional test with multiple streams or modifying this test would make the testing process more robust, we can expand it.

@chrispaquot In these tests we mainly wanted to verify functionality of the hipEventSynchronize API, as extensive joint testing of multiple Event Management APIs is present in the Unit_hipEvent.cc. The point of the implemented test was to see if further execution will be blocked until specified event is completed, I have expanded it to include the case when kernel is launched in a different stream. Please review the changes.


Problems in SequentialTrainer.lua

line 45: _db_name, should be self._db_name
136: _optimState should be self._optimState

Thank you for pointing my attention to this. I modified the resume training option. The issues are addressed in commit 84f3fc2.

I've pulled your new commits, but it seems there's no error when resume training; however, the loss shows that the model is still trained from scratch.
I'm not sure if it's my problem; can you put more detail on resume training in README? Thank you.

Sure I will. If you trained a model with this code you should be able to start from the model again. Each time the code saves the network, three files are written to disk, one contains the model weights, one contains the optimizer state which has the gradient information for applying momentum and a text file that shows what was the configuration during the training.
For resuming the training, you should set the -resume_training flag to true in the config file and set the -pre_trained_file to the model that you want to continue training from (not the imagenet pre-trained model). If everything goes well you should see these lines in your terminal: "Copying weights from the classifier layer!"; "Copying weights from regressor layer!"; "Preparing the regression layer weights...". Also, if you see "The optimizer state is not found for continuing the training, a new state is being used!" it means that  the optimizer state could not be accessed. Can you check the terminal output to see if these lines are printed or not?

It's correct now, thank you very much.


angular: scope is missing from breaking changes

On angular preset, the generated breaking changes don't reference the scope for each change.
This should be the default behaviour IMO. Thx.

I don't understand. Can you elaborate?

Sure. You can see here what I mean.
If a commit message is like:
fix(ngAnimate): ....

Breaking chnage: due to .....

I expect on Breaking Changes section, the ngAnimate to be referenced, like:
* ngAnimate: due to ...

and not just
* due to ...

like it is happening now.

This is defined by the angular project. Only if your "type" is feat, fix or perf it will appear in the changelog. If you think this is not correct, you should probably open an issue there.
Or use the angular preset and change the settings to your needs.

I referring only to the Breaking Changes section, not the Bug fixes or the Features. On the Breaking changes section, for each breaking change there is not reference to the <scope> of the commit.

@bekos are you saying that breaking changes should be added regardless of the type of change, that is - also for refactor in this case?
I would argue that other types of changes (refactor, docs, tests) shouldn't actually contain breaking changes. Otherwise it is more than just a refectoring.... Just my $0.03....

@pkozlowski-opensource Nope.
I'm just telling that if the  of the commit message, ie ngAnimate is not referenced in the Breaking changes` section. Just look at my example above :-)

I see what you mean now. I don't think angular projects are doing this. You can easily modify the template to achieve this.

For Angular 1.x this is the way they used to do this and are still doing this. Angular 2.x is not, but I think this is a problem, because if you forget to explicitly repeat the <scope> of the commit, in the text of the breaking change, it can be very confusing.
I argue about the default usage, but if this is something I can easily override on my side, through the templates, Im fine with it :-)

@pkozlowski-opensource is it the expected behaviour? The template for the presets can be changed.

@bekos @pkozlowski-opensource I think it was modified so that they don't care about the scope in the breaking changes any more? Or we just forgot it in the default?

My opinion is that without scope you lose the context, and you cannot easily find out if the breaking change affects you.

scope of breaking change does not appear in the changelog, but it is not in bold as it should be.
Scopes are in bold in the changelog, except in the breaking change. I think this is a bug. Isn't it?

Angular team didn't give me the requirement on this. So this whole issue wasn't a bug. I just thought it made sense to add them. I don't know what format it should be.

As you include the scope in the breaking change, I thought it would be more consistent that scope would always be in bold, in bugfixes, features and in breaking change. Don't you think ?

@oupala PR welcome :) I don't know what the angular team whats so I don't have any thoughts.


Crosscat installation problem

I have been trying to install BayesDB using python2.7and it gets stuck at crosscat installation. I already have it and all the dependencies required by it are already there.
Here is the error that I got:
Installed /usr/local/lib/python2.7/site-packages/bayeslite-0.1.4.post1+gfb66992-py2.7.egg
Processing dependencies for bayeslite==0.1.4.post1+gfb66992
Searching for crosscat>=0.1.48
Reading https://pypi.python.org/simple/crosscat/
No local packages or download links found for crosscat>=0.1.48
error: Could not find suitable distribution for Requirement.parse('crosscat>=0.1.48')

This is what I get when I try installing crosscat:
Requirement already satisfied (use --upgrade to upgrade): crosscat in /usr/local/lib/python2.7/site-packages
Requirement already satisfied (use --upgrade to upgrade): cython>=0.20.1 in /usr/local/lib/python2.7/site-packages (from crosscat)
Requirement already satisfied (use --upgrade to upgrade): numpy>=1.7.0 in /usr/local/lib/python2.7/site-packages (from crosscat)
Requirement already satisfied (use --upgrade to upgrade): six in /usr/local/lib/python2.7/site-packages (from crosscat)

That's neat. I didn't think I had published that bayeslite 0.1.4 version. Where'd you get it? Did you install from the git repo? In that case, you'll want the head version of crosscat, bayeslite-apsw, and bdbcontrib too.

Yeah I cloned the repository. Is there any other way to install it?

The usual is via pip, but the repo is fine, you just then need all the relevant repos, as noted above.

I tried installing it with pip too. Its the same error:
Collecting bayeslite-apsw>=3.8.0 (from bayeslite)
  Downloading bayeslite-apsw-3.9.2-r1+pypiisbroken.tar.gz (1.9MB)
    100% |████████████████████████████████| 1.9MB 252kB/s
Collecting crosscat>=0.1.48 (from bayeslite)
  Could not find a version that satisfies the requirement crosscat>=0.1.48 (from bayeslite) (from versions: 0.1.0.dev0, 0.1.2, 0.1.5, 0.1.25, 0.1.27, 0.1.28, 0.1.36, 0.1.43)
No matching distribution found for crosscat>=0.1.48 (from bayeslite)

Start with crosscat. Clone it, build it (you need to have boost, and
specify BOOST_ROOT) and install it. Then bayeslite-apsw, then bayeslite,
then bdbcontrib.
Gregory Marton
617-858-0775
Calendar: http://csail.mit.edu/~gremio/cal
Plus: http://goo.gl/Ne09o
Web: http://csail.mit.edu/~gremio
On Jan 7, 2016 6:42 PM, "Prateek"<EMAIL_ADDRESS>wrote:

I tried installing it with pip too. Its the same error:
Collecting bayeslite-apsw>=3.8.0 (from bayeslite)
Downloading bayeslite-apsw-3.9.2-r1+pypiisbroken.tar.gz (1.9MB)
100% |████████████████████████████████| 1.9MB 252kB/s
Collecting crosscat>=0.1.48 (from bayeslite)
Could not find a version that satisfies the requirement crosscat>=0.1.48 (from bayeslite) (from versions: 0.1.0.dev0, 0.1.2, 0.1.5, 0.1.25, 0.1.27, 0.1.28, 0.1.36, 0.1.43)
No matching distribution found for crosscat>=0.1.48 (from bayeslite)
—
Reply to this email directly or view it on GitHub
https://github.com/probcomp/bayeslite/issues/331#issuecomment-169841973.

That works! Thanks.


first commit on feature responsiveAccordionTabs

#8651
@zurb/yetinauts
maybe some native english speaker could take a look at the docs page english

@colin-marshall
just added
Accordion markup should have an ID, if no ID is present, an ID is autogenerated for switch to Tabs.
good? ;-)

First sentence:
The Accordion should have an id specified, but the plugin will automatically generate one if the id is omitted.
The second sentence I think I know what you're saying but am not positive. I will have to come back to this when I have a minute to dig into the functionality of this new component.

Damn - this is a hot feature! Let's get this across the line for 6.3.  @colin-marshall Great feedback!
@designerno1 How can we support you on this?

@rafibomb
some real example testing could be fine, if there should be JS Unit Test it would be great if someone could support the feature there

@Owlbertz can you take a dig into this, particularly with the unit testing in mind as you've been doing such good work on other unit tests?

Given that all we are lacking at this point is testing, I'm going to add @colin-marshall's docs feedback and merge this to make it easier to test.

Merged!


arch.regs should be all regs, and the limited variant should be a list

still need to make sure all regs are included
this won't include SSE regs yet and this all depends on the unicorn cpu state api to be performant

also need a list of caller-saved regs for signal handling


Notification Types suggestion

Currently all notification types are exported in a union type Notification. Since the properties in each type are not consistent the following code is needed to access non universal properties:
const notif = notifications[0];
if ("submittedAt" in notif && "transaction" in notif && "receipt" in notif) {
  // access all properties
}

Currently only type is accessible without a type guard. Without such type gaurd we get Property 'submittedAt' does not exist on type 'WalletConnected'
I think it would be cleaner to have type removed from the notification types and instead export the interfaces for each notification. Then we can use:
if(notif instanceof TransactionFailed){
    // access all properties
}

I can possibly create a PR, let me know your thoughts

You don't have to check each property.
It's enough if you check if if(notif.type === "transactionFailed")
Types here are probably not final yet, but in this situation, I don't think that's really an issue.

Yes thats fair point, closing.


Temporarily remove map on home page

As of the time this PR is being written, the gem we are using for the map on our homepage is [https://travis-ci.org/apneadiving/Google-Maps-for-Rails/builds/129553203](not working). Let's remove the map until it is fixed upstream.

Update: My bad, looks like the errors in their build are unrelated to Google updating their API.

:+1:
On Wed, May 11, 2016 at 6:49 PM, Max Wofford<EMAIL_ADDRESS>wrote:

Update: My bad, looks like the errors in their build are unrelated to
Google updating their API.
—
You are receiving this because you were assigned.
Reply to this email directly or view it on GitHub
https://github.com/hackclub/website/pull/234#issuecomment-218638871

--
Zach Latta
https://zachlatta.com http://zachlatta.com


Docker run environment

Who have downloaded docker runtime environment? Trank you for giving me a download address

Are you referring to the docker command line?

https://docs.docker.com/engine/install/ubuntu/
https://github.com/NVIDIA/nvidia-docker


feature: Review and create issues for index file

Contact Details
<EMAIL_ADDRESS>Feature Description
Standardize and more completely fill the index file.
Why?
There are a lot of places that the index file isn't consistent across distributions or packages which limits the effectiveness of the project.
What-should happen?
This should wait until after the initial move to golang, though it doesn't need to wait for a complete migration. It would mostly benefit from waiting until the index file is in yaml instead of json.
All packages should have either a "curl" (which should be renamed to binary) package-manager-install-option or a valid object for every other package-manager-install-option, if possible. If it's not possible that should be put in the notes with a link to create an issue should it become possible to make it easy for end users to report a change in package availability.
Also, as this is being done add issues for those that need testing on the different architectures. Covers this TODO.
This may need to be split into multiple issues.
Steps to Test

curl should be renamed to binary throughout the program and index file
all packages should have a binary package or all other package managers, if neither of those can happen the notes should mention why and have a link to create an issue to request new package manager support because of a change in availability

Code of Conduct

[X] I agree to follow this project's Code of Conduct

This also covers these TODO's:

curl awscli: NEEDS TESTING
curl code: support checkums
curl code: We should support this
curl code: support Mac here, the below is only for Linux
curl go: NEEDS TESTING
curl go: implement checksum verification (https://golang.org/dl/?mode=json)
curl helm: NEEDS TESTING
curl k9s: support checkums
curl k9s: NEEDS TESTING
curl kpt: support checkums
curl kpt: NEEDS TESTING
curl kubectl: NEEDS TESTING
curl terraform: support checkums
curl terraform: NEEDS TESTING
curl tldr: support checkums
curl tldr: NEEDS TESTING
yq tldr: support checkums
yq tldr: NEEDS TESTING

TODO: move everything that isn't NEEDS TESTING into it's own issue


feat(imgaug): add 3d mode

3d mode applies the same aug across sections

Why this is related is beyond me (┬┬﹏┬┬)
zetta_utils/internal/segmentation/embedding/utils.py:117: error: Argument 1 to "_compute_affinity" has incompatible type "ndarray[Any, dtype[Any]]"; expected "Tensor"  [arg-type]
zetta_utils/internal/segmentation/embedding/utils.py:117: error: Argument 2 to "_compute_affinity" has incompatible type "ndarray[Any, dtype[Any]]"; expected "Tensor"  [arg-type]


Build out db seeders to create an org and some org related data

I want to make slugs optional and auto-generate them on entity create if the slug value is empty to make dealing with them easier


about dataset input value

Hi, I noticed that your example dataset and default grid range are within the range of [-1, 1]. Should we always try to normalize input value to that range or model will handle by itself?
E.g., Now I have 3 input variables of which range are different. They are [-5, 5] [-4, 4] [-1, 1], I don't know how to initialize the spline grid_eps and grid_range.

Hi, update_grid_from_samples handles that. if you use the train method, it calls update_grid_from_samples every few steps. With that being said, it's still advisible to normalize your data to O(1), although not necessarily strictly in [-1,1].

Thanks for kindly answering.
After training some steps, I suppose that the model has updated grid several times. Is it possible to know current grid range? If yes, does it mean that I don't need update grid manually?
Or, if I want to update the grid by myself,  is it the best way to turn off that update grid argument?

Yes, it is possible to know current grid range by calling model.act_fun[0].grid, for later layers, use model.act_fun[l].grid with l=1,2,.... Yes, it means you don't need to update grid manually. If you are still worrying about this, you can check grids right after initialization the model and after training.
Yes, if you want to update the grid by yourself, you could simply set train(...., stop_grid_update_step=1000000000) or any extremely large number.

Thank you! This is very helpful for understanding the implementations. I'm applying KAN to my domain to see if I can rediscover the same formula. Interpretability is what I have been concerned about for a long time. Thanks again for this great work!


running build_binary_graphs > error: [WinError 2] The system cannot find the file specified

I'm trying to build MediaPipe Python Package on Windows by following the corresponding tutorial.
When executing the last step: python setup.py install --link-opencv, the following error information comes:

(mp_env) C:\Users\chenl\Desktop\mediapipe>python setup.py install --link-opencv
C:\Users\chenl\Desktop\virtual\mp_env\lib\site-packages\setuptools\dist.py:477: UserWarning: The version specified ('dev') is an invalid version, this may not work as expected with newer versions of setuptools, pip, and PyPI. Please see PEP 440 for more details.
warnings.warn(
running install
running build_binary_graphs
error: [WinError 2] The system cannot find the file specified

I checked my bazel version, and it could compile hello-world successfully.
(mp_env) C:\Users\chenl\Desktop\mediapipe>bazel --version
bazel 4.2.1
Can anyone help me solve this problem?

Hi @chenlize96 , duplicate issue.

@sureshdagooglecom It is not duplicate.
Firstly, the outputs are quite different.
Secondly, I have tried "bazel clean --expunge", but it shows the same error.

@chenlize96, Which version of Python are you using? Did you refer this https://github.com/google/mediapipe/issues/1325

@chenlize96, Which version of Python are you using? Did you refer this #1325

It is 3.8.10. #1325 is not the same issue. I do not want to use 'pip3 install mediapipe' to install mediapipe. Instead, I use the installation tutorial to install it step by step.

@chenlize96, as per the document. Did you  Download the latest protoc win64 zip and which release you have downloaded?.

@sureshdagooglecom yes, I downloaded 3.19.4-64. Also I added it into my path

@chenlize96 ,did you install OpenCV if yes what is the version?

@sureshdagooglecom The version of OpenCV is 3.4.10, and it was installed at C:\opencv. However, when I tried "import cv2" in python, it raised an error 'no module found', so I inputted 'pip3 install opencv-python'. Both two ways led to the same error mentioned above

@chenlize96 ,
1)First do run these commands inside Terminal/CMD:
conda update anaconda-navigator
conda update navigator-updater
2)For windows if you have anaconda installed, you can simply do:
pip install opencv-python or conda install -c https://conda.binstar.org/menpo opencv.
please try above commands.

@sureshdagooglecom Thanks. However, I need to avoid using anaconda. I have already tried 'pip3 install opencv-python' then 'import cv2' works. My question is that there is an error when I run 'running build_binary_graphs' OR 'python setup.py install --link-opencv'

@sureshdagooglecom My pip is the latest version.

Can anyone help me please?

Solved. Find subprocess.py in your lib，CTRL+f to search 'class Popen', replace 'shell = False' with 'shell = True' in init.
However, new errors happened. They are about python path invalid. I closed this issue about this topic. Thanks all your help.

If someone meets the similar issue like invalid python path, please add 'PYTHON_LIB_PATH' into your environmental variables, its value is 'C:\Users\chenl\AppData\Local\Programs\Python\Python38\Lib'  (this path is the lib in my laptop. also, python without anaconda works great)

All these are run on the system terminal of Windows. Thanks.

Solved. Find subprocess.py in your lib

Where? I'm using a venv and I cant seem to find it there.


rpcmessage: obsole some and change how UserCode is documented

The obsolete errors are not used at the moment and from point of RPC they makes no sense.
Some of the implementation specific errors are switched in here to be documented as the error reserved to the implementation. They won't be used in the communication.
The user specific error codes are modified to be application specific and all of them are now tied to be more specific of MethodCallException.

@fvacek ?


bug path is not shown on the UI

If I select a report, all of the reports in the file are shown, on the bug tree view.
The report steps are closed for all of the reports, for the currently selected one too.

I can't reproduce this error. Does it still occur at you?

Yes, I can still reproduce it.


Remove duplicate async dependency

It seems that async is declared in both dependencies and devDependencies (with different versions no less).

Coverage remained the same at 100.0% when pulling 5c97eb3984a316385667932f40a09b3626b7cb60 on whymarrh:patch-1 into ee768652c2674d46d25f2f3600942ac58085bdab on MitMaro:master.


Execution issue: n.durationBetween is not a function

We are seeing an error when calling fqm-execution with the measure bundle and patient bundle attached.

We are seeing this with v1.0.2, v1.0.4 and v1.0.5. (v1.0.3 was not specifically tested).
Please let us know if you need any other info or resources, thanks!
durationBetween-issue.zip

Hi @nmorasb, thanks for the issue.
A stack trace like this usually leads us to cql-execution and eventually to the ELM itself. "durationBetween is not a function" means that the left hand side of that invocation is not a CQL Date or DateTime type. After some brief debugging, the engine is indeed not receiving either of those types, specifically for the patient's birthDate property. Here is the relevant CQL snippet from your measure:
/* From library PCMaternal version '5.0.000' */
define "Encounter with Age Range":
  ["Encounter": "Encounter Inpatient"] EncounterInpatient
                      where EncounterInpatient.status = 'finished'
 /* causes error --> */ and AgeInYearsAt(date from start of EncounterInpatient.period) in Interval[ 8, 65 )
                        and EncounterInpatient.period ends during day of "Measurement Period"

I took a look at the corresponding ELM, specifically around the birthDate property expression (snipped for brevity):
{
  "localId": "28",
  "locator": "35:29-35:86",
  "resultTypeName": "{urn:hl7-org:elm-types:r1}Integer",
  "precision": "Year",
  "type": "CalculateAgeAt",
  "operand": [
    {
      "path": "birthDate",
      "type": "Property",
      "source": {
        "name": "Patient",
        "type": "ExpressionRef"
      }
    },
    // snip

}

Per the CQL spec, the only valid signatures for CalculateAgeAt are (Date, Date) and (DateTime, DateTime). I would expect the above expression to be surrounded with a FHIRHelpers.ToDate call (that the translator should be inserting during translation) to ensure that the signature is satisfied. This is what I've seen in other chunks of ELM:
Snipped from MATGlobalCommonFunctionsFHIR4 version 6.1.000:
{
  "localId": "316",
  "locator": "215:8-215:44",
  "name": "ToDate",
  "libraryName": "FHIRHelpers",
  "type": "FunctionRef",
  "operand": [
    {
      "localId": "315",
      "locator": "215:27-215:43",
      "path": "birthDate",
      "type": "Property",
      "source": {
        "localId": "314",
        "locator": "215:27-215:33",
        "name": "Patient",
        "type": "ExpressionRef"
      }
    }
  ]
}

Having a FHIRHelpers.ToDate call in the ELM will resolve this issue. We should confirm if this is an error in the translator with @brynrhodes before proceeding anywhere in cql-execution.

Initiated by https://oncprojectracking.healthit.gov/support/browse/BONNIEMAT-1343

I've extracted the contents of the attached bundles and tested them in the current version of the VSCode translator (2.4.0) and it does produce ELM that correctly accesses the birthDate (either using a FHIRHelpers, or by accessing the value property, depending on how it's referenced).

I do know that this is an issue that was recently addressed, so my guess is that the version of the translator producing the ELM that is causing the issue is one that still had that issue.

Here is the issue: https://github.com/cqframework/clinical_quality_language/issues/822, and it was included in translator release 2.4.0: https://github.com/cqframework/clinical_quality_language/releases/tag/v2.4.0

MADiE is currently on Translator v2.4.0 - has been since Jan 11 release (MADiE v1.0.3) - https://www.emeasuretool.cms.gov/sites/default/files/2023-01/MADiE 1.0.3 Release Notes.pdf. @mgramigna , possible that this issue occurs in an earlier version of fqm-execution? MADiE is currently on fqm-execution v1.0.2.

MADiE is currently on Translator v2.4.0 - has been since Jan 11 release (MADiE v1.0.3) - https://www.emeasuretool.cms.gov/sites/default/files/2023-01/MADiE 1.0.3 Release Notes.pdf. @mgramigna , possible that this issue occurs in an earlier version of fqm-execution? MADiE is currently on fqm-execution v1.0.2.

@JSRankins this issue is independent of fqm-execution. The ELM causes an issue even when calculating the raw results of the engine with cql-execution.
I tried translating the ELM directly, both in VS Code extension and locally with the CQL to ELM CLI, and I still haven't seen proper output with translator 2.4.0. @brynrhodes -- are we missing something? What is the expected ELM for this? I see
{
  "path": "value",
  "type": "Property",
  "source": {
    "path": "birthDate",
    "type": "Property",
    "source": {
      "name": "Patient",
      "type": "ExpressionRef"
    }
  }
}

So accessing birthDate.value on the Patient. Is this an intended difference to not use FHIRHelpers.ToDate?

I re-translated the posted measure locally and re-ran it through fqm-execution without changing anything and the error no longer happens.

Hi @mgramigna. I'm just throwing out ideas. Sounds like Bryn is saying that the ELM issue is fixed in 2.4.0, which is what was in use by MADiE when this issue was encountered by the user. The issue was encountered while running fqm-execution. Since you were able to recreate against ELM JSON generated by Translator 2.4.0, this is the source of my confusion.

Hi @mgramigna. I'm just throwing out ideas. Sounds like Bryn is saying that the ELM issue is fixed in 2.4.0, which is what was in use by MADiE when this issue was encountered by the user. The issue was encountered while running fqm-execution. Since you were able to recreate against ELM JSON generated by Translator 2.4.0, this is the source of my confusion.

My confusion is that I don't think the ELM posted in the attached measure bundle was actually translated using 2.4.0.  On the left is the ELM on the Measure attached to this issue, on the right is the measure translated with 2.4.0

The ELM on the left throws an error during execution, the ELM on the right doesn't

@nmorasb will need to comment/provide update. I just know that @p9g reported issue, and his measure would have been using 2.4.0 Translator. Refer to https://oncprojectracking.healthit.gov/support/browse/BONNIEMAT-1343.

If it helps, here's a full diff of the ELM side by side (link expires in 24h): https://www.diffchecker.com/Ks0HVrlX/
left is extracted from PCMaternal in the posted measure bundle, right is PCMaternal translated using cql-to-elm CLI 2.4.0

Per discussion yesterday, closing due to translator version issue identified in a dependent library


Possible loadAsFile inconsistencies

It seems that if resolving a path that matches the regexp
e.g. ./node_modules/jquery/jquery.min - then loadAsFile(String, Function) is called
however if the id takes format of "jquery/jquery.min", then loadAsFile(String, undefined, Function) is called
This results in opts.package being used only when the RegExp is matched. This has an upstream effect on browser-resolve and module-deps whereby if I use the first format, my transforms will apply inside of node_modules, but if I use the second, they don't... is this possibly a bug, or is there a reason behind it? Thanks!

@jaredhanson any thoughts on this one? I had a look at the changeset and it looks like it was a conscious choice to send undefined as 2nd param if id does not match regexp.

I think this change was implemented here - which links to https://github.com/defunctzombie/node-browser-resolve/issues/56.
I'm not sure what's specifically actionable here, so I'm going to close it, but I'm more than happy to reopen it if there's something specific we should be doing.


Preserve directory structure

It would be great to preserve the directory structure of the base_path

sorry, I didn't notice that there's a %{dirname}


License updates for dependabot/github_actions/actions/setup-dotnet-2.1.1

This PR was auto generated by the licensed-ci GitHub Action.
It contains updates to cached github/licensed dependency metadata to be merged into dependabot/github_actions/actions/setup-dotnet-2.1.1: (branch, PR).
The licensed-ci action will add comments to this PR with the status of license checks.  Please make any changes needed to fix any status failures on this branch, then merge license updates into your branch.
If updates are unexpected, please check the github/licensed changelog for recent updates.

The licensed-ci GitHub Action updated cached license metadata based on changes to dependabot/github_actions/actions/setup-dotnet-2.1.1: (branch, PR).
licensed status result: SUCCEEDED
All status checks succeeded, please merge when ready.
Checking cached dependency records for licensed
....................
20 dependencies checked, 0 errors found.

/cc @dependabot[bot]


Use throwing iterator to avoid hidden force try

Using the default iterator on a statement uses a force try, which crashes (although I'm not sure the circumstances). Replace that with a long-form while rather than for-in to re-throw exception rather than crashing.
See https://github.com/stephencelis/SQLite.swift/pull/1030 for discussion
Issue #, if available:
Description of changes:
Check points:

[ ] Added new tests to cover change, if needed
[ ] All unit tests pass
[ ] All integration tests pass
[ ] Documentation update for the change if required
[ ] PR title conforms to conventional commit style

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Hello, Could you please add information about when a situation would arise that'd cause a crash and possible ways to test it?

SQLite codebase uses a force try in default iterator which causes a crash.
I tried to reproduce by calling Datastore.clear() on Auth signedOut events but couldn't reproduce.
Merging this in because it is difficult to replicate the crash in an integration test having a clear use case.
A unit test to cover the change is difficult as it is difficult to mock Statement and Connection classes because they are marked final and part of SQLite codebase.
Verified that existing integration tests and unit tests are passing for this PR.


ponyc build fails recently with: make: *** No rule to make target 'src/libponyc/codegen/genjit.c'

There seems to be a recent regression where ponyc build fails on Fedora 26:
ponyc]$ git checkout 41b80eeb6e3 ponyc]$ make Makefile:259: WARNING: LLVM 4 support is experimental and may result in decreased performance or crashes make: *** No rule to make target 'src/libponyc/codegen/genjit.c', needed by 'build/release/obj/libponyc/codegen/genjit.o'.  Stop.
(Same situation for current master, 922c028cb72d.)
It builds fine with using the commit before that:
ponyc]$ git checkout 123c4a40427a ponyc]$ make Makefile:259: WARNING: LLVM 4 support is experimental and may result in decreased performance or crashes genjit.c codegen.c Linking libponyc Linking ponyc util.cc codegen.cc Linking libponyc.tests Linking libponyc.benchmarks

You need to clear your build directory. The genjit.c file was recently removed from the project.

Thanks, it worked after doing 'make clean'.
Out of curiosity, without 'make clean', why does it try to build something on which it does not depend?

You had a left over .o file that make didn't know what to do with


Add RELAX NG schemas to resources catalog

Description
We currently ship 2 RELAX NG schemas that can be used to validate plug-ins and project files:

resources/project.rnc
resources/plugin.rnc

In https://github.com/dita-ot/website/pull/455, these schemas were published to public URLs, and https://github.com/dita-ot/docs/commit/08e4f4ea6aab69d18a089b98a1ae77cac7aeac62 updated the corresponding <?xml-model> processing instructions in the docs to allow editors to validate these files when stored outside of DITA-OT.
Motivation and Context
This PR adds a catalog to the resources folder that maps these public URLs to the local resources and extends the base catalog-dita.xml with this new catalog entry so that catalog-aware editors will load the local resource instead of accessing the public URL over the network.
How Has This Been Tested?
Ran the distribution build in a fresh worktree, and verified that the catalog is added and included in plugins/org.dita.base/catalog-dita.xml.
Type of Changes

Enhancement (non-breaking change which improves existing functionality)

Documentation and Compatibility

What documentation changes are needed for this feature?
Release Notes entry
Will this change affect backwards compatibility or other users' overrides?
No.

Fixes #3641.

@raducoravu These changes should allow you to remove your previous catalog workarounds from oXygen's DITA framework, as the new public IDs are now referenced in plugins/org.dita.base/catalog-dita.xml.

@infotexture right, thanks, I will add a note to look over this once I integrate the DITA OT 3.6 in our Oxygen project.


Extending Route configuration

New Feature
We need a way to access the custom properties defined for Routes when using Ocelot configuration merge option.
Motivation for New Feature
When having multiple ocelot configuration files and using Ocelot configuration merge option, custom configurations defined for Routes are lost in the merged ocelot.json.
Steps to Reproduce the Problem

Create a ocelot.xservices.json file

{
  "Routes": [
    {
      "DownstreamPathTemplate": "/xservice.svc/serviceMethodA/byNumber/{number}",
      "UpstreamPathTemplate": "/gw/serviceMethodA/byNumber/{number}",
      "UpstreamHttpMethod": [
        "Get",
        "Post"
      ],
      "DownstreamScheme": "http",
      "DownstreamHostAndPorts": [
        {
          "Host": "host",
          "Port": 80
        }
      ],
      "CustomStrategyProperty": {
        "GET": [
          "SomeCustomStrategyMethodA"
        ],
      "POST": [
          "SomeCustomeStrategyMethodB"
        ]
      }
    },
  ]
}

Create a ocelot.yservices.json file
Use AddOcelot to merge configurations and create ocelot.json config file
The merged file created at the root directory, and ocelot.json does not contain custom property (e.g. "CustomerStrategyProperty") added to any Route item.

Specifications

Version: 12.0.1 - 19.0.2, latest

@nurhat this isn't something I intend to change at the moment, however happy to accept a PR :)
I will keep the issue open!

@nurhat
Congrats! 🎉
Your issue has been accepted because of created a nice PR #1183 by @jlukawska 😺

#1183

@TomPallister commented on Oct 7, 2018:

@nurhat this isn't something I intend to change at the moment, however happy to accept a PR :)
I will keep the issue open!

It is accepted twice! 😉
God blesses you, because you have been keeping the issue open these 5 years! 🤣
We have some progress nowadays in 2023... :blush:

Hey @nurhat !
Are you still with Ocelot?

Hi @guerinsylvain!
Are you @nurhat ? 🤣
Do you know @nurhat? 😄

Dear @nurhat,
The functionality you requested is already in place as the Metadata feature, introduced by PR #1843 (refer to commit https://github.com/ThreeMammals/Ocelot/commit/573a9d98e3eb730a53a555d881df15de9017b43d) and included in release 23.3.0. However, we utilize a different concept inspired by MS Reverse Proxy metadata.
The same, in Ocelot all extended properties are stored within the Metadata property →
https://github.com/ThreeMammals/Ocelot/blob/da9d6fab6c704baa925d3c1c5fc984f6a9e9c717/src/Ocelot/Configuration/File/FileRoute.cs#L65
Your feature request is somewhat unique, and in light of the ongoing PR #1183, we aim to accommodate it.


[otbn] Should we supply PC or other information on error?

Currently on an error only an error code is provided, which can make debugging tricky. Do we want to extend the error information?
The simplest thing to do would be to include an error PC, simply the PC of the instruction executing at the time the error is seen. We could add further information, e.g. the DMem address of a read that resulted in an error. I feel a PC should be sufficient though.

estimate 4

Sounds like a good idea. Maybe we should just expose "stopping pc" on the register interface, updated with the current PC whenever OTBN stops. Since we don't have any exception handling, I think this will give what we want (and should be pretty trivial to plumb in).
There is a slight information leak, which might be a problem on the security side. For example, suppose that my code does some data dependent control flow from which I can extract secret information. Then I can fire alpha particles (or whatever!) at the chip during the run and read out the PC at my leisure. Maybe we just need to have an enable register that turns this feature on or off? Then the Ibex code can turn it on when doing on-silicon debug, and leave it turned off the rest of the time.
This question probably needs some security review.
@felixmiller: Do we have any code that does data-dependent branching? Will we?
@moidx, @cdgori: This looks like it would be helpful for debugging. Are there any other "security elephant traps" that we should be careful to skirt?

We could factor in life-cycle state and avoid providing the error PC in certain life-cycle states.

I agree that being able to read the current PC is a useful feature. In the current security model, I don't consider Ibex access to the OTBN PC to be an issue.
Under what conditions would an attacker be able to read any OTBN control/status register? The current requirement is that only a crypto library running in M mode will be able to interact with OTBN. For private key operations, such library will be integrity protected as part of the secure boot implementation. I suspect that an attacker being able to read any OTBN register in a production state device will probably have the ability to successfully attack other assets.
@GregAC, do you have an initial list of internal state error codes to review as part of this issue? For example, illegal instruction, *overflows, *integrity, etc?

Cool, that sounds like we should just add a "stop PC" CSR.
I think that this issue probably isn't the right place to discuss the list of error codes. I don't think anything is written down about them other than what appears in the documentation. We'll probably have something firmer in a week or so (since Greg's working on the RTL for error handling and pinning down the loose corners of that bit of the spec at the moment)

@rswarbrick we currently only have data dependent branching in the ECDSA verification routine but leakage is uncritical there. General guideline for OTBN SW is that data dependent branching is forbidden in any OTBN code dealing with secrets.
I agree with @moidx that in our current model we do not consider OTBN and Ibex as different security perimeters and Ibex has full control on any data exchanged with OTBN. It would be a different story if we planned to enable self-contained isolated algorithms on OTBN (with side loading of assets) or if we considered OTBN as an isolated execution environment for user code. In such a case it would be preferable if the feature could be disabled permanently via fuses.

I generally agree with @moidx - I don't see a problem with Ibex access to OTBN PC.  But, @felixmiller has a valid point about isolation/side-loading considerations, so I would be interested in doing what @GregAC proposed about considering LC state and only allowing this in dev, just to prevent any potential future issues were we to pursue such sideload/isolation methods in the future.  It seems like that's not too much of a restriction since most developer samples will be in dev mode already...

hey all,
this is a bit late, but do you guys remember this proposal (it's rough and outdated at this point)
https://docs.google.com/document/d/1KnmIt7vj8vkfqgAIcx6HJo8KH6zirFNF1TxlL5K6Aso/edit?usp=sharing
The idea was also to expose some kind of useful information from a processing element, and make it only available only through specific life cycle states.
In addition to life cycle states though, how do we feel about letting software control this visibility really early in the boot stack? (for example ROM_ext or probably more realistically boot loader?)  That will mean if this needed to be debugged due to a boot failure, we'd need a specifically signed image.

Probably no surprise, but that freaks me out, since it could easily capture non-sideloaded keys.  We'll have to then consider all the surface area of this and how strong our LC-based locking of it is, and if that can be thwarted.
If we do want it, I'd probably go even farther and make the enabling of this functionality somehow folded into the attestable state that diversifies the keymgr derivation paths (at minimum), so that devices with this turned on are even more non-interoperable with those that have it turned off.

hey chris,
the stuff in the doc is way out dated.  Instead of capturing transactions, it's more going down the line of letting different processing element deciding what to supply as crash information.  But capturing the raw data on the bus wouldn't be a good idea... it would more or less be metadata about that data (for example PC).
That crash information would then be modulated via fuses or early boot chain setting. If it's tied to life cycle fuse, it's already probably the identity process. Similarly, if we leave it as software controlled in the boot chain, we can do something similar as part of software binding in key manager (i think it can be used as that).

This discussion is a bit outdated and the specific question about stop address was split out (by accident) into #7158, which we've just resolved for now. Let's close this issue since we seem to have agreement that we're not going to report anything extra at the moment.


health check params configured in rancher-compose is not honored.

Server Version - Build from master
Used the rancher-compose file below which had health_check enabled to create service.
Service does not have  health checks enabled.
sangeethahariharan1@sangee01:~$ cat health-rc.yml
health4:
  scale: 10
  health_check:
    port: 80
    interval: 2000
    unhealthy_threshold: 3
    request_line: GET /name.html HTTP/1.0
    healthy_threshold: 2
    response_timeout: 2000
health1:
  scale: 5
  health_check:
    port: 80
    interval: 2000
    unhealthy_threshold: 3
    request_line: GET /index.html HTTP/1.0
    healthy_threshold: 2
    response_timeout: 2000
health3:
  scale: 5
  health_check:
    port: 80
    interval: 2000
    unhealthy_threshold: 3
    request_line: GET /name.html HTTP/1.0
    healthy_threshold: 2
    response_timeout: 2000
health2:
  scale: 5
  health_check:
    port: 80
    interval: 2000
    unhealthy_threshold: 3
    request_line: GET /index.html HTTP/1.0
    healthy_threshold: 2
    response_timeout: 2000
sangeethahariharan1@sangee01:~$

Tested with Rancher compose version - V0.1.2
Using the below generated rancher and docker compose yml from rancher server for a environment that had services with health check enabled , I am able to start services with health check enabled from rancher compose command line tool.
test1:
scale: 6
health_check:
port: 80
interval: 2000
unhealthy_threshold: 4
request_line: GET /name.html HTTP/1.0
healthy_threshold: 3
response_timeout: 2000


merge package template from Kronos-Tools/npm-package-template

Updated standard to latest version

Codecov Report

Merging #130 into master will not change coverage.
The diff coverage is n/a.

@@          Coverage Diff           @@
##           master    #130   +/-   ##
======================================
  Coverage    97.5%   97.5%
======================================
  Files           1       1
  Lines         120     120
======================================
  Hits          117     117
  Misses          3       3

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6b9037e...fc0ac63. Read the comment docs.


Add support for displaying message/rfc822 parts

When Emails are forwarded as attachment in outlook, the attached mail is shown by k9 as an attachment called noname.eml. K9 can not open this attachment, but hands it over to the standard mail client, which DOES show it. (Strangely, android mail can not open this attachment directly...)
Is it a bug or a missing feature that k9 can't open these attachments?

It's a missing feature.

Some spam filters attach potential spammy emails as .eml attachments. Unfortunately, I cannot read those in k-9 mail.

Is there another (very small) app that does it?

In case k9-mail gets support to open/view attached eml-files please also register it for this file-type so it will be offered upon touching on a eml-file in the file browser.
I often get emails with eml-attachments because people want me to assess possible spam emails. I only can do it by looking into the mail headers which requires the original mail to forwarded as an attachment (=eml). Currenty I have to save that eml-file from k9 and than to start the android file browser and than to display the eml-file in an text-editor.
(PS. Unluckily gmail-app DOES support both (open attached eml and open eml from file browser or k9) but has no option to display mail headers. Additionally it has a bug and won't display mails with eml-attachments at all. )

I wonder how best to do this. There are two ways I can think of, both with different problems:

insert the message into the database in some sort of temporary state or special folder. this immediately brings up the question which account and folder to associate it with, but all other loading logic can stay identical

change the loader to load a message from outside sources. this is relatively easy itself, just need to substitute the LocalMessageLoader in MessageLoaderHelper for another one with the appropriate logic. however it can't return a LocalMessage, so places where the actual LocalMessage methods (like makeMessageReference in onReply) are required that need different handling will naturally come up. It's just possible that way too many of these come up :)

Option 1 seems to be the easier path right? The message should be associated with the account the parent emails has been received with. In case the message has been opened from outside, e.g. file browser - the last used account / default account? For the folder I think something like a temporary folder (not synced via imap) will do.
PS. As for Gmail for Android, it will just open the eml in a limited viewer mode - no options to reply or forward - only viewing and printing. While I don't like this approach it could be considered as option 3 if the other ones bring to much problems.

Hello!
Are there plans to implement this. Spamassassin usually attaches originals as message/rfc822.

Is there a known workaround for retrieving attachments from an attached email?

Hello,
As a workaround you can install applications like "Letter Opener" or other .eml file viewers.
I did not find yet decent open source .eml file viewer but I'll keep on searching.
If would be better of course to have such feature right there in k-9 (and it would save me a tap ^^) but it makes .eml file viewing more comfortable than text file viewers.
Kind regards,

Very often, I save those attachments to the mailbox, i.e. extract them from the enclosing email. This would require K9 to handle them internally.

might be unrelated to this issue, however I always see all attached .eml files with 0 B in size.

on Aug 4, 2015

It is clear that there is no will to implement this.
Opening rfc822 part is for technical people only thus 0.00000% of users.

Opening .eml (which is the point there) files though is not only for technical people.
Forwarding mails as enclosed documents is really frequent. Some mail system do that by default
For example if you create rules in MS Exchange to forward urgent emails to your private mailbox the rule can only forward them as enclosed doc.
A lot of people save important mail as local .eml files and send them later by email as enclosed files

Opening rfc822 part is for technical people only thus 0.00000% of users.

That seems vastly exaggerated, if not just wrong. See comments further above in this issue regarding forwarded email, email classified as spam, or email messages saved to files.

@fajabird I saw your comment from Feb 21, 2017
You say that gmail-app supports eml attachments. However, I noticed that this is (no longer?) the case now in 2024. For messages which have an eml attachment, neither the body (new text added while forwarding) is shown nor is it possible to open the attachment (forwarded message). I'm now wondering if there maybe is some special action or setting that is necessary to view such messages. Or whether this indeed got broken in the last 7 years.
Thanks for any info

Mhh fellas, this has been working for months at least for me.

Just to clarify: my question to fajabird was about gmail-app, not k9mail. I just posted here, because there seems to be no way of contacting him using other means (no e-mail, nor private message within github). Sorry for the confusion.


[SPARK-28699][Core] Fix a corner case for aborting indeterminate stage

What changes were proposed in this pull request?
Change the logic of collecting the indeterminate stage, we should look at stages from mapStage, not failedStage during handle FetchFailed.
Why are the changes needed?
In the fetch failed error handle logic, the original logic of collecting indeterminate stage from the fetch failed stage. And in the scenario of the fetch failed happened in the first task of this stage, this logic will cause the indeterminate stage to resubmit partially. Eventually, we are capable of getting correctness bug.
Does this PR introduce any user-facing change?
It makes the corner case of indeterminate stage abort as expected.
How was this patch tested?
New UT in DAGSchedulerSuite.
Run below integrated test with local-cluster[5, 2, 5120], and set spark.sql.execution.sortBeforeRepartition=false, it will abort the indeterminate stage as expected:
import scala.sys.process._
import org.apache.spark.TaskContext

val res = spark.range(0, 10000 * 10000, 1).map{ x => (x % 1000, x)}
// kill an executor in the stage that performs repartition(239)
val df = res.repartition(113).map{ x => (x._1 + 1, x._2)}.repartition(239).map { x =>
  if (TaskContext.get.attemptNumber == 0 && TaskContext.get.partitionId < 1 && TaskContext.get.stageAttemptNumber == 0) {
    throw new Exception("pkill -f -n java".!!)
  }
  x
}
val r2 = df.distinct.count()

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/14415/
Test PASSed.

Test build #109349 has started for PR 25498 at commit 08bba60.

good catch! LGTM

Test build #109349 has finished for PR 25498 at commit 08bba60.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/109349/
Test PASSed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/14440/
Test PASSed.

Test build #109378 has started for PR 25498 at commit 485157a.

Test build #109378 has finished for PR 25498 at commit 485157a.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/109378/
Test PASSed.

thanks, merging to master!
@xuanyuanking can you send PRs for branch 2.3 and 2.4? the code conflicts.

Sure, I'm doing the backport now.

Thank you, @xuanyuanking , @cloud-fan and @viirya !
I'm waiting for backporting PRs.
Also, cc @kiszk for branch-2.3.


Change matching HTML tag when either is edited

This is a re-open of #43402. (The conversation for the prior issue is locked.) It is worth looking into whether the extension system supports implementing this feature effectively. In the year plus since that issue was created, no one has created a working extension to implement the feature.
There is an extension, Auto Rename Tag, that tries to do this, but it only works if you type slowly (formulahendry/vscode-auto-rename-tag#62) and has lots of issues reported for high CPU load. I don't know if it's a matter of the developer knowing how to use the VS Code API correctly or the API not supporting well what the extension is trying to do.
This issue requests that the VS Code team look into (1) making sure a suitable extension is practical to develop and (2) making such an extension available.

We talked internally a bit on this. Makes sense to make this built-in.
We'd likely need some incremental parsing before this can take place.

dup of #76619

#76619 is about CSS and looks to me to be distinct from this issue, which is about HTML close tags (which CSS doesn't even have). @aechan Can you elaborate on the dup?

@aeschli Linked to wrong issue? Should be dup of this one: https://github.com/microsoft/vscode/issues/47069

sorry for that, thanks for the correction


prepare for 1.0.0 release

Related Issue(s):

n/a

Proposed Changes:

update to 1.0.0
update maturity to Stable
Clarify character + in URLs
remove GET form discussion, as this is never used.

PR Checklist:

[X] This PR has no breaking changes.
[X] I have added my changes to the CHANGELOG or a CHANGELOG entry is not required.

also fixes https://github.com/stac-api-extensions/sort/issues/6


Fix length calculation and some cleanup

This a 🐛 bug fix.
The doc says len() is the number of bits and this matches the code
in https://github.com/mafintosh/sparse-bitfield.git
Checklist

[x] tests pass
[x] tests and/or benchmarks are included

Also, tests in tree-index are failing with sparse-bitfield v0.5.0 but passing with these changes.


Add Developer Disk Images to XCode section

Project URL
https://github.com/haikieu/xcode-developer-disk-image-all-platforms
Category
XCode-other
Description
Add Developer Disk Images item to XCode-other section
Why it should be included to awesome-ios (mandatory)
Xcode Developer Disk Images is needed when you want to put your build to the device, however sometimes XCode is not updated with the latest Disk Images, you could find them here for convenience.
Checklist

[Not to close] Has 50 GitHub stargazers or more
[X ] Only one project/change is in this pull request
[X ] Isn't an archived project
[Will have more contributor in the future] Has more than one contributor
[Not Applicable] Has unit tests, integration tests or UI tests
[X ] Addition in chronological order (bottom of category)
[Not Applicable ] Supports iOS 9 / tvOS 10 or later
[Not Applicable ] Supports Swift 4 or later
[X] Has a commit from less than 2 years ago
[X] Has a clear README in English

Thanks for contributing! I always install the beta on my personal device and I've been using this technique for years.
The problem is that I always needed to install the new Xcode to get those files, so your repo definitely makes it easier!

Should be Xcode, not XCode 🙏
Sent with GitHawk

Should be Xcode, not XCode 🙏
Sent with GitHawk

Corrected

@lfarah  The travis-ci is failing due to some other reason, but the PR should be good to merge. Thanks

@haikieu Thanks for contributing!


Is it possible convert the hex string to integer value?

Hi, I'm trying to convert the hex string to integer value, is it possible?
For example:
def hexstr_to_int(n1 : string) : int{
//
}
print hexstr_to_int("ABCD");

Since this is a toy language, I didn't include any constructs which allow you to get part of a string, unfortunately strings are atomic objects which can only be added. In theory it is possible to implement such a function (since the language is Turing Complete) but you'd have to implement strings in a non-atomic way.

Hmm...I like your resource, it clear and simple, makes me easy understand, but I still cannot understand how to implement, it might be modify the source code? I think it's difficult to me, could you give me some advice step by step? Thanks.
For example:

The Lexer : The lexer should add ...
The Parser : The parser ...
Semantic Analysis: In semantic ...
The Interpreter : ...

Depends, if you want to be able to extract a character from a string, you might want to do something like my_string[1] to get the character in position 1. Thus you'd have to modify the Lexer to look out for [ and ], adding a rule to the EBNF and adapting the parser accordingly. Then the changes you make in the semantic analysis and the interpreter will be analogous, where the interpreter will use C++'s [] operator.
Does this give you a good idea of what to do?

Great!~I think i can understand what you say. But It sounds difficult in EBNF and semantic analysis. I don't think I am good enough to finish the goal... l will try it. Thanks your advice, drmenguin :)


deis run need pods-create permissions (rbac)

deis run needs pods - create permissions in charts/controller/templates/controller-clusterrole.yaml
Error:
Running 'bundle exec rake db:migrate'...
Error: Unknown Error (503): {"detail":"assessments-service-config-run-f1dc5 (run): Expecting value: line 1 column 1 (char 0)"}```

Logs
```INFO:scheduler:[assessments-service-config]: run assessments-service-config-run-5ouaj, img quay.io/welltok/assessments:f56e975, entrypoint ['/bin/bash', '-c'], cmd "['bundle exec rake db:migrate']"
ERROR:root:Expecting value: line 1 column 1 (char 0)
Traceback (most recent call last):
  File "/app/api/models/app.py", line 749, in run
    **data
  File "/app/scheduler/__init__.py", line 311, in run
    self.pod.create(namespace, name, image, **kwargs)
  File "/app/scheduler/resources/pod.py", line 42, in create
    raise KubeHTTPException(response, 'create Pod in Namespace "{}"', namespace)
  File "/app/scheduler/exceptions.py", line 10, in __init__
    data = response.json()
  File "/usr/local/lib/python3.5/dist-packages/requests/models.py", line 885, in json
    return complexjson.loads(self.text, **kwargs)
  File "/usr/lib/python3.5/json/__init__.py", line 319, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python3.5/json/decoder.py", line 339, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python3.5/json/decoder.py", line 357, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/rest_framework/views.py", line 486, in dispatch
    response = handler(request, *args, **kwargs)
  File "/app/api/views.py", line 241, in run
    rc, output = app.run(self.request.user, request.data['command'])
  File "/app/api/models/app.py", line 755, in run
    raise ServiceUnavailable(err) from e
api.exceptions.ServiceUnavailable: assessments-service-config-run-5ouaj (run): Expecting value: line 1 column 1 (char 0)```

Creating PR

This issue was moved to teamhephy/controller#15


[BUG] [Urgent] Cifar-10/100 sample dataset can not be loaded in 0.14.2

System information

OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Ubuntu 20.04
FiftyOne installed from (pip or source): pip
FiftyOne version (run fiftyone --version): 0.14.2
Python version: 3.8.3

Commands to reproduce
# pip install fiftyone==0.14.2
import fiftyone as fo
import fiftyone.zoo as foz

dataset = foz.load_zoo_dataset("cifar10", split="test")
print(dataset)

Describe the problem
Exception raised:
/ipykernel_864/2572829385.py in <module>
      2 import fiftyone.zoo as foz
      3
----> 4 dataset = foz.load_zoo_dataset("cifar10", split="train", drop_existing_dataset=True, overwrite=True, cleanup=True)
      5 print(dataset)

~/miniconda3/envs/edge/lib/python3.8/site-packages/fiftyone/zoo/datasets/__init__.py in load_zoo_dataset(name, split, splits, label_field, dataset_name, dataset_dir, download_if_necessary, drop_existing_dataset, overwrite, cleanup, **kwargs)
    195         )
    196
--> 197         info, dataset_dir = download_zoo_dataset(
    198             name,
    199             splits=splits,

~/miniconda3/envs/edge/lib/python3.8/site-packages/fiftyone/zoo/datasets/__init__.py in download_zoo_dataset(name, split, splits, dataset_dir, overwrite, cleanup, **kwargs)
    114         name, dataset_dir, **kwargs
    115     )
--> 116     return zoo_dataset.download_and_prepare(
    117         dataset_dir=dataset_dir,
    118         split=split,

~/miniconda3/envs/edge/lib/python3.8/site-packages/fiftyone/zoo/datasets/__init__.py in download_and_prepare(self, dataset_dir, split, splits, overwrite, cleanup)
   1002                     num_samples,
   1003                     classes,
-> 1004                 ) = self._download_and_prepare(split_dir, scratch_dir, split)
   1005
   1006                 # Add split to ZooDatasetInfo

~/miniconda3/envs/edge/lib/python3.8/site-packages/fiftyone/zoo/datasets/torch.py in _download_and_prepare(self, dataset_dir, scratch_dir, split)
    191         get_class_labels_fcn = _parse_classification_labels
    192         sample_parser = foud.ImageClassificationSampleParser()
--> 193         return _download_and_prepare(
    194             dataset_dir,
    195             scratch_dir,

~/miniconda3/envs/edge/lib/python3.8/site-packages/fiftyone/zoo/datasets/torch.py in _download_and_prepare(dataset_dir, scratch_dir, download_fcn, get_class_labels_fcn, sample_parser)
    534         )
    535     else:
--> 536         raise ValueError("Unsupported label class %s" % label_cls)
    537
    538     # Write the formatted dataset to `dataset_dir`

TypeError: not all arguments converted during string formatting

What areas of FiftyOne does this bug affect?

[ ] App: FiftyOne application issue
[x] Core: Core fiftyone Python library issue
[ ] Server: Fiftyone server issue

Willingness to contribute
The FiftyOne Community encourages bug fix contributions. Would you or another
member of your organization be willing to contribute a fix for this bug to the
FiftyOne codebase?

[ ] Yes. I can contribute a fix for this bug independently.
[ ] Yes. I would be willing to contribute a fix for this bug with guidance
from the FiftyOne community.
[x] No. I cannot contribute a bug fix at this time.

Also broken in v0.14.1, but work in v0.14.0.

Duplicate of https://github.com/voxel51/fiftyone/issues/1451. This is fixed by https://github.com/voxel51/fiftyone/pull/1452 on develop and will be included in the next release.


add  flag for OSLogWriter to retain log in release mode

as os_log document pointed, os_log will cover log as <private> when reviewed from console while in release mode
this option will give developer the right on whether to hide message on release-mode

need this <3

@lolicon Really need this. Could I test somehow with our app ?


Removed SIGHUP handler on win32.

This PR fixes https://github.com/taskiq-python/taskiq/issues/333.

Codecov Report
Attention: Patch coverage is 0% with 3 lines in your changes missing coverage. Please review.

Project coverage is 77.93%. Comparing base (20f92b0) to head (769e5f3).
Report is 1 commits behind head on develop.

Files
Patch %
Lines

taskiq/cli/worker/process_manager.py
0.00%
3 Missing :warning:

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@             Coverage Diff             @@
##           develop     #334      +/-   ##
===========================================
+ Coverage    77.75%   77.93%   +0.17%
===========================================
  Files           61       62       +1
  Lines         1803     1849      +46
===========================================
+ Hits          1402     1441      +39
- Misses         401      408       +7

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Automate retrieval of NOAA Obspack?

Automated retrieval not possible / fair, we've added some sample obspack data to the example_data repo for now, can add documentation for correct retrieval and processing.


gnu/linux build documment

Thanks for open the sources!
Please, add some docs to build it on gnu/linux!

Currently the only way would be to remove Eto.Wpf dependency (as this requires windows to be built), but otherwise should be compilable by installing latest mono-complete from here and running msbuild on Source/PabloDraw/PabloDraw.csproj  You could also use MonoDevelop as the IDE.
Once Eto 2.4 is released I'll switch it over to nuget dependencies so it should be fully compilable on any platform.


APEx benchmarks: add regression test

Add regression tests to the initial benchmarks.

To generate reference data, run benchmark once
Upload reference data to s3 object storage

What endpoint/bucket?

From then on every benchmark run should also compare its output to the reference data
Alerting when output does not match?

For the UDP owner, openEO team?
How to configure?

Save comparison result (Success, failure) to s3 object storage

So we can see a timeseries for each benchmark
Which endpoint/bucket?

Requirements:

Should be easy to update reference data.
Component that we can reuse in our own integrationtests.
Mostly via configuration, try to avoid having to write custom code for every scenario.

Component that we can reuse in our own integrationtests.

I started with reusable job result comparison utilities in the python client at https://github.com/Open-EO/openeo-python-client/pull/587

merged https://github.com/Open-EO/openeo-python-client/pull/587 no

Sync with Thomas for s3 buckets.

got s3 bucket info, and set up initial poc for comparing job results with reference data: bf11993827fd7f9d990724dbc4eaf0fc787d6207

First (intendedly) failing benchmark using reference data on S3 and brand new (unreleased) assert_job_results_allclose functionality of python client:
https://github.com/ESA-APEx/apex_algorithms/actions/runs/9961442215/job/27522945120
E           AssertionError: Issues for file 'openEO.tif':
E           Coordinates mismatch for dimension 'x': [295355. 295365. 295375. 295385. 295395. 295405. 295415. 295425. 295435.
E            295445. 295455. 295465. 295475. 295485. 295495. 295505. 295515. 295525.
...
E            296705. 296715. 296725. 296735. 296745. 296755. 296765. 296775. 296785.
E            296795. 296805. 296815. 296825. 296835.] != [644525. 644535. 644545. 644555. 644565. 644575. 644585. 644595. 644605.
E            644615. 644625. 644635. 644645. 644655. 644665. 644675. 644685. 644695.
...
E            645785. 645795. 645805. 645815. 645825. 645835. 645845. 645855. 645865.
E            645875. 645885. 645895. 645905. 645915. 645925. 645935. 645945. 645955.
E            645965. 645975. 645985.]
E           Coordinates mismatch for dimension 'y': [5679485. 5679475. 5679465. 5679455. 5679445. 5679435. 5679425. 5679415.
E            5679405. 5679395. 5679385. 5679375. 5679365. 5679355. 5679345. 5679335.
...
E            5677405. 5677395. 5677385. 5677375. 5677365. 5677355. 5677345. 5677335.
E            5677325. 5677315. 5677305. 5677295. 5677285. 5677275. 5677265. 5677255.
E            5677245. 5677235. 5677225. 5677215. 5677205.] != [5677475. 5677465. 5677455. 5677445. 5677435. 5677425. 5677415. 5677405.
E            5677395. 5677385. 5677375. 5677365. 5677355. 5677345. 5677335. 5677325.
E            5677315. 5677305. 5677295. 5677285. 5677275. 5677265. 5677255. 5677245.
...
E            5675395. 5675385. 5675375. 5675365. 5675355. 5675345. 5675335. 5675325.
E            5675315. 5675305. 5675295. 5675285. 5675275. 5675265. 5675255. 5675245.
E            5675235. 5675225. 5675215.]
E           Shape mismatch: (1, 229, 149) != (1, 227, 147)
E           Issues for metadata file 'job-results.json':
E           Differing 'derived_from' links (0 common, 24 only in actual, 24 only in expected):
E             only in actual: {'/eodata/Sentinel-2/MSI/L2A/2023/08/20/S2A_MSIL2A_20230820T1...
E             only in expected: {'/eodata/Sentinel-2/MSI/L2A/2023/09/17/S2B_MSIL2A_20230917T1....

first actual result uploaded automatically from failing test: https://s3.waw3-1.cloudferro.com/APEx-benchmarks/gh-10090720287!tests_test_benchmarks.py__test_run_benchmark_max_ndvi_fail_intentionally_!actual/openEO.tif

merged #20 which added the functionality (pytest plugin) to automatically upload generated assets on a failed test

added some docs too at https://github.com/ESA-APEx/apex_algorithms/blob/main/docs/benchmarking.md
I think that's enough scope for this ticket


Minimal Ancestors not identified?

Hi!
I'm experimenting with the package on a causal inference project I'm working on. I'm trying to identify confounders using the utility identify_confounders but I keep getting an empty set despite the manually proposed set of backdoor variables being confirmed by the  DoWhy method CausalGraph.check_valid_backdoor_set.
So I'm a bit confused. I would have expected that the identify_confounders utility to provide a set of confounders if they exist. the method from DoWhy seems to confirm that they do in fact exist.
Consequently, I'm wondering if I'm misunderstanding something in the utility method.
here is a minimum example:
import networkx as nx
from cai_causal_graph.causal_graph import CausalGraph

from cai_causal_graph.identify_utils import identify_confounders

dag_edges = [
    ('T','m'),
    ('m','e'),
    ('T','tt'),
    ('tt','treatment'),
    ('I','treatment'),
    ('I','ic'),
    ('treatment','ic'),
    ('ic', 'p'),
    ('I','p'),
    ('p','e'),
    ('p','ce'),
    ('I','soc'),
    ('soc','c'),
    ('c','ce'),
    ('ce','outcome'),
    ('e','outcome'),
]

G = nx.from_edgelist(dag_edges, create_using=nx.DiGraph)
cG = CausalGraph.from_networkx(G)

nx.draw_kamada_kawai(G, labels={k:k for k in G.nodes.keys()})

print(identify_confounders(cG, 'treatment', 'outcome'))

from dowhy.causal_graph import CausalGraph

cg = CausalGraph(treatment_name='treatment', outcome_name='outcome', graph=G)

print(cg.check_valid_backdoor_set(['treatment'], ['outcome'], set(['T','I']), dseparation_algo="naive"))

output:
[]
{'is_dseparated': True}

What am I missing here?
Thanks and happy new year!

Hey Andrew,
Thanks for the quick response!
Cheers,
Andrew

Hi Andrew (@arainboldt), we fixed this in the linked PR and released a patch version v0.3.14 with the fix. The docs were deployed as well to show the bug fix in the changelog. Thanks again for finding the issue and let us know if you notice anything else.

Wow! That's an amazing turn-around. And without an SLA! Too bad you guys don't run government services 😭


WSO2 AM Pre-Packaged Identity version 5.4.0/5.4.1

Description:

WSO2 AM Pre-Packaged Identity version 5.4.0/5.4.1
Even a version of the Keymanager Feature that will install on those versions would help.

By Chris Martens from https://wso2.org/jira/browse/APIMANAGER-6060

Hi Chris,
According to the WSO2 product Release Matrix, there is no compatible version of WSO2 API Manager for Identity Server 5.4.0 and 5.4.1. Therefore, key manager releases for IS 5.4.0 and 5.4.1 are not available and we do not recommend using IS 5.4.0 and 5.4.1 as the Key Manager for any API Manager version.
The recommended approach is to upgrade both Identity Server and API Manager for later released versions, APIM 2.6.0 and IS 5.7.0 as APIM 2.6.0 is compatible with the pre-packaged IS 5.7.0 release.


Installs 9.1 when 9.3 already installed, and no version specified

With the simple puppet manifest of
class { 'postgresql::server': }

when PostgreSQL 9.3 is installed results in an action of installing 9.1 with the same configured port number. Thus the new server cannot start, and the functioning server may fail to start the next time init scripts are run.
Defaults for database management should be safe.

Hello! We are doing some house keeping and noticed that this issue has been open for a long time.
We're going to close it but please do raise another issue if the issue still persists. 😄


On every action a "@@redux/INIT" action type is also triggered

I caught this by using the following in my reducer:
  switch(action.type) {
    default: {
      console.warn('uncaught action', action);
      return state;
    }
  }

This causes anything listening to the state change to trigger twice every time a single action is performed.

This causes anything listening to the state change to trigger twice every time a single action is performed.

Are you saying that you think you found the cause?

Nevermind, I get what you're saying. This bug is causing state change listeners to be called twice for every action. I think I know what's happening, it was part of some unfinished work that I pushed up and have since rethought.