Initialize formatted datetime on load

Why doesn't this directive render the formatted datetime on init?  There doesn't seem to be any simple way of doing this.  This should be default behavior.  Render datestring provided in "moment-picker" on init.

Hi @Yugloocamai, let me find a way to simplify this process...

In the next release this should be fixed.
<div ng-init="ctrl.stringDate = '2017-01-20';">
    <input class="form-control"
           placeholder="Select a date..."
           moment-picker="ctrl.stringDate"
           format="YYYY-MM-DD"
           ng-model="ctrl.momentDate"
           ng-model-options="{ updateOn: 'blur' }">
    <!-- ctrl.momentDate is undefined -->
</div>

Latest release v0.9.8 is now out.

Hello Indri,
Thanks for this fantastic date picker - I feel it's one of the best ones out there.
To offer you a little bit of context, I believe that most people developing applications prefer keeping one standard regarding time, which is the ISO8601 format (eg. 1980-12-20T00:00:00.000Z). There are many libraries out there which can format this for viewing purposes, hence the popularity of it.
I think that the expectation of many of my other fellow colleagues, including mine, is that when they pass a variable to your date picker, they expect it to be read automatically, and then display it in the format specified, but still keeping the actual model data in the initial ISO8601 way.
For example, if vm.dob = 1980-12-20T00:00:00.000Z, then <input moment-picker="vm.dob" format="MM-DD-YYYY"> should display the data as 12-20-1980, without having to manually format the string variable in the controller, or set the vm.dobModel to be equal to moment(). Then, when one would check vm.dob after the date-picker loaded, it would still be 1980-12-20T00:00:00.000Z.
The ultimate goal with this is to be able to control the format the data is displayed, while the model/data in itself maintains the ISO8601 format, and when sending that data back to the server, there is no extra formatting that needs to be done.
I believe that this works automatically with the 'YYYY-MM-DD' format, but I can confirm that it is not working with 'MM-DD-YYYY', since moment.js doesn't know how to pick it up, and it forces us to pre-format the text to MM-DD-YYYY, pass it to moment-picker and then reformat it to ISO8601 before sending it to the API, which is a cumbersome experience, it involves adding more code to controllers, when I think it can be nicely solved in the component.
Best,
Robert


Zip for iOS is not working

When trying to zip a file for iOS, it keeps giving me the following error:
JS ERROR TypeError: SSZipArchive.createZipFileAtPathWithContentsOfDirectoryKeepParentDirectoryCompressionLevelPasswordAESProgressHandler is not a function. (In 'SSZipArchive.createZipFileAtPathWithContentsOfDirectoryKeepParentDirectoryCompressionLevelPasswordAESProgressHandler(tempDestinationPath.path, folderPath, keepParent, -1, password, true, onZipProgress)', 'SSZipArchive.createZipFileAtPathWithContentsOfDirectoryKeepParentDirectoryCompressionLevelPasswordAESProgressHandler' is undefined

Looks like a wrong definition in the zip function

Found the error. Not sure if the pod is outdated or what could be the problem.
Anyway, what I did was change the node_module to the correct implementaiton ( I know is wrong )
I changed:
THIS
SSZipArchive.createZipFileAtPathWithContentsOfDirectoryKeepParentDirectoryCompressionLevelPasswordAndProgressHandler(tempDestinationPath.path, folderPath, keepParent, -1, password, true, onZipProgress);

to THIS
SSZipArchive.createZipFileAtPathWithContentsOfDirectoryKeepParentDirectoryWithPasswordAndProgressHandler(tempDestinationPath.path, folderPath, keepParent, password, onZipProgress);

fixed: https://github.com/triniwiz/nativescript-zip/commit/c109fc3c295f9ad2460b5259b652340b7a90ce47


Open Source Project Review

Items we can review on a live stream:

Open-source project: https://github.com/flyimg/flyimg : dockerized application can run as a microservice to resize and crop images on the fly.

Thank you for sharing 👍 . I will review in a live stream or video soon 🤓

Thank you for sharing 👍 . I will review in a live stream or video soon 🤓


[npm] Remove Babel from npm-shrinkwrap.json

Babel was removed from package.json; remove it from npm-shrinkwrap.json.
Fix #3549

cc @vjeux @martinbigio

cc @tadeuzagallo

This is the shrinkwrap that was automatically generated, did you manually remove it? It seems that it's still a transitive dependency.

Oh yeah you're right. I got confused since it looked like a top-level dependency and I didn't see it in package.json.


feat: add ignoreErrors flag to LlamaParse

This also fixes an error that would not create the directory if it did not exist.
#961

@marcusschiesser Applied your review, my bad I missed that. Thanks!


Fix broken links

./docs/about/FAQ.md

[ ] https://www.sciencedaily.com/releases/2016/11/161110154942.htm => 405

See also: https://travis-ci.org/sonarwhal/sonar/jobs/249880967

Closing in favor of #363.


enabled custom filters in metric-aggr

Fixing https://github.com/splunk/splunk-connect-for-kubernetes/issues/621

I have rebased with develop.

let's add aggrCustomFilters: to values.yaml so that user can specify a different custom filter.
There is already custom filter example in values.yaml file (https://github.com/splunk/splunk-connect-for-kubernetes/blob/d8a62ec04d13499ed675dc521672e6b827b6bf53/helm-chart/splunk-connect-for-kubernetes/values.yaml#L489-L501).
It is same for all SCK plug-in. Same example would apply for this plugin as well.
What are your thoughts?

I think @rockb1017 is right on this one, we need to differentiate between filters for metrics and metrics-agg

Hey @rockb1017, added customFiltersAggr config


UX

Afternoon all,
I’ve made the amendments Matt has suggested to the Google Form Application and added comments to the Word doc outlining the changes.
https://app.zenhub.com/files/201485031/bb582f66-3620-48c9-9547-15fac515f869/download
The links to the google form can be found on the document or below:
https://docs.google.com/forms/d/1yJ6Qgy5Ndn8pNuuIyFkQY8376bJI004wfnK_1DNlKR0/edit?usp=drive_web
https://forms.gle/DfCCpG2uhfYSisZs8
Kind regards,
Elliot Koh
Selection Exercise Executive Officer | Judicial Appointments Commission | M:07591 994182
www.judicialappointments.gov.uk | LinkedIn | Twitter |
From: Howley, Matt<EMAIL_ADDRESS>Sent: 17 October 2022 13:46
To: Koh, Elliot<EMAIL_ADDRESS>Cc: Isaac, Andrew | (He/They<EMAIL_ADDRESS>Russell, Tom<EMAIL_ADDRESS>Subject: RE: JAC Application (Welsh) - Google Forms
This is looking great, thanks Elliot and apologies for taking so long to respond. I’ve made some comments/suggestions against some of your notes on the attached. If you could have a look at these and make any changes I’ve suggested that would be great.
Copying in Andrew and Tom just for info at this stage, but if you could copy them into your response once the changes are made Elliot that would be fab.
Andrew/Tom – if Elliot is able to make the changes before the Digital Team meeting on Wednesday 26th Oct, please could you share with Rob and discuss at the meeting whether you think any other changes are required before I send for translation when I’m back from leave the week after next.
Kind regards
Matt Howley
Head of Digital Delivery and Programme Office | Judicial Appointments Commission | M: 07719 098105
If you receive this email outside regular working hours it means I am working flexibly. Flexibility works for me but please do not feel any pressure to respond outside of your own working pattern.
www.judicialappointments.gov.uk | LinkedIn | Twitter |
From: Koh, Elliot<EMAIL_ADDRESS>Sent: 27 September 2022 16:47
To: Howley, Matt<EMAIL_ADDRESS>Subject: JAC Application (Welsh) - Google Forms
Hi Matt,
I’ve finished creating a generic JAC application on Google Forms. I’ve made some notes on each section comparing the platform and google forms functionality, and raising any issues identified. The links to the google form can be found on the document or below:
Admin Link
Applicant Link
Kind regards,
Elliot Koh
Selection Exercise Executive Officer | Judicial Appointments Commission | M:07591 994182
www.judicialappointments.gov.uk | LinkedIn | Twitter |

https://welearnwelsh.com/grammar/how-to-say-yes-and-no-in-welsh/
https://digitalpublicservices.gov.wales/resources/digital-service-standards/#design-in-Welsh-and-English
https://www.gov.uk/government/organisations/valuation-office-agency/about/welsh-language-scheme
https://gov.wales/welsh-language-technology

Closed as archived in this document


[Feature][Sort] Add LessThanOperator definition to support transform

Description
Add LessThanOperator definition to support transform
Use case
No response
Are you willing to submit PR?

[X] Yes, I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

Same as https://github.com/apache/incubator-inlong/pull/3820.


num-results option for az storage blob list

Hello,
My blob container is expected to have more than 100,000+ blob files. I would like to use az storage blob list to recursively get a list of all blobs in the container. If I do not specify --num-results would I receive the entire list of blobs with my request?
Thanks very much for your help
Chanaka

Environment summary
Install Method (e.g. pip, interactive script, apt-get, Docker, MSI, edge build) / CLI version (az --version) / OS version / Shell Type (e.g. bash, cmd.exe, Bash on Windows)

Related to #3481.

Hello TJPREScott,

Is this different from #5474?

Hello TJPREScott,
Yes, az storage blob list does seem to work recursively by listing filenames but the documentation is confusing on --num-results option. I would like to get all the results without paging. My container is expected to have over 100,000 + blob files. Will this work if I don't include --num-results option?
Thanks for your help

@troydai

@chanaka-jostle yes, az storage blob list will by default list all the blobs in a container (note this might take some time depending on how many blobs you have), we page for all the blobs for you before returning everything.
Including the --num-results parameter will limit the result to whatever you set, with a maximum of 5000 items.

Thanks for the reply!


Duplicated namespace in redis keys and issues when using a single store for different caches.

Bug description
Every newly instantiated Keyv cache sets the namespace attribute of the underlying store, which has two implications (at least when using the KeyvRedis backend):

the namespace is duplicated. Redis keys have names like sets:namespace:<namespace>:<namespace>:<key> instead of sets:namespace:<namespace>:<key> (I wonder if we need such a long prefix at all, vs just <prefix>:<namespace>:<key>, where <prefix> can be configured by the user when instantianing the store, but that's a different story...)
it's not possible to safely use the same store for multiple caches, as instantiating a new cache modifies the namespace of the underlying store, which is used to retrieve keys, thus breaking previous stored entries.

How to reproduce
Given this snippet which instantiates two Keyv caches which use the same store (which, in turn, share the same pre-existing Redis connection).
import KeyvRedis from '@keyv/redis';
import Redis from 'ioredis';
import Keyv from 'keyv';

const redisClient = new Redis(process.env.REDIS_URI);
const redisStore = new KeyvRedis(redisClient, { useRedisSets: false });

const KEY = 'jane';

const addressCache = new Keyv({
  store: redisStore,
  namespace: 'addresses',
  ttl: 24 * 60 * 60, // 1 day
});
await addressCache.set(KEY, 'Somewhere in London');

const professionCache = new Keyv({
  store: redisStore,
  namespace: 'professions',
  ttl: 10 * 24 * 60 * 60, // 10 days
});
await professionCache.set(KEY, 'Doctor');

console.log(await addressCache.get(KEY));
console.log(await professionCache.get(KEY));

await redisClient.disconnect();

Running the script you'll get
undefined
'Doctor'

Instead of the expected:
'Somewhere in London'
'Doctor'

And this is the output of redis MONITOR command while running the script:
1727813797.264412 [0 <IP_ADDRESS>:52014] "info"
1727813797.267410 [0 <IP_ADDRESS>:52014] "set" "sets:namespace:addresses:addresses:jane" "{\"value\":\"Somewhere in London\",\"expires\":1727813883685}" "PX" "86400"
1727813797.270006 [0 <IP_ADDRESS>:52014] "set" "sets:namespace:professions:professions:jane" "{\"value\":\"Doctor\",\"expires\":1727814661298}" "PX" "864000"
1727813797.271760 [0 <IP_ADDRESS>:52014] "get" "sets:namespace:professions:addresses:jane"
1727813797.275019 [0 <IP_ADDRESS>:52014] "get" "sets:namespace:professions:professions:jane"

You can see the duplicated namespace and the full key for the address cache entries changing after the professions cache is instantiated.
Suggested resolution
Keyv should not set the namespace of the underlying store at all. Store namespace (or, better, prefix) should be optional and configurable by the user at the store level.  This would result in much cleaner redis keys like <store-prefix>:<key cache namespace>:<key>. It would also allow users to configure short prefixes/namespaces to save memory, which can be quite important for redis and other stores where key size is consequential.

@leo-fresha thanks for posting this and we are working on an update around this to make it better.

@leo-fresha - we are working on the rewrite now. I am going to close this issue and add it as an item to resolve on the main thread.


Popup buttons not aligned center

Description:
When sync popup is shown the buttons are not aligned center in the popup on my S22

QA passed


Add Linux requirments info to README

Example fails to build without these installed, and it was not yet mentioned.

I don't think I should add it in the readme, because window_manager doesn't need to install any dependencies, this dependency is needed by tray_manager in the example, maybe I should put this description in example/README.md

I see what you mean, and yeah that might make better sense.
I just fired up the example to check out window_manager and met the failure.


Fix reserved-word guard

There were two regressions in the babel port:

the name option is not a prequisite for the reserved words being reserved.
Only the name key is reserved if the enum is given a custom name. The rest of the keys are reserved unconditionally.
Array.prototype.includes has virtually zero browser support (it's not in
Chrome, Firefox, Safari, Opera, or IE) nor is it even in Node (not even iojs). It was never caught in the tests, though, because there weren't any tests that provided a custom name. (Which normally shouldn't have been a prerequisite, but was in this case being masked by bug # 1 above.)


token contains an invalid number of segments

I'm hitting this error in the cognito callback function. Looks like it's coming from here https://github.com/tmaiaroto/aegis/blob/master/framework/cognito_client.go#L287

The issue is actually here.
https://github.com/tmaiaroto/aegis/blob/master/framework/cognito_client.go#L254
I get the following response body.
{
    "error": "invalid_grant"
}

It decodes properly and continues without error. but the Token.IDToken field is blank. I updated my grants and it works now. Might be worth checking the response code to catch the error.

Have you set up your Cognito pool?
The example will try to figure out the well known keys and everything else
based on the user pool ID and also the client ID. Take a look at the readme.
And if so, how do you have your pool set up?
On Wed, Jul 4, 2018 at 3:27 PM Andy Gorman<EMAIL_ADDRESS>wrote:

I'm hitting this error in the cognito callback function. Looks like it's
coming from here
https://github.com/tmaiaroto/aegis/blob/master/framework/cognito_client.go#L287
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/tmaiaroto/aegis/issues/34, or mute the thread
https://github.com/notifications/unsubscribe-auth/AAMXnb7MIiYCDWgSmJ1msJ7d-a3HQq2Cks5uDUE9gaJpZM4VDD87
.

Definitely will, thanks. And thank you for testing this all out and bearing
with me on the issues, it really helps.
On Wed, Jul 4, 2018 at 3:40 PM Tom Maiaroto<EMAIL_ADDRESS>wrote:

Have you set up your Cognito pool?
The example will try to figure out the well known keys and everything else
based on the user pool ID and also the client ID. Take a look at the readme.
And if so, how do you have your pool set up?
On Wed, Jul 4, 2018 at 3:27 PM Andy Gorman<EMAIL_ADDRESS>wrote:

I'm hitting this error in the cognito callback function. Looks like it's
coming from here
https://github.com/tmaiaroto/aegis/blob/master/framework/cognito_client.go#L287
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/tmaiaroto/aegis/issues/34, or mute the thread
https://github.com/notifications/unsubscribe-auth/AAMXnb7MIiYCDWgSmJ1msJ7d-a3HQq2Cks5uDUE9gaJpZM4VDD87
.

Can you let me know the options you had for your Cognito client app? It'll let me test this better.
For now, I've included an Error field in the CognitoToken struct so that it will take the "error" from the response body and put it somewhere accessible to your code. I've also updated the Cognito example to reflect how you'd do this. https://github.com/tmaiaroto/aegis/blob/master/examples/cognito/main.go#L90
So handlers can now more easily do something about this situation. I may need to adjust the service itself to handle more types of auth flows or distinctly label things to make sure it's clear about the authorization code grant flow being used.

For the client app I can reproduce by changing the seeings under App client settings -> OAuth 2.0. Check only Authorization code grant and aws.cognito.signin.user.admin.

Ok, I changed the example to the more common access token. Arguably the right one to be using. If you use openid then you'd have the ID token as well. Left some comments in there, it's simply a matter of using the "proper" token (or the only one you might have I suppose depending on options).
I could get this working (with tokens.AccessToken) using aws.cognito.signin.user.admin grant alone. Let me know if there are any other configuration options that cause further issues. Thanks!


A way to set-up repository credentials without breaking configuration cache

Expected Behavior

Configuration cache hit when setting repository credentials.
Via Env variables
ORG_GRADLE_PROJECT_repoAuthHeaderName: "username"
ORG_GRADLE_PROJECT_repoAuthHeaderValue: "value"

or gradle.properties
Current Behavior
Configuration cache can't be re-used:
Calculating task graph as configuration cache cannot be reused because the set of environment variables prefixed by 'ORG_GRADLE_PROJECT_' has changed.

or
Calculating task graph as configuration cache cannot be reused because file '../../../.gradle/gradle.properties' has changed.

Context

Our CI server generates a short-lived 'JOB_TOKEN' that is generated for each job to access Maven repositories. But setting Gradle's repository credential for each job makes configuration caches non-reusable between CI jobs. It would be nice to have a way to set the repository credential without breaking the configuration cache.
Repositories configured like this in settings.gradle.kts:
dependencyResolutionManagement {
    repositories {
        <...>
        exclusiveContent {
            forRepository {
                maven {
                    name = "repo"
                    setUrl("<...>")
                    credentials(HttpHeaderCredentials::class)
                    authentication {
                        create<HttpHeaderAuthentication>("header")
                    }
                }
            }
            filter { includeGroup("some.group") }
        }
}

Thank you for your interest in Gradle!
This issue needs a decision from the team responsible for that area. They have been informed, response time may vary.

To CC team:
In our understanding, the reporter wants to reuse CC-entry while using different credentials for jobs.

This is how GitHub works too... Makes me think that the credentials are still getting stuck and the configuration cache.

We face a similar problem with AWS CodeArtifact which doesn't allow credentials to last longer than 12h approx, so the longest we can get our configuration cache to stay valid is that. Being able to have those credentials change and now invalidate the cache would be great.
We currently have a daily setup script that stores all those temp creds in files, and then settings.gradle just reads those files, but I think we could easily adjust to use env variables/properties or whatever the options are.


Implement Pagination for /view endpoint

/view endpoint currently returns ALL of the past updates. I plan to use this bot for YEARS which means every single day the data returned by /view will increase and increase

Be careful not to use limit&skip methods in mongodb to add this
BAD :
https://www.npmjs.com/package/mongoose-paginate-v2
https://github.com/aravindnc/mongoose-paginate-v2/blob/master/src/index.js#L177
GOOD :
https://github.com/mixmaxhq/mongo-cursor-pagination
API Pagination is typically implemented one of two different ways:
Offset-based paging. This is traditional paging where skip and limit parameters are passed on the url (or some variation such as page_num and count). The API would return the results and some indication of whether there is a next page, such as has_more on the response. An issue with this approach is that it assumes a static data set; if collection changes while querying, then results in pages will shift and the response will be wrong.
Cursor-based paging. An improved way of paging where an API passes back a "cursor" (an opaque string) to tell the caller where to query the next or previous pages. The cursor is usually passed using query parameters next and previous. It's implementation is typically more performant that skip/limit because it can jump to any page without traversing all the records. It also handles records being added or removed because it doesn't use fixed offsets.

https://swr.vercel.app/docs/pagination
Sent from my iPhone

On Jul 10, 2021, at 09:31, Matt @.***> wrote:

Be careful not to use limit&skip methods in mongodb to add this
BAD :
https://www.npmjs.com/package/mongoose-paginate-v2
https://github.com/aravindnc/mongoose-paginate-v2/blob/master/src/index.js#L177
GOOD :
https://github.com/mixmaxhq/mongo-cursor-pagination
API Pagination is typically implemented one of two different ways:
Offset-based paging. This is traditional paging where skip and limit parameters are passed on the url (or some variation such as page_num and count). The API would return the results and some indication of whether there is a next page, such as has_more on the response. An issue with this approach is that it assumes a static data set; if collection changes while querying, then results in pages will shift and the response will be wrong.
Cursor-based paging. An improved way of paging where an API passes back a "cursor" (an opaque string) to tell the caller where to query the next or previous pages. The cursor is usually passed using query parameters next and previous. It's implementation is typically more performant that skip/limit because it can jump to any page without traversing all the records. It also handles records being added or removed because it doesn't use fixed offsets.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

I think we want a separate page for each user ? Kind of like a blog site tbh

I think we want a separate page for each user ?  My main use case would be to visit the site 1x a day after 11 to view everyones updates  (and another time to post my update, and ideally a way to respond to other peoples updates)

So is the current collapse view annoying? Would it be better to have just one table showing the last update for each person? And then maybe expand each row to show more updates from them

So is the current collapse view annoying?

Yes

Would it be better to have just one table showing the last update for each person?

Yes

And then maybe expand each row to show more updates from them
Yes

I fixed the current view to show latest 1 update, now just need to make the "more" button work to show the remaining updates

site is deprioritized


Why softmax is applied in the _local_layer function?

https://github.com/RandolphVI/Hierarchical-Multi-Label-Text-Classification/blob/843aeaa6f7c17937fc137cef02b4d0191dfb35bc/HARNN/text_harnn.py#L101-L103
May I ask why softmax is applied here? Based on my understanding from your paper, the calculated 'score' should directly be multiplied with 'W_att^h' and then average(mean) is applied to generate 'visual' for the next layer.
But from your code, additional softmax is applied before average(mean).
May I ask why softmax is applied here and please correct me if my understanding is wrong and thanks!

@XinyiZ001
Sorry for my mistakes. I used the score directly in the paper.
And I guess is that when I try to verify adding the softmax or not would benefit the final performance  (the effect is basically the same). And the code uploaded is my local version (which is added the softmax function) rather than the code version in my server, so there was a deviation.
To summarize, the details you mentioned are my mistakes, but should not affect the use of the model (because the results are basically the same). Also, thank you for your attention to my work.


Add different color for different kinds of modules

💡 Describe the solution you'd like
I think a really cool feature would be to make a visual distinction between Android apps, Android libs, Kotlin/Jvm modules (maybe others related with KMP? 🤔 I feel these ones would be a good start though).
🤚 Do you want to develop this feature yourself?
I might look into it if I have time, but right now it's impossible. Between my own library, work, family, I'm totally booked 😬

Cool idea! This can definitely be useful, especially when you're dealing with big projects. We got a couple of options here - change the node colors or the edges.
This could let folks personalize their node color settings. I mean, I was thinking of letting users pick a color for the nodes using a regex pattern. This would allow all something-feature modules to show up in red.
What do you think?

The regex option is good, but I feel like an option specifically for android apps / android libs / kotlin/jvm modules would still be nice in some cases. For example in our project, there's no naming convention that lets us know about this characteristic of the module.
I am assuming it isn't hard to know what kind of module it is? Maybe based on which plugins are applied? (agp app, agp lib, etc?).

Hey @raamcosta , I've given this a try on #35
It'd be nice to have your thoughts on the overall API and features.


about usage

Hi, @hehefan ,
Thanks for releasing such a useful package. However, there's lack of guide on how to use it. Could you provide some guide for it (such as visualization on the prediction, training steps, etc.)?
THX!

Hi @amiltonwong,

You need to compile the operations in "modules/tf_ops" by executing the "make" command. The " CUDA_HOME" and "CUDNN_HOME" in Makefiles should be set in each directory before compiling.

The visualization script is "scripts/visualization.py", which is used for real-world dataset visualization. For this script, you need a Ubuntu Desktop and MayaVi should be installed. This script takes npy files as inputs.

For Moving MNIST Point Cloud training, just directly run "train-mmnist.py". For Argoverse and nuScenes training, you first need to extract point cloud by yourself and then run "train-argo-nu.py". The "scripts/extract-nu-pc.py" provides the script to extract point clouds from nuScenes.  For Argoverse, you can directly obtain point clouds in the "lidar" directories.

Best regards.

Hello @hehefan,
First thank you for releasing your code!
I'm trying to use it and first I am trying to compile operations in modules/tf_ops.
I change the " CUDA_HOME" and "CUDNN_HOME" in Makefiles and checked the architecture.
For every operation, except 3d_interpolation, I get the following errors:
make: Dépendance circulaire tf_nndistance_g.cu <- tf_nndistance_g.cu.o abandonnée. /usr/include/c++/8/type_traits(1049): error: type name is not allowed /usr/include/c++/8/type_traits(1049): error: type name is not allowed /usr/include/c++/8/type_traits(1049): error: identifier "__is_assignable" is undefined 3 errors detected in the compilation of "/tmp/tmpxft_000039a3_00000000-6_tf_nndistance_g.cpp1.ii".
Do you have any idea about what is happening?
I use tensorflow 1.9.0, cuda 9.0, gcc 8.3.
Thank you.

Hi @IdeGelis,
I have not compiled the operations with gcc 8.3. Could you please try gcc 5.4?
Best regards.

Hi,
Yes, I moved to gcc 6.3 and with little modification in makefiles, it worked! Thank you!
Regards.

Cheers! @IdeGelis

Hi @hehefan ,
like IdeGelis I'm trying to reproduce your interesting result. After changing the " CUDA_HOME" and "CUDNN_HOME" in Makefiles I'm trying to compile operations in modules/tf_ops.
Unfortunately I'm getting for all modules the following error:
C:/Users/localuser/Anaconda3/envs/tensorflow/lib/site-packages/tensorflow/include/tensorflow/core/framework/op_def.pb.h:9:42: fatal error: google/protobuf/stubs/common.h: No such file or directory  #include <google/protobuf/stubs/common.h>                                           ^ compilation terminated. make: *** [Makefile:17: tf_interpolate_so.so] Error 1
I've already read some related posts such as https://github.com/protocolbuffers/protobuf/issues/5805 , where it' s stated that this issue should be solved by using the cuda update.
I'm using tensorflow 1.13.1, gcc 6.3, cuda 10.1 update2 .

Hi @Philipp03 ,
Could you please have a try on tensorflow 1.9 or tensorflow 1.12 ?

Hi @hehefan ,
I gave it a try. With tensorflow 1.12 the error still occurs. With tf 1.9, cuda 10.1 update 2 and gcc 5.4 I'm getting another error:
g++ -std=c++11 -shared -fPIC -o tf_interpolate_so.so tf_interpolate.cpp -IC:/Users/localuser/Anaconda3/envs/py36/lib/site-packages/tensorflow/include -IC:/Programme/NVIDIA_GPU_Computing_Toolkit/CUDA/v10.1/include -L-LC:/Programme/NVIDIA_GPU_Computing_Toolkit/CUDA/v10.1/lib -LC:/Programme/NVIDIA_GPU_Computing_Toolkit/CUDA/v10.1/lib/x64 -LC:/Users/pmerk/cuda/lib -LC:/Users/pmerk/cuda/lib/x64 -lcudart -L C:/Users/localuser/Anaconda3/envs/py36/lib/site-packages/tensorflow -ltensorflow_framework -lcublas -O2 -D_GLIBCXX_USE_CXX11_ABI=0
tf_interpolate.cpp:1:0: warning: -fPIC ignored for target (all code is position independent)
 #include <cstdio>
 ^
In file included from C:/Users/localuser/Anaconda3/envs/py36/lib/site-packages/tensorflow/include/tensorflow/core/platform/mutex.h:31:0,
                 from C:/Users/localuser/Anaconda3/envs/py36/lib/site-packages/tensorflow/include/tensorflow/core/framework/op.h:32,
                 from tf_interpolate.cpp:6:
C:/Users/localuser/Anaconda3/envs/py36/lib/site-packages/tensorflow/include/tensorflow/core/platform/default/mutex.h:143:8: error: 'cv_status' in namespace 'std' does not name a type
   std::cv_status wait_for(mutex_lock& lock,
        ^
C:/Users/localuser/Anaconda3/envs/py36/lib/site-packages/tensorflow/include/tensorflow/core/platform/default/mutex.h:158:8: error: 'cv_status' in namespace 'std' does not name a type
   std::cv_status wait_until_system_clock(
        ^
C:/Users/localuser/Anaconda3/envs/py36/lib/site-packages/tensorflow/include/tensorflow/core/platform/default/mutex.h: In function 'tensorflow::ConditionResult tensorflow::WaitForMilliseconds(tensorflow::mutex_lock*, tensorflow::condition_variable*, tensorflow::int64)':
C:/Users/localuser/Anaconda3/envs/py36/lib/site-packages/tensorflow/include/tensorflow/core/platform/default/mutex.h:166:3: error: 'cv_status' is not a member of 'std'
   std::cv_status s = cv->wait_for(*mu, std::chrono::milliseconds(ms));
   ^
C:/Users/localuser/Anaconda3/envs/py36/lib/site-packages/tensorflow/include/tensorflow/core/platform/default/mutex.h:167:11: error: 's' was not declared in this scope
   return (s == std::cv_status::timeout) ? kCond_Timeout : kCond_MaybeNotified;
           ^
C:/Users/localuser/Anaconda3/envs/py36/lib/site-packages/tensorflow/include/tensorflow/core/platform/default/mutex.h:167:21: error: 'std::cv_status' has not been declared
   return (s == std::cv_status::timeout) ? kCond_Timeout : kCond_MaybeNotified;
                     ^
In file included from C:/Users/localuser/Anaconda3/envs/py36/lib/site-packages/tensorflow/include/tensorflow/core/platform/notification.h:26:0,
                 from C:/Users/localuser/Anaconda3/envs/py36/lib/site-packages/tensorflow/include/tensorflow/core/lib/core/notification.h:21,
                 from C:/Users/localuser/Anaconda3/envs/py36/lib/site-packages/tensorflow/include/tensorflow/core/framework/cancellation.h:22,
                 from C:/Users/localuser/Anaconda3/envs/py36/lib/site-packages/tensorflow/include/tensorflow/core/framework/op_kernel.h:24,
                 from tf_interpolate.cpp:7:
C:/Users/localuser/Anaconda3/envs/py36/lib/site-packages/tensorflow/include/tensorflow/core/platform/default/notification.h: In member function 'bool tensorflow::Notification::WaitForNotificationWithTimeout(tensorflow::int64)':
C:/Users/localuser/Anaconda3/envs/py36/lib/site-packages/tensorflow/include/tensorflow/core/platform/default/notification.h:69:20: error: 'class tensorflow::condition_variable' has no member named 'wait_for'
                cv_.wait_for(l, std::chrono::microseconds(timeout_in_us)) !=
                    ^
C:/Users/localuser/Anaconda3/envs/py36/lib/site-packages/tensorflow/include/tensorflow/core/platform/default/notification.h:70:25: error: 'std::cv_status' has not been declared
                    std::cv_status::timeout);
                         ^
make: *** [Makefile:17: tf_interpolate_so.so] Error 1

Hi @Philipp03 ,
For TF 1.9,  please have a try on gcc 5.4.0, CUDA 9.0. It there are still some errors, please try to use virturalevn to build the environment, rather than anaconda.

I tried it with CUDA 9.0 and after that failed I tried to use virtualenv to build the environment still getting the same error from above

Hi @Philipp03,
To make it work on gcc 6.3 I removed -D_GLIBCXX_USE_CXX11_ABI=0 in every Makefile.
Regards

Is there a Windows version of how to use it? Please.

Is there a Windows version of how to use it? Please.

Hi @ZhaoPengpeng1116 .
I am sorry that there has not been a Windows version.  I strongly suggest you set up a Ubuntu desktop, which is also convenient for visualization.
Thank you.

Hi @ZhaoPengpeng1116 .
I am sorry that there has not been a Windows version.  I strongly suggest you set up a Ubuntu desktop, which is also convenient for visualization.
Thank you.

Thank you very much, I have compiled the .so files on Ubuntu.
Thank you!

Thank you very much, I have compiled the .so files on Ubuntu.
Thank you!

Hi @amiltonwong,

You need to compile the operations in "modules/tf_ops" by executing the "make" command. The " CUDA_HOME" and "CUDNN_HOME" in Makefiles should be set in each directory before compiling.
The visualization script is "scripts/visualization.py", which is used for real-world dataset visualization. For this script, you need a Ubuntu Desktop and MayaVi should be installed. This script takes npy files as inputs.
For Moving MNIST Point Cloud training, just directly run "train-mmnist.py". For Argoverse and nuScenes training, you first need to extract point cloud by yourself and then run "train-argo-nu.py". The "scripts/extract-nu-pc.py" provides the script to extract point clouds from nuScenes.  For Argoverse, you can directly obtain point clouds in the "lidar" directories.

Best regards.

When you say that you first need to extract pointcloud by yourself for nuscenes, do you mean just extract from every file in the lidar folder and store it in trainval and test folder? If you do that, when do you synchronize the pointclouds? Also, you have not calibrated the pointclouds too, where do you do that?

I met an error that "Could not create cudnn handle: CUDNN_STATUS_NOT_INITIALIZED, Possibly insufficient driver version: 418.152.0"
It looks like that cuda9.0 is supported by driver418. I run the codes in a docker "tensorflow/tensorflow:1.12.0-gpu-py3", and I changed cuDNN from 7.0 to 7.4.2, which is didn't work all.
Could you please tell me the accurate version of cuda, cudnn, tensorflow-gpu, and the nvidia driver? Thanks!

By the way, I noticed that Makefiles of tf_ops say cuda_path=/usr/local/cuda-9.0 cudnn_path=/usr/local/cudnn7.4-9.0
It is usually method of installing cudnn that put the "include"and "lib64"folders in cuda's path(/usr/local/cuda-9.0) , so why seperate the folders?

hello ,
when I try to run train-mnist.py in autodl,it says ' /root/PointRNN-master/modules/tf_ops/sampling/tf_sampling_so.so: cannot open shared object file: No such file or directory',what should I do about this issue?

Hi,Prof@@hehefan,
I am a little confused about the codes:"def get_batch(dataset, batch_size):
batch_data = []
for i in range(batch_size):
sample = dataset[0]
batch_data.append(sample)
return np.stack(batch_data, axis=0)
" in the train-argo-nu.py.what does it mean when it keep getting dataset[0]?


HttpError: Not Found in gradle-build-action post-action

Hello there!
Thanks for the development of this plugin, and especially for the recent Dependency Graph Submission feature - it's pretty useful to make Dependabot work with Gradle.
While configuring the workflow, I experienced the issue that happens in the post-action stage:
Warning: Unhandled error in Gradle post-action - job will continue: HttpError: Not Found
HttpError: Not Found
    at /runner/_work/_actions/gradle/gradle-build-action/v2/dist/post/index.js:50[98]
    at processTicksAndRejections (node:internal/process/task_queues:96:5)

Some context:

it happens for the GitHub Enterprise Server (we using v.3.8.1)
our workflow setup looks like this:

name: Gradle Dependency Submission
on:
  push:
    branches:
         - main

jobs:
  gradle-action-detection:
    runs-on: [ ubuntu-latest ]
    permissions:
      contents: write
    steps:
      - name: 'Checkout Repository'
        uses: actions/checkout@v4

      - name: Setup Java 17
        uses: actions/setup-java@v3
        with:
          java-version: '17'
          distribution: corretto

      - name: Setup Gradle to generate and submit dependency graphs
        uses: gradle/gradle-build-action@v2
        with:
          dependency-graph: generate-and-submit

      - name: Run a build, generating the dependency graph snapshot which will be submitted
        run: ./gradlew build

The provided error happens for the Post Setup Gradle to generate and submit dependency graphs step.
Based on the logs, I can see that the error is originated here:
await submitDependencyGraphs(await uploadDependencyGraphs())

when the submitDependencyGraphs method is called.
I noticed that there is the exact endpoint hardcoded, so I wonder whether that could be somehow related to the differences between the regular GitHub and GitHub Enterprise: according to the GitHub Docs, there is a slight diff:

regular GitHub: https://api.github.com/repos/OWNER/REPO/dependency-graph/snapshots
GitHub Enterprise: http(s)://HOSTNAME/api/v3/repos/OWNER/REPO/dependency-graph/snapshots

Maybe, it seems like this might be somehow resolved by the Octokit library you are using under the hood. However, I didn't manage to find the answer while browsing the codebase.
Also, I double-checked that:

the Dependency Submission endpoint is available within our GitHub server
tried calling it with different scenarios via curl (without a token, without /api/v3, with the wrong token), and I can't get a Not found response from it - so it doesn't seem to be some permissions-related issue.

Question:
Is it me doing something wrong, or is it something that is not supported by this action?
Any help here is appreciated.
Thanks a lot!

Thanks for your detailed report. I'm not able to test this action on GitHub Enterprise, so it's quite possible you've discovered a genuine issue.
It looks like we'll need to initialize Oktokit with your GitHub Enterprise server URL: https://github.com/octokit/octokit.js#constructor-options.
The value should be available in the [GITHUB_SERVER_URL](https://docs.github.com/en/enterprise-cloud@latest/actions/learn-github-actions/variables#default-environment-variables) environment variable.
If you could fork the action, try this out, and submit a PR, that would really help.

Thanks for your detailed report. I'm not able to test this action on GitHub Enterprise, so it's quite possible you've discovered a genuine issue.
It looks like we'll need to initialize Oktokit with your GitHub Enterprise server URL: https://github.com/octokit/octokit.js#constructor-options. The value should be available in the [GITHUB_SERVER_URL](https://docs.github.com/en/enterprise-cloud@latest/actions/learn-github-actions/variables#default-environment-variables) environment variable.
If you could fork the action, try this out, and submit a PR, that would really help.

Hey! Thanks for the reply. I'll try to do the suggested fork approach, but it will take some time due to the security processes.

No problem. This should already be fixed on the main branch, and I plan to release a 2.8.1 with the fix shortly.

@n0rdy I've released v2.8.1 (aliased to v2) with the fix. It would be helpful if you could test and confirm that Dependency Submission now works for you on GitHub Enterprise.

I gave it a try: the error is gone and the dependency graph was submitted - thanks a lot!
The Dependabot security update failed with the same error as I described within the issue report to another dependency submission action library, but I guess it's the issue on the Dependabot end.
Thanks again for the fix!


Add sync query params support

Referencing this issue here
https://github.com/Developer-DAO/pixel-avatars/issues/27
Added Vue Router and query params sync support, they can now share links to their friends and display the pixel avatar based on query param id

We're happy to have you too! Welcome


Update gRPC Version

The gRPC Java libraries have a dependency on Netty. This PR is to update to the latest gRPC Java version to match with the version of Netty being used by the rest of Helidon.

@thegridman you might want to do this on the helidon-1.x branch as well (Netty was upgraded there as well).

@barchetta could you please have a look at the dependency version and merge if OK? Thanks!


add outputs.tf file to use as Terraform modules

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.:x: manoatsas sign nowYou have signed the CLA already but the status is still pending? Let us recheck it.


Add integration tests for UI

Added integration tests for the below scenarios:

Add client side libraries from the UI by invoking the command from project root
Add client side libraries from UI by invoking the command from a folder

        Thread.Sleep(1000); // This can be removed after bug 624281 get fixed

I don't know what that bug is, but barf :)

Refers to: test/LibraryManager.IntegrationTest/VisualStudioLibmanHostTests.cs:45 in f044332. [](commit_id = f044332e0e3197b1c72f375581689c3ee9b00530, deletion_comment = False)

        Thread.Sleep(1000); // This can be removed after bug 624281 get fixed

Hmmm. The issue is open in TFS and from the call stack and code:
https://github.com/aspnet/LibraryManager/blame/397bf9e8e522f9bfef66529164448f252b702bc7/src/LibraryManager.Vsix/Json/Completion/CompletionController.cs#L147
doesn't look like we made any changes to fix it. So I'm leaving this as is.

In reply to: 408099745 [](ancestors = 408099745)

Refers to: test/LibraryManager.IntegrationTest/VisualStudioLibmanHostTests.cs:45 in f044332. [](commit_id = f044332e0e3197b1c72f375581689c3ee9b00530, deletion_comment = False)

        Thread.Sleep(1000); // This can be removed after bug 624281 get fixed

Adding @YangLyu89, as he added this and opened the bug.

In reply to: 408151791 [](ancestors = 408151791,408099745)

Refers to: test/LibraryManager.IntegrationTest/VisualStudioLibmanHostTests.cs:45 in f044332. [](commit_id = f044332e0e3197b1c72f375581689c3ee9b00530, deletion_comment = False)

        Thread.Sleep(1000); // This can be removed after bug 624281 get fixed

We should have resolved this one now as removing this line has made tests passed. I think we can remove it now.

In reply to: 408152450 [](ancestors = 408152450,408151791,408099745)

Refers to: test/LibraryManager.IntegrationTest/VisualStudioLibmanHostTests.cs:45 in f044332. [](commit_id = f044332e0e3197b1c72f375581689c3ee9b00530, deletion_comment = False)

        Thread.Sleep(1000); // This can be removed after bug 624281 get fixed

Spoke to<EMAIL_ADDRESS>offline. He mentioned it used to fail consistently without this. Something has changed I guess. We were able to get the tests to pass without this. So removing this in iteration 2.

In reply to: 408152450 [](ancestors = 408152450,408151791,408099745)

Refers to: test/LibraryManager.IntegrationTest/VisualStudioLibmanHostTests.cs:45 in f044332. [](commit_id = f044332e0e3197b1c72f375581689c3ee9b00530, deletion_comment = False)


Open as a Menubar Application in macOS.

I think that instead of opening on the taskbar, the keybase client should open up exclusively into the menubar. I personally don't like cluttering my taskbar with lots of applications.
I may be completely missing the feature to disable it, but as of now, it seems as if that doesn't exist.

I think by "taskbar", you mean the Dock?
If so, there's already an issue open for this: here.

@HFreni As noted in the application itself, there are currently no options that can be changed in the GUI application itself.


wip: no_sig_flattening option

Was exploring https://github.com/bytecodealliance/wit-bindgen/issues/524 today, and was considering if we should disable all flattening by default as well.
This PR implements a no_sig_flattening option for the C generation as suggested.
I did explore entirely removing the flattening based on the argument that one might want a more "low level" interface, but at the end of the day even the C interface is a high level interface! And if anything, the flattening is actually slightly more optimal from a bindings perspective (while trivial, it does avoid unnecessary allocations of stack objects in lifted interfaces).
So I think having this option may well just be the best approach here.
This is just a draft initial exploration of the work, not ready to merge.

For no_sig_flattening = true, can we remove all the option flattening for both arguments and results?
I am still seeing the optional arguments being flattened even if the flag is set to true.
interface option-result {
	type o1 = option<u32>
    o1-arg: func(x: o1)
    o1-result: func() -> o1

    record empty {}
    type o2 = option<empty>
    o2-arg: func(x: o2)
    o2-result: func() -> o2
}

default world my-world {
	import imports: self.option-result
	export exports: self.option-result
}

// Imported Functions from `imports`
void imports_o1_arg(uint32_t *maybe_x);
void imports_o1_result(imports_o1_t *ret);
void imports_o2_arg(imports_empty_t *maybe_x);
void imports_o2_result(imports_o2_t *ret);

// Exported Functions from `exports`
void exports_o1_arg(uint32_t *maybe_x);
void exports_o1_result(exports_o1_t *ret);
void exports_o2_arg(exports_empty_t *maybe_x);
void exports_o2_result(exports_o2_t *ret);

Expected

I pushed a commit to fix the above issue.

All the runtime and codegen tests for go bindgen are passing for me.

@Mossaka would you be up for rebasing this and handling my comment above?

@Mossaka would you be up for rebasing this and handling my comment above?

Yes, WIP I am trying to enable all the codegen tests for go bindgen.
What comment were you referring to? I didn't see you have any comment above.

I've enable all the codegen go tests except issue 544 and issue 551. I intend to follow up with another PR to resolve these two tests. For now, I dis-abled them in the codegen tests, and thus this PR is ready to review!


Conflicts with vscode's own TS/JS intellisense

Without TabNine:

With TabNine:

with TabNine enabled (+semantic completion with flow), vscode only give suggestions from TabNine. but I can manually trigger a list containing all suggestions with Control+space
Expected behavior:
The completion list should contain all suggestions from TabNine and other sources when typing on-the-fly.

Same thing is happening with me when editing Kotlin files using the Kotlin extension (it has other issues so I suspected it); also experiencing the same Ctrl+Space behavior, if I catch it before TabNine shows its suggestions. IIRC with other languages, including TypeScript, TabNine was not previously broken for me.

I'm having the exact same issue

This one issue alone makes the program unusable. Other than this it's fantastic, but this is a deal-breaker.

I've found that when you keep deleting and re-typing part of the word (can just be a letter), the intellisense suggestions suddenly appear on top of the tabnine ones again. Weird.

I wrote a plugin that basically solves this problem  https://marketplace.visualstudio.com/items?itemName=llej.suggest-plus

https://github.com/zxqfl/tabnine-vscode/commit/03d2f57c558d441bbac5b635ad97699be9efb2ea tries to fix this issue, please update to tabnine-vscode version 2.1.16 and reply to let me know whether it worked.

03d2f57 tries to fix this issue, please update to tabnine-vscode version 2.1.16 and reply to let me know whether it worked.

it works like a charm! thanks for the long-awaited fix😀

@zxqfl  This is still an issue. That PR didn't fix the issue.

tries to fix this issue, please update to tabnine-vscode version 2.1.16 and reply to let me know whether it worked.
It didn't :(

I wrote a plugin that basically solves this problem https://marketplace.visualstudio.com/items?itemName=llej.suggest-plus
This works for me

Hi, any update on this issue? I'm using v3.2.17 and still only getting tabnine suggestions and not Typescript suggestions for example

This should be reopened, it blocks usage.

I think we need a new issue.


Refactored Litmus Book - Changed the kubectl command used in shell module to k8s , k8s_facts module

What this PR does / why we need it:
Which issue this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged): fixes #
Checklist

[x] Does this PR have a corresponding GitHub issue?
[ ] Have you included relevant README for the chaoslib/experiment with details?
[x] Have you added debug messages where necessary?
[ ] Have you added task comments where necessary?
[x] Have you tested the changes for possible failure conditions?
[x] Have you provided the positive & negative test logs for the litmusbook execution?
[ ] Does the litmusbook ensure idempotency of cluster state?, i.e., is cluster restored to original state?
[x] Have you used non-shell/command modules for Kubernetes tasks?
[ ] Have you (jinja) templatized custom scripts that is run by the litmusbook, if any?
[ ] Have you (jinja) templatized Kubernetes deployment manifests used by the litmusbook, if any?
[ ] Have you reused/created util functions instead of repeating tasks in the litmusbook?
[ ] Do the artifacts follow the appropriate directory structure?
[ ] Have you isolated storage (eg: OpenEBS) specific implementations, checks?
[ ] Have you isolated platform (eg: baremetal kubeadm/openshift/aws/gcloud) specific implementations, checks?
[ ] Are the ansible facts well defined? Is the scope explicitly set for playbook & included utils?
[x] Have you ensured minimum/careful usage of shell utilities (awk, grep, sed, cut, xargs etc.,)?
[ ] Can the limtusbook be executed both from within & outside a container (configurable paths, no hardcode)?
[ ] Can you suggest the minimal resource requirements for the litmusbook execution?
[ ] Does the litmusbook job artifact carry comments/default options/range for the ENV tunables?
[ ] Has the litmusbooks been linted?

Special notes for your reviewer:
CHANGES:
Changed the kubectl command used in shell module to k8s,k8s_facts module in the following utils.
/utils/fcm/update_litmus_result_resource.yml
/utils/k8s/pre_create_app_deploy.yml
/utils/k8s/create_ns.yml
/utils/k8s/deploy_single_app.yml
/utils/scm/openebs/check_replica_count.yml
/utils/k8s/fetch_app_pod.yml
/utils/scm/applications/mysql/check_db_connection.yml
/utils/scm/openebs/target_affinity_check.yml
/utils/k8s/deprovision_deployment.yml
/utils/k8s/status_testns.yml
TESTS:
After changing, following test were performed:
i.) In Percona  - Provision [deployment]
ii.) In Percona  - Deprovision [deployment]
iii.) In Busybox  - Provision [statefulset]
iv.) In Busybox - Deprovision [statefulset]
Test_logs.txt
The logs of all the 4 tests has been uploaded please review in case you need it .
TO DO:

In util - k8s/deprovision_deployment.yml
Task name - Delete the application deployment
is left in shell module as by using k8s module the pvc get struct in terminating state

The util  status_app_pod.yml is left in shell module because of this issue - https://github.com/litmuschaos/litmus/issues/734

Hi @uditgaurav
Are you working on changing the remaining files as well?
I've started working on changing chaoslib/litmus/pod_failure_by_litmus.yml before I saw this PR. If you are not already working on that, I will complete it and create a separate PR with the changes.


Premerge test with combined jar

Is your feature request related to a problem? Please describe.
Currently the premerge just uses the individual profile to run the tests, it would be nice to use the combined jar -Psnapshots to make sure no issues with multiple jars combined.  The problem is we run jacoco and it doesn't seem to like the combined jar.
10:12:54  java.lang.IllegalStateException: Can't add different class with same name: com/nvidia/spark/rapids/AvoidAdaptiveTransitionToRow
10:12:54  	at org.jacoco.core.analysis.CoverageBuilder.visitCoverage(CoverageBuilder.java:106)
10:12:54  	at org.jacoco.core.analysis.Analyzer$1.visitEnd(Analyzer.java:99)
10:12:54  	at org.objectweb.asm.ClassVisitor.visitEnd(ClassVisitor.java:377)
10:12:54  	at org.jacoco.core.internal.flow.ClassProbesAdapter.visitEnd(ClassProbesAdapter.java:100)
10:12:54  	at org.objectweb.asm.ClassReader.accept(ClassReader.java:725)
10:12:54  	at org.objectweb.asm.ClassReader.accept(ClassReader.java:401)
10:12:54  	at org.jacoco.core.analysis.Analyzer.analyzeClass(Analyzer.java:116)
10:12:54  	at org.jacoco.core.analysis.Analyzer.analyzeClass(Analyzer.java:132)
10:12:54  Caused: java.io.IOException: Error while analyzing /var/jenkins/jobs/rapids_premerge-g

Investigate solutions or perhaps run jacoco differently.

@sameerz One option to fix this is to just remove all of the JaCoCo code coverage. No one really looks at the numbers and enough has changed with the build, 3.2 and splitting the tests into separate builds, the numbers no longer really show that much of a full picture of what is happening.  Should we spend more time to make this useful? Or do we just remove it all together?

Short term we can disable JaCoCo.  Mid term we should re-investigate how we measure code coverage.


How to generate the annotation file?

Hi, this work is pretty cool !
I come across several problems when I use the repo.

What's the difference between ava_train_v2.2.pkl & ava_train_v2.2_with_fair_0.9.pkl ? What's the meaning of "with_fair_0.9" or "with_fair_0.85"? I print some data of these two pkl file mentioned above, and found that the one with "with_fair_0.9" got more bbox but the corresponding person_idx value is -1.

How can I generate the pkl file from original ava csv format?

How can I test arbitrary video input? It seems it can only support the ava val dataset which already got the bounding box value.

Thanks!

Hello, have you sloved this problem?

Hello, have you sloved this problem?


Problem rebuilding styles.css.

Please excuse this potentially naive question. I am trying to regenerate style.css from the sccs files and I am getting nothing but grief.
u153:src paulmacp$ ls
components	core		styles.scss
u153:src paulmacp$ sass --scss styles.scss styles.css
Error: File to import not found or unreadable: compass/css3.

No problem! :D


Support color icons

As of right now, all icons inherit their color from the current context, but there are some icon packs with colored icons, or even icons with multiple colors.  Investigate supporting those icons and design an API for enabling / disabling colors.

That would be a very useful feature! Also, thanks for all the work you're putting into styled-icons. v7 was a great update!


Launching ur10e with humble in simulation mode

I was trying to get a simple simulation running with the universal robot ur10e, but it took many hours to get working due to misleading/wrong documentation.
Firstly, to start the driver, I ran
ros2 launch ur_robot_driver ur_control.launch.py ur_type:=ur10e robot_ip:=yyy.yyy.yyy.yyy use_fake_hardware:=true launch_rviz:=true

but when I checked active controllers, I see that scaled_joint_trajectory_controller is active which doesn't work in simulation:
ros2 control list_controllers
forward_position_controller[position_controllers/JointGroupPositionController] inactive
io_and_status_controller[ur_controllers/GPIOController] active
scaled_joint_trajectory_controller[ur_controllers/ScaledJointTrajectoryController] active
speed_scaling_state_broadcaster[ur_controllers/SpeedScalingStateBroadcaster] active
joint_state_broadcaster[joint_state_broadcaster/JointStateBroadcaster] active
force_torque_sensor_broadcaster[force_torque_sensor_broadcaster/ForceTorqueSensorBroadcaster] active

The use_fake_hardware argument is supposed to change the default controller from scaled_joint_trajectory_controller to joint_trajectory_controller, but for some reason it doesn't work.
My workaround solution for this is to add 'initial_joint_controller:=joint_trajectory_controller' to the arguments. That works to activate the correct controller for simulation:
ros2 control list_controllers
joint_trajectory_controller[joint_trajectory_controller/JointTrajectoryController] active
joint_state_broadcaster[joint_state_broadcaster/JointStateBroadcaster] active
force_torque_sensor_broadcaster[force_torque_sensor_broadcaster/ForceTorqueSensorBroadcaster] active
speed_scaling_state_broadcaster[ur_controllers/SpeedScalingStateBroadcaster] active
io_and_status_controller[ur_controllers/GPIOController] active
forward_position_controller[position_controllers/JointGroupPositionController] inactive

Next, the moveit node needs be executed. The only documentation about that one is to run this line:
ros2 launch ur_moveit_config ur_moveit.launch.py ur_type:=ur10e launch_rviz:=true

However, that doesn't work for simulation. I checked the source code and saw that it also includes use_fake_hardware. So, I thought I could just add that to the arguments and that would allow me to run it in simulation. However, that only gave me a new error:
Action client not connected to action server: scaled_joint_trajectory_controller/follow_joint_trajectory

After a lot of debugging, I finally noticed another argument use_sim_time that needs to be manually set to true, when using simulation. Finally, when adding that to my arguments, I could plan and execute plans.
So to summarize. The switch of controllers in ur_robot_driver, when using fake hardware, needs to be fixed. Also, better documentation for ur_moveit_config is needed. I would also think that the 'use_fake_hardware' should activate 'use_sim_time' automatically, unless there is a situation that I'm missing where you need them to be separate.

This sounds like it should be resolved by #480. If you build from the current main branch, does that still persist?

Yes, I tried both building from source and binary build and the problem remains in both.

Checking this again, I find the error. As you have correctly said, currently it is needed to specify  initial_joint_controller:=joint_trajectory_controller when using fake_hardware. However, MoveIt! does need to get the information about fake_hardware, as well, in order to connect to the correct controller (The change mentioned in #480).
So, the following two launches should get you running:
ros2 launch ur_robot_driver ur_control.launch.py ur_type:=ur10e robot_ip:=yyy.yyy.yyy.yyy use_fake_hardware:=true launch_rviz:=true initial_joint_controller:=joint_trajectory_controller

ros2 launch ur_moveit_config ur_moveit.launch.py ur_type:=ur10e launch_rviz:=true use_fake_hardware:=true

You are correct, the docs do not reflect that correctly at the moment. That's definitely something for #494, although this will change in the near future again, as soon as https://github.com/ros-controls/ros2_control/pull/822 gets merged.

ros2 launch ur_moveit_config ur_moveit.launch.py ur_type:=ur10e launch_rviz:=true use_fake_hardware:=true

If I'm not mistaken, this line needs to be:
ros2 launch ur_moveit_config ur_moveit.launch.py ur_type:=ur10e launch_rviz:=true use_fake_hardware:=true use_sim_time:=true

no, use_sime_time is not really connected to that, as this is not a real simulation such as gazebo, but (as the name suggests) a fake hardware. It acts like a real hardware interface at the communication level, but it is not a full robot simulation.

hmm, weird then that it didn't work for me until I added use_sim_time:=true

I noticed that, if I copy the ur_moveit.launch.py exactly as it is on github to my own package and run it from there, then I don't need to have use_sim_time:=true for it to execute a plan. Running it from my binary install still fails in execution without use_sim_time:=true

The necessary changes will come with the next sync. I don't quite understand why the use_sim_time parameter should make things work without those changes, but since it is working with the recent version, I'm going ahead and close this.


New Command Ideas

Hello! I have thought of some new command ideas! They will be in a list:

Some Bot Owner Only Commands

- Utility Commands

This is too generic.

Ok


The L2-projection operator in projected replicator dynamics

Please see issue 308.

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).
📝 Please visit https://cla.developers.google.com/ to sign.
Once you've signed (or fixed any issues), please reply here with @googlebot I signed it! and we'll verify it.
What to do if you already signed the CLA
Individual signers

It's possible we don't have your GitHub username or you're using a different email address on your commit. Check your existing CLA data and verify that your email is set on your git commits.

Corporate signers

Your company has a Point of Contact who decides which employees are authorized to participate. Ask your POC to be added to the group of authorized contributors. If you don't know who your Point of Contact is, direct the Google project maintainer to go/cla#troubleshoot (Public version).
The email used to register you as an authorized contributor must be the email used for the Git commit. Check your existing CLA data and verify that your email is set on your git commits.
The email used to register you as an authorized contributor must also be attached to your GitHub account.

ℹ Googlers: Go here for more info.

@googlebot I signed it

Thanks @rezunli96 for this contribution! I outlined a few suggestions below.

@dhennes Hi thanks for your helpful suggestions! I have modified the code based on these and please take a look.


Getting 401 error after following docs

This was very well written and I've followed all the steps on this page, but, I am getting a 401 unauthorized error even though I'm authenticating with an Oauth2.0 token.
Steps I took:

Gave my app the SecurityRecommendation.Read.All permission
Generated Bearer token via Oauth2.0
Tested this endpoint in postman with Bearer token and received the 401 error.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 213cc23c-1986-b6c3-80c6-beb4eed99b6c
Version Independent ID: 7433acd8-95d5-06ca-15c2-2899ad1d2ed5
Content: List all recommendations - Windows security
Content Source: windows/security/threat-protection/microsoft-defender-atp/get-all-recommendations.md
Product: w10
Technology: windows
GitHub Login: @DulceMontemayor
Microsoft Alias: dolmont

I have figured out the problem... pebkac
My scope was wrong. For anyone encountering the same issue here is the scope I used:
https://api.securitycenter.windows.com/.default


No support for long project names with GitLab integration?

I'm trying to add an integration with GitLab and our group is managed in a way so that we have a clear understanding of what the repo is related to. It looks something like this:
(company)slug
  teamnameA
    clientX
      projectX
      projectY
    clientY

  teamnameB
    toolX
    toolY
    clientX
  teamname C

Now when I try to add an integration to platform for Project X for client X in team A, I use the following command:
platform integration:add --type=gitlab --token=[MY-GITLAB-TOKEN] --base-url=https://gitlab.com/ --gitlab-project=slug/teamA/clientX/projectX
The CLI now throws an InvalidValueException:

Invalid value for 'Project': slug/teamA/clientX/projectX

Is there a work around for this?

Fixed in version 3.32.2

Wow. That's quick 👍
Thanks a lot!


Makefile: add update- for verify-

i.e. update-license for verify-license

I'd like to work on this issue, please assigned to me. :)

Thanks. Assigned.

@lingsamuel Could you describe the issue deliberately?

@mangoGoForward We just need to add some update- action.
Currently we have verify-license but adding the license header still needs to be done manually. This issue hopes to add an update-license action to automatically complete the addition of the license header.

Got it, thanks.

We may also need a unified formatting tool (just a suggestion).
Example script:
#!/bin/bash

set -e

go install github.com/incu6us/goimports-reviser/v2@latest

PROJECT_NAME=github.com/apache/apisix-ingress-controller
while IFS= read -r -d '' file; do
  goimports-reviser  -file-path "$file" -project-name $PROJECT_NAME
done <   <(find . -name '*.go' -not -path "./test/*" -print0)

PROJECT_NAME=github.com/apache/apisix-ingress-controller/test/e2e
while IFS= read -r -d '' file; do
  goimports-reviser  -file-path "$file" -project-name $PROJECT_NAME
done <   <(find . -name '*.go' -path "./test/*" -print0)

We may also need a unified formatting tool (just a suggestion). Example script:
#!/bin/bash

set -e

go install github.com/incu6us/goimports-reviser/v2@latest

PROJECT_NAME=github.com/apache/apisix-ingress-controller
while IFS= read -r -d '' file; do
  goimports-reviser  -file-path "$file" -project-name $PROJECT_NAME
done <   <(find . -name '*.go' -not -path "./test/*" -print0)

PROJECT_NAME=github.com/apache/apisix-ingress-controller/test/e2e
while IFS= read -r -d '' file; do
  goimports-reviser  -file-path "$file" -project-name $PROJECT_NAME
done <   <(find . -name '*.go' -path "./test/*" -print0)

How about gofmt or goimports to sort import packages?

goimports-reviser can sort project packages as a separate group, e.g.:
package testdata

import (
	"bytes"
	"log"

	"github.com/abcabc/abcabc"
	"github.com/pkg/errors"

	"github.com/incu6us/goimports-reviser/testdata/innderpkg"
)

instead of

package testdata

import (
	"bytes"
	"log"

	"github.com/abcabc/abcabc"
	"github.com/incu6us/goimports-reviser/testdata/innderpkg"
	"github.com/pkg/errors"
)

IIRC gofmt or goimports can't do this.

Of course we can. Just create it and cross reference this issue.


Use of Context.SaveChanges() in Create() method of EF Repository

Related to Create(T entity) method within EF repository to what extend is useful to use Context.SaveChanges() within Create method?
I mean, I understand you use it for having available for next queries the entitity you just added. But it is an expesive price for having this commodity. In your opinion is this commodity really a need?
I mean in Orchard 1.8 the current code that consumes Repository really harness this commodity?
How much code? Can you point me an example? Will it be difficult to change that code that adds and consumes data to delegate on it the decission of when to call context.SaveChanges()?

Yes it is an expensive commodity - it's one of the things I really dislike about EF - there is a lot I like, but this one area does bug me. I think it should be added in by settings, like "AutoFlush" : true within a config file.

What exactly you mean with the Autoflush feature? What do you expect it does? When would it save data? With this  autoflush features you would expect that when you query the context any created data but not saved in db would be returned because if it still is not saved previously EF will save it? Is this what you mean?
Enviado desde mi Windows Phone

De: Nicholas<EMAIL_ADDRESS>Enviado: 31/01/2015 11:31
Para<EMAIL_ADDRESS>CC<EMAIL_ADDRESS>Asunto: Re: [Brochard] Use of Context.SaveChanges() in Create() method of EF Repository (#10)
Yes it is an expensive commodity - it's one of the things I really dislike about EF - there is a lot I like, but this one area does bug me. I think it should be added in by settings, like "AutoFlush" : true within a config file.

Reply to this email directly or view it on GitHub:
https://github.com/OrchardCMS/Brochard/issues/10#issuecomment-72312552

The idea of AutoFlush would be that if you add/update an object, a call to SaveChanges is made.
Or alternatively, if you add/update an object we set a isdirty flag, then when making a query that involves that object, we call save changes and then the query will return everything.
Not sure...

I see, I think both alternatives are expensive.
The point is why do you need that? Do you have many code where you need to mix in memory data pending of being added with data in the db? Can you point me code in Orchard that has this need. I don't know too many of Orchard but I suspect we are worried for something we don't have any need.

Removed Repository pattern.


Remove cdscan for e3sm_diags

Resolves #346. Resolves #80. Follow-up to #519.

@tomvothecoder Re: https://github.com/E3SM-Project/zppy/pull/519#issuecomment-2127712530 -- this PR includes the changes that couldn't be included in #519 since the e3sm_diags refactor isn't finished yet. My initial testing suggests this PR's implementation will be sufficient to fully remove CDAT dependencies from zppy. But we'll have to wait to merge it until after the e3sm_diags refactor is done.

@tomvothecoder I created a new "min case" cfg for testing sets that have already been migrated. They appear to generate alright using E3SM Diags built off the latest CDAT branch: https://web.lcrc.anl.gov/public/e3sm/diagnostic_output/ac.forsyth2/zppy_min_case_e3sm_diags_cdat_migrated_www/issue-346-diags-20240802/v3.LR.historical_0051/e3sm_diags/

I ran zppy -c tests/integration/generated/test_min_case_e3sm_diags_cdat_migrated_sets_chrysalis.cfg.

 The cfg
[default]
case = "v3.LR.historical_0051"
constraint = ""
dry_run = "False"
environment_commands = ""
input = /lcrc/group/e3sm2/ac.wlin/E3SMv3/v3.LR.historical_0051
input_subdir = archive/atm/hist
mapping_file = "map_ne30pg2_to_cmip6_180x360_aave.20200201.nc"
output = "/lcrc/group/e3sm/ac.forsyth2/zppy_min_case_e3sm_diags_cdat_migrated_output/test-diags-no-cdat-20240917/v3.LR.historical_0051"
partition = "debug"
qos = "regular"
www = "/lcrc/group/e3sm/public_html/diagnostic_output/ac.forsyth2/zppy_min_case_e3sm_diags_cdat_migrated_www/test-diags-no-cdat-20240917"
years = "1985:1989:2",

[climo]
active = True
walltime = "00:30:00"

  [[ atm_monthly_180x360_aave ]]
  frequency = "monthly"
  input_files = "eam.h0"
  input_subdir = "archive/atm/hist"
  vars = ""

  [[ atm_monthly_diurnal_8xdaily_180x360_aave ]]
  frequency = "diurnal_8xdaily"
  input_files = "eam.h3"
  input_subdir = "archive/atm/hist"
  vars = "PRECT"

  [[ land_monthly_climo ]]
  frequency = "monthly"
  input_files = "elm.h0"
  input_subdir = "archive/lnd/hist"
  mapping_file = "map_r05_to_cmip6_180x360_aave.20231110.nc"
  vars = ""

[ts]
active = True
e3sm_to_cmip_environment_commands = ""
walltime = "00:30:00"

  [[ atm_monthly_180x360_aave ]]
  frequency = "monthly"
  input_files = "eam.h0"
  input_subdir = "archive/atm/hist"
  ts_fmt = "cmip"

  [[ rof_monthly ]]
  extra_vars = 'areatotal2'
  frequency = "monthly"
  input_files = "mosart.h0"
  input_subdir = "archive/rof/hist"
  mapping_file = ""
  vars = "RIVER_DISCHARGE_OVER_LAND_LIQ"

  [[ land_monthly ]]
  extra_vars = "landfrac"
  frequency = "monthly"
  input_files = "elm.h0"
  input_subdir = "archive/lnd/hist"
  mapping_file = "map_r05_to_cmip6_180x360_aave.20231110.nc"
  ts_fmt = "cmip"
  vars = "FSH,RH2M,LAISHA,LAISUN,QINTR,QOVER,QRUNOFF,QSOIL,QVEGE,QVEGT,SOILICE,SOILLIQ,SOILWATER_10CM,TSA,TSOI,H2OSNO,TOTLITC,CWDC,SOIL1C,SOIL2C,SOIL3C,SOIL4C,WOOD_HARVESTC,TOTVEGC,NBP,GPP,AR,HR"

[tc_analysis]
active = True
scratch = "/lcrc/globalscratch/ac.forsyth2/zppy_weekly_comprehensive_v3_scratch/test-diags-no-cdat-20240917/v3.LR.historical_0051"
walltime = "00:30:00"

[e3sm_diags]
active = True
environment_commands = "source /home/ac.forsyth2/miniconda3/etc/profile.d/conda.sh; conda activate e3sm_diags_cdat_branch_20240916"
grid = '180x360_aave'
multiprocessing = True
num_workers = 8
partition = "compute"
qos = "regular"
ref_final_yr = 1986
ref_start_yr = 1985
ref_years = "1985-1986",
# Also lat_lon_land for mvm
#sets="lat_lon","lat_lon_river","cosp_histogram","zonal_mean_xy","zonal_mean_2d","zonal_mean_2d_stratosphere","aerosol_aeronet","polar","meridional_mean_2d","area_mean_time_series","aerosol_budget","annual_cycle_zonal_mean","diurnal_cycle","tc_analysis","enso_diags", "streamflow"
sets="lat_lon","tc_analysis","enso_diags","streamflow",
short_name = "v3.LR.historical_0051"
ts_num_years = 2
walltime = "5:00:00"
years = "1987:1989:2"

  [[ atm_monthly_180x360_aave ]]
  climo_diurnal_frequency = "diurnal_8xdaily"
  climo_diurnal_subsection = "atm_monthly_diurnal_8xdaily_180x360_aave"
  climo_subsection = "atm_monthly_180x360_aave"
  dc_obs_climo = '/lcrc/group/e3sm/public_html/e3sm_diags_test_data/unit_test_complete_run/obs/climatology'

  [[ atm_monthly_180x360_aave_mvm ]]
  # Test model-vs-model using the same files as the reference
  climo_diurnal_frequency = "diurnal_8xdaily"
  climo_diurnal_subsection = "atm_monthly_diurnal_8xdaily_180x360_aave"
  climo_subsection = "atm_monthly_180x360_aave"
  diff_title = "Difference"
  partition = "compute"
  qos = "regular"
  ref_name = "v3.LR.historical_0051"
  reference_data_path = "/lcrc/group/e3sm/ac.forsyth2/zppy_min_case_e3sm_diags_cdat_migrated_output/test-diags-no-cdat-20240917/v3.LR.historical_0051/post/atm/180x360_aave/clim"
  run_type = "model_vs_model"
  short_ref_name = "same simulation"
  swap_test_ref = False
  tag = "model_vs_model"
  ts_num_years_ref = 2
  ts_subsection = "atm_monthly_180x360_aave"

#   [[ lnd_monthly_mvm_lnd ]]
#  # Test model-vs-model using the same files as the reference
#   climo_subsection = "land_monthly_climo"
#   diff_title = "Difference"
#   partition = "compute"
#   qos = "regular"
#   ref_name = "v3.LR.historical_0051"
#   reference_data_path = "/lcrc/group/e3sm/ac.forsyth2/zppy_min_case_e3sm_diags_cdat_migrated_output/test-diags-no-cdat-20240917/v3.LR.historical_0051/post/lnd/180x360_aave/clim"
#   run_type = "model_vs_model"
#   sets = "lat_lon_land",
#   short_ref_name = "same simulation"
#   swap_test_ref = False
#   tag = "model_vs_model"
#   ts_num_years_ref = 2

I specifically set:
sets="lat_lon","tc_analysis","enso_diags","streamflow",

to test the latest sets added on https://github.com/E3SM-Project/e3sm_diags/commits/cdat-migration-fy24 (apparently missing qbo as a recent set). [Note I still included lat_lon as a set since I recall E3SM Diags sometimes running into problems if that set isn't run first.]
These three sets unfortunately ran into some errors. https://web.lcrc.anl.gov/public/e3sm/diagnostic_output/ac.forsyth2/zppy_min_case_e3sm_diags_cdat_migrated_www/test-diags-no-cdat-20240917/v3.LR.historical_0051/e3sm_diags/atm_monthly_180x360_aave/model_vs_obs_1987-1988/viewer/ only shows lat_lon. Now, by #625, apparently tc_analysis wasn't even showing up correctly on main. But that still means enso_diags and streamflow ran into some issues on the migrated branch.
$ cd /lcrc/group/e3sm/ac.forsyth2/zppy_min_case_e3sm_diags_cdat_migrated_output/test-diags-no-cdat-20240917/v3.LR.historical_0051/post/scripts

$ grep -v "OK" *status
# No outright errors

$ grep -in tc_analysis e3sm_diags_atm_monthly_180x360_aave_model_vs_obs_1987-1988.o585928
$ grep -in enso_diags e3sm_diags_atm_monthly_180x360_aave_model_vs_obs_1987-1988.o585928
$ grep -in streamflow e3sm_diags_atm_monthly_180x360_aave_model_vs_obs_1987-1988.o585928

tc_analysis:
IndexError: list index out of range

This error matches the one on main, as noted in #625
enso_diags:
OSError: No files found for target variable TAUX or derived variables ([('tauu',), ('surf_mom_flux_\
U',)]) in ts.
OSError: No files found for target variable LHFLX or derived variables ([('hfls',), ('QFLX',), ('su\
rface_upward_latent_heat_flux',)]) in ts.
OSError: No files found for target variable SHFLX or derived variables ([('hfss',), ('surf_sens_flu\
x',)]) in ts
OSError: No files found for target variable LHFLX or derived variables ([('hfls',), ('QFLX',), ('su\
rface_upward_latent_heat_flux',)]) in ts.
OSError: No files found for target variable SHFLX or derived variables ([('hfss',), ('surf_sens_flu\
x',)]) in ts.
OSError: No files found for target variable NET_FLUX_SRF or derived variables ([('FSNS', 'FLNS', 'Q\
FLX', 'PRECC', 'PRECL', 'PRECSC', 'PRECSL', 'SHFLX'), ('FSNS', 'FLNS', 'LHFLX', 'SHFLX'), ('FSNS', \
'FLNS', 'QFLX', 'SHFLX'), ('rsds', 'rsus', 'rlds', 'rlus', 'hfls', 'hfss')]) in ts.
OSError: No files found for target variable PRECT or derived variables ([('PRECT',), ('pr',), ('PRE\
CC', 'PRECL'), ('sat_gauge_precip',), ('PrecipLiqSurfMassFlux', 'PrecipIceSurfMassFlux')]) in ts.
OSError: No files found for target variable TAUX or derived variables ([('tauu',), ('surf_mom_flux_\
U',)]) in ts.
OSError: No files found for target variable TAUY or derived variables ([('tauv',), ('surf_mom_flux_\
V',)]) in ts.
OSError: No files found for target variable FSNS or derived variables ([('sfc_net_sw_all_mon',), ('\
rsds', 'rsus')]) in ts.
OSError: No files found for target variable NET_FLUX_SRF or derived variables ([('FSNS', 'FLNS', 'Q\
FLX', 'PRECC', 'PRECL', 'PRECSC', 'PRECSL', 'SHFLX'), ('FSNS', 'FLNS', 'LHFLX', 'SHFLX'), ('FSNS', \
'FLNS', 'QFLX', 'SHFLX'), ('rsds', 'rsus', 'rlds', 'rlus', 'hfls', 'hfss')]) in ts.

streamflow:
OSError: No time series `.nc` file was found for 'RIVER_DISCHARGE_OVER_LAND_LIQ' in 'rof'

The enso_diags and streamflow errors seem to suggest missing data, but this confuses me because the comprehensive_v3 test uses the same simulation output as its data source...

@chengzhuzhang @tomvothecoder
https://web.lcrc.anl.gov/public/e3sm/diagnostic_output/ac.forsyth2/zppy_min_case_e3sm_diags_cdat_migrated_www/test-diags-no-cdat-20240917/v3.LR.historical_0051/e3sm_diags/atm_monthly_180x360_aave/model_vs_obs_1987-1988/viewer/ shows only lat_lon despite tests/integration/template_min_case_e3sm_diags_cdat_migrated_sets.cfg having sets="lat_lon","tc_analysis","enso_diags","streamflow",.
The relevant output directory can be seen here, and is also mentioned in a code block above.
$ cd /lcrc/group/e3sm/ac.forsyth2/zppy_min_case_e3sm_diags_cdat_migrated_output/test-diags-no-cdat-20240917/v3.LR.historical_0051/post/scripts

$ grep -v "OK" *status
# No obvious failures

# Possibly useful commands:
$ grep -in error e3sm_diags_atm_monthly_180x360_aave_model_vs_obs_1987-1988.o585928
$ grep -in tc_analysis e3sm_diags_atm_monthly_180x360_aave_model_vs_obs_1987-1988.o585928
$ grep -in enso_diags e3sm_diags_atm_monthly_180x360_aave_model_vs_obs_1987-1988.o585928
$ grep -in streamflow e3sm_diags_atm_monthly_180x360_aave_model_vs_obs_1987-1988.o585928

Note https://web.lcrc.anl.gov/public/e3sm/diagnostic_output/ac.forsyth2/zppy_min_case_e3sm_diags_cdat_migrated_www/test-diags-no-cdat-20240917/v3.LR.historical_0051/e3sm_diags/atm_monthly_180x360_aave_mvm/model_vs_model_1987-1988/viewer/ also shows only lat_lon in the final viewer.

@forsyth2 I'm looking at the log files (as below), it appears that time series files are not correctly feed into e3sm_diags, which means the code change that removes cdscan step is not working with e3sm_diags correctly.
>cp: cannot create regular file 'ts/FSNTOA_1987*.nc': No such file or directory
cp: cannot create regular file 'ts/FLUT_1987*.nc': No such file or directory
cp: cannot create regular file 'ts/FSNT_1987*.nc': No such file or directory
cp: cannot create regular file 'ts/FLNT_1987*.nc': No such file or directory
cp: cannot create regular file 'ts/FSNS_1987*.nc': No such file or directory
cp: cannot create regular file 'ts/FLNS_1987*.nc': No such file or directory
cp: cannot create regular file 'ts/SHFLX_1987*.nc': No such file or directory
cp: cannot create regular file 'ts/QFLX_1987*.nc': No such file or directory
cp: cannot create regular file 'ts/TAUX_1987*.nc': No such file or directory
cp: cannot create regular file 'ts/TAUY_1987*.nc': No such file or directory
cp: cannot create regular file 'ts/PRECC_1987*.nc': No such file or directory
cp: cannot create regular file 'ts/PRECL_1987*.nc': No such file or directory
cp: cannot create regular file 'ts/PRECSC_1987*.nc': No such file or directory
cp: cannot create regular file 'ts/PRECSL_1987*.nc': No such file or directory
cp: cannot create regular file 'ts/TS_1987*.nc': No such file or directory
cp: cannot create regular file 'ts/TREFHT_1987*.nc': No such file or directory
cp: cannot create regular file 'ts/CLDTOT_1987*.nc': No such file or directory
cp: cannot create regular file 'ts/CLDHGH_1987*.nc': No such file or directory
cp: cannot create regular file 'ts/CLDMED_1987*.nc': No such file or directory
cp: cannot create regular file 'ts/CLDLOW_1987*.nc': No such file or directory
cp: cannot create regular file 'ts/U_1987*.nc': No such file or directory
cp: target 'RIVER_DISCHARGE_OVER_LAND_LIQ_*.nc' is not a directory

Thanks @chengzhuzhang. I haven't had problems with previous added sets though. I'll try to debug this further to see if it's something with how this PR handles these sets in particular...

@tomvothecoder we will need to coordinate on this issue with e3sm_diags refactoring.
The original cdscan codes basically subset and extract the metadata from input file for the specified time period and generate an xml file with file name as T_startTime_endTime.xml which cdms2 is able to read and process.
In the new e3sm_diags code, for time-series, we use xarray.open_dataset, and the input needs to be one single .nc file. I think we may need to update to xarray.open_mfdataset which allows multiple files, and also to support subset time based on start and end years (i think the later is already supported by https://github.com/E3SM-Project/e3sm_diags/blob/7b53932d72ef7782818a5bd1e4e9d527420fec49/e3sm_diags/driver/utils/dataset_xr.py#L1185 ?).

@tomvothecoder I created a new "min case" cfg for testing sets that have already been migrated. They appear to generate alright using E3SM Diags built off the latest CDAT branch: https://web.lcrc.anl.gov/public/e3sm/diagnostic_output/ac.forsyth2/zppy_min_case_e3sm_diags_cdat_migrated_www/issue-346-diags-20240802/v3.LR.historical_0051/e3sm_diags/

Thanks @chengzhuzhang. I haven't had problems with previous added sets though. I'll try to debug this further to see if it's something with how this PR handles these sets in particular...

Could you clarify "the previous added sets", i think the issue only relevant for time-series based diagnostics sets. For sets depends on climo datasets, they should be fine.

@forsyth2 I saw the test results from Aug 6, yes by then those sets only rely on climo files.

@chengzhuzhang Does this mean the bug is for certain in the Diags code or should I dive deeper into the zppy changes in this PR?

@chengzhuzhang Does this mean the bug is for certain in the Diags code or should I dive deeper into the zppy changes in this PR?

It is not a bug in diags. We can also add the time selecting and slicing code in zppy to be able to remove cdscan. We should have a discussion with @tomvothecoder on how best to approach this.

@chengzhuzhang Does this mean the bug is for certain in the Diags code or should I dive deeper into the zppy changes in this PR?

It is not a bug in diags.

To summarize this issue, based on what I'm reading:

lat_lon works fine because the refactored e3sm_diags uses xc.open_mfdataset() for climatology datasets (link), which supports directories or globs for .nc files (like how zppy specifies here)
tc_analysis , enso_diags, and streamflow fail because they use time series datasets (via a directory). The refactored e3sm_diags uses xc.open_dataset() for time series files (link).

I will open a PR to update e3sm_diags to use xc.open_mfdataset() for time series files (here).

also to support subset time based on start and end years (i think the later is already supported by<EMAIL_ADDRESS>?).
We can also add the time selecting and slicing code in zppy to be able to remove cdscan. We should have a discussion with @tomvothecoder on how best to approach this.

The time slicing and subsetting code you listed should work. No need to add this in zppy.

@forsyth2 Can you provide the input paths for the test data? I am trying to reproduce your errors by running prov/e3sm.py but the test_ts is a local reference.

enso_diags and streamflow fail because they use time series datasets (via a directory). The refactored e3sm_diags uses xc.open_dataset() for time series files ([link](https://github.com/E3SM-Project/e3sm_diags/blob/22cd0f569e7df4be147a8fe6fae3e00d5aeb9d8c/e3sm_diags/driver/utils/dataset_xr.py#L1035-

Note, opening multiple time series files is actually supported already if deriving from other variables.

@tomvothecoder

Can you provide the input paths for the test data?

Everything is specified in the cfg I have copied in https://github.com/E3SM-Project/zppy/pull/598#issuecomment-2384025994. Specifically:
input = /lcrc/group/e3sm2/ac.wlin/E3SMv3/v3.LR.historical_0051
output = "/lcrc/group/e3sm/ac.forsyth2/zppy_min_case_e3sm_diags_cdat_migrated_output/test-diags-no-cdat-20240917/v3.LR.historical_0051"

For the test_ts directory specifically, I believe that would be copied from:
/lcrc/group/e3sm/ac.forsyth2/zppy_min_case_e3sm_diags_cdat_migrated_output/test-diags-no-cdat-20240917/v3.LR.historical_0051/post/atm/180x360_aave/ts/monthly/2yr

TODO:

Address merge conflicts
Once https://github.com/E3SM-Project/e3sm_diags/pull/866

Address https://github.com/E3SM-Project/e3sm_diags/pull/866#issuecomment-2405981379
Update & run zppy_min_case_e3sm_diags_cdat_migrated

There have been a number of code changes since original work on this; I started fresh with #651, so closing this PR.


Redo versionning

Fixes #233

Switch to main branch
On each commit to main, after tests pass, (re-)tag the action as vx, vx.y and vx.y.z

When this is merged, it's probably worth doing a v3.1.0. Also, we need to change the default branch and update the existing PRs, and delete the v3 branch
At this point I have no idea how to handle minor or major versions (e.g. how to make it so when we merge a PR that warrants a version change, it's not immediately released as incremental)

I thought about it a bit more and somehow this versioning system doesn't feel convincing to me. We create a new tag for each commit (for which the CI is running) and I don't quite see how our workflow should work when we want to increase the major or minor version. And I'm not so convinced about pushing tags around that may be linked to official releases either.
What if we do it a bit differently and use the following system:

After the CI runs, we delete all tags ending with -development.
Then we take the last tag with schema v*.*.* and add a -development after it and use that as the tag for the latest commit.
If we want to increase the major or minor version, we just tag the latest commit with the new version number

So the basic idea would be that we have a development tag that we push around so that everyone who just uses @v3 is always up to date and the "real" version tags always stay in the same place.
That might also have disadvantages that I'm not thinking of right now, but that would somehow appeal to me as a system better than moving the "real" release tags around or creating a whole lot of tags.

Well, what I suggested is what GitHub says we should do, unless I'm mistaken:

We recommend creating releases using semantically versioned tags – for example, v1.1.3 – and keeping major (v1) and minor (v1.1) tags current to the latest appropriate commit. For more information, see "About custom actions" and "About semantic versioning.
source

The only change I did was adding minor tags too but I can remove that. The part where we force-push tags is their idea.
That said, we can discuss alternatives for sure!
Can you expand on the benefit from your solution compared to having people use @main? Using @vx.y.z-development in your workflow will only work until we release the next incremental which feels a bit like a pain, no ? Plus dependabot is likely to be confused and suggest to people to switch from dev to non-dev tag, though I can't say for sure.

Another alternative could be to just say we release manually once in a while (which we haven't been doing a lot) and have our workflow repush the vX tag when we push a release. This means people on v3 will get the latest release instead of the bleeding edge, so it may take a while to figure out if there's a problem.

I just thought of 2 other possibilities:

we keep a 2-number version instead of a 3 number. So there's just minor and major. We do a minor tag every commit on main, major tags are handled by hand. Or:
calver. But given we don't plan releases in advance, that might get weird.

Can you expand on the benefit from your solution compared to having people use @main?

Compared to that, there is really no advantage. If we were to come up with a concept for an action that no one uses yet, I would absolutely be in favour of people who want bleeding-edge functions simply using @main, and we would keep manually publishing a release every once in a while without force pushing the tags, so that everyone else gradually gets the new functions as well.
But the problem is that pretty much all the people who use the action are already using @v3. I also thought that was the problem we were trying to solve. Namely, that everyone who uses @v3 should continue to get bleeding-edge. That was the intention behind my suggestion, that we find a way between force pushing tags and having everyone switch to @main, by effectively making @v3 an @main by always adding a tag at the most recent commit that redirects the @v3 to an @main.

But then I'm not sure what usecase the "development" tags would help ?

(Switched to vx.y without incremental. This eliminates the need for distinguishing incremental releases from minor ones. If we're releasing with every PR and always keeping backwards compat anyway, then there's no real difference)

But then I'm not sure what usecase the "development" tags would help ?

Ah no, my mistake, I'm sorry. I made some wrong assumptions and thought that Github would have more logic for finding a specific version. Just ignore what I suggested.