tools get so slow outside MSYS2 shell

Hi,
I am sorry if this is not the proper place for this question.
The tools (like cp, mv, cat, etc.) work smoothly under the MSYS2 shell, but when I try to launch them under Windows cmd, they will take too long to response. For example, open Windows cmd console and cd to %MSYS2_ROOT%\usr\bin, input mv and hit enter, it takes almost 10 seconds to output the help info of mv...
I am new to MSYS2 and it is really great, I just want to use it everywhere, not only under MSYS2 shell, is it possible? Or, is there any workaround to achieve this?
Thanks,
Kelvin

I see the same problem for a few weeks now on my work machine in our corporate network where we have an Active Directory with a very elaborated group membership structure, which means I'm a member in tons of groups and to start a MSYS2 shell takes minutes (5-10 min), which makes it unusable in such an environment. To be honest, this is a weak design/implementation by the cygwin folks.

@juergenpf Checkout this issue https://github.com/Alexpux/MSYS2-packages/issues/138 where users seems to have workaround the problem by commenting db in their nsswitch.conf file.
Not sure if this will work on your case, good luck.

@maoueh Thanks for the tip, it did solve the issue for me, but with a small tweak, which I'd written on #138's page.

Yeah, after disable db in nsswitch.conf, those commands work great also in Windows cmd! Also, there is a great article describes this problem and gives a solution: http://bjg.io/guide/cygwin-ad/.
Thanks for all of you, closing the long live issue now.

@kelvinh
Thank you, the article saved me. I also got issue and after comment the db option.


broken links

hey, the index.html file links to https://github.com/fellu/coinbin/ and https://github.com/fellu/coinbin/releases returning 404s but is should link to https://github.com/BitcoinInterestOfficial/coinbin and https://github.com/BitcoinInterestOfficial/coinbin/releases instead.
i know its trivial but it might be confusing to some users, regards.

Thanks, will try to update those fast. :)


pytest: resolve deprecated nose-specific method warnings

Coverage: 100.0%. Remained the same when pulling 055d96737a13faf9518422621f4339dc3f95e575 on jvoss/fix_deprecation_warnings into 100babb00776fcd0077152109000602bd77b1b16 on master.


Test existence of holdout using 'is not None'.

Fixes #78 only :)

Looks great! Thanks for fixing this, and for splitting up the PRs!


suggestation to improve the jsonpath_mini function

I was just wondering since REST API gives a json object where all the keys are always string so why do you need a int function inside the query dictionary. Please see the below function
def query_dictionary(query, dictionary, delimiter='.'):
    """ Do an xpath-like query with dictionary, using a template if relevant """
    # Based on
    # http://stackoverflow.com/questions/7320319/xpath-like-query-for-nested-python-dictionaries

    try:
        stripped_query = query.strip(delimiter)
        if stripped_query:
            for x in stripped_query.split(delimiter):
                try:
                    x = int(x) # why this int function is needed here?
                    dictionary = dictionary[x]
                except ValueError:
                    dictionary = dictionary[x]
    except:
        return None
    return dictionary

@rsukla-handy A REST API can return an array rather than a dictionary (the use of 'dictionary' as the argument name here is a bit misleading).  This is a little bit of magic to do lookup on that array by index.
In general though, the jsonpath_mini extractor is only for a minimal case where you need to do trivial validation without access to extra libraries.  More complex validation should use the JSONSchema validator or JMESpath extractor (which does far more advanced querying).  See the advanced.md docs for details.
I'm going to go ahead and close this one out, since I think that answers the question here, but would be happy to reopen it if you feel there's more to it.
Thanks,
Sam

could you please provide an example as to how to use it inside tests? So basically I need to install this python library JMESpath and then use the same syntax for jsonpath_mini like   - compare: {jmespath: "postal_code", comparator: "equals", expected: "10001"}


Add option for visitor to become a host of a new chain

If there is no chain in the users area, the user can start a new chain and becomes a host.

Decide where to put this option on the website.
The user gets a user account on the website.
The user can start a new chain. (Options: in the same form as the user account creation, or as a separate form when logged in).

Duplicate of https://github.com/pim0223/clothing-chain-frontend/issues/4; only the user accoung locig needs to be figured out.


Plugins lazy module loading

Tracking issue
Fixes https://github.com/flyteorg/flyte/issues/4401
Why are the changes needed?
This change is primarily to simplify the dependencies needed for things like generating API reference documentation for flytekit. Before one needed all dependencies of all plugins to render API docs.
What changes were proposed in this pull request?
Use flytekit.lazy_module to lazily load modules in the plugin packages.
How was this patch tested?
Run tests locally with pytest

I had heard stories about lazy importing breaking some workflows. For example in flytekit: https://github.com/flyteorg/flyte/issues/3853 .

I also fixed that in this PR

Overall, I am okay with making plugin imports lazy. Moving forward, how do we make sure plugins always lazy import dependencies? I suspect the monodocs will fail with an "can not import ___" error, which does not directly connect to the solution of lazy loading.

Short of catching this in a review, not sure how we can handle this.

Created this issue @thomasjpfan: we can discuss the plugin convention issue there - https://github.com/flyteorg/flyte/issues/4619

Lazy-loading pyspark.ml causes issues on executions of Spark tasks on real clusters. More details in https://github.com/flyteorg/flytekit/pull/2184. Reverting just that particular line works, but I'm not sure what the repercussions of this to monodocs.


peerDependencies error when installing

This error is from a project using sails 0.12.0-rc2:

npm WARN peerDependencies The peer dependency<EMAIL_ADDRESS>included from sails-permissions will no
npm WARN peerDependencies longer be automatically installed to fulfill the peerDependency
npm WARN peerDependencies in npm 3+. Your application will need to depend on it explicitly.
npm ERR! Darwin 14.3.0
npm ERR! argv "node" "/Users/rquinn/.nvm/versions/node/v0.12.5/bin/npm" "install" "sails-permissions" "--save"
npm ERR! node v0.12.5
npm ERR! npm  v2.11.3

npm ERR! code EPEERINVALID
npm ERR! peerinvalid The package sails does not satisfy its siblings' peerDependencies requirements!
npm ERR! peerinvalid Peer<EMAIL_ADDRESS>wants<EMAIL_ADDRESS>npm ERR! peerinvalid Peer<EMAIL_ADDRESS>wants<EMAIL_ADDRESS>

I blame npm. I removed the offending peerDependencies, which were there mostly due to anal-retentiveness anyway.


Uncaught TypeError: Cannot read property 'domain' of undefined - with Ngrok

Ziggy version
v1.2.0
Laravel version
v8.45.1
Description
When sharing a local domain with Ngrok.io the routes do not work.
Uncaught TypeError: Cannot read property 'domain' of undefined
Ziggy call and context
submit() {
    this.form.post(this.route('sign', route().params), { preserveState: false })
}

Ziggy configuration
{
defaults: {}
port: null
routes: {ignition.healthCheck: {…}, ignition.executeSolution: {…}, ignition.shareReport: {…}, ignition.scripts: {…}, ignition.styles: {…}, …}
url: "http://myrtr.test"
}

Route definition
Route::post('s/{document}',[DocumentController::class, 'update'])->middleware(['web'])->name('sign');

Can you share the complete section of the config object at routes.sign, the complete URL when that error happens, and the value returned from route().params on that page? Thanks.

Can you share the complete section of the config object at routes.sign, the complete URL when that error happens, and the value returned from route().params on that page? Thanks.

routes.sign: {
    bindings: {document: "id"}
    methods: (2) ["GET", "HEAD"]
    uri: "s/{document}"
}

route().params just returns the document ID

And the complete URL of the page?

And the complete URL of the page?

The url is myrtr.test/s/<id> and I'm trying to post to the same url by using that this.form.post(this.route('sign', route().params), { preserveState: false })

I thought I was able to reproduce this for a few minutes but now I can't anymore. How are you using Ngrok, with valet share or another way? Looking at the output you provided again it looks like your app isn't being served through Ngrok, or doesn't know it is—at least not on the pages you got that output from. Ziggy.url should be the Ngrok URL.


Use summary instead of full body when rendering blog posts in homepage activity feed

Default to showing the full body though if summary isn't available

Looking good on v1.laddr.io. Thanks!

fixed upstream


⚠ Jenkins Plugins Artifactory has degraded performance

In cc90053, Jenkins Plugins Artifactory (https://get.jenkins.io/plugins/artifactory/) experienced degraded performance:

HTTP code: 200
Response time: 1138 ms

Resolved: Jenkins Plugins Artifactory performance has improved in 80789ee after 16 minutes.


bumped compiler version

Description
Bumped compiler version
part of inmanta/inmanta-service-orchestrator#273
Self Check:
Strike through any lines that are not applicable (~~line~~) then check the box

[ ] Attached issue to pull request
[ ] Changelog entry
[ ] Type annotations are present
[ ] Code is clear and sufficiently documented
[ ] No (preventable) type errors (check using make mypy or make mypy-diff)
[ ] Sufficient test cases (reproduces the bug/tests the requested feature)
[ ] Correct, in line with design
[ ] End user documentation is included or an issue is created for end-user documentation (add ref to issue here: )

Reviewer Checklist:

[ ] Sufficient test cases (reproduces the bug/tests the requested feature)
[ ] Code is clear and sufficiently documented
[ ] Correct, in line with design

Processing this pull request

Processing #4050.


Fixes #1360. Fix NAG build bug

Description

Fixes a build bug with NAG. More of a workaround than a fix I suppose, but it might take @weiyuan-jiang level knowledge to make a reproducer for NAG.
So for now, we implement a fix from @tclune
Related Issue

Closes #1360
Motivation and Context

Lets MAPL build with NAG 7.1
How Has This Been Tested?

Built the branch with Intel, GNU, and NAG. All worked.
I'll also build and run with GEOSgcm, but it seems trivial for that (as it only really affects base/cub2latlon_regridder.F90. But I'll also tag @bena-nasa on this so he can make sure it's good by him.
Types of changes

[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] Trivial change (affects only documentation or cleanup)

Checklist:

[x] I have tested this change with a run of GEOSgcm (if non-trivial)
[x] I have added one of the required labels (0 diff, 0 diff trivial, 0 diff structural, non 0-diff)
[x] I have updated the CHANGELOG.md accordingly following the style of Keep a Changelog

This is zero-diff as expected.


Doesn't work in IE11. Object doesn't support this action

Hello,
embed() function doesn't work in IE 11. Got 'Object doesn't support this action' error. It works in other browsers.
Could you please advise?
ERROR TypeError: Object doesn't support this action
"ERROR"
{
[functions]: ,
proto: { },
description: "Object doesn't support this action",
message: "Object doesn't support this action",
name: "TypeError",
ngDebugContext: { },
number: -2146827843,
stack: "TypeError: Object doesn't support this action
at EmbedVideoService.prototype.embed (http://localhost:4200/vendor.js:102331:9)
at PortfolioItemPageComponent.prototype.getVideoFrame (http://localhost:4200/main.js:3563:9)
at Anonymous function (http://localhost:4200/main.js:3659:21)
at PortfolioItemPageComponent.prototype.updateVideolinksToFrame (http://localhost:4200/main.js:3655:9)
at PortfolioItemPageComponent.prototype.test (http://localhost:4200/main.js:3560:9)
at Anonymous function (Function code:691:15)
at handleEvent (http://localhost:4200/vendor.js:49067:9)
at callWithDebugContext (http://localhost:4200/vendor.js:50160:9)
at debugHandleEvent (http://localhost:4200/vendor.js:49863:5)
at dispatchEvent (http://localhost:4200/vendor.js:46526:9)",
Symbol(rxSubscriber)_g.jc4utky2gqc: undefined,
Symbol(rxSubscriber)_h.jc4utky2gqc: undefined
}

Hi @adamkovuch, In order to support IE11 (or other older browsers) some js features need to be polyfilled.
Can you try adding this to polyfill.ts inside your project:
import 'core-js/es6/object';
// or
import 'core-js/es7/object';

Let me know if this helped you.

It's already included. Also, the application works ok, but when I call embed() function, I get such error.

Afaik, you should also need this meta header in your index.html
<meta http-equiv="X-UA-Compatible" content="IE=edge" />

It doesn't help, the same problem.
Thanks

Hello,
I used https://www.npmjs.com/package/embed-video and this problem doesn't appear. So the issue is resolved for me.
Thanks.

#onedotover thanks its worked you save my time


Update indexing settings

Description

Add the indexing settings to the update APIs
Split the update APIs to one route per updatable field to avoid side effects (e.g not specifying the retention policy drops it)
Simplify the CLI to be a very basic client for the APIs that read the input body from files

How was this PR tested?
Unit tests and validated that swagger works fine.

what do we do with this PR @rdettai @guilload ?

It's the next thing in my backlog after SQS. It needs a pretty big rebase and quite a few changes we agreed upon.

This is superseeded by https://github.com/quickwit-oss/quickwit/pull/5078


[Feature]Inlong manager put inlong group id in dataflow info

Description
Sort will use inlong group id for reporting metrics, so manager should put this in the dataflowInfo.
Use case
No response
Are you willing to submit PR?

[ ] Yes, I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

please put inlong group id in the map

Solved by https://github.com/apache/incubator-inlong/pull/3052


docs: :memo: updated flutter-freezed plugin guide

Description
Explained the various blocks and Patterns that could be used to configure them
Related # (issue)
Type of change

[x] This change requires a documentation update

Checklist:

[x] I have followed the CONTRIBUTING doc and the style guidelines of this project
[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[x] I have made corresponding changes to the documentation
[x] My changes generate no new warnings
[x] I have added tests that prove my fix is effective or that my feature works
[x] New and existing unit tests pass locally with my changes
[x] Any dependent changes have been merged and published in downstream modules

@saihaj please can we get the plugin docs updated too

Standing by to provide any solution


fix wrong line numbers

Hey @long1eu ! Thanks for this ! LGTM :)

no prob. the official plugin is close to 0


[SEDONA-441] Add ST_LineLocatePoint

Did you read the Contributor Guide?

Yes, I have read Contributor Rules and Contributor Development Guide

Is this PR related to a JIRA ticket?

Yes, the URL of the associated JIRA ticket is https://issues.apache.org/jira/browse/SEDONA-441. The PR name follows the format [SEDONA-XXX] my subject.

What changes were proposed in this PR?
Adds Vector function ST_LineLocatePoint

Returns a double between 0 and 1, representing the location of the closest point on the LineString as a fraction of its total length.

How was this patch tested?

Passes new and existing tests

Did this PR include necessary documentation updates?

Yes, I am adding a new API. I am using the current SNAPSHOT version number in since vX.Y.Z format.

Thank you @prantogg  and @jiayuasu. This is great to see!


How to Handle Voice Search?

Thank you for the great SearchView.
I am having a problem with Voice Search. Your sample application does not include its working. Could you please explain its implementation?

Hi,
There is no direct support for voice search. Instead, you have to implement it yourself by adding it as a MenuItem  in xml and then respond to its clicks from whithin:
mSearchView.setOnMenuItemClickListener(new FloatingSearchView.OnMenuItemClickListener() {
      @Override
      public void onMenuItemSelected(MenuItem item) {

      }
   });

Please refer to README.

Here is a method to handle Voice Search
final int VOICE_SEARCH_CODE = 3012;
public void startVoiceRecognition() {         Intent intent = new Intent("android.speech.action.RECOGNIZE_SPEECH");         intent.putExtra("android.speech.extra.LANGUAGE_MODEL", "free_form");         intent.putExtra("android.speech.extra.PROMPT", "Sepeak Now");         this.startActivityForResult(intent, VOICE_SEARCH_CODE);     }
And in onActivityResult(..)....
@Override     public void onActivityResult(int requestCode, int resultCode, Intent data) {         if (requestCode == VOICE_SEARCH_CODE && resultCode == RESULT_OK) {             ArrayList<String> matches = data                     .getStringArrayListExtra("android.speech.extra.RESULTS");             searchView.setSearchText(matches.get(0));         }         super.onActivityResult(requestCode, resultCode, data);     }


ESP32 Wroom Pin For CLK

Hello,
Your library is great, but unfortunately I stuck in one situation, where i would like to have you help in that. In my project I used 3.5 Touch Screen Display(ILI9488), For That i used following pins
T_IRQ	-
T_DO	19
T_DIN	23
T_CS	13
T_CLK	18
SCK	           18
SDI(MOSI)   23
DC	           4
RESET	   2
CS	           15
I was using following pis for HX711_ADC
HX711_dout = 36
HX711_sck = 23
Now due to Pin 23 is in common between Both, I can't Use Pin 23 for HX711, I tried Pin 18 but that also not resolve my problem. Also I had tried all other pins  but didn't got any success.
So my question is Which Other pin i can use for HX711_sck (other than display pins)?
Thanks and Regards
Sudarshan Patil

Hi!
One of the following GPIO pins should work for HX711 sck: 5, 14, 21, 22, 25, 26, 27, 32, 33

Hello,
Thank you very much for your timely reply, I used Pin 33 as your suggested and it's work.
Thanks and regards
Sudarshan Patil


Some styles are not applied in release, but works perfectly in debug.

Describe the bug
NativeWind Version: 4.0.1
React Native Version: 0.73.6
In release configuration looks like some styles are randomly not being applied.
Looks very strange especially because in other places same styles are applied correctly.
This is example of the code, but it looks totally random.
For example here, w-32, h-32 are not applied, while other styles applied correctly. For example colors, border-colors, as well as rounded-full are applied correctly.
export const FullScreenLoadingModal = ({ isLoading = true, loadingMessage = ""}) => {
return (

<Pressable className="flex-1 justify-center items-center" style={{ backgroundColor: 'rgba(0, 0, 0, 0.7)' }}>

<LottieView
source={require("./../../assets/loading.json")}
style={{ width: 300, height: 300, marginLeft: -110, marginTop: -110}}
autoPlay
loop
/>

);
};
In other screens, mt-16, mt-20 are not applied, and again, the rest of the styles are applied correctly.
To Reproduce
Steps to reproduce the behavior:
In Debug configuration everything is working correctly as expected.
In Release configuration, some styles seems are not applied.
Question
Could you please explain how to debug the release build.
Is it possible to output the final NativeWind styles output for each of the files in project?

If you're talking about <LottieView />, you might need to wrap it with a cssInterop since it's a 3rd party component that uses the style prop.

https://www.nativewind.dev/v4/api/css-interop
https://www.nativewind.dev/v4/guides/third-party-components

For example:
import LottieView from 'lottie-react-native';
import { cssInterop } from "nativewind";

const WrappedLottieView = cssInterop(LottieView , { className: "style" });

And use it as:
<LottieView className="" />
``

I have similiar issue, its related to #886

We are closing all pre-v4.1 issues now that v4.1 is released. If the problem is still happening with the latest v4.1, please open a new issue and reference this one in the description.


Start registering KPIs

Now that we have the KPIs module, we need to start registering data.

Hey, @srodriguezOPT
I've been reading the examples, and I have some doubts. For example, I want to register the number of packages submitted, some will be ok, others will be rejected because of an error or lack of permission, others will be duplicate, etc.

What should I send in job and instance fields?
What are the base_labels you mention, are they like {'created_at':'...', 'user_uuid':'...', ...}?

Thanks

Hi @jbonnet ,
Let me show you an example of how prometheus will store the counter/gauge:
gk_maxApiCalls(exported_instance="gatekeeper", exported_job="api calls",..., "base_labels": {"api name": "services", "method": "GET"})
gk_maxApiCalls(exported_instance="gatekeeper", exported_job="api calls",..., "base_labels": {"api name": "services", "method": "POST"})
gk_maxApiCalls(exported_instance="gatekeeper", exported_job="api calls",..., "base_labels": {"api name": "services", "method": "PUT"})
gk_maxApiCalls(exported_instance="gatekeeper", exported_job="api calls",..., "base_labels": {"api name": "functions", "method": "GET"})
So, we can use instance and job to identify the origin and type of the KPI.
Another example of an existing vim KPI: vim_maxServerGroupMembers{exported_instance="POP_2",exported_job="limits"...
With the base_labels, prometheus will create a counter/gauge with every set of base_labels. In the example:
a gk_maxApiCalls gauge/counter with the api GET /services count
a gk_maxApiCalls gauge/counter with the api POST /services count
....
We can use the base labels for the service status, the http error code, etc.

@srodriguezOPT
Can you please add some KPIs registration code.. to the KPIs service?
That would be nice playing as examples...
Thanks,

Hi @jbonnet ,
In http://sp.int3.sonata-nfv.eu:9091/ you can check the counter/gauges created in the pushgateway.
For example, the job="sonata", instance="gtkkpi" both examples (example_gauge and example_counter) were created with the https://github.com/sonata-nfv/son-tests/tree/master/int-gtkkpi-prometheus tests; In test1.sh and test2.sh, you can check that the REST invocations are like
curl -H 'Content-Type: application/json' -X PUT -d '{ "metric_type": "counter", "job": "sonata", "instance": "gtkkpi", "name": "example_counter", "docstring": "metric counter test", "base_labels": {"label1": "value1", "label2": "value2"}}' http://sp.int3.sonata-nfv.eu:32001/api/v2/kpis)

Super!
Thanks, consider it done!
:-)

On 21 Mar 2017, at 16:55, Santiago Rodríguez<EMAIL_ADDRESS>wrote:
Hi @jbonnet https://github.com/jbonnet ,
In http://sp.int3.sonata-nfv.eu:9091/ http://sp.int3.sonata-nfv.eu:9091/ you can check the counter/gauges created in the pushgateway.
For example, the job="sonata", instance="gtkkpi" both examples (example_gauge and example_counter) were created with the https://github.com/sonata-nfv/son-tests/tree/master/int-gtkkpi-prometheus https://github.com/sonata-nfv/son-tests/tree/master/int-gtkkpi-prometheus tests; In test1.sh and test2.sh, you can check that the REST invocations are like
curl -H 'Content-Type: application/json' -X PUT -d '{ "metric_type": "counter", "job": "sonata", "instance": "gtkkpi", "name": "example_counter", "docstring": "metric counter test", "base_labels": {"label1": "value1", "label2": "value2"}}' http://sp.int3.sonata-nfv.eu:32001/api/v2/kpis http://sp.int3.sonata-nfv.eu:32001/api/v2/kpis)
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/sonata-nfv/son-gkeeper/issues/447#issuecomment-288144624, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAuQ2hSUl97EP3gTor2vv9ffbd-jmsWks5roAEJgaJpZM4MP1_K.

Closed by #461


 component

Follow up, clean up to #367.
Description

This pull request:

Refactors out some of the code in the EnvBuild component into EnvBuildStatus
Renames the export of BuildDropdown.tsx to match the file name
Some code clean up for clarity

Pull request checklist

[ ] Did you test this change locally?
[ ] Did you update the documentation (if required)?
[ ] Did you add/update relevant tests for this change (if required)?

Additional information

Marking this as draft until I update Storybook

Tests pass, runs locally, looks good in Storybook. Merging in now.


[BUG]: If there is a file name that is too long, I cannot click the pin.

How are you running AnythingLLM?
AnythingLLM desktop app
What happened?

If there is a file name that is too long, I cannot click the pin.
Are there known steps to reproduce?
No response

I cannot repro this and from the screenshot I can tell that this is most likely an older version of AnythingLLM Desktop. In the latest version we have select checkboxes on each row for moving around multiple files and we truncate more of the text so it does not overlap anymore.
Please update to the latest version and this will be resolved.


Bugfix: fix Houdini publish button bugs

Changelog Description
Self publish had few bugs

Unnecessary faulty logic in find_rop_input_dependencies function which caused karma and HDA publishes to fail.
Recursive rop rendering as each extractor was rendering all input nodes!

Solution:

refactor find_rop_input_dependencies to find_rop_inputs_chain with much simpler logic.
adding ignore_inputs flag in render_rop function

Testing notes:

use self publish buttons to publish karma and HDA
use different combinations and it should still work
examples:

@BigRoy @fabiaserra
I've launched this PR that only focuses on fixing the issues.
Many thanks for your guidance.

Could you show what's the result of clicking submit on those examples you sent? I assume only the ones that have Deadline support are getting tasks there but they aren't dependencies of the parent ROPs?

I have a question on that but it's not quite related to this PR. Is it possible to not limit the hda creation within /obj level? Sometimes let's say you need to create some reuseable digital assets for use in the stimulation, user can always load and update the hda for the sim.

This question has come up before, see https://github.com/ynput/OpenPype/issues/4880

Could you show what's the result of clicking submit on those examples you sent? I assume only the ones that have Deadline support are getting tasks there but they aren't dependencies of the parent ROPs?

Disclaimer: Order is not considered, I can detect it but it doesn't exist yet and if it exists, no way I can tell publisher to use it.
Unless I publishes these instances one by one in separate publish sessions (which I failed to do it as function need to wait till the current publish is done otherwise it will reset the publisher!)

So, if this PR solves the problem with publish button.
Do you think we need the OTLs added in this PR https://github.com/ynput/OpenPype/pull/5871 ? or should I just close it?

I think we're best of removing the custom nodes - unless they provide special abilities not common to Houdini itself.
Thinking about it now - we might be able to implement functionality like what e.g. Arnold and Redshift do to add their custom attributes to built-in houdini nodes dynamically.
Say we do that same thing, by registering custom parms for built-in types then we could do so for RopNode types in general (or specific ones) and add it to e.g. node types outside of OpenPype ROP nodes as well - including the built-in Merge or Fetch nodes and thereby need no custom HDA?
However, for those cases it's still needed to first resolve the publishing to be able to better adapt/support Houdini's ROP node ordering and for that likely has to stay closer to Houdini's ROP workflow like @fabiaserra described a few times.

Could you show what's the result of clicking submit on those examples you sent? I assume only the ones that have Deadline support are getting tasks there but they aren't dependencies of the parent ROPs?

Disclaimer: Order is not considered, I can detect it but it doesn't exist yet and if it exists, no way I can tell publisher to use it. Unless I publish these instances one by one in separate publish sessions (which I failed to do it as function need to wait till the current publish is done otherwise it will reset the publisher!)

Thanks for the useful images, it confirms my experience and the limitations that I have raised with this. In those situations what you'd expect is to have a network that for example does the following this in order:

Run a simulation
Cache all your outputs as geo
Render the outputs
Create a flipbook

That's not possible with the OP Deadline integration of Houdini but very easy to set with the vanilla Deadline submitter.
On any case, the PDG Deadline scheduler has gotten even better in Houdini 20 so I think we should maybe focus our efforts on providing support for PDG in Houdini 20 and make use of that scheduler to expose the potential of Houdini and PDG's by adding OP TOP nodes that allow us to do operations with it. For example, look at the TOP nodes that SideFX ships for Shotgrid integration: https://www.sidefx.com/docs/houdini/nodes/top/shotgunserver.html and look at the Shotgun provided HIP example https://www.sidefx.com/docs/houdini/tops/examples.html#shotgun to get an idea of what I mean

@fabiaserra would you be ok if it worked like this:

Remove all current Houdini extractor that in essence just do RopNode.render()
Create on ExtractROPs extractor that is context plug-in and forces it extracts in the order of Deadline's dependencies.

class RopRenderExtractor(pyblish.api.ContextPlugin):
    def process(self, context):
        rop_instances = [inst for inst in context if inst.get("instance_node")]
        rop_nodes = [inst.get("instance_node")] for inst in rop_instances]
        rop_nodes.sort(sort_by_dependency)

        processed = set()
        for rop in reversed(rop_nodes):
            # We render the ROP node furthest down dependency chains *WITH*
            # all its connected inputs in one batch. Then we track what we
            # have rendered to avoid re-rendering them for other instances
            if rop.path() in processed:
                continue

            rop.render()
            processed.add(rop.path())
            dependency_nodes, dependency_frames
            for node in dependency_nodes:
                processed.update(node.path())

So that we basically only trigger RopNode.render() on the last Rop Node in each tree and allow Houdini to do whatever it wants upstream so that it renders in the order like Deadline would?

Happy to give it a shot and compare it with the behavior we have right now yeah but this would only work on the local evaluation workflow? Most of the times we want to submit the whole ROP graph into Deadline as individual tasks

The idea would be to take that concept over to the Deadline submissions as well - but since local testing of the implementation would be more trivial I'd recommend prototyping that first, confirm it does what it needs and then see how we can best translate the logic to Deadline.
I'd expect that in Deadline you actually want also the input dependencies submitted as jobs themselves as "dependent jobs", right? So if triggering the render in deadline we should there at least run the render with RopNode.render(ignore_inputs=True) so that its up to the deadline jobs to be dependencies and not a single job doing more than one ROP node. I suspect that's also what Deadline itself does - if not let me know, we should at least stay close to what they are doing - by doing so we should also make it much more trivial to actually use their submission logic more, etc.
You mentioned "individual tasks" but I suspect those dependent rops would be individual jobs right?

Correct yeah, for the Deadline case we would mostly just submit the corresponding Deadline task for each node with its inputs as dependencies of that job. For most ROPs that would just be a matter of cooking the ROP through a Houdini plugin (running as chunks if not a sim or a single task for a simulation) and for the render ROPs there would be two separate jobs (export + render). That would already get us much closer to a working solution

Happy to give it a shot and compare it with the behavior we have right now yeah but this would only work on the local evaluation workflow? Most of the times we want to submit the whole ROP graph into Deadline as individual tasks

The idea would be to take that concept over to the Deadline submissions as well - but since local testing of the implementation would be more trivial I'd recommend prototyping that first, confirm it does what it needs and then see how we can best translate the logic to Deadline.
I'd expect that in Deadline you actually want also the input dependencies submitted as jobs themselves as "dependent jobs", right? So if triggering the render in deadline we should there at least run the render with RopNode.render(ignore_inputs=True) so that its up to the deadline jobs to be dependencies and not a single job doing more than one ROP node. I suspect that's also what Deadline itself does - if not let me know, we should at least stay close to what they are doing - by doing so we should also make it much more trivial to actually use their submission logic more, etc.
You mentioned "individual tasks" but I suspect those dependent rops would be individual jobs right?

Correct yeah, for the Deadline case we would mostly just submit the corresponding Deadline task for each node with its inputs as dependencies of that job. For most ROPs that would just be a matter of cooking the ROP through a Houdini plugin (running as chunks if not a sim or a single task for a simulation) and for the render ROPs there would be two separate jobs (export + render). That would already get us much closer to a working solution

Out of curiosity - when doing a deadline submission like that; does it allow setting a chunksize per ROP node on the submission? And if so, how does that look in the Deadline Submitter UIs?

Out of curiosity - when doing a deadline submission like that; does it allow setting a chunksize per ROP node on the submission? And if so, how does that look in the Deadline Submitter UIs?

You are able to define all Deadline settings on the upstream nodes through the vanilla Deadline ROP so you have full flexibility on your hands directly on the graph. You can have multiple Deadline ROPs on the same submission to set the different chunk sizes upstream: https://docs.thinkboxsoftware.com/products/deadline/10.3/1_User Manual/manual/app-houdini.html#rop-node It requires a bit of a technical user to know how to do these and it could certainly be streamlined to set defaults but most Houdini TDs are plenty capable of understanding this, that's why the current Deadline OP blackbox doesn't fit their needs

A question came up to my mind (irrelevant to the PR but relevant to the previous discussion)
Could we have run publish jobs from inside houdini ?
I mean deadline runs some command line to trigger the publishing, why don't we write the same command inside a houdini python node ?

A question came up to my mind (irrelevant to the PR but relevant to the previous discussion) Could we have run publish jobs from inside houdini ? I mean deadline runs some command line to trigger the publishing, why don't we write the same command inside a houdini python node ?

Could you explain to me what it solves?
Anyway, that's actually what we did initially for farm publishing during Avalon which made it work automatically for all output types instead of having to write farm submissions for each. Basically what we did was this:

Submit the scene to the farm with a publish script which just triggers headless publishing (only on specified instances so we would still have a job PER publish instance)
The same job would then also be integrating the files because it'd basically be running a regular publish.

Upside of that is:

On the farm machine the publish validations also trigger (not sure if that's currently the case?) so that e.g. any file checks whether a file exist are done from the machine extracting the files.

There are some downsides however:

It's not trivial to implement "chunks" since it's a single publish session that also extracts it.
It's not trivial to separate the extracting from the integrating. As such to just republish the files you'd still need to run the full publish including the integration. We could implement the publishing so that it allows "publish existing frames" so that we'd submit the extraction separate from the integrating job, but by that stage - why not make it so that the integrating of the files is actually a separate process. Which I think is currently the case in the farm submissions?

And of course, if we rely more on the "just render a rop", "just integrate the files" we should be able to make a very simple entry point so that we can stay pretty close to what Deadline does, which is rendering a ROP.

from openpype.pipeline.create import CreateContext
from openpype.pipeline import registered_host

host = registered_host()
context = CreateContext(host)
for instance in context.instances:
    instance.run_validations()

For what it's worth - this is all possible via pyblish.util, for example:
import pyblish.util
from openpype.pipeline.create import CreateContext
create_context = CreateContext(host)

pyblish_context = pyblish.api.Context()
pyblish_context.data["create_context"] = create_context
pyblish_plugins = create_context.publish_plugins

# Run collect and validate order of pyblish plugins
pyblish_context = pyblish.util.collect(pyblish_context, pyblish_plugins)
pyblish_context = pyblish.util.validate(pyblish_context, publish_plugins)

Likely something like this - which from an API standpoint could be simplified if the publishing is also available on the CreateContext preferably with a end order you can set, e.g. a up_to_order or last_order or stop_at_end_of_order.
import pyblish.api
from openpype.pipeline.create import CreateContext
create_context = CreateContext(host)

create_context.publish(
    stop_at_end_of_order=pyblish.api.ValidatorOrder
)

Even more madness if you could specify the start and stops

```python
import pyblish.api
from openpype.pipeline.create import CreateContext
create_context = CreateContext(host)

create_context.publish(
    start = pyblish.api.CollectorOrder - 0.5
    end = pyblish.api.ValidatorOrder + 0.5
)

A question came up to my mind (irrelevant to the PR but relevant to the previous discussion) Could we have run publish jobs from inside houdini ? I mean deadline runs some command line to trigger the publishing, why don't we write the same command inside a houdini python node ?

Could you explain to me what it solves?
Anyway, that's actually what we did initially for farm publishing during Avalon which made it work automatically for all output types instead of having to write farm submissions for each. Basically what we did was this:

Submit the scene to the farm with a publish script which just triggers headless publishing (only on specified instances so we would still have a job PER publish instance)
The same job would then also be integrating the files because it'd basically be running a regular publish.

Upside of that is:

On the farm machine the publish validations also trigger (not sure if that's currently the case?) so that e.g. any file checks whether a file exist are done from the machine extracting the files.

There are some downsides however:

It's not trivial to implement "chunks" since it's a single publish session that also extracts it.
It's not trivial to separate the extracting from the integrating. As such to just republish the files you'd still need to run the full publish including the integration. We could implement the publishing so that it allows "publish existing frames" so that we'd submit the extraction separate from the integrating job, but by that stage - why not make it so that the integrating of the files is actually a separate process. Which I think is currently the case in the farm submissions?

And of course, if we rely more on the "just render a rop", "just integrate the files" we should be able to make a very simple entry point so that we can stay pretty close to what Deadline does, which is rendering a ROP.

wow!
My intention was to breakdown the publishing process into pieces that can be converted into Houdini nodes. but yeah you have shown me that even these nodes exist, there will be non trivial problems.
I really wish if we could find a middle ground because @fabiaserra mentioned in many discussions before, we need to find means to adopt current Houdini solutions instead of reinventing new ones!

from openpype.pipeline.create import CreateContext
from openpype.pipeline import registered_host

host = registered_host()
context = CreateContext(host)
for instance in context.instances:
    instance.run_validations()

For what it's worth - this is all possible via pyblish.util, for example:
import pyblish.util
from openpype.pipeline.create import CreateContext
create_context = CreateContext(host)

pyblish_context = pyblish.api.Context()
pyblish_context.data["create_context"] = create_context
pyblish_plugins = create_context.publish_plugins

# Run collect and validate order of pyblish plugins
pyblish_context = pyblish.util.collect(pyblish_context, pyblish_plugins)
pyblish_context = pyblish.util.validate(pyblish_context, publish_plugins)

Likely something like this - which from an API standpoint could be simplified if the publishing is also available on the CreateContext preferably with a end order you can set, e.g. a up_to_order or last_order or stop_at_end_of_order.
import pyblish.api
from openpype.pipeline.create import CreateContext
create_context = CreateContext(host)

create_context.publish(
    stop_at_end_of_order=pyblish.api.ValidatorOrder
)

Even more madness if you could specify the start and stops
import pyblish.api
from openpype.pipeline.create import CreateContext
create_context = CreateContext(host)

publish_context = create_context.publish(
    start = pyblish.api.CollectorOrder - 0.5
    end = pyblish.api.ValidatorOrder + 0.5
)

So that if ok one could e.g. use the publish context to continue if there are not errors at e.g. start=pyblish.api.ValidationOrder + 0.5

This is quite interesting!

A few thoughts:

In my opinion there's little benefits on adding yet another pass of validations in the farm. Validations should run before you submit your job to the farm to give the user a peace of mind that the process will likely run successfully, any other errors that happen in the farm are going to be mostly handled by TDs/developers so at that point what benefit do you get from doing those extra validations?
Why would you want to waste a Houdini license while doing an OpenPype publish that shouldn't need Houdini for anything?
The current design of OP handling both the extraction/render on the ROPs and the integration for local executions is a BIG limitation. There's so many occasions where OP errors out during the publish, this means that even though you have already successfully cached or rendered all your files, just because OP failed to do an integrate step you need to re-run your whole render/cache?
OpenPype shouldn't add even more ways of things breaking during a render/cache, that's the bread and butter of Houdini and farm processes. I think OP should only help you streamline and prepare your scene for success, run some validations before submission and then run other separate tasks that publishes the outputs.
The part of submitting the scene and job dependencies to a farm is solved with the third-party ROP/TOP schedulers. Do you still think we want to duplicate that work and handle it in OP?

A question came up to my mind (irrelevant to the PR but relevant to the previous discussion)
Could we have run publish job
s from inside houdini ?
I mean deadline runs some command line to trigger the publishing, why don't we write the same command inside a houdini python node ?

I strongly think we should start leveraging the PDG framework and just steal from the ideas of others who have already thought about these problems. Moreover, in PDG you have the flexibility of doing a lot of the things you describe. You can have a Python Processor TOP node or a Python Script TOP node and do whatever OP process you want and in a very efficient way, with chunk size and dependencies being first class citizens of the framework. If we provided the equivalent nodes that there are for Shotgrid but for Ayon, we can start creating TOP graphs that evaluate in Deadline and handle the publish and everything without needing to reinvent the wheel on how to submit jobs in the farm!!

I strongly think we should start leveraging the PDG framework and just steal from the ideas of others who have already thought about these problems. Moreover, in PDG you have the flexibility of doing a lot of the things you describe. You can have a Python Processor TOP node or a Python Script TOP node and do whatever OP process you want and in a very efficient way, with chunk size and dependencies being first class citizens of the framework. If we provided the equivalent nodes that there are for Shotgrid but for Ayon, we can start creating TOP graphs that evaluate in Deadline and handle the publish and everything without needing to reinvent the wheel on how to submit jobs in the farm!!

I really like the idea to have AYON tools like these available in PDG however, TOPs/PDG I personally haven't really used much nor have I found debugging it really that trivial. I'm wondering, would you be able to make a prototype yourself @fabiaserra? Even if rough around the edges it could kickstart the approach.

I really like the idea to have AYON tools like these available in PDG however, TOPs/PDG I personally haven't really used much nor have I found debugging it really that trivial. I'm wondering, would you be able to make a prototype yourself @fabiaserra? Even if rough around the edges it could kickstart the approach.

Yeah, I already filed a couple bugs to SideFX regarding the use of the File Cache SOP node in PDG but I have been able to submit other PDG graphs successfully to Deadline. I think H20 and the new version of Deadline have fixed some other bugs and added other features (https://www.sidefx.com/changelog/?journal=&categories=&body=PDG&version=&build_min=&build_max=&show_versions=on&show_compatibility=on&items_per_page=) so I'd like to update to H20 before I start prototyping but I should be able to have a look at those Shotgrid TOP nodes and see if I can write equivalent ones. I was actually already planning on doing this as the evolution of that OP Publisher ROP node that we are using so it's easier to leverage Python code executing both locally and in the farm, I will keep you posted

This comment shows https://github.com/ynput/OpenPype/pull/5909#issuecomment-1809744762
that find_rop_inputs_chain needs to be fixed.

This comment shows #5909 (comment) that find_rop_inputs_chain needs to be fixed.

@BigRoy Revisiting this issue!
I've updated find_rop_inputs_chain algorithm to fix cyclic dependencies
This is can be tested directly in Houdini's python source editor.
import hou

def find_rop_inputs_chain(node):
    all_input_nodes = []
    input_nodes = node.inputs()
    for input_node in input_nodes:

        nodes = find_rop_inputs_chain(input_node)
        # Filter existed nodes while keeping the order
        # It's a solution for cyclic dependencies
        nodes = [n for n in nodes if n not in all_input_nodes]
        all_input_nodes += nodes
        all_input_nodes.append(input_node.path())

    return all_input_nodes

node = hou.node('/out/staticMeshObjTest')
all_inputs = find_rop_inputs_chain(node)
all_inputs.append(node.path())

print (all_inputs)

Example tests:
['/out/usdTest',
 '/out/reviewTest',
 '/out/staticMeshSopTest',
 '/out/staticMeshObjTest']

['/out/mantra_ropTest',
 '/out/usdTest',
 '/out/reviewTest',
 '/out/staticMeshSopTest',
 '/out/staticMeshObjTest']

Another Test: I'm using shell fail report to check the dependency graph

find_rop_inputs_chain Function output:
['/out/karma_ropTest',
 '/out/vdbcacheTest',
 '/out/mantra_ropTest',
 '/out/usdTest',
 '/out/pointcacheABCTest',
 '/out/reviewTest',
 '/out/staticMeshSopTest',
 '/out/staticMeshObjTest']

Shell nodes output
Failed to run echo_karma_ropTest with error 2: The system cannot find the file specified.
Argument list:
   0: echo_karma_ropTest
Failed to run echo_vdbcacheTest with error 2: The system cannot find the file specified.
Argument list:
   0: echo_vdbcacheTest
Failed to run echo_mantra_ropTest with error 2: The system cannot find the file specified.
Argument list:
   0: echo_mantra_ropTest
Failed to run echo_usdTest with error 2: The system cannot find the file specified.
Argument list:
   0: echo_usdTest
Failed to run echo_pointcacheABCTest with error 2: The system cannot find the file specified.
Argument list:
   0: echo_pointcacheABCTest
Failed to run echo_reviewTest with error 2: The system cannot find the file specified.
Argument list:
   0: echo_reviewTest
Failed to run echo_staticMeshSopTest with error 2: The system cannot find the file specified.
Argument list:
   0: echo_staticMeshSopTest
Failed to run echo_staticMeshObjTest with error 2: The system cannot find the file specified.
Argument list:
   0: echo_staticMeshObjTest

That cyclic check should really be checking using a set, not a list. You're doing lookups against it. Since this can get slow very quickly with a lot of ROPs, if you really want to keep the ordering. Then create a set for the lookups and use a set for the ordering.
Anyway, having the algorithm is nice - but next step is supporting:

Ordering multiple ROP nodes, e.g. like publishing in itself would need - e.g. for #5909
Fetch nodes? -> since it'd basically be taking a ROP from elsewhere?
Switch nodes? -> since not all inputs are relevant

Ordering multiple ROP nodes, e.g. like publishing in itself would need

I believe this already happens. the returned list is sorted following the dependency graph execution order.

That cyclic check should really be checking using a set, not a list. You're doing lookups against it. Since this can get slow very quickly with a lot of ROPs, if you really want to keep the ordering. Then create a set for the lookups and use a set for the ordering.

I thought about it using sets, but I didn't figure out how to do so without losing the element order
https://stackoverflow.com/questions/4211209/remove-all-the-elements-that-occur-in-one-list-from-another

@MustafaJafar I think this also a good case to write test functions to ensure we can expand on the 'expected behavior' through tests over time (if we find edge cases down the line, we can fix the function and add a test for the edge case).
Here's a quick start:
import hou

def _rop(parent, name):
    return parent.createNode("geometry", node_name=name)

def find_rop_inputs_chain(node):
    processed = {node.path()}
    queue = [node]
    for node in queue:
        input_nodes = node.inputs()
        input_nodes = [n for n in input_nodes if n.path() not in processed]
        processed.update({n.path() for n in input_nodes})
        queue.extend(input_nodes)

    return list(reversed(queue))

def connect_chain(chain):
    iterable = iter(chain)
    source = next(iterable)
    for target in iterable:
        target.setNextInput(source)
        source = target

def test_order_simple():

    # Create nodes
    out = hou.node("/out")
    a = _rop(out, "a")
    b = _rop(out, "b")
    c = _rop(out, "c")
    d = _rop(out, "d")
    e = _rop(out, "e")
    connect_chain([a, b, c, d, e])
    assert find_rop_inputs_chain(e) == [a, b, c, d, e], "Failed"

    # Clear all nodes for next tests
    out.deleteItems([a, b, c, d, e])

def test_order_complex():

    # Create nodes
    out = hou.node("/out")
    a = _rop(out, "a")
    b = _rop(out, "b")
    c = _rop(out, "c")
    d = _rop(out, "d")
    e = _rop(out, "e")
    b_a = _rop(out, "b_a")
    b_b = _rop(out, "b_b")
    b_c = _rop(out, "b_c")
    d_a = _rop(out, "d_a")
    d_b = _rop(out, "d_b")
    d_c = _rop(out, "d_c")

    connect_chain([a, b, c, d, e])
    connect_chain([d_a, d_b, d_c, d])
    connect_chain([b_a, b_b, b_c, b])
    order = find_rop_inputs_chain(e)

    print(", ".join(n.name() for n in order))

    # The exact order here isn't crucial, as long as:
    # - b_* become before b and the rest
    # - d_* become before d and the rest
    # - a, b, c, d, e should process in that order
    assert order.index(b_a) < order.index(b_b), "b_a before b_a"
    assert order.index(b_b) < order.index(b_c), "b_b before b_c"
    assert order.index(b_c) < order.index(b), "b_c before b"
    assert order.index(d_a) < order.index(d_b), "d_a before d_b"
    assert order.index(d_b) < order.index(d_c), "d_b before d_c"
    assert order.index(d_c) < order.index(d), "d_c before d"
    assert order.index(a) < order.index(b), "a before b"
    assert order.index(b) < order.index(c), "b before c"
    assert order.index(c) < order.index(d), "c before d"
    assert order.index(d) < order.index(e), "d before e"

    out.deleteItems([a, b, c, d, e, b_a, b_b, b_c, d_a, d_b, d_c])

print("Testing simple")
test_order_simple()
print("Testing complex")
test_order_complex()

Note also here I refactored the find order function to use a set, etc.

I believe this already happens. the returned list is sorted following the dependency graph execution order.

Yes, but now given ROP nodes A, B, C, D, E give me the render order - it's not super difficult but you'd need to filter the nodes that are already an input of another to avoid duplicates, etc.
Additionally, we might also need to be able to detect which ROP depended on which ROP (not just the order but the actual dependency) because for Deadline submissions we'll want to mark those jobs dependent.
So we should design a call that would be able to provide the info for those use cases.
Likely we'll need to have a dedicated
def get_rop_inputs(node):
    # Here implement special cases for `fetch` and `switch`?
    return node.inputs()

def find_rop_inputs_chain(node):
    # implementation as we have it, but using `get_rop_inputs()` call

def get_sorted(nodes):
    # given multiple nodes provide me the final ordering to render all nodes
    # or, return separate branches for rendering in the order of `find_rop_inputs_chain`

# So that we can do
ordered = get_sorted(rop_nodes)
for node in ordered:
    render(node)
    link_job_dependency(node, get_rop_inputs(node))

And voila?

I might be playing devil's advocate here but let me try throw some more thoughts here.
If you want to run a local execution of ROP nodes, you would use Houdini's HQueue and just click the vanilla "Render" node on any of the nodes you want to cook. That already handles the dependencies, switches, fetches, bypasses... and it already has everything you'd need to run some local tests before you submit your job to the farm (very rarely one would cook a ROP graph locally, it's mostly just for testing and validating). Moreover Houdini will probably try to be smart in certain situations about parallel execution and multi-threading on HQueue. Abstracting that logic away and controlling the order and render of the nodes locally I think is regressive and not even on-par with the existing Houdini feature set. No one should be having to mix workflows when they want to just do a local execution vs when they want to publish. I shouldn't need to publish my outputs when I test a ROP render!
Collecting the dependency ROP graph to map it to a farm job or run validations or just publish the existing rendered outputs could make sense but as it is, I still don't see any value on using OP's Houdini ROP extract/render/publish integration.
For example, what this guy did https://github.com/fxnut/hou_farm makes sense: it simplifies the submission to Deadline by not having to use the Deadline ROP workflow and abstracting some of the common settings for IFD/ASS extraction. That's a pretty old repo and the Deadline ROP is more advanced now as it supports Fetch ROPs but the idea is useful and it's directly oriented at a Houdini production workflow -> using the FARM!
So again, why are we trying to solve the problem when we have the solution already in the SubmitDeadlineROP.py module of Deadline?

For example, what this guy did https://github.com/fxnut/hou_farm makes sense: it simplifies the submission to Deadline by not having to use the Deadline ROP workflow and abstracting some of the common settings for IFD/ASS extraction. That's a pretty old repo and the Deadline ROP is more advanced now as it supports Fetch ROPs but the idea is useful and it's directly oriented at a Houdini production workflow -> using the FARM!

I feel like that linked repo is an exact argument/case for what we're doing here now instead of against. Or at least, I mean when referring to at least defining logic to figure out the ordering of ROPs since that repository also implements that logic, e.g. here.
If we want to be creating Deadline jobs, etc - in order, I feel like at the very least we'll need to know what jobs we might need (or if we somehow want to really really want to patch into whatever code Deadline provides itself) we'll still need to figure out what belongs where, etc. for publish job dependencies.

So again, why are we trying to solve the problem when we have the solution already in the SubmitDeadlineROP.py module of Deadline?

You're bringing in hqueue and PDG here as well - I feel like those are also both things that the Deadline code does not 'work with' (even though Houdini does offer its own TOPs node?)

If you want to run a local execution of ROP nodes, you would use Houdini's HQueue and just click the vanilla "Render" node on any of the nodes you want to cook. That already handles the dependencies, switches, fetches, bypasses... and it already has everything you'd need to run some local tests before you submit your job to the farm (very rarely one would cook a ROP graph locally, it's mostly just for testing and validating). Moreover Houdini will probably try to be smart in certain situations about parallel execution and multi-threading on HQueue. Abstracting that logic away and controlling the order and render of the nodes locally I think is regressive and not even on-par with the existing Houdini feature set. No one should be having to mix workflows when they want to just do a local execution vs when they want to publish. I shouldn't need to publish my outputs when I test a ROP render!

This has a totally valid point. But:

Allow an artist to just do regular renders without publishing. That's totally possible already, clicking 'render' on ROP node or ROP chain of nodes in /out works fine, generates no publish.
Similarly, you can use hqueue now - it just won't publish.
Similarly, you can use deadline submissions now - it just won't publish.

However I see no mention here of if you want all of these to just be regular Houdini workflows, which as you can see they can be. When/how would you then define what the user publishes? And, what's the easiest workflow from a Houdini artist perspective where they wouldn't always need to go through five hoops just to end up publishing after a render.
The whole intent it to smooth out that so that if they want to render and publish, they can do so quickly. I must admit that's 95% of our workflow, create output ASAP - validators capture actual pipeline errors; with the data available in the pipeline the artists can can check and confirm the output is as needed; whether that's the FX artist or the lighting artist they can then all access it and review it.
Sure, you wouldn't want 500 water sims to be published - but I don't see any need for an artist to touch the publisher UI.
Having said that, the next bigger issue is to allow the artist to at some point say "I've generated this, and it's ok as is. - Publish it without rerendering" - That's currently a big missing feature for heavy sims / long renders.
Anyway, I feel like now there's a lot of what if's on the table but none seem to really solve what you are describing to be lacking.
Given the provided options, are you able to maybe provide a TL;DR workflow of what you want it to be like from an artist perspective? How do you want the artists to work 99% of the time. You can comment on how that might work technically but to me that's less crucial to the issue at hand. First is, how do artists want to work, then we try to make that work pipeline-wise, etc.
Of course preferably that way artists want to work is designed in a way that's also automatable (e.g. has some predefined steps they follow).
It could be along the lines of:

Create 10 rops (where? in /out or also elsewhere? what decides where it goes? case-specific?)
Put them in any order (allow branching in /out; if so, do we always want to process all upstream nodes, or skip the bypassed ones or?)
Queue the ROPs to render (where? hqueue? local background? pdf? deadline?)
Wait
Check (preferably this can be 'skipped' so that it can be automated)
Publish

I feel like that linked repo is an exact argument/case for what we're doing here now instead of against. Or at least, I mean when referring to at least defining logic to figure out the ordering of ROPs since that repository also implements that logic, e.g. here.
If we want to be creating Deadline jobs, etc - in order, I feel like at the very least we'll need to know what jobs we might need (or if we somehow want to really really want to patch into whatever code Deadline provides itself) we'll still need to figure out what belongs where, etc. for publish job dependencies.

Yeah but it does it in quite a similar way to how Deadline provides that and at any point is trying to abstract the local execution of the nodes.

You're bringing in hqueue and PDG here as well - I feel like those are also both things that the Deadline code does not 'work with' (even though Houdini does offer its own TOPs node?)

HQueue is just an alternative job scheduler to Deadline and I might be wrong but I believe that's what Houdini uses when you execute locally and in the background. Regardless whether it's HQueue what's used, Deadline doesn't need to "work with" the local execution as it's not required to do that. On the contrary, OP is trying to provide an alternative to local execution of your nodes! As for PDG, there is already a list of provided Schedulers https://www.sidefx.com/docs/houdini/tops/schedulers.html that do the work for you to map your PDG graph into the farm. Do you also think that OP should abstract that away instead of using the existing schedulers?

This has a totally valid point. But:

Allow an artist to just do regular renders without publishing. That's totally possible already, clicking 'render' on ROP node or ROP chain of nodes in /out works fine, generates no publish
Similarly, you can use hqueue now - it just won't publish.
Similarly, you can use deadline submissions now - it just won't publish.

Sure, and that's how it should be, but the problem is that when you want to publish, OP is forcing you to re-render/re-cache the outputs! How do you distinguish both workflows to an artist? How do you explain them that when they need to submit jobs to the farm they use the vanilla Deadline ROP and that when they want to publish they go to OP publish menu and re-render everything?

However I see no mention here of if you want all of these to just be regular Houdini workflows, which as you can see they can be. When/how would you then define what the user publishes? And, what's the easiest workflow from a Houdini artist perspective where they wouldn't always need to go through five hoops just to end up publishing after a render.

The publish process should be an addition on top of the existing workflows that work. The publish existing frames target, as you also said, it's a must have in order for this to play together in a less friction way. Alternatively, provide something similar to what I already did, split the publishing to a separate node so it's explicit when they choose to publish as part of the graph evaluation:

The same way that there's the Shotgrid TOP nodes in PDG so you have explicit control when you are publishing outputs.

Sure, you wouldn't want 500 water sims to be published

Exactly the reason why we need to split the creation of outputs from the publishing.

Given the provided options, are you able to maybe provide a TL;DR workflow of what you want it to be like from an artist perspective? How do you want the artists to work 99% of the time. You can comment on how that might work technically but to me that's less crucial to the issue at hand. First is, how do artists want to work, then we try to make that work pipeline-wise, etc.
Of course preferably that way artists want to work is designed in a way that's also automatable (e.g. has some predefined steps they follow).

I recommend reading this three part blog posts https://www.andynicholas.com/post/customising-houdini-nodes-part-1-hdas-or-dynamic-ui as it gives good insight of some of the problems of providing a pipeline in Houdini and the different ways to wrap the system with custom functionality.
Going through the OP create menu is very non-Houdini friendly as an artist knows what you are creating as soon as you put down the nodes in the node graph (i.e., I don't need to go through the list and choose that I want to publish an Arnold ROP if I have already created an Arnold ROP!).
Similarly, having the OpenPype shelve create menus to create the equivalent OP wrapped node is not intuitive and requires a workflow change that's very error prone because it doesn't bridge the vanilla workflow (I have created all my setup using vanilla nodes and set all the parms, oops, how do I now make them publishable?)
Instead, we should automatically add an "OpenPype" tab on the supported vanilla publishable nodes as soon as they get created so artists just work the way they are used to. Or on the contrary, as explained on that article, if we want to avoid messing up with vanilla nodes we could make the "Create" action wrap the EXISTING node with the custom parms that make that node publishable.
I think this is already a big improvement on wrapping OP on top of the vanilla workflow so it's not as disruptive.
Secondly, we should allow and expect that outputs to come from multiple different ways, either locally or through Deadline or any other custom way the studio has implemented. The publishing process should be less opinionated and able to adapt to that and take any output and publish it to OP.
If we think that the Deadline ROP workflow is non-intuitive and we want to abstract that away (in a similar fashion that https://github.com/fxnut/hou_farm/blob/master/python/hou_farm does), we can provide something like that but that should be separate from the publish workflow, that's just a helper to facilitate the creation of dependency jobs.

Thanks - something good to read!

If we think that the Deadline ROP workflow is non-intuitive and we want to abstract that away (in a similar fashion that https://github.com/fxnut/hou_farm/blob/master/python/hou_farm does), we can provide something like that but that should be separate from the publish workflow, that's just a helper to facilitate the creation of dependency jobs.

Just wanted to quickly comment on this. That, one hundred percent. We should make that simple and if the publisher ends up doing anything with complexity to define any ordering it should basically be using that lib.get_order() logic.
Anyway - will need to read more thoroughly and distill the details - but thanks for putting this together here.

Very quick parm prototype of the kind of things that would be very useful to expose on the ROP nodes by default in an OpenPype/Ayon tab on all the nodes that are publishable. The Deadline tab is just a rip off that hou_farm repo, it could be a good utility as an alternative to the Deadline ROP workflow, but could be something to add in the future (that tab would exist on all ROP nodes, not just the publishable ones)

For cameras or other publishable nodes that need a link to a ROP node to export/publish the tab would simply just have a reference to that ROP node so it's easier to jump directly to the exporter/publisher

Very quick parm prototype of the kind of things that would be very useful to expose on the ROP nodes by default in an OpenPype/Ayon tab on all the nodes that are publishable. The Deadline tab is just a rip off that hou_farm repo, it could be a good utility as an alternative to the Deadline ROP workflow, but could be something to add in the future (that tab would exist on all ROP nodes, not just the publishable ones)

For cameras or other publishable nodes that need a link to a ROP node to export/publish the tab would simply just have a reference to that ROP node so it's easier to jump directly to the exporter/publisher

Then if we wanted to also support this workflow with the OP Publisher UI, it would just list all the nodes that contain this OpenPype Publish tab and show as active the ones that aren't bypassed

I really love the conversations on this PR!
Let's wrap it up, please correct me.

The goal of this PR: In essence, fixing find_rop_input_dependencies  logic, which may be used for something like this PR. In my opinion

Publish in a more Houdini native way: There are some suggestions:
i. Sandwiching vanilla houdini rops with Ayon parameters, example customising-houdini-nodes

ii. Publisher Node: That's capable of running the actual publish logic not just triggering the publisher for the input nodes.

Just for context - I feel like this PR doesn't make sense to merge if it doesn't solve any of point 2). We're much better of having 1) if actually used for 2) or the other PR so that it can run accurate tests in production for it as well. Without doing so it basically means we're only fixing "self publish" and not the regular publishing which would still show the same problems.

So, the only reason to merge this PR if we are able to reuse some of its code for 2 ?

Because we're splitting OpenPype into ayon-core and individual host addons, this PR would have to be re-created to target one of those.
We're closing it down, but we'll he happy for a new PR to ynput/ayon-core or the host addon repository once it's up.


NaN problem of dcrec_seq model

Firstly, I am grateful for your valuable contribution to applying self-supervised learning in the recommendation system.
Problem Description
I discovered that the kl_loss of the DCRec model is yielding a value of nan during the training of the DCRec_seq model.
Steps to reproduce

Run the code by the python main.py --model dcrec_seq
Check out the log of loss, which is [Epoch  49 /  50] loss: 5.5758 cl_loss: 0.0003 kl_loss: nan  at the last epoch

Problem Cause
Upon delving into the code, I found the main reason for the problem is the following code:
expected_weights_distribution = torch.normal(
            self.weight_mean, 0.1, size=mainstream_weights.size()).sort()[0].to(self.device)
        kl_loss = self.kl_weight * F.kl_div(F.log_softmax(
            mainstream_weights, dim=0).sort()[0], expected_weights_distribution, reduction="batchmean")

the expected_weights_distribution is a normal distribution with the mean of self.weight_mean and the std of 0.1. However, we should notice that the value of the expected_weights_distribution, obtained by torch.normal, may be non-positive, which will cause the nan problem during the calculation of KL divergence.
Problem Solution
Due to the problem cause, I modify the code as follows:
        expected_weights_distribution = torch.normal(
            self.weight_mean, 0.1, size=mainstream_weights.size()).sort()[0].to(self.device)
        kl_loss = self.kl_weight * F.kl_div(F.log_softmax(
            mainstream_weights, dim=0).sort()[0], F.softmax(expected_weights_distribution, dim=-1), reduction="batchmean")

I simply apply the softmax function to ensure the value of expected_weights_distribution is positive. However, I have no idea whether the distribution of the handled one is still normal.
I would appreciate your reply.

Hi,
Thanks for issuing this bug. This NaN bug caused by KL Divergence calculation in DCRec was suspected and fixed in the individual repository of the paper: link
And you are right the reason is that torch.normal can have non-positive values for log(). A simple workaround is to truncate these values to a tiny number (like 1e-8):
def cal_kl_1(target, input):
    target[target<1e-8] = 1e-8
    target = torch.log(target + 1e-8)
    input = torch.log_softmax(input + 1e-8, dim=0)
    return F.kl_div(input, target, reduction='batchmean', log_target=True)

I test this and it works well without a noticeable performance decline. I'm not quite sure if applying softmax is another effective workaround. However, I have now updated this bugfix for DCRec here, and please feel free to test and compare. :)

Got it! Thank you for your reply.


Cannot roll over when CFD is already closed

Observed in the testnet maker logs
Relevant log: Cannot roll over when CFD is already closed and NOT NULL constraint failed
Relevant CFDs where this happened (both on 18.07.2022):

9069c43c-b0c9-4fc6-bc9d-d85d828741d6 - taker_id = 064e561cc7d1bb5c80ba7e51a68e9c3e63a2704f6a18a770620e8e6586604031
1db397b3-ebb8-432b-91b5-ff1252031354 - taker_id = c92f72932f6d0aa07e51624e237da9b5980c433ffb44c525487516bd8b8a5520

Behavior:
The CFD gets collaboratively closed by the taker. The collaborative close succeeds on the maker side.
The taker keeps triggering rollovers, indicating that somehow the collaborative close failed after sending the signature to the maker.
The CFDs were opened from the same IP-address by different takers during a load test run of multiple takers using the same wallet extprvkey.

closing until it happens again


Show which files are uploading in the log and in filestatus

My log currently reads:
2021-09-02 10:44:45 sync DEBUG: Created "[ABRIDGED]" on Dropbox
2021-09-02 10:44:48 sync DEBUG: Created "[ABRIDGED]" on Dropbox
2021-09-02 10:44:48 sync INFO: Syncing ↑ 75/82

But, regarding the last line, it looks like it's uploading something, but which file is it uploading?
It doesn't say.
Also:
$ maestral filestatus .
uploading

I propose to show which file it is uploading both (i) in the log in and (ii) in filestatus.

In a sense this is already possible:

If you set the log level to DEBUG, logs for individual files are shown. This is however very verbose, giving details on conflict resolution, etc.
The command maestral activity will show a live view of all files currently being uploaded or downloaded, together with progress for large files.

The maestral filestatus command shows the status of an individual file or folder, similarly to what a badge in a file manager might show. maestral filestatus '~/Dropbox (Maestral)/my file' will show the status of just that file.
I do see a use case for showing logs with a file level granularity at a level of INFO, without all of the noise of DEBUG.

I didn't know maestral activity.
Thanks!

A display of the maestral activity output in an expandable GUI menu would be fantastic as a future enhancement/feature!


Clean up unused formatters

These have been sitting around, commented out for a while, let's remove them to tidy up the code base a bit.

I've left the implementations intact though I'm open to removing them too.

It seems like the implementations were active a year or more ago and then the commands were disabled.
In terms of removing the implementations, I don't really see the use of the PIRST_* ones so they could be removed in a second PR IMO. CAPITALIZE_FIRST_WORD possibly makes sense to keep though.


ODBC External Database

Description
Howdy 👋,
Love the project! Do you have any plans to add support for external databases over ODBC? That would enable most databases to be queried with GlareDB and specific databases could then have their own driver for performance/specific features.

We can definitely look into this. Is there a particular database you'd want to see supported through odbc?

Amazing 🙏
IBM Db2 would be top of the list. Followed by SQL Server and Oracle.

Sql server and Oracle are top of my list.


hashable-<IP_ADDRESS> will break build purity of Haskell packages

Describe the bug
hashable-<IP_ADDRESS> has been released on Hackage and may be propagated into a Stackage LTS and the Haskell package set here some time later. It includes a package flag arch-native that defaults to True, which means it enables -march=native in order to harvest SIMD speedup when compiling the vendored xxhash cbits.
This will cause the binaries built by Hydra depend on the specific CPU models of the builders. AFAIK in nixpkgs, the C/C++ compiler wrapper scripts do not contain logic to filter out -march=native, and packages that involve -march=native should be explicitly marked with impureUseNativeOptimizations. Given hashable is a very common Haskell dependency, before the real damage is caused downstream, I'd like to raise awareness of the potential issue here so the nixpkgs Haskell maintainers can apply the appropriate overrides when hashable is bumped in the package set.
Upstream issue (closed at the time of writing this one): https://github.com/haskell-unordered-containers/hashable/issues/308
Notify maintainers
@domenkozar @maralorn @sternenseemann

Add a :+1: reaction to issues you find important.

Update: after taking a quick look at output of $CC -march=native -v in a nix shell, the -march=native seems to be automatically rewritten to '-mtune=generic' '-march=x86-64'. So it seems we're good here, feel free to close this issue any time, but it would be nice for someone more familiar with wrapper logic to take a closer look to confirm.

Yes, that is correct. Also it does seem unlikely we'll end up with <IP_ADDRESS> in an LTS snapshot. Current LTS seems to be sticking to <IP_ADDRESS> for some reason and Nightly is already using <IP_ADDRESS>.

Can this be closed? Both LTS and Nightly use versions that are not affected by this: https://www.stackage.org/lts-23.1/package/hashable-<IP_ADDRESS>
Since they both use versions >=<IP_ADDRESS>, which is the one that disables arch-native by default: https://hackage.haskell.org/package/hashable-<IP_ADDRESS>/changelog

Yes, we are using <IP_ADDRESS>, but will be jumping to <IP_ADDRESS> (which is already being used for GHC 9.10) when upgrading to LTS 23.


video_player I cannot play m3u8 video with the IOS platform

video_player I cannot play m3u8 video with the IOS platform

Duplicate of #48670


save the visualization

Hello, thank you very much for your excellent work, successfully trained and tested the model using the code on github, but wondering how you can visualize the results of the predictions and save them?

Did you solve it?

No, I tried to use the visualization method in PCDET, but it doesn't look like the 3D box is aligned

I wonder if you've solved it. Would you mind sharing the solution?


White space / Indentation checking configuration

Suggestion: Expose white space and indentation checking options, to allow customisation of which rules are applied when "apply strict whitespace rules" is chosen. This would allow white space checks to be more useful in places with different coding standards.
Having the white space rule set is very helpful for projects with many developers, to highlight and encourage coding standards. However, under JSLint (like many of its features), the option is a boolean black box where you either apply the JSLint way or turn it off.
For example, the Switch statement is commonly used in two forms, depending on what coding standards a company may have chosen to adopt:
// Java coding standards indent (and JSLint)

switch (foo) {
case "bar":
    break;
case "baz":
    break;
default:
    break;
}

// Alternative (and common) indentation style

switch (foo) {
    case "bar":
        break;
    case "baz":
        break;
    default:
        break;
}

If you are in an environment where the latter standard is in place, the white space check becomes unusable because it will always fail on switch indentation, and therefore the only choice is to disable it completely, even though the rest of the white space checks may be useful and desired by the team.
Other examples of "whitespace" rules I think would have value being exposed as options are the ones dealing with anonymous functions:
function() {} // common
function () {} // crockford

(function (){})() // common
(function (){}()) // crockford

And paren spacing:
if ( something ) {
if (something) {

Being able to turn "use whitespace checking" on, but have the ability to configure what whitespace checking "means" for your particular use case, dev standards and work environment would be very helpful in JSHint being a usable style checking tool for improved code quality.
Dummy configuration example:
var whiteSpaceSettings = {
    switchIndent: false,
    anonFnSpace: false,
    anonFnWrap: false,
    parenSpace: true
}

Good luck with the project, I think there's a lot of value in a community driven version of JSLint.
Ian

+1 for this, especially "anonFnSpace: false".

Any updates on this?  Without this I basically can't use whitespace checking since everyone on my team (including me) omits the space between "function" and its parenthesis.
I can take a crack at it if you want.

This fix (is we want it to be flexible) requires some substantial refactoring. We're working on that.

thanks!
On Dec 29, 2011, at 12:24 PM, Anton Kovalyov wrote:

This fix (is we want it to be flexible) requires some substantial refactoring. We're working on that.

Reply to this email directly or view it on GitHub:
https://github.com/jshint/jshint/issues/28#issuecomment-3304830


handle preflight requests

Added to the cors code to allow for preflight requests because getting preflight errors when sending request from Cyrus' latest frontend.

Good work handling this.


When can we expect USAID data to be updated?

Describe the bug
USAID refreshed most of their data on the 11/02/2021, this bulk update was to fix a bug.
The datastore doesn't yet have their updated data e.g.

Here we can see the iati-identifiers are not properly formed.
The new data has correctly formed identifiers.
To Reproduce
Steps to reproduce the behavior:

Search for activity id: US-GOV-1-%3CElement%20'iati-identifier'%20at%200x0000019D25640728%3E
Compare to updated activity on d-portal: http://d-portal.org/ctrack.html?text_search=Costes administrativos&reporting_ref=US-GOV-1#view=act&aid=US-GOV-1-01FB3AFBBEB3
Or the xml file on the Registry: https://iatiregistry.org/dataset/usaid-multiple-13
See error

Expected behavior
That the datastore takes less than a week to update.
When can we expect the data in the datastore to be refreshed?

This is related to a bug in the incremental updating process. This should be picked up soon. We will monitor this and are working on updating our monitoring process so we are able to catch this specific service should if fail.


sqlite3.OperationalError: no such table: terms_to_entries

hi
im running Python 3.8.9 .
is there any fix to this error?
python cpe_search.py -q "Sduo 1.8.2"

Traceback (most recent call last):
  File "cpe_search.py", line 1097, in <module>
    results[query] = search_cpes(query, args.number, -1, config).get('cpes', [])
  File "cpe_search.py", line 991, in search_cpes
    cpes = _search_cpes([query], count=count, threshold=CPE_SEARCH_THRESHOLD_ALT, config=config)
  File "cpe_search.py", line 660, in _search_cpes
    db_cursor.execute(db_query, (word, ))
sqlite3.OperationalError: no such table: terms_to_entries

Hi, cpe_search changed its database format some time ago. I think that's the reason for your error.
I think performing an update of the local database with -u should fix the error. Alternatively, you can also download a prebuilt version from my other repository search_vulns: https://github.com/ra1nb0rn/search_vulns/releases/download/v0.5.0/cpe-search-dictionary.db3
Let me know if this fixed your issue or if you need any more help.
PS: I think you're looking for Sudo and not Sduo ;)

yes, Sduo  is my typo, thanks.
after downlaod cpe-search-dictionary.db3
I can run like this
python cpe_search.py -q "Sudo 1.8.2"
cpe:2.3:a:sudo_project:sudo:1.8.2:*:*:*:*:*:*:*
[('cpe:2.3:a:sudo_project:sudo:1.8.2:*:*:*:*:*:*:*', 0.8812565009687703),
 ('cpe:2.3:a:sudo_project:sudo:1.8.2:*:*:*:*:*:*:*', 0.674613486972654),
 ('cpe:2.3:a:sudo_project:sudo:1.6.8:p10:*:*:*:*:*:*', 0.6525629975748669)]

But updating DB still error
python cpe_search.py -u
[+] Getting NVD's official CPE data (might take some time)
[-] No API Key found - Requests will be sent at a rate of 5 per 30s. To lower build time, consider getting an NVD API Key.
[-] Failed updating the local CPE database!

after apply api key
look good but still not update success
python cpe_search.py -uv -k ******
[+] Getting NVD's official CPE data (might take some time)
[+] API Key found - Requests will be sent at a rate of 25 per 30s.
Got the following exception when downloading CPE data via API: Cannot connect to host services.nvd.nist.gov:443 ssl:default [Connect call failed ('<IP_ADDRESS>', 443)]
Got the following exception when downloading CPE data via API: Cannot connect to host services.nvd.nist.gov:443 ssl:default [Connect call failed ('<IP_ADDRESS>', 443)]
Got the following exception when downloading CPE data via API: Cannot connect to host services.nvd.nist.gov:443 ssl:default [Connect call failed

I'm glad downloading the prebuilt database worked for you and solved the issue.
Regarding the failing update, I don't think there is much I can do. The underlying issue is that the tool cannot connect to the NVD's API on a network level. This is not directly related to the cpe_search, however. I've been experiencing similar problems from time to time, but can't reproduce it now. How I usually get around this problem is trying a couple of times (with an API key) or using a VPN / different network. What might also help is decreasing the rate limit here, even though it's already within the NVD's guidelines: https://github.com/ra1nb0rn/cpe_search/blob/master/cpe_search.py#L343
Otherwise, my tool search_vulns rebuilds its CPE database weekly, so you can always download a recent version from https://github.com/ra1nb0rn/search_vulns/releases/latest/download/cpe-search-dictionary.db3 . I'm sorry I couldn't be of more help and hope my suggestions work for you. I don't really know how else to fix this problem, since I perceive it to be a fault of the NVD API.
Let me know if these suggestions work out for you or if you have any more insights / ideas or issues.

Hi @er1chsu , since I haven't heard back from you, I'm closing this issue.
I'm sorry if I couldn't be of much help. As I've said though, I perceive this to be an issue with the NVD API.
I hope my suggested workarounds have helped you. If anything is still unclear or needs to be resolved, respond here and we'll reopen the issue.


Fix misspelling

 Issue by awulkan
Monday Sep 18, 2017 at 19:20 GMT
Originally opened as https://github.com/Wyamio/Wyam.Web/pull/52

awulkan included the following code: https://github.com/Wyamio/Wyam.Web/pull/52/commits

 Comment by daveaglick
Friday Oct 06, 2017 at 20:10 GMT

Thanks :)


feat: added support for A21 and Amazon Titan models via bedrock api

Closes #1076
📑 Description
Included support for A21 & Amazon Titan Model through the bedrock api, using a case structure to enable future flexibility to continue adding other models available. As Bedrock supports quite a few models not adding them all in a single merge.
✅ Checks

[ ] My pull request adheres to the code style of this project
[ ] My code requires changes to the documentation
[ ] I have updated the documentation as required
[ ] All the tests have passed

ℹ Additional Information

@AlexsJones did you get a chance to review this for the changes and if it is ok to merge now ?

I need to test it first, will try to get around to it ASAP

I need to test it first, will try to get around to it ASAP

@AlexsJones just adding some screenshots of the runs if that helps.

If you're looking for a workaround, would suggest using LiteLLM to proxy Bedrock

Can you respond to comments and we will merge this next week @awsyshah ?

made the key suggest improvement of moving some of the constants to config. The other vars are as per the API or not need outside of the case and hence declared within it. Please review.

comments are not fixed

comments are not fixed

Hi @JuHyung-Son ,
Sorry for the trouble there, can I request which are the specific ones you want changed.
The first action of moving the constants like maxTokens to config is done right ? or is my understanding / implementation incorrect ?
For Comment 2 , the stop_reason , that call used is as per the documentation from :https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-anthropic-claude-messages.html
For Comment 3:  The type you have commented on is only used once and that is if it is a a21 model so its defined within the area so as its only ever used there. The rest of the code has not use of it , is it still better to just define it up top even is only ever to be used if the model is one of the A21 one and may not be used otherwise ?
Let me know and I will make the changes so we can have this merged.
Thanks


Add Azure terraform/ansible support

I do not have access to Azure account, assigned to you @jrx then

implemented


feat: Support electron 7

Closes https://github.com/getsentry/sentry-electron/issues/193

Thanks for this.
I think we're going to have to do some fudging/merging with the type definitions from multiple Electron versions to get this compiling.

@timfish I've just done that. The build is now passing.

Cool thx!


Can't apply custom fonts and insert images

After downloading fonts and mapping them in the pubspec file, they wouldn't display on my app, still no errors popped up

Hi @gustavo440
could you please provide your updated  flutter doctor -v,
your pubspec.yaml and a minimal code sample to reproduce the problem?
Thank you

Flutter Doctor
`[flutter] flutter doctor -v
[✓] Flutter (Channel dev, v1.12.5, on Linux, locale en_US.UTF-8)
• Flutter version 1.12.5 at /home/gustavo.alves/flutter
• Framework revision 2c7af1e24e (2 days ago), 2019-11-20 12:18:44 -0800
• Engine revision 176f563296
• Dart version 2.7.0
[!] Android toolchain - develop for Android devices (Android SDK version 29.0.2)
• Android SDK at /home/gustavo.alves/Android/Sdk
• Android NDK location not configured (optional; useful for native profiling support)
• Platform android-29, build-tools 29.0.2
• Java binary at: /snap/android-studio/81/android-studio/jre/bin/java
• Java version OpenJDK Runtime Environment (build 1.8.0_202-release-1483-b49-5587405)
! Some Android licenses not accepted.  To resolve this, run: flutter doctor --android-licenses
[✓] Chrome - develop for the web
• Chrome at google-chrome
[✓] Android Studio (version 3.5)
• Android Studio at /snap/android-studio/81/android-studio
• Flutter plugin version 41.1.2
• Dart plugin version 191.8593
• Java version OpenJDK Runtime Environment (build 1.8.0_202-release-1483-b49-5587405)
[✓] VS Code (version 1.40.1)
• VS Code at /usr/share/code
• Flutter extension version 3.6.0
[✓] Connected device (2 available)
• Chrome     • chrome     • web-javascript • Google Chrome 78.0.3904.108
• Web Server • web-server • web-javascript • Flutter Tools
! Doctor found issues in 1 category.
exit code 0`
pubspec.yaml
`name: expense_app
description: A new Flutter project.
version: 1.0.0+1
environment:
sdk: ">=2.1.0 <3.0.0"
dependencies:
flutter:
sdk: flutter
intl: ^0.16.0
flutter_masked_text: ^0.8.0
cupertino_icons: ^0.1.2
dev_dependencies:
flutter_test:
sdk: flutter
flutter:
uses-material-design: true
assets:

assets/images/waiting.png

fonts:
- family: Roboto
fonts:
- asset: assets/fonts/Roboto-Regular.ttf
- asset: assets/fonts/Roboto-Bold.ttf
weight: 700
`
And here is the sample code:
`import 'package:expense_app/widgets/add_transaction_form.dart';
import 'package:flutter/material.dart';
import 'dart:math';
import './models/transaction.dart';
import './widgets/chart.dart';
import './widgets/transaction_list.dart';
void main() => runApp(MyApp());
class MyApp extends StatelessWidget {
@override
Widget build(BuildContext context) {
return MaterialApp(
theme: ThemeData(
fontFamily: 'Roboto',
),
title: 'Sample App',
home: MyHomePage(),
);
}
}
class MyHomePage extends StatelessWidget {
@override
Widget build(BuildContext context) {
return Scaffold(
appBar: AppBar(
title: Text('Sample App'),
),
body: Center(
child:
Text('Should be in Roboto Regular'),
),
);
}
}
`

Hi @gustavo440
just tested on the latest master and seems to work just fine both on mobile and web
sample code

can you please double check your folder asset
thank you

I know it sounds stupid, but I'm really stuck on this. Can you please take a look at my repo and tell me what I'm doing wrong?
https://github.com/gustavo440/flutter_sample_app.git

Hey @iapicca, I've just found out that cleaning and reloading my flutter project solved it. In the end there was nothing wrong with my code.
Thank you very much for all the help :smiley:

Hi @gustavo440
I'm glad to hear that,
thank you for letting me know


Validate gas_per_byte

As per the specs, a transaction is invalid if bytePrice != gasPrice * GAS_PER_BYTE. We need to enforce this rule in tx validation.

We want this validated at the base tx format / consensus layer for now instead of just the txpool to mitigate the following concern from @adlerjohn:

It's not the fee deduction that's troublesome (that's easy and as you say already done), but rather that the logic for optimal packing across multiple dimensions is non-trivial and therefore a centralization concern. More effective packers will accumulate more stake over time. Whether this is actually a problem long-term remains to be seen (similar to MEV bleeding into the work validators need to do to create optimal blocks), but if we keep things simple then that gives us time to explore the more complex solutions at our leisure.


Fcrepo4 mapping

Change deposit pipeline to use RDF for Premis events.

Trying to merge against the wrong branch


Modify script that executes BlueXolo to enable or not the chat bot

Implementation Done, need to review with Fer before submitting pull request, this could fix issue #393


feat(pipeline executions/front50) : Added code to save multiple pipelines at once to sql database.

feat(pipeline executions/front50) : Added code to save multiple pipelines at once to sql database.
This is part of: spinnaker/spinnaker#6147.
Enhanced PipelineController.java to
Added new rest api method bulksave(List pipelineList, Boolean staleCheck)
This method accepts a list of pipelines json.
This method checks each and every pipeline for authorization.
If the pipeline has valid authorization, then it will be saved to a savedPipelineList.
If the pipeline do not have a valid authorization, then it will be saved to a errorPipelineList
and an error message is also populated.
All the savedPipelinesList will be saved to sql database using bulk insert.
This method returns a Map object having the below data:
{
“Successful”: ,
“Failed”: ,
“Failed_list”: [<array of failed pipelines - (application, pipeline name, etc) and the error message]
}
Enhanced AuthorizationSupport.java to
Added code to accept the pipeline list and authorize the pipelines.
Enhanced PipelineControllerSpec.java to
Added bean of FiatPermissionEvalutor.java as it is used in the PipelineController constructor.
Enhanced PipelineControllerTck.groovy to
Added bean of FiatPermissionEvalutor.java as it is used in the PipelineController constructor.

@robzienert @ajordens ....Can you please review this PR


Feature/578 attributes fix when editing

Make sure that the edited attributes get saved when another gets deleted.

:tada: This PR is included in version 2.13.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket: