feat(aws-ecs): add ScheduledEc2Task L3 construct

Defined a new construct to enable creating a ec2 task to be run at a scheduled interval
Fixes #2352

Pull Request Checklist

[X] Testing

Unit test added (prefer not to modify an existing test, otherwise, it's probably a breaking change)
CLI change?: coordinate update of integration tests with team
cdk-init template change?: coordinated update of integration tests with team

[X] Docs

jsdocs: All public APIs documented
README: README and/or documentation topic updated

[X] Title and Description

Change type: title prefixed with fix, feat will appear in changelog
Title: use lower-case and doesn't end with a period
Breaking?: last paragraph: "BREAKING CHANGE: <describe what changed + link for details>"
Issues: Indicate issues fixed via: "Fixes #xxx" or "Closes #xxx"

[ ] Sensitive Modules (requires 2 PR approvers)

IAM Policy Document (in @aws-cdk/aws-iam)
EC2 Security Groups and ACLs (in @aws-cdk/aws-ec2)
Grant APIs (only if not based on official documentation with a reference)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license.

To be clear, this is not possible in Fargate, correct?

To be clear, this is not possible in Fargate, correct?

Correct, to use Fargate, I would need to add a network configuration as networkmode is awsvpc. This is not currently supported by Cloudformation.

To be clear, this is not possible in Fargate, correct?

Correct, to use Fargate, I would need to add a network configuration as networkmode is awsvpc. This is not currently supported by Cloudformation.

What exactly isn't supported by CFN? We currently allow for awsvpc network mode for both Fargate Services and regular EC2 Task Definitions.

@SoManyHs

What exactly isn't supported by CFN?

The NetworkConfiguration property is currently missing in CloudFormation and needed for Fargate. See the differences between Amazon CloudWatch Events Rule EcsParameters (CloudFormation) and EcsParameters (CloudWatch PutTargets

To be clear, this is not possible in Fargate, correct?

Correct, to use Fargate, I would need to add a network configuration as networkmode is awsvpc. This is not currently supported by Cloudformation.

What exactly isn't supported by CFN? We currently allow for awsvpc network mode for both Fargate Services and regular EC2 Task Definitions.

CloudFormation currently only supports clusters, services and taskdefinitions: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/cfn-reference-ecs.html.
TaskDefinitions contain a network mode, and for EC2, we can use none, bridge or the many others, but in order to use awsvpc, you need to create a NetworkConfiguration, which is not supported in the TaskDefinition, but is supported in the Service.
For this construct, we're creating a scheduled task (that is standalone task, not backed with a service), and for Fargate to support this feature, we'd need to have a CloudFormation create the concept of a standalone Task (not Task Definition) that can take a NetworkConfiguration property.

@pkandasamy91  Ah, right -- I was more referring to the CFN support on CWE mentioned by @jogold (thanks for the comment). Appreciate the clarification.

Squashed all commits

Unit tests have been updated to handle merged changes.

Doh, more merge conflicts.

As per my discussion with @rix0rrr, created a new module (aws-ecs-patterns) and added the ScheduledEc2Task to that module. I'll refactor other L3 ECS constructs into this module in a separate PR.

I notice you made these changes on the master branch in your repo. That might cause you some trouble when trying to resync with upstream master later on. Don't forget to git reset --hard :).


Display "Function definition for '***' not found.C/C++" ERROR

Type: Bug
Display "Function definition for '***' not found.C/C++" ERROR
, although it has been defined.
VS Code version: Code 1.78.2 (b3e4e68a0bc097f0ae7907b217c1119af9e03435, 2023-05-10T14:47:05.613Z)
OS version: Linux x64 5.11.0-051100-generic
Modes:
Sandboxed: No

System Info

Item
Value

CPUs
Intel(R) Xeon(R) W-11955M CPU @ 2.60GHz (16 x 2600)

GPU Status
2d_canvas: enabledcanvas_oop_rasterization: disabled_offdirect_rendering_display_compositor: disabled_off_okgpu_compositing: enabledmultiple_raster_threads: enabled_onopengl: enabled_onrasterization: enabledraw_draw: disabled_off_okvideo_decode: enabledvideo_encode: disabled_softwarevulkan: disabled_offwebgl: enabledwebgl2: enabledwebgpu: disabled_off

Load (avg)
2, 1, 0

Memory (System)
30.57GB (24.31GB free)

Process Argv
.

Screen Reader
no

VM
0%

DESKTOP_SESSION
ubuntu

XDG_CURRENT_DESKTOP
Unity

XDG_SESSION_DESKTOP
ubuntu

XDG_SESSION_TYPE
x11

Extensions (27)

Extension
Author (truncated)
Version

vscode-bazel
Baz
0.7.0

doxdocgen
csc
1.4.0

vscode-markdownlint
Dav
0.50.0

githistory
don
0.6.20

gitlens
eam
13.6.0

LogFileHighlighter
emi
2.16.0

shell-format
fox
7.2.5

latex-workshop
Jam
9.10.0

better-cpp-syntax
jef
1.17.2

vsc-python-indent
Kev
1.18.0

vscode-leetcode
Lee
0.18.1

git-graph
mhu
1.30.0

vscode-docker
ms-
1.25.0

vscode-language-pack-zh-hans
MS-
1.78.2023051009

python
ms-
2023.8.0

vscode-pylance
ms-
2023.5.20

jupyter
ms-
2023.4.1011241018

jupyter-keymap
ms-
1.1.0

jupyter-renderers
ms-
1.0.15

vscode-jupyter-cell-tags
ms-
0.1.8

vscode-jupyter-slideshow
ms-
0.1.5

cpptools
ms-
1.15.4

copy-file-name
nem
1.2.0

autodocstring
njp
0.6.1

cmake
twx
0.0.17

markdown-all-in-one
yzh
3.5.1

vscode-proto3
zxh
0.5.5

It looks like this is caused by the C++ extension. Please file the issue to the C++ extension repository. Make sure to check their issue reporting template and provide them relevant information such as the extension version you're using. See also our issue reporting guidelines for more information.
Happy Coding!


CBA-chrisda to Main

Learn Build status updates of commit 0676639:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

exchange/docs-conceptual/app-only-auth-powershell-v2.md
:white_check_mark:Succeeded
View (exchange-ps)

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the learn.microsoft.com contributor guidesPost your question in the Learn support channel


Broadcasting seems to mess with gradients

Hello,
I'm not entirely sure this is a bug or if this is me just misusing the API.
I am training a pretty simple full connected layer with some data irrelevant to this issue (issue persists with small amount of random data). My inputs and training targets are of shape [x, 773] and [x, 1] where x is the amount of samples.
I have noticed that the network really struggles to converge when having the shape of the target be [x] instead of [x, 1].
Here is the code of my training.
Notice that on line 56 I set the training target to random data, for the sake of this example.
Here is the loss if the y_full Tensor is changed to shape [x, 1] instead of being [x]:

And here is the loss if the y_full Tensor is of shape [x] (remove the .view([-1, 1]) from line 57).

I assume this is due to broadcasting as the output tensor of the network (forward on line 63) is of shape [batch_size, 1].
Could this be?

Maybe you want to try and reproduce the same behavior with the python api to see if it's something wrong with this crate vs with the way PyTorch is used in your example.


Github module: Empty notifications

I can't seem to customize the empty notifications on the new github module.
[module/github]
type = internal/github
token = "something"

label = "%notifications%"
format-prefix = ""

This setting above alone will not print anything to the bar which is nice as I would prefer to not display anything if I don't have any notification. But unless I misread the code, I should be reading '0'.
Changing the value of empty-notifications to anything doesn't make the value show on the bar either.

Also related to the github module, is there a way I can use an environment variable value there? Ideally I would like to keep my polybar configuration public and using the token value from an environment variable would allow me to do that.

@bpinto The config parameter empty-notifications = true would display the 0 count. This defaults to false.

Thanks!

not running displaying entire the module if there isn't any notification seems better, no ?

it is not

@eoli3n yes, or at least clear the label (which will hide it) to allow more stuff to be added other than %notifications%.. untli more tokens are implemented

I think i did'nt understand well your answer so,
is there a way to hide the entire module when there isn't any notification ?
with
empty-notifications = "true"

it displays  0
with
;empty-notifications = "true"

it displays

actually, by setting format-underline, format-prefix and format-prefix-foreground, it's working without your commit :)
[module/github]
type = internal/github
token = ${env:GITHUB_API_TOKEN}

;label = "%{u#2aa198}%{F#5b} %{F-}%notifications%%{u-}"
label = "%notifications%"
label-prefix =
format-underline = #2aa198
format-prefix = " "
format-prefix-foreground = #5b
;empty-notifications = true

@eoli3n yeah that's what i meant, but it felt like an unnecessary workaround


Sequel instrumentation adapter

Convert Datadog instrumentation for Sequel to OpenTelemetry.
See

Datadog porting guide
Semantic Conventions
Datadog implementation

Datadog porting guide

This Datadog porting guide link is broken, updated url is https://github.com/open-telemetry/opentelemetry-ruby/blob/main/instrumentation/datadog-porting-guide.md (I think)


Show a link to recent expenses

Currently, if there aren't more than 3 pending expenses, we don't show a link to /collective/expenses. Add a link to the bottom to always take you to recent expenses

And would be even cooler if this page doesn't have any expenses that we give you suggestions on what you can file...

I'd argue that we should always show the 3 latest expenses irrespective of their status (no reason to limit this to the pending ones).
i.e. the 2 last one could be pending, and the 3rd one is approved and paid, or if there are no pending expenses, just show the 3 latest approved/paid expenses.

That's a good idea
On Wed, Mar 15, 2017 at 7:14 PM Xavier Damman<EMAIL_ADDRESS>wrote:

I'd argue that we should always show the 3 latest expenses irrespective of
their status (no reason to limit this to the pending ones).
i.e. the 2 last one could be pending, and the 3rd one is approved and
paid, or if there are no pending expenses, just show the 3 latest
approved/paid expenses.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/OpenCollective/OpenCollective/issues/326#issuecomment-286909074,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AF0vPVQp_5aAmySx44bnfM_Fojyn5yGPks5rmHDhgaJpZM4MeoGa
.

encapsulated here: https://github.com/OpenCollective/OpenCollective/issues/336


Get getGovernedAssets (full list) working in GE Omas

Continue testing on getGovernedAssets working in GE Omas.
This is the second capability needed to support tagsync/ranger

Now able to retrieve a list of assets and classifications. For example getAssets on GEOMAS now returns
{
    "relatedHTTPCode": 200,
    "governedAssetList": [
        {
            "guid": "6b6ed55c-bdda-49aa-a2b6-f0afae6c9c54",
            "type": "Database",
            "assignedGovernanceClassifications": [
                {
                    "name": "Confidentiality",
                    "attributevalues": {}
                }
            ],
            "fqName": "MyResourceWithAnotherName <placeholder>"
        },
        {
            "guid": "3d76cd7c-3e0a-4c24-8525-6ef1a4675c57",
            "type": "Database",
            "assignedGovernanceClassifications": [
                {
                    "name": "Confidentiality",
                    "attributevalues": {}
                },
                {
                    "name": "Retention",
                    "attributevalues": {}
                }
            ],
            "fqName": "MyResourceWithAnotherName <placeholder>"
        },
        {
            "guid": "b8a0ac52-da7d-4351-a10d-8f9615399eb0",
            "type": "Database",
            "assignedGovernanceClassifications": [
                {
                    "name": "Confidentiality",
                    "attributevalues": {}
                }
            ],
            "fqName": "MyResourceWithAnotherName <placeholder>"
        }
    ]
}

Properties are not correctly getting decoded (though code is in place - not working!) - will open a seperate todo item to track next step
This is the first stage only


Issue

Code doesn't work even though followed instructions to a tee...
Challenge Target the same element with multiple jQuery Selectors has an issue.
User Agent is: Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.96 Safari/537.36.
Please describe how to reproduce this issue, and include links to screenshots if possible.
My code:

jQuery Playground

#left-well

        <button class="btn btn-default target" id="target1">#target1</button>
        <button class="btn btn-default target" id="target2">#target2</button>
        <button class="btn btn-default target" id="target3">#target3</button>

#right-well

        <button class="btn btn-default target" id="target4">#target4</button>
        <button class="btn btn-default target" id="target5">#target5</button>
        <button class="btn btn-default target" id="target6">#target6</button>

Dear @jstacksx thanks for reporting this issue.
Unfortunately your code is incorrect since, as the challenge state:

Only add one class with each of your three selectors.

If you are in need of assistance please don't hesitate to ask for help in one of the Gitter's official chat

Hi @jstacksx
Thanks for reporting this issue.
Here are some next steps that you should follow before we can help you with your query:

Check these troubleshooting steps first.
Please use the Chat Rooms or try looking through our forum for help with a specific challenge and any coding assistance.

IMPORTANT:
Please NOTE that this tracker is for reporting bugs and enhancements to freeCodeCamp's code base only.
Happy Coding!


scratchattach._project.Project object at 0x0000016F34008F40

I get this when i try to fetch all my projects do anyone know how to fix this

Whats your code?

Umm thats a class object

scratchattach creates a Project class object for each project so it can be easier to get the project info for example
if you are searching for projects using
import scratchattach as scratch3

result = scratch3.search_projects(query="query", mode="trending", language="en", limit=40, offset=0)

you will get a list back looking like this
[<scratchattach._project.Project object at 0x1061656d0>, <scratchattach._project.Project object at 0x106794510>, <scratchattach._project.Project object at 0x106794610>, <scratchattach._project.Project object at 0x1067947d0>, etc....

to get the info from the list use this
result[0].<Functions or variables>

All of the Functions can be found here

I hope that helps 😄

The attributes and method of the scratchattach._project.Project class are documented here: https://github.com/TimMcCool/scratchattach#projects

Hey tim this issue needs to be labeled as "Support"

This repo currently doesn't have a support label, I'll create one
Anyways I'll close this issue because of inactivity

Thank you so much for helping me


About build and deploy sppkg file, how to discriminate uat and production package ?

Category

[x] Question
[ ] Typo
[ ] Additional article idea

Question
I have only one SharePoint online, that's two site collections

sites/UAT
sites/production.

When I develop spfx and only want to test the UAT environment, how can I deploy to appcatalog? And does not affect the production environment
when cmd run "gulp package-solution --ship", modify id,name in config/package-solution.json every time ?

I would use Site Collection App Catalog in your UAT site if your environments are in the same tenant. This will allow you to have a Production app living in the Tenant level, and then you can separately deploy the app into your UAT site collection.
https://docs.microsoft.com/en-us/sharepoint/dev/general-development/site-collection-app-catalog

Vert thanks. That's great


Support Debian 10

Pull Request Checklist

Description

Add Debian 10 support
Related Issue

Fixes #1121
Motivation and Context

Sensu Go added support for Debian 10.

This is WIP because right now the only Puppet repos for Debian 10 are nightlies.

Great work @treydock

Removed WIP as now am testing Debian 10 against just Puppet 6.

Travis failures are unrelated and due to a bug introduced in Puppet 6.8.0.
https://tickets.puppetlabs.com/browse/PUP-9975


Support Dark Theme

Can you add support for dark theme?

I think the best option for this at the moment is shinyswatch.theme.darkly()
https://rstudio.github.io/py-shinyswatch/reference/theme.darkly.html


⚠ Bofrost.lu has degraded performance

In 5c6050e, Bofrost.lu (https://www.bofrost.lu) experienced degraded performance:

HTTP code: 200
Response time: 1578 ms

Resolved: Bofrost.lu performance has improved in dad0278 after 10 minutes.


Plugin does not handle multiple solidity versions

Try to compiling the Uniswap project (https://github.com/Uniswap/v2-core)
Notice this error:

Error: Source file requires different compiler version (current compiler is 0.8.12+commit.f00d7308.Darwin.appleclang) - note that nightly builds are considered to be strictly less than the released version
 --> test/ERC20.sol:1:1:
  |
1 | pragma solidity =0.5.16;

Looking at code, it seems like we only using in version in the compile call

Bump

figured this one out, some debug needed further than can push

Can now compile https://github.com/Uniswap/v2-core using https://github.com/ApeWorX/ape-solidity/pull/33


Preload Data & Ajax Support

I use select2 to allow a user to select multiple employees from an ajax request.
Once saved, if they ever come back to the tool I need to pre-populate the fields with the employees they selected prior.
I have completed this however when putting an Ajax function in there as well, it doesn't  work. The presets are the only options in the select and it wont let me use the external data source as well.
Is there a way to accomplish this?

+1

This was almost 3 years ago.  In an update, it was possible to "initialize" the select with values, but then in a recent update, that feature was broken.  The following code used to work for me but now does not:
var employeeList = $.parseJSON(expenseDetailRow.IncludedEmployees);
$('#expenseEmployeeList').select2().val(employeeList).trigger('change');

There are multiple posts on StackOverflow regarding this change.  Any possibility of reverting back to the previous version so this functionality works again?

For the moment, the best way to achieve this is by retrieving the individual items and adding them as new options.


Feature/add event

Add log index and tx hash to returned events of rest endpoints
Based on https://github.com/trustlines-protocol/relay/pull/472

Codecov Report

Merging #476 into master will decrease coverage by 0.2%.
The diff coverage is 55.31%.

@@            Coverage Diff             @@
##           master     #476      +/-   ##
==========================================
- Coverage   64.46%   64.25%   -0.21%
==========================================
  Files          53       53
  Lines        4336     4404      +68
==========================================
+ Hits         2795     2830      +35
- Misses       1541     1574      +33

Impacted Files
Coverage Δ

src/relay/api/app.py
25.86% <0%> (ø)
:arrow_up:

src/relay/blockchain/events.py
97.87% <100%> (+0.04%)
:arrow_up:

src/relay/ethindex_db.py
25.62% <12.5%> (-0.7%)
:arrow_down:

src/relay/api/resources.py
41.87% <15.38%> (-1.54%)
:arrow_down:

src/relay/relay.py
30.52% <40%> (+0.01%)
:arrow_up:

src/relay/blockchain/events_informations.py
84.87% <72%> (-0.99%)
:arrow_down:

src/relay/blockchain/proxy.py
83.91% <91.3%> (+0.39%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update bb785f4...7ce766a. Read the comment docs.


feat: add curriculum feature flag

Checklist:

[X] I have read freeCodeCamp's contribution guidelines.
[X] My pull request has a descriptive title (not a vague title like Update index.md)
[X] My pull request targets the main branch of freeCodeCamp.
[X] I have tested these changes either locally on my machine, or GitPod.

Closes #XXXXX

After discussion with @ojeytonwilliams, we decided to extract this from #44183 into it's own PR.
This PR adds an env value that can be used as a feature-flag to gate the new curriculum. Currently this feature flag does absolutely nothing, but will be integrated into the above PR to allow us to control the deployment of the new curriculum - The goal here is to be able to display the new curriculum on English, but not on i18n deployments until we begin translating.

@raisedadead we'll need to update the pipelines with the variable.  Let me know if you're happy with me handling this.

@raisedadead we'll need to update the pipelines with the variable.  Let me know if you're happy with me handling this.

Please go ahead.


Change thread names to fit into 15 symbols

Before
After

The reason to touch dash-q-phase (even though it looks ok on screenshots above) is the way it works on regtest.
Before:
dash-q-phase-10 | RenameThread: thread new name dash-q-phase-100
dash-q-phase-10 | q-phase-100 thread start
dash-q-phase-10 | RenameThread: thread new name dash-q-phase-102
dash-q-phase-10 | q-phase-102 thread start

After:
  dash-llmq-100 | RenameThread: thread new name dash-llmq-100
  dash-llmq-100 | llmq-100 thread start
  dash-llmq-102 | RenameThread: thread new name dash-llmq-102
  dash-llmq-102 | llmq-102 thread start

~Based on #3896 atm due to merge conflicts.~

dash-q-cachepop is ok?


Restart ghost step on Installation

I'm not sure I want to add this. The documentation is already quite extensive. Is not it too obvious?

Yeah, it's a bit obvious. I hope in future it's not necessary anymore. See here here

Then I prefer don't merge this PR and don't mix Ghost issue with theme documentation. Thanks!


fix fix no cluster for common user

Proposed changes
Issue Number: close #xxx

Further comments
If this is a relatively large or complex change, kick off the discussion at<EMAIL_ADDRESS>by explaining why you chose the solution you did and what alternatives you considered, etc...

run buildall

run buildall


Comprehensive Example for Drift Injection Methods

We should probably add a notebook which covers at least the more complex drift injection functions implemented from literature. Can use other examples for reference.

@951378644 let me know if you get alerted to this comment. Sometimes we use this webpage for tracking the issue and communicating with others with questions/answers/design discussions.
We might brainstorm some code like so:
# should I do this?
x = 2

# or this?
x = 3

Or note some sub-tasks we don't want to forget:

[ ] a

(typically we get emails about comments on issues we're assigned to)


App Freeze on huge JS files

VSCode Version: Code 1.4.0 (6276dcb0ae497766056b4c09ea75be1d76a8b679, 2016-08-04T16:45:31.680Z)
OS Version: Windows_NT ia32 10.0.10586

Having multiple large (3000+ lines) JavaScript files open seems to crash the language server. The code colorization goes haywire coloring single letters randomly and the whole app freezes.
ReOpen or Wait does not work, only Close will kill the app.

@con-ssc thank you for opening this issue. Do you have a repo I can clone to reproduce? We have not observed this on our end and we regularly test with many large repositories. I can't speak to the size of each file though.

No I cannot since it is for a customer. I also guess that it might have something to do with some code lens extensions and not with vscode directly maybe. I don't know how to monitor the crashes though.

@con-ssc can you test your theory by running code from the command line with --disable-extensions

Closing until more info is provided.


Need a Way to Stop, Not Just Pause, Currently Playing Audio

Similar/almost exactly the same to issue #118 with the stop feature:
While testing out the audios, I listened half way through and wanted to restart/stop, but I couldn't find a way to. I would just wait for the current audio to finish playing. An additional of a stop feature for this case would be really helpful for me to restart playing the current row audio.
This would probably be another button.

Something like this (minus the wrapping somehow)?

Yeah, something like that.

:-1: Buttons

How about a drop down with options: (1) Play, (2) Pause, (3) Stop, and one button for it to work.

I will just add the visual of the three buttons.

When you make them, set the onclick to player.play() player.stop() and player.pause() for the corresponding buttons. I will work on modifying the audio functions to handle this.

Okay.

Original code for play/pause button:
        <button id="playButton" onclick="playStopAudioButton()" aria-label="Play Pause Toggle">
          <i id="icon" class="fa fa-eject fa-2x fa-rotate-90" ></i></button>

I'm not sure if the id="icon" is used for anything, but I've changed it to id="playToggle" for the play button.

How does the buttons on the right versus on the left look?
[Left]
(1)

(2)

[Right]
(1)

(2)

I'd probably say left because everything else is aligned to the left

Okay.


GLM keeps crashing when running a loop. GPU memory gets full

Kernel died after few hundred iterations with wide data
def lasso_search(m, n, s, verbose = True):
    # generating the data
    X = np.random.uniform(-100, 100, size = (m, n))
    coefs = np.random.randn(n)
    const_coef = np.random.randn(1)

    # index of sparse coefficients
    zero_coef_loc = random.sample(range(n), s)
    coefs[zero_coef_loc] = 0

    y = np.dot(X, coefs) + const_coef

    start = time.time()
    lasso = h2o4gpu.Lasso()
    lasso_model = lasso.fit(X, y)
    print('time to train:', time.time() - start)

    zero_coef_index = np.where(lasso_model.X[0] == 0)

    if verbose:
        print('original coeffs', coefs)
        print('Lasso coefficients:', lasso_model.X[0])
        print(const_coef)
        print(zero_coef_loc)
        print(zero_coef_index[0])

    if np.array_equiv(zero_coef_index[0], np.sort(zero_coef_loc)):
        sr = 1
    else:
        print(zero_coef_index[0], s)
        sr = 1.0 * zero_coef_index[0].shape[0] / s

    return sr

m_list = []
s_list = []
sr_list = []

n= 1000
m_ratio = 2
s_ratios = [0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8]

m = int(m_ratio * n)
for s_ratio in s_ratios:
    s = int(s_ratio * n)
    s_list.append(s)
    for i in range(100):
        print(i, m, n, s)
        sr = lasso_search(m, n, s, verbose = False)
        sr_list.append(sr)

The Kernel crash is due to GPU memory being full. After the model is trained, it's not released from GPU memory. Therefore, the memory gets full very quickly when training multiple models in a loop.
Also, trying del model in python doesn't work

Tried this yesterday and noticed the same problem (though with notebooks, so memory was increasing only by ~2M with each fit). The finish methods (which are supposed to free up all the pointers) also didn't seem to work.

Any solution to this?
Still present in <IP_ADDRESS>99

https://github.com/h2oai/h2o4gpu/issues/204
It's a long-term bug.  I recommend using xgboost's glm.

This bug is extremely inconvenient. My jupyter notebook crashes every few iterations and was unable to finish the job..

@socathie Could you try Bleeding edge version.


Eigenvalues and vectors from JAMA implementation are wrong

After some failed tests with CRubix interface I realized the expected eigenvalues and vectors from MatrixTest are actually wrong compared to other libraries. The following matrix was used:
[
      [22, -17, 12],
      [4, 11, -2],
      [20, -6, -9],
]

Tensor with JAMA
$values = [25.108706520450326, -15.096331148319537, 13.9876246278692];
$vectors = [
            [-0.5029346679560592, -0.1309992382037118, -0.33107976181279675],
            [0.15580805853732102, -0.08643645234319261, -0.6918777439682378],
            [0.8501650243704214, 0.987607178637524, 0.641631809310763],
        ];

Tensor with CArray
$values = [-15.09633115,  25.10870652,  13.98762463];
$vectors = [[  0.25848695, -0.86227193, -0.66844722 ]
            [ -0.11314538, -0.17721180, -0.61268791 ]
            [ -0.95936574,  -0.47442924, -0.42165370 ]];

Computed using LAPACK_dgeev
NumPy
values = [-15.09633115,  25.10870652,  13.98762463]
vectors = [[ 0.25848695, -0.86227193, -0.66844722],
           [-0.11314538, -0.1772118 , -0.61268791],
           [-0.95936574, -0.47442924, -0.4216537 ]]

Computed using LAPACK_dgeev

Interesting @henrique-borba nice catch, that library is pretty old. Do you think it's using a different method to obtain the eigenvaues?
Let's not worry too much about it, I'm willing to remove the dependency on JAMA and just throw an exception in the polyfill if need be. We only use eig for PCA and LDA in the main package anyway.

I'm getting the same results for eigenvalues as you are ... haven't looked at eigenvectors yet
It looks like the eigenvalues are just in different order (probably not an issue). Also, eigenvectors are not unique so that's not necessarily a problem either. Is it safe to assume different algorithms can produce a different set of eigenvectors?


Make FlyteFile compatible with Annotated[..., HashMethod]

See issue #3424
TL;DR
This PR makes FlyteFiles usable with offloaded hash calculation via Annotated[FlyteFile, HashMethod(...)], so that they can be cached based on file contents, e.g.
Type

[x] Bug Fix
[ ] Feature
[ ] Plugin

Are all requirements met?

[x] Code completed
[x] Smoke tested
[x] Unit tests added
[ ] Code documentation added
[ ] Any pending items have an associated Issue

Complete description
In FlyteFilePathTransformer.to_literal(), Annotated type aliases are now
handled by unwrapping the first type argument to the alias, before checking if
they are a subclass of FlyteFile.
Tracking Issue
https://github.com/flyteorg/flyte/issues/3424
Follow-up issue
NA

import Annotated from typing_extensions so that we can use it in python 3.8.

Fixed here and in test code in aa18baa6


Fix flaky AmqpchannelProcessor  test

Description
Please add an informative description that covers that changes made by the pull request and link all relevant issues.
If an SDK is being regenerated based on a new swagger spec, a link to the pull request containing these swagger spec changes has been included above.
All SDK Contribution checklist:

[ ] The pull request does not introduce [breaking changes]
[ ] CHANGELOG is updated for new features, bug fixes or other significant changes.
[ ] I have read the contribution guidelines.

General Guidelines and Best Practices

[ ] Title of the pull request is clear and informative.
[ ] There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, see this page.

Testing Guidelines

[ ] Pull request includes test coverage for the included changes.

API change check
API changes are not detected in this pull request.

Run with the test repeated 1000 times with no issues - https://dev.azure.com/azure-sdk/public/_build/results?buildId=3479167&view=results


Update s2i image from library for ppc64le

What type of PR is this?
/kind bug
What does does this PR do / why we need it:
Similar to PR #3873 . The upstream library updated the link for s2i imagestreams. There are errors when running odo ci script on OCP/P. Errors of following types are observed.
++ oc apply -n openshift -f https://raw.githubusercontent.com/openshift/library/master/community/dotnet/imagestreams/dotnet-centos7.json
error: unable to read URL "https://raw.githubusercontent.com/openshift/library/master/community/dotnet/imagestreams/dotnet-centos7.json", server reported 404 Not Found, status code=404
++ sleep 15
++ oc annotate istag/dotnet:latest --namespace=openshift tags=builder --overwrite
Error from server (NotFound): imagestreamtags.image.openshift.io "dotnet" not found

Which issue(s) this PR fixes:
Fixes #?
PR acceptance criteria:

[ ] Unit test

[ ] Integration test

[ ] Documentation

[ ] I have read the test guidelines

How to test changes / Special notes to the reviewer:

/area system-pz

/ok-to-test

/retest

/retest

/retest

/retest

/retest

/retest

/approve
/lgtm

one word change and does not affect the prow CI
/override v4.5-integration-e2e


Only send one notification unless --wait is used.

When helm is used without --wait, it's basically impossible for the install or upgrade to fail. Helm doesn't wait for the pods to start up or anything, it just returns to the user immediately. There is no information available in the second notification that couldn't have been sent in the first.
When --wait is used, it makes sense to send two notifications because the upgrade or install can fail and the second notification can indicate that to the user.
As far as I can tell, there is no way to determine if --wait was used or not from within the cluster. Might need to talk to the Helm owners.

@dtuite How about global way to disable wait behaviour to reduce noice?

And/or set max retries?

@dtuite How about global way to disable wait behaviour to reduce noice?

Yes. This is another option. I presume that most teams will have scripted their helm deployments and either will always be using --wait or never using --wait. There shouldn't be many teams using a mix outside of dev clusters.

I would like to see this as well, the extra "Upgraded" notification is not very useful to us. We are using helm-operator, and generally have to watch its logs or k describe helmrelease xxxxxx to see whats happening :)


(almost) unbounded decimals by default (instead of floating point)

How about becoming the first modern language with safe numbers?
mpdecimal could be used as the backend (of course, if static analysis proves smaller boundaries, this should be refined to a much more close-to-bare-metal number type).
Explanation with numerous examples in https://github.com/vlang/v/issues/5180#issuecomment-675732199
Or in the worst case at least implement https://github.com/vlang/v/issues/9915 .

Zig (and probably me) plans to implement the compiler_rt decimal routines (https://github.com/ziglang/zig/tree/d6b430b520103fb6691b2c18ae06c2f2a360e806/lib/compiler_rt#readme).
Keep in mind however, that decimals have no hardware acceleration for common hardware (instructions are in x86, but not x86_64).
The main caveat of using those is that they make things unpredictably slow due to the necessary first check + allocation of how to represent the number sufficiently accurately.
This could also be the reason, why libgcc has The software decimal floating point library supports either DPD (Densely Packed Decimal) or BID (Binary Integer Decimal) encoding as selected at configure time. to make perf at least somewhat more deterministic (you either need general absolute accuracy or only accuracy wrt 2s complement rational numbers).
Sidenode: Zig already has arbitrary precision integers, which have simpler checks and are planned to be exported.


Added Docker support for creating a cabby image for easy access to th…

…e commandline tools.

Coverage remained the same at 87.99% when pulling 3eed2370a3e146d485a7b65eaf889f739c8b93e0 on add_docker into b4a73b1e730e43098dbda6b497e2e32af1506bd4 on master.


[Health Gorilla] Use bot identifiers for subscriptions

Hi @codyebberson was this deployed or just closed? It's marked as completed but I'm not seeing a related PR or update to documentation. Was looking for this specific functionality, want to be sure I'm not missing something. Thanks!


apparent invalid sealifebase spec code for Tylosurus acus: http://sealifebase.org/summary/acus

Steps to reproduce:

use Web UI to create a default configuration for Gulf of Mexico
download resulting osmose_config.zip (see attached)
inspect functional_groups.csv

expected:
all species in functional groups have valid sealifebase / fishbase summaries.
actual:
on row 317, I found:

functional group name
functional group type
species name
species url

fishreefassociatedlg
focal_functional_group
Tylosurus acus
http://sealifebase.org/summary/acus

note the species url https://sealifebase.org/summary/acus
osmose_config.zip

Please note that other values look ok, e.g.,

functional group name
functional group type
species name
species url

fishflatfishsmmd
focal_functional_group
Paralichthys albigutta
http://fishbase.org/summary/980

fishflatfishsmmd
focal_functional_group
Citharichthys macrops
http://fishbase.org/summary/4214

fishflatfishsmmd
focal_functional_group
Achirus lineatus
http://fishbase.org/summary/4256

cephalopods
focal_functional_group
Octopus vulgaris
http://sealifebase.org/summary/58190

cephalopods
focal_functional_group
Thysanoteuthis rhombus
http://sealifebase.org/summary/58136

cephalopods
focal_functional_group
Todarodes sagittatus
http://sealifebase.org/summary/57559

cephalopods
focal_functional_group
Illex coindetii
http://sealifebase.org/summary/58130

cephalopods
focal_functional_group
Illex illecebrosus
http://sealifebase.org/summary/58129

The unexpected reference to "http://sealifebase.org/summary/acus" appeared in the request sent from the Web UI (
request-from-UI.json.gz
) . Also, the reference appears to be the only malformed fishbase/sealifebase species url judging from the result of below
$ zcat ~/Desktop/request-from-UI.json.gz  | jq .  | grep -P  "base" | grep -P  "summary/([a-z,\"])+$"
          "url": "http://sealifebase.org/summary/acus"
$

Hello @jhpoelen,
Thank you for pointing this out. I updated the wizard to pass a correct link for this.

@FIN-casey thanks! I confirmed that Tylosurus acus acus now points to http://fishbase.org/summary/976 .


round cpu usage to 2 decimals.

In some cases I was finding CPU being reported to 18 decimal places.  When querying data via Echo is seems like 2 decimal places is probably sufficiently accurate.

Interesting, I definitely didn't hit that one in my testing but it does seem unnecessary 😄 Thanks!


Bump zwave-js-server-python to 0.39.0

It's a minor bump because I find it helpful for the versions (both server and lib) to map well to the schema version. let me know if you disagree

@MartinHjelmare  I guess we should keep this to 0.39.0 instead of skipping to 0.40.0 even though we are skipping straight to schema 20 huh?

We could introduce a version bump schema that keeps some identifiers in sync with the server. But then I think we should first document that.

Agreed. Seems simple though - we can just keep major and minor versions in sync, and say that minor version is the max schema number currently supported


chore: update submodules

Thank you for your contribution to the IC Developer Portal. This repo contains the content for https://internetcomputer.org and the ICP Developer Documentation, https://internetcomputer.org/docs/.
If you are submitting a Pull Request for adding or changing content on the ICP Developer Documentation, please make sure that your contribution meets the following requirements:

[ ] Follows the developer docs style guide.
[ ] Follows the best practices and guidelines.
[ ] New documentation pages include document tags.
[ ] New documentation pages include SEO keywords.
[ ] New documents are in the .mdx file format to support the previous two components.
[ ] New documents are registered in /sidebars.js, otherwise, it will not appear in the
side navigation bar.
[ ] Make sure that the .github/CODEOWNERS file is
filled with new documents that you added. This way we can ensure that future Pull Requests are reviewed by the right people.

submodule needs to be updated again once https://github.com/dfinity/examples/pull/1055 is merged

superseded by https://github.com/dfinity/portal/pull/3834


Measure code coverage as part of CI

Relevant course page: https://skaftenicki.github.io/dtu_mlops/s5_continuous_integration/unittesting/

Github Action for coverage can be found here: https://github.com/marketplace/actions/pytest-coverage-commentator


Have the Seti Icon Theme use the json icon for jsonc files

Currently the built in Seti Icon Theme does not support a custom icon for jsonc files. I believe it would be better to use the json icon for jsonc files instead of the "standard" generic file icon.

See here: https://github.com/AndyBay/vscode/blob/2c63821d7decad388d6a6b58eb1aa313f2a1b2cf/extensions/theme-seti/icons/vs-seti-icon-theme.json#L2
This file is generated and your changes would get lost. We need to find an other way.


Add Vuejs example

Improve the documentation by adding a Vuejs sandbox.

Made a sample
I'm not a Vue expert, so maybe we can get feedback from more experienced devs before posting.

I really love this library and would like to help flesh out the Vue use cases.
I changed @eyalcohen4 sample to use more idiomatic Vue, let me know what you think.
Sample

LGTM
Thanks, Max. I'll update the sandbox


Lag causing server to hang for a few seconds every x minutes

Every x minutes my server is getting a big lag spike of 5 seconds and timings show it is this
https://i.imgur.com/yRYElEY.png
I believe it has something to do with this line
https://github.com/lucko/LuckPerms/blob/0f23b7a460af276659d0122c2b7842194103cd47/bukkit/src/main/java/me/lucko/luckperms/bukkit/listeners/BukkitCommandListUpdater.java#L100
As it is running on the main thread. Any help would be appreciated.

You can fix this issue in two ways:

Update to use Paper instead of Spigot, you can just swap the jars (reccomended)
Set this option to false in the LP config: https://github.com/lucko/LuckPerms/blob/0f23b7a460af276659d0122c2b7842194103cd47/bukkit/src/main/resources/config.yml#L626-L627


:sparkles: Add username, password, vhost to RabbitMQ

#740
#739
#738  👈 (View in Graphite)
main

This stack of pull requests is managed by Graphite. Learn more about stacking.


Memory storage moveTableItemAtIndexPath:toIndexPath: bug

When moving to the last row, content offset changes incorrectly:

This is probably UITableView/UIScrollView bug. DTTableViewManager does not contain any code, that influences layout, specifically content offsets.
This code is called, if you use default UITableView reordering behaviour. Using this method is usually what you want if reordering is initiated by user.
And this method is usually good, when reordering is initiated by programmer.
I'm closing this issue for now, because it seems framework has nothing to do with this.
Please let me know if i can help you more on this issue, or reopen, if you think there are things we can do from framework standpoint.


[nice to have] avoid using trailing comma directly after the shared link

Hey Samuel I just had a look on the pydantic PR. Looks simple to setup and use! Well done!
I just noticed that in Github actions the generated message is
Site created with root smokeshow.helpmanual.io/2u3e441k2v375b2t1r0v/, uploading 63 files...

When you click on the link it hence goes to smokeshow.helpmanual.io/xyz/, with the trailing comma, which is annoying.
Maybe a trailing dot would be better as it's automatically removed by the browser?
Ref: https://github.com/samuelcolvin/smokeshow/blob/2985a676ff057394e032a4713c5d8c572bb40744/cli/smokeshow/main.py#L113
EDIT: just saw that there is at the end
upload complete ✓ site size 2.9MB
go to smokeshow.helpmanual.io/xyz

but could still be nice to have the working link at the beginning as well

Thanks, I'll fix. I put the link at the bottom of the output for that reason, but I agree removing it from the first line would be good too.


Forward the used HTTPS protocol and cipher suite as header

Do you want to request a feature or report a bug?
Feature
What did you expect to see?
Forward the used HTTPS protocol (TLS 1.2 / 1.3) and cipher suite as header. Is this feature useful to be included? Am I right that this is not available yet? Finally, if this feature could be added, I was wondering: would this have to be added to the forwardedheaders or headers middleware?
Why?
The backing service / application might want to know/log internally what type of connections are reaching it. This helps determining (future) strategies how to handle connections for certain protocols or cipher suites, especially if we remember the drop of < TLS 1.2 and SSL protocols.

I was going to ask the same question! My application also need this functionality to have the full view of connection information. Both TLS version and used cipher would be useful.
Is this something that could make sense as custom middleware #1336 or even core functionality in traefik?

I would also love to have this feature.
AWS is offering this in their Load Balancer too. Excerpt from their specs about the setting (https://docs.aws.amazon.com/elasticloadbalancing/latest/APIReference/API_LoadBalancerAttribute.html)

routing.http.x_amzn_tls_version_and_cipher_suite.enabled - Indicates whether the two headers (x-amzn-tls-version and x-amzn-tls-cipher-suite), which contain information about the negotiated TLS version and cipher suite, are added to the client request before sending it to the target. The x-amzn-tls-version header has information about the TLS protocol version negotiated with the client, and the x-amzn-tls-cipher-suite header has information about the cipher suite negotiated with the client. Both headers are in OpenSSL format. The possible values for the attribute are true and false. The default is false.

Hello @holgerniehoff,
We are interested in this issue, and we would love some help from the community.
If you or another community member would like to contribute on this, let us know, and we will work with you to make sure you have all the information needed.
Let us know here before you start. We prefer to work with our community members to ensure that we are aligned and can move quickly with the review and merge process.


Generate FCL version info dynamically

Single quotes? Really? :-)
On Fri, Nov 20, 2020, 16:41 Joaquim Rocha<EMAIL_ADDRESS>wrote:

You can view, comment on, or merge this pull request online at:
https://github.com/kinvolk/docs/pull/6
Commit Summary

Generate FCL's front-matter dynamically
docs/flatcar-container-linux: Use latest as the verison name
DISCARD_ME: For preview purposes only

File Changes

M Makefile
https://github.com/kinvolk/docs/pull/6/files#diff-76ed074a9305c04054cdebb9e9aad2d818052b07091de1f20cad0bbac34ffb52
(1)
M docs/flatcar-container-linux/.gitignore
https://github.com/kinvolk/docs/pull/6/files#diff-28db231966ae0b4d2f3bd714608b26e40370205668eb480e0078bf8e34ef2c5d
(4)
R docs/flatcar-container-linux/_index.md.in
https://github.com/kinvolk/docs/pull/6/files#diff-6c3257ff6fa6eb443d7a13f5db33ffe5e5cc339280bd20d22ca9551402221703
(11)
A tools/fcl-fetch-version-data.py
https://github.com/kinvolk/docs/pull/6/files#diff-07627ac55a6e05c6df09316ad07a2b456e91896464326e6ebe6a62c37f0e8085
(112)

Patch Links:

https://github.com/kinvolk/docs/pull/6.patch
https://github.com/kinvolk/docs/pull/6.diff

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/kinvolk/docs/pull/6, or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAAQL2I5YCYM3RBIERY5AUDSQ3O73ANCNFSM4T5JEK4A
.

this stuff needs some github action magic :wink:


systemd: lnd signals readiness before the RPC server has started

When no wallet exists, this call to NotifyReady happens before the REST server has started.
This makes it impossible for wallet-creating services to wait for the server via the systemd notification system.
lnd should signal readiness only after the server has started.
cc @Kixunil @guggero

Sorry, there was an misunderstanding on my part.
The server is indeed ready to serve initwallet requests, but it fails for function macaroon.
The the upcoming lndinit tool will greatly help this use case.


Remove unnecessary assignments in Paraiso style

Remove the duplicated assignments to background_color and highlight_color in both paraiso_light and paraiso_dark styles.

Nice catch, thanks!


When closing the picker, camera gets frozen

I open the picker with:
fusuma.defaultMode = .camera
then i close the picker and open again, it's like it took a photo, and the camera no longer works, until i change to library and go back.
If i try to take a picture without changing first to library, i got the following error:
017-10-24 13:25:03.865282-0500 DicLarousse[3022:1065125] *** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '*** +[AVCaptureStillImageOutput jpegStillImageNSDataRepresentation:] NULL sample buffer.'
The version i'm using is 1.2.0, and the device is an iphone 5s with 11.0.3, my proyect currently is developed using swift 3.2.

Same here.


user-select

user-select CSS3 propert not being processed, not mentioned in the code.

I have the same issue of mpa3b

I was going to look into why my user-select wasn't being prefixed, but I noticed that it was while testing PR #6147. So I reverted that PR and it was still working, although I had the unrelated PR #6145 in effect.

Can somebody provide a testcase (e.g. in a dabblet or codepen) so I (or someone else) can look into it?


How to install it on ubuntu 20.04?

Is there any easy to understand guide for installing this on Ubuntu 20.04

Is there any easy to understand guide for installing this on Ubuntu 20.04

You can follow the steps for Debian 10 and the Notes is for Ubuntu 20.04

I followed the steps. Is there something missing? Sorry I am annoying. I am just new to linux.

Don’t worry, we can help. You use virtual environment and install the requirements?

Create the virtual environment and activate

Inside the virtual environment, install requirements and yt-dlg

@insanitywanted Can run the GUI too:
python3 -m youtube_dl_gui


Custom SVG Renderer

Hi,
Is there a way to use SVGRenderer inside Angular Three ?
I have a requirement where I need to use the LineBasicMaterial with its linewidth property set to >1.
Basically, it should work like this one in the official ThreeJS example.
https://threejs.org/examples/?q=line#svg_lines
Any guide on this please.
Than you.

Hi,
Please check the example here: https://github.com/angular-threejs/angular-three/blob/main/apps/kitchen-sink/src/app/misc/svg-renderer/svg-renderer.ts
Demo: https://demo.angularthree.org/misc/svg-renderer ported from the ThreeJS SVG Lines example.
Let me know if you have further questions.

Hello nartc,
Thanks a lot for your quick response. I tried the code that you linked above, however I hit a 'gl.setClearAlpha is not a function' error when running it.
main.ts:5 ERROR TypeError: gl.setClearAlpha is not a function
    at Object.configure (angular-three.mjs:2401:14)
    at _NgtCanvas.noZoneResizeEffect (angular-three.mjs:2590:25)
    at effect.injector [as effectFn] (angular-three.mjs:2574:16)
    at EffectHandle.runEffect (core.mjs:37277:12)
    at Object.fn (core.mjs:37272:52)
    at Object.run (signals.mjs:507:12)
    at EffectHandle.run (core.mjs:37288:18)
    at ZoneAwareEffectScheduler.flushQueue (core.mjs:37255:14)
    at core.mjs:37245:31
    at _ZoneDelegate.invoke (zone.js:369:28)

I'm not really sure what did I do wrong. I just cloned your threejs template and added svgRendererFactory as you suggested.
@Component({
  selector: 'app-root',
  standalone: true,
  template: `
    <ngt-canvas [sceneGraph]="sceneGraph"  [gl]="svgRendererFactory" />
  `,
  host: { class: 'block h-dvh w-full' },
  imports: [NgtCanvas],
})
export class AppComponent {
  sceneGraph = Experience;

  svgRendererFactory = (canvas: NgtCanvasElement) => {
		const renderer = new SVGRenderer();

		if (canvas instanceof HTMLCanvasElement) {
			canvas.style.display = 'none';
			canvas.parentElement?.appendChild(renderer.domElement);
		}

		return renderer;
	};
}

ah I did changed something internal. Lemme cut a patch release

Give 2.3.2 a try

Hello nartc,
I ended up extending the existing SVGRenderer and creating an empty function for "setClearAlpha" instead since I have difficulty installing the patch version.
Thank you so much for this great library by the way. This opens a lot of avenues in the Angular community.

@reyryan can you open a new issue on what problems you ran into trying to instal the patch version?
Extending the SVGRenderer is a nice workaround! I'll close this issue.


Feature/Product-sales-backend

This a draft PR. It's not ready to merge yet./ Created a Products sales table that stores the sales of products

created new PR having issues with this one.


Chore: Wrapper for click proposal

Changelog Description
This is a proposal how to resolve issues with click python module. Issue https://github.com/ynput/OpenPype/issues/5921 reported that in Houdini 20+ is our click clashing with click in houdini, where is expected higher version. We can't update our version to support older pythons (NOTE older Python 3).
Possible solutions

Use this wrapper (in this PR) that mimics click approach how to define commands and their options. This gives a little bit more options to enhance cli callbacks.
Move cli definition out of the addon main file and import it only when needed.

def cli(self, click_group):
    from .cli import cli_main
    click_group.add_command(cli_main)

This PR solution
Created wrapper to mimic click behavior, with small enhancement to be able to use chain function calls which simplify registering methods as cli functions, that might be helpfull in cli commands that need to create ModulesManage. Used the wrapper in ftrack addon and it works...
Now the question is if we should use option n1, option n2 or we should look for option n3.
Note: This PR does not resolve the issue fully, we would still need to remove click from dependencies and move it next to pyside.

Hmm - I wouldn't push this direction. We're now basically maintaining our own vendorized copy of click via this - which is something we should rather not touch if not absolutely necessary.
Honestly if this is the hack we'd allow I'd say that maybe we should put a click.py replacement namespace package on path that just registers the package as if it was the original package and then does:
click.py:
try:
    import click
except ImportError as exc:
    # Import our own vendorized version, e.g. adding it on path
    import_our_click()

I recall there being a feature where you could do that so that that file acts as if it was whatever it imported, it can act like click you imported exactly. That way, if click is available from the DCC - we use that, otherwise load our own.
I'd consider that less maintenance likely.
Still a hack, but easier to maintain + no other code changes needed?

Hmm - I wouldn't push this direction. We're now basically maintaining our own vendorized copy of click via this - which is something we should rather not touch if not absolutely necessary.

The wrapper is not even 5% of click logic. I don't think this wrapper needs more maintainance, only what could need maintainance in future is convert_to_click function.

I recall there being a feature where you could do that so that that file acts as if it was whatever it imported, it can act like click you imported exactly. That way, if click is available from the DCC - we use that, otherwise load our own.

To be honest, that is more complicated than this wrapper. Also more complicated to understand what is used when. I would rather explicitly name it differently and use always only the one, or use proposed option 2.

If I need to write CLI code for an addon and I see click I assume it's the click API - if it then doesn't have half its features I'd just be confused.
I feel like if we're differentiating we might as well just put this into openpype.lib as openpype.lib.cli_utils or whatever - the lib file itself could still say it's based on click but then at least there's not the expectancy that it's click itself.
If anyone then wants to instead expose API using click for a custom addon then indeed it's their mess to solve the lib dependencies, etc.
So if what you're saying is just using a thin command line interface that we just put in openpype.lib I think that's fine as long as we make it super clear (e.g. differentiating name) that it's not click.

Oh wait, no - this does still import click and use it; but then, how does it solve the dependency? :) How do we make sure that cli still has access to the click that it imports when used?

Oh wait, no - this does still import click and use it; but then, how does it solve the dependency? :) How do we make sure that cli still has access to the click that it imports when used?

The difference is that it does not need click to define cli commands, groups, options etc. it needs click only when you want to convert it for click. Look at change in ftrack addon, specifically to cli method on the addon, the click is imported only when to_click_object is called.

With that, we can remove click from pyproject toml dependencies to runtime dependencies, and problem solved...

Yes, I was missing this part - thanks. Then I suppose the approach is fine with me. Would still love if we can improve on the dependency hell in a better way; and even better, drop all Py2 support too.

and even better, drop all Py2 support too.

We have to keep Python 2 in OpenPype (not in AYON).

Not sure, if my comment related to this PR or not but here are my 2 cents.
I'm not able to launch Houdini 20 without https://github.com/ynput/OpenPype/pull/5932
so, I merged both this PR and https://github.com/ynput/OpenPype/pull/5932
and Here's what I see in Houdini 20 console on startup.
In essence, the problem with these:
  File "C:\PROGRA~1\SIDEEF~1\HOUDIN~1.506\python310\lib\site-packages\flask\cli.py", line 15, in <module>
    from click.core import ParameterSource
ImportError: cannot import name 'ParameterSource' from 'click.core' (E:\Ynput\OpenPype\.venv\lib\site-packages\click\core.py)

  File "C:\Program Files\Side Effects Software\Houdini 20.0.506\python310\lib\site-packages-forced\shiboken2\files.dir\shibokensupport\__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "E:\Ynput\OpenPype\.venv\lib\site-packages\PIL\Image.py", line 103, in <module>
    from . import _imaging as core
ImportError: cannot import name '_imaging' from 'PIL' (E:\Ynput\OpenPype\.venv\lib\site-packages\PIL\__init__.py)

Full log:
Could not initialize the help server:
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\PROGRA~1/SIDEEF~1/HOUDIN~1.506/houdini/python3.10libs\houdinihelp\api.py", line 126, in initialize
    bookish_app = get_houdini_app(use_houdini_path=bool(hou))
  File "C:\PROGRA~1/SIDEEF~1/HOUDIN~1.506/houdini/python3.10libs\houdinihelp\server.py", line 25, in get_houdini_app
    from bookish import flaskapp, flasksupport
  File "C:\Program Files\Side Effects Software\Houdini 20.0.506\python310\lib\site-packages-forced\shiboken2\files.dir\shibokensupport\__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "C:\PROGRA~1/SIDEEF~1/HOUDIN~1.506/houdini/python3.10libs\bookish\flaskapp.py", line 37, in <module>
    import flask
  File "C:\Program Files\Side Effects Software\Houdini 20.0.506\python310\lib\site-packages-forced\shiboken2\files.dir\shibokensupport\__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "C:\PROGRA~1\SIDEEF~1\HOUDIN~1.506\python310\lib\site-packages\flask\__init__.py", line 2, in <module>
    from .app import Flask as Flask
  File "C:\Program Files\Side Effects Software\Houdini 20.0.506\python310\lib\site-packages-forced\shiboken2\files.dir\shibokensupport\__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "C:\PROGRA~1\SIDEEF~1\HOUDIN~1.506\python310\lib\site-packages\flask\app.py", line 34, in <module>
    from . import cli
  File "C:\Program Files\Side Effects Software\Houdini 20.0.506\python310\lib\site-packages-forced\shiboken2\files.dir\shibokensupport\__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "C:\PROGRA~1\SIDEEF~1\HOUDIN~1.506\python310\lib\site-packages\flask\cli.py", line 15, in <module>
    from click.core import ParameterSource
ImportError: cannot import name 'ParameterSource' from 'click.core' (E:\Ynput\OpenPype\.venv\lib\site-packages\click\core.py)

Nothing to change, Houdini vars are already up to date.
= = = Resources registration = = =
ERROR: C:/PROGRA~1/SIDEEF~1/HOUDIN~1.506/packages/apex/viewer_states/apexanimate.py
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\PROGRA~1/SIDEEF~1/HOUDIN~1.506/houdini/python3.10libs\viewerstate\utils.py", line 967, in register_pystate
    mod = importlib.import_module(module_name)
  File "C:\PROGRA~1\SIDEEF~1\HOUDIN~1.506\python310\lib\importlib\__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "C:\PROGRA~1/SIDEEF~1/HOUDIN~1.506/packages/apex/viewer_states\apexanimate.py", line 15, in <module>
    from apex.ui.selectionmanager import model as ssm
  File "C:\Program Files\Side Effects Software\Houdini 20.0.506\python310\lib\site-packages-forced\shiboken2\files.dir\shibokensupport\__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "C:\PROGRA~1/SIDEEF~1/HOUDIN~1.506/packages/apex/python3.10libs\apex\ui\selectionmanager\model.py", line 9, in <module>
    from poselib.itemlibrary import IdentityProxyModel as plIdentityProxyModel
  File "C:\Program Files\Side Effects Software\Houdini 20.0.506\python310\lib\site-packages-forced\shiboken2\files.dir\shibokensupport\__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "C:\PROGRA~1/SIDEEF~1/HOUDIN~1.506/houdini/python3.10libs\poselib\itemlibrary.py", line 16, in <module>
    from . import plglobals
  File "C:\Program Files\Side Effects Software\Houdini 20.0.506\python310\lib\site-packages-forced\shiboken2\files.dir\shibokensupport\__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "C:\PROGRA~1/SIDEEF~1/HOUDIN~1.506/houdini/python3.10libs\poselib\plglobals.py", line 23, in <module>
    from PIL import Image
  File "C:\Program Files\Side Effects Software\Houdini 20.0.506\python310\lib\site-packages-forced\shiboken2\files.dir\shibokensupport\__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "E:\Ynput\OpenPype\.venv\lib\site-packages\PIL\Image.py", line 103, in <module>
    from . import _imaging as core
ImportError: cannot import name '_imaging' from 'PIL' (E:\Ynput\OpenPype\.venv\lib\site-packages\PIL\__init__.py)

ERROR: C:/PROGRA~1/SIDEEF~1/HOUDIN~1.506/packages/apex/viewer_states/apexposestate.py
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\PROGRA~1/SIDEEF~1/HOUDIN~1.506/houdini/python3.10libs\viewerstate\utils.py", line 967, in register_pystate
    mod = importlib.import_module(module_name)
  File "C:\PROGRA~1\SIDEEF~1\HOUDIN~1.506\python310\lib\importlib\__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "C:\PROGRA~1/SIDEEF~1/HOUDIN~1.506/packages/apex/viewer_states\apexposestate.py", line 7, in <module>
    import apex.ui.selectionmanager.model as ssm
  File "C:\Program Files\Side Effects Software\Houdini 20.0.506\python310\lib\site-packages-forced\shiboken2\files.dir\shibokensupport\__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "C:\PROGRA~1/SIDEEF~1/HOUDIN~1.506/packages/apex/python3.10libs\apex\ui\selectionmanager\model.py", line 9, in <module>
    from poselib.itemlibrary import IdentityProxyModel as plIdentityProxyModel
  File "C:\Program Files\Side Effects Software\Houdini 20.0.506\python310\lib\site-packages-forced\shiboken2\files.dir\shibokensupport\__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "C:\PROGRA~1/SIDEEF~1/HOUDIN~1.506/houdini/python3.10libs\poselib\itemlibrary.py", line 16, in <module>
    from . import plglobals
  File "C:\Program Files\Side Effects Software\Houdini 20.0.506\python310\lib\site-packages-forced\shiboken2\files.dir\shibokensupport\__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "C:\PROGRA~1/SIDEEF~1/HOUDIN~1.506/houdini/python3.10libs\poselib\plglobals.py", line 23, in <module>
    from PIL import Image
  File "C:\Program Files\Side Effects Software\Houdini 20.0.506\python310\lib\site-packages-forced\shiboken2\files.dir\shibokensupport\__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "E:\Ynput\OpenPype\.venv\lib\site-packages\PIL\Image.py", line 103, in <module>
    from . import _imaging as core
ImportError: cannot import name '_imaging' from 'PIL' (E:\Ynput\OpenPype\.venv\lib\site-packages\PIL\__init__.py)

This PR is not fixing the issue, it is first step to be able to fix it...

Actually - it's worse. As Mustafa's logs show there is also a conflict with the PIL library from OpenPype.

Actually - it's worse. As Mustafa's logs show there is also a conflict with the PIL library from OpenPype.

One at a time, please add it to the issue, not related to this PR.

Actually - it's worse. As Mustafa's logs show there is also a conflict with the PIL library from OpenPype.

Side note: Just heard from @fabiaserra that the PIL lib is only used for the TVPaint and Photoshop integration. For their case they'd just removed it as a required dependency to OpenPype and it fixed the PIL thing for them.

@fabiaserra could you check out this proposal and provide your own feedback? Then maybe @iLLiCiTiT can finish the PR so it actually fixes the issue, because last he said:

This PR is not fixing the issue, it is first step to be able to fix it...

I suppose the bug is still there and would need more input to continue.

Just a note, we're AYON focused now, and in AYON is Pillow already not added to PYTHONPATH, so it is resolved for us. Change of dependencies would require to bump minor version of OpenPype.
This PR needs final decisions: Where to put the wrapper, and how to name it.

Current location is in root of openpype. I think it can be moved to openpype.lib or openpype.modules.
Current name is click_wrap. Here I didn't find better name.

Just a note, we're AYON focused now, and in AYON is Pillow already not added to PYTHONPATH, so it is resolved for us. Change of dependencies would require to bump minor version of OpenPype.
This PR needs final decisions: Where to put the wrapper, and how to name it.

Current location is in root of openpype. I think it can be moved to openpype.lib or openpype.modules.
Current name is click_wrap. Here I didn't find better name.

Isn't it here https://github.com/ynput/OpenPype/blob/develop/pyproject.toml#L50 ? that will be picked up by the dependency package for Ayon as well? I removed it from there and confirmed that I no longer get the Pillow error running latest OP minor (haven't tested in Ayon because I was lazy to add H20 and update the dependency package but I'd assume it's the same). I now only get the click error reported here.
As for my preferred solution, I think here's where rez would provide the proper solution to have independent environments per DCC but given the lack of it, I'd try prio the python packages from the DCC when there's conflicts like @BigRoy was suggesting than creating extra wrappers and more technical debt because this is not only going to happen with click but numerous other packages... In fact I had numpy installed in OP as a dependency and that was another conflict that I was getting so I now need to come up with some hack to roll that only for Hiero. Is there a solution in Ayon to set packages only for specific DCCs?

In fact I had numpy installed in OP as a dependency and that was another conflict that I was getting so I now need to come up with some hack to roll that only for Hiero. Is there a solution in Ayon to set packages only for specific DCCs?

Is relying on environment variables enough?

I think the target of this PR should be openpype.

I opened Houdini 20 from develop branch and faced no issues in Ayon.
In openpype I faced the same error mentioned in this PR https://github.com/ynput/OpenPype/pull/5928#issuecomment-1819096151.

then I checked out this PR and didn't face any errors in Ayon.
but, in openpype I faced the following error.
Could not initialize the help server:
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\PROGRA~1/SIDEEF~1/HOUDIN~1.502/houdini/python3.10libs\houdinihelp\api.py", line 126, in initialize
    bookish_app = get_houdini_app(use_houdini_path=bool(hou))
  File "C:\PROGRA~1/SIDEEF~1/HOUDIN~1.502/houdini/python3.10libs\houdinihelp\server.py", line 25, in get_houdini_app
    from bookish import flaskapp, flasksupport
  File "C:\Program Files\Side Effects Software\Houdini 20.0.502\python310\lib\site-packages-forced\shiboken2\files.dir\shibokensupport\__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "C:\PROGRA~1/SIDEEF~1/HOUDIN~1.502/houdini/python3.10libs\bookish\flaskapp.py", line 37, in <module>
    import flask
  File "C:\Program Files\Side Effects Software\Houdini 20.0.502\python310\lib\site-packages-forced\shiboken2\files.dir\shibokensupport\__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "C:\PROGRA~1\SIDEEF~1\HOUDIN~1.502\python310\lib\site-packages\flask\__init__.py", line 2, in <module>
    from .app import Flask as Flask
  File "C:\Program Files\Side Effects Software\Houdini 20.0.502\python310\lib\site-packages-forced\shiboken2\files.dir\shibokensupport\__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "C:\PROGRA~1\SIDEEF~1\HOUDIN~1.502\python310\lib\site-packages\flask\app.py", line 34, in <module>
    from . import cli
  File "C:\Program Files\Side Effects Software\Houdini 20.0.502\python310\lib\site-packages-forced\shiboken2\files.dir\shibokensupport\__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "C:\PROGRA~1\SIDEEF~1\HOUDIN~1.502\python310\lib\site-packages\flask\cli.py", line 15, in <module>
    from click.core import ParameterSource
ImportError: cannot import name 'ParameterSource' from 'click.core' (E:\Ynput\OpenPype\.venv\lib\site-packages\click\core.py)

Nothing to change, Houdini vars are already up to date.
= = = Resources registration = = =
ERROR: C:/PROGRA~1/SIDEEF~1/HOUDIN~1.502/packages/apex/viewer_states/apexanimate.py
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\PROGRA~1/SIDEEF~1/HOUDIN~1.502/houdini/python3.10libs\viewerstate\utils.py", line 967, in register_pystate
    mod = importlib.import_module(module_name)
  File "C:\PROGRA~1\SIDEEF~1\HOUDIN~1.502\python310\lib\importlib\__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "C:\PROGRA~1/SIDEEF~1/HOUDIN~1.502/packages/apex/viewer_states\apexanimate.py", line 15, in <module>
    from apex.ui.selectionmanager import model as ssm
  File "C:\Program Files\Side Effects Software\Houdini 20.0.502\python310\lib\site-packages-forced\shiboken2\files.dir\shibokensupport\__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "C:\PROGRA~1/SIDEEF~1/HOUDIN~1.502/packages/apex/python3.10libs\apex\ui\selectionmanager\model.py", line 9, in <module>
    from poselib.itemlibrary import IdentityProxyModel as plIdentityProxyModel
  File "C:\Program Files\Side Effects Software\Houdini 20.0.502\python310\lib\site-packages-forced\shiboken2\files.dir\shibokensupport\__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "C:\PROGRA~1/SIDEEF~1/HOUDIN~1.502/houdini/python3.10libs\poselib\itemlibrary.py", line 16, in <module>
    from . import plglobals
  File "C:\Program Files\Side Effects Software\Houdini 20.0.502\python310\lib\site-packages-forced\shiboken2\files.dir\shibokensupport\__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "C:\PROGRA~1/SIDEEF~1/HOUDIN~1.502/houdini/python3.10libs\poselib\plglobals.py", line 23, in <module>
    from PIL import Image
  File "C:\Program Files\Side Effects Software\Houdini 20.0.502\python310\lib\site-packages-forced\shiboken2\files.dir\shibokensupport\__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "E:\Ynput\OpenPype\.venv\lib\site-packages\PIL\Image.py", line 103, in <module>
    from . import _imaging as core
ImportError: cannot import name '_imaging' from 'PIL' (E:\Ynput\OpenPype\.venv\lib\site-packages\PIL\__init__.py)

ERROR: C:/PROGRA~1/SIDEEF~1/HOUDIN~1.502/packages/apex/viewer_states/apexposestate.py
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\PROGRA~1/SIDEEF~1/HOUDIN~1.502/houdini/python3.10libs\viewerstate\utils.py", line 967, in register_pystate
    mod = importlib.import_module(module_name)
  File "C:\PROGRA~1\SIDEEF~1\HOUDIN~1.502\python310\lib\importlib\__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "C:\PROGRA~1/SIDEEF~1/HOUDIN~1.502/packages/apex/viewer_states\apexposestate.py", line 7, in <module>
    import apex.ui.selectionmanager.model as ssm
  File "C:\Program Files\Side Effects Software\Houdini 20.0.502\python310\lib\site-packages-forced\shiboken2\files.dir\shibokensupport\__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "C:\PROGRA~1/SIDEEF~1/HOUDIN~1.502/packages/apex/python3.10libs\apex\ui\selectionmanager\model.py", line 9, in <module>
    from poselib.itemlibrary import IdentityProxyModel as plIdentityProxyModel
  File "C:\Program Files\Side Effects Software\Houdini 20.0.502\python310\lib\site-packages-forced\shiboken2\files.dir\shibokensupport\__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "C:\PROGRA~1/SIDEEF~1/HOUDIN~1.502/houdini/python3.10libs\poselib\itemlibrary.py", line 16, in <module>
    from . import plglobals
  File "C:\Program Files\Side Effects Software\Houdini 20.0.502\python310\lib\site-packages-forced\shiboken2\files.dir\shibokensupport\__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "C:\PROGRA~1/SIDEEF~1/HOUDIN~1.502/houdini/python3.10libs\poselib\plglobals.py", line 23, in <module>
    from PIL import Image
  File "C:\Program Files\Side Effects Software\Houdini 20.0.502\python310\lib\site-packages-forced\shiboken2\files.dir\shibokensupport\__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "E:\Ynput\OpenPype\.venv\lib\site-packages\PIL\Image.py", line 103, in <module>
    from . import _imaging as core
ImportError: cannot import name '_imaging' from 'PIL' (E:\Ynput\OpenPype\.venv\lib\site-packages\PIL\__init__.py)

First of all, target of this PR is to fix click import issues inside DCCs.

Isn't it here https://github.com/ynput/OpenPype/blob/develop/pyproject.toml#L50 ? that will be picked up by the dependency package for Ayon as well?

Not is is not. That is only OpenPype build related. OpenPype addon has different pyproject.toml.

I think the target of this PR should be openpype.

The target is AYON and could be used in OpenPype too (with another PR). This PR is to avoid import of click in addons which have cli interface, because click import breaks inside DCCs which have incompatible python, in both AYON and OpenPype.
The PIL is NOT handled by this PR, and won't be, please stop to mention it here as it's not relevant. First crash in your traceback in because of click, that is what this PR is about.

I opened Houdini 20 from develop branch and faced no issues in Ayon.

Did you have enabled an addon that is using click, like ftrack or kitsu, when you've tried?

As for my preferred solution, I think here's where rez would provide the proper solution to have independent environments per DCC but given the lack of it, I'd try prio the python packages from the DCC when there's conflicts like @BigRoy was suggesting than creating extra wrappers and more technical debt because this is not only going to happen with click but numerous other packages...

That's a long run task. This PR is solution for current issue without reimplementing whole applications launch logic and all addons and figuring out how to deploy rez to workstations and many other issues that must be solved.

No, it is not. That is only OpenPype build related. OpenPype addon has different pyproject.toml.

Oh nice, I hadn't seen that other pyproject, good to know

That's a long run task. This PR is solution for current issue without reimplementing whole applications launch logic and all addons and figuring out how to deploy rez to workstations and many other issues that must be solved.

Yeah I understand, that's why I think as a first stop solution it's better if we just hack it by prioritizing the DCC click than creating the wrapper, both should be short-term solutions until we come up with a better plan

it's better if we just hack it by prioritizing the DCC click than creating the wrapper

I'm afraid we cannot hack it. The error can happen before our plugin is loaded. It happens when flask application inside houdini is started. Since the click is in PYTHONPATH there is not much ways how to fix it...

I don't know why the click error doesn't show up any more in H20 Ayon Mode.

I think it is because we're using newer version of click in ayon launcher, which lead to accidental compatibility. In other words we're lucky. I think if you would update venv in openpype it would be fixed too.

I'm afraid we cannot hack it. The error can happen before our plugin is loaded. It happens when flask application inside houdini is started. Since the click is in PYTHONPATH there is not much ways how to fix it...

Even if we prepend the Houdini python libs to the PYTHONPATH?

Even if we prepend the Houdini python libs to the PYTHONPATH?

We can't determine the path in reliable way. We have clients which use custom launch scripts instead of houdini executable in application settings.

Even if we prepend the Houdini python libs to the PYTHONPATH?

We can't determine the path in reliable way. We have clients which use custom launch scripts instead of houdini executable in application settings.

is $HHP resolved too late?

is $HHP resolved too late?

You'd need to set the PYTHONPATH before the actual houdini launch to ensure anything houdini does, accesses the right entry.
I assume this $HHP is a houdini var and thus would be too late.

You'd need to set the PYTHONPATH before the actual houdini launch to ensure anything houdini does, accesses the right entry. I assume this $HHP is a houdini var and thus would be too late.
Then there might also be a case where $HHP gets set DIRECTLY by Houdini and before the call to click but you'd still need to somehow get Python access directly after Houdini sets that var - nothing is ensuring if you do it e.g. in pythonlibs or whatever startup procedure that those run before Houdini does any call to click for example. (And it's not about an actual call to the functions of click; but when it imports it.)

Yeah makes sense. On the interim I was able to hack it in our env on the H20 entry, pretty straightforward:

You'd need to set the PYTHONPATH before the actual houdini launch to ensure anything houdini does, accesses the right entry. I assume this $HHP is a houdini var and thus would be too late.
Then there might also be a case where $HHP gets set DIRECTLY by Houdini and before the call to click but you'd still need to somehow get Python access directly after Houdini sets that var - nothing is ensuring if you do it e.g. in pythonlibs or whatever startup procedure that those run before Houdini does any call to click for example. (And it's not about an actual call to the functions of click; but when it imports it.)

Yeah makes sense. On the interim I was able to hack it in our env on the H20 entry, pretty straightforward:

As long as the bug is documented and the different possible solutions on how to hack it... I think it would be perfectly fine as a temporary solution until we can tackle rez or a better system

Yeah makes sense. On the interim I was able to hack it in our env on the H20 entry, pretty straightforward:

For what it's worth - we ended up doing the same thing as it felt most reliable since there were multiple conflicts we had to manage where click was only one of them.

For what it's worth - we ended up doing the same thing as it felt most reliable since there were multiple conflicts we had to manage where click was only one of them.

exactly, adding the wrapper is just a tiny bandaid that doesn't fix the actual root of the problem...


storage: simplify backend

I intended to do this right at the beginning but then focused on the crucial parts.

rip out all the unnecessary log-wrapping stuff: We can achieve the same with simply logging directly in the functions like we e.g. do in lxd/container_lxc.go.
move separate interfaces into separate files

jenkins: test this please

Hmm, so the removal of the logwrapper stuff means we're loosing the context whenever logging something from a storage driver... We don't know what driver is used or what pool it is.
I'm not saying that the wrapper was a good approach to the problem, but I think we should have a custom logger be defined and used so that we always log what driver it is, what pool we're dealing, what volume, ...
Our logs already don't have much context as they are right now, would be nice not to make them worse :)

Nah, we're not loosing "context". The way I implemented it will log the storage driver and pool the operation occurs on everytime. So this provides a lot more information as before. Can you look at the testsuite run and confirm that this isn't enough.

@brauner your branch is doing things like this all over the place:
-			s.log.Warn("Received value over semaphore. This should not have happened.")
 +			shared.LogWarnf("Received value over semaphore. This should not have happened.")

s.log used to be intiialized with extra context providing "driver" for all log entries.
How is this not being lost?

Anyway. You're right that you do add some useful logging to make up for that and merely logging the driver name in log.Ctx isn't all that useful now that you can have multiple storage pools using the same driver.
I still think it'd be a good idea to have custom log context used there so for any error/warning/... we log we know what pool it was, what driver it was and what volume it was, without having to make a guess based on previous log entries (which may be of a different log level and therefore not logged at all).

If you want pool contex, fine. :) I'm ready to add that. The logwrapper stuff is (in my opinion) just overhead. :)


Accessibility parameters not being handled

Hi,
This is not handling attributes which are required for accessibility . Without these our links will not be accessibility compatible. Please handle these attributes:

aria-haspopup
id
aria-posinset
aria-setsize
aria-label

Thank you!


Add waiters for athena

I added two waiters for athena(https://github.com/boto/boto3/issues/1212).
For single query, query_succeeded only comes back without error iff query succeeded. Otherwise, WaitError is raised and one may call get_query_execution to examine the details.
For multiple queries, queries_finished is used. When it comes back, it means none of the queries is in RUNNING state. One should call batch_get_query_execution to examine the details.

I assume delay=15 means a delay of 15 seconds, that seems quite a long time for cases where queries return relatively quickly?

@jamesjrg you are right. While the default values work for to my cases, they may not be practical for many. I'd love to change them if that makes more sense.
Fortunately, the default values can be overwritten if needed:
waiter.wait(WaiterConfig={
            'Delay':5,
 })

I'm just some random internet passer-by, but 5 seconds sounds like a reasonable default to me

+1 on 5 secs. in our use case, 1 sec seems too much API calls and 15 secs are just too long especially you have a chain of lambda functions that execute different athena queries

I've actually now gone for 2 seconds, though depending on the use case I don't know if that would work for others. A few extra HTTP calls don't really cost very much and I don't like waiting for results!

This is awesome, looks like code coverage reduced, maybe there are example tests for other waiters that could provide an example of how to test it. That's probably the main reason why its not been merged.

is this feature abandoned?

Hi all,
Thanks again for the request. We are now submitting requests for paginators and waiters upstream so that all AWS SDKs benefit from a consistent implementation. I will pass on the request for these waiters to the service team. We will also update the guidance in the contributing guide to better reflect this policy. I've added an issue to track this here:
aws/aws-sdk#80
Thanks again for bringing this to our attention!


xml namespace support in sitemaps

i ran into some sites that have a sitemap with xml prefix/namespace support
i now monkeypatched my code with following snippet to work around it
from trafilatura import sitemaps
sitemaps.LINK_REGEX = re.compile('<(?:\w+:)?loc>(?:<!\\[CDATA\\[)?(http.+?)(?:\\]\\]>)?</(?:\w+:)?loc>')
sitemaps.SITEMAP_FORMAT = re.compile(r'^.{0,5}<\?xml|<sitemap|<(?:\w+:)?urlset')

obviously this is not really maintanable on my side, so it would be better to get this upstreamed.
i can make a PR with this fix, but it's too simple in the sense that it does not limit and/or validate the namespaces.
eg when you would have a <ns1:loc> and also a <ns2:loc> in the same sitemap, this simple solution does not care and just grabs it all
it also only deals with urlset being namespaced atm

Hi @stdweird, extraction using regexes is brittle but I thought it's a reasonable way to tackle sitemaps which are fairly regular in their form. It'd be great if you could write a PR to fix the issue you ran into. Please also write a few tests to make sure everything works as intended.


Material to getting start

Hi!
Sorry, I understand that this is not the right place where to ask this.
I'm a big fan of processing.org and toxiclibs but I'm a little bit bored about java as a language itself.
So here is a question: is there any tutorial or getting started or any other kind of advice to use thi.ng?

Hi @nkint, tutorials are non-existing so far, but are on the TODO list... However for examples, there're various snippets lingering throughout the codebase and various workshop repos:
https://github.com/thi-ng/geom/blob/master/src/index.org#example-usage
https://github.com/thi-ng/geom/blob/master/geom-viz/src/core.org#example-usage
https://github.com/thi-ng/geom/blob/master/geom-svg/src/examples.org
https://github.com/thi-ng/ws-ldn-1
https://github.com/thi-ng/ws-ldn-2
https://github.com/learn-postspectacular/resonate-workshop-2015
https://github.com/learn-postspectacular/resonate-workshop-2014

It would be nice if a slack channel was created for thi.ng. I'm also a beginner and want to learn more and have sometimes questions. But I just ask here then; for svg/path, how do I use is, as string or map?
(svg/path {:M "150 0" :L "75 200 225 200 Z"} {:fill "green"})
or
(svg/path "M 150 0 L 75 200 225 200 Z" {:fill "green"})
Neither seems to work.

Hi @hlolli - I personally don't like slack due to its way too truncated history. we're using http://gitter.im/thi-ng/geom, which has unlimited history and is also linked to from the main readme... There're currently ~30 users and I try to reply as often as I can (mostly within a few hours).
As for your SVG question. The whole point of the SVG ns is to avoid having to work directly with the SVG internals. The fns provided all work seamlessly with the geom types and I'd really recommend reading the source code and looking at the various examples.
hth! (if not, see you on the chat :) )