Split container app deployment out from provisioning

Currently, azd deploys container apps (non-Aspire) by updating the container app properties in the main provisioning flow, then patching the container app image during  azd deploy.
The primary pain point is that users do not have the ability to patch the container app in a single revision with the provision + deploy flow (#3116). A secondary pain point is that if the user doesn't store the container app existence state in CI/CD scenarios, the container app image may be reset during CI/CD provisioning. The UX is also slightly more challenging since multiple parts in the application have to be set up and coordinate well together to build up the overall experience.
There are two ways we can consider splitting out container app deployment from provisioning:

Container app deployed using Bicep
Container app deployed using YAML

Container app deployed using Bicep
This is familiar for a person that is using and knowing Bicep. It is also quite common to have multiple Bicep modules that are deployed and managed independently. Bicep allows the user to reference any existing provisioned resources and link it to fill application environment settings.
The main drawback from azd's perspective is that azd interacts with Bicep opaquely. azd doesn't know if the user had provisioned other resources within the Bicep module. azd can only communicate with the Bicep module via input/outputs just like other provisioning modules.
Historically, this deployment mechanism was previously used by azd. It was noted to be slow, however this may have changed recently.
Container app deployed using YAML
This is familiar for a person that has prior K8s experience. A manifest style approach for an app operator that doesn't need to touch infrastructure. This provides the right amount of separation for the app developer without having to learn IaC.
The expectation would then be that azd needs to provide and document templating functions to "tie-in" the container app to the rest of the application. At the very least, I could imagine:

The context environment variables (especially a managed identity, if being used)
The container app environment ID
A way to reference KeyVault secrets
A way to reference properties (perhaps $.jsonpath) of arbitrary ARM resources by resource ID

Discussion
Should container app deployment be split from provisioning?
Do you prefer deploying container app using Bicep or YAML?

I would be hesitant to complicate the ACA deployment scenario with another YAML spec.
Can you provide an example of:

Customer complaints/concerns on this
Demo of existing issues
Demo of proposed solution

Thanks

Agree on the pain points listed above.  It is a tricky situation since the Bicep (CAE + ACA) today is a full provision+deploy that can contain references to the container images being used by the service.
When your application container images aren't available yet then you won't have a final container image to use and pass to your bicep module to set the parameter(s).
When thinking about the separation of provisioning & deployment when it comes to ContainerApps, the ACA environment is the resource that theoretically could be provisioned, but the ACA apps themselves are not created until the deploy stage (similar to Aspire).
Then in the deploy stage we could then choose to either user bicep or yaml.  Some customers may prefer bicep while other prefer yaml and maybe we need to support both.
In a bicep world the module can take many parameters used to set container image references as well as other configurable properties.  Also - if azd is moving to deployment stacks, then each ACA deployment would need to be a separate stack and deployed independently.  This opens the door for root level infra to be considered "platform" infra while each service can provide service level infra like ACA apps, etc.
In Yaml, I agree that we would need some text templating language to allow the injection of azd environment/configuration to form a complete well formed yaml document.
If I had to pick one path to start with I'd opt for CAE to be provisioned during standard provision stage and ACA apps to be created via YAML deployment via azd deploy stage.

If I had to pick one path to start with I'd opt for CAE to be provisioned during standard provision stage and ACA apps to be created via YAML deployment via azd deploy stage.

This is what we did for Aspire.  For .NET 9 we are changing the model such that instead of using YAML for deployment (which required us to talk to the container apps RP directly, and invent a templated yaml logic in the style of helm) to generate a bicep file for just the container app which we will deploy using the ARM control plane, during the deploy phase.
Work for that is being prototyped in #4286.
We moved away from YAML and towards Bicep because Aspire is working with the .NET Team to produce a CDK which makes it easy to generate Bicep using C# and we felt this would end up being more natural for users long term instead of yet another IaC format designed specifically by azd.


Partially fix #819

@jbogard, I have added one unit test that covers the situation

Sometimes I think that the issue should not be resolved on Mediator end, as some other containers (not MS.Ext.DI, but for example Lamar) does support contravariance and it was working before the proposed patch.
This patch should only be applied to MS.Ext.DI and not all other containers, @asimmon could you suggest me how to do such?
cc @jbogard

Yeah I'm more inclined to let 3rd-party containers take care of this. They'll do the job much better than I can, and I've already taken on a bit much by adding the scanning for handlers.

For MS.Ext.DI I have already (re-)opened the issue: https://github.com/dotnet/runtime/issues/82372
They already said "wontfix" back in 2016, but I have hope for such change today.

@tvardero Before 12.x, MediatR was container-agnostic_-ish_. If you look at v11.1.0/src/MediatR/Mediator.cs, there was a ServiceFactory acting as a DI abstraction layer.
In 12.x, IServiceProvider from Ms.Ext.DI is the built-in way of resolving dependencies. I personally think this a valid approach as ASP.NET Core, hosting extensions and many Microsoft and third-party libraries rely on this by default. It's part of the .NET ecosystem. Too ensure projects longevity and smooth migrations to more recent .NET versions in the next 5 years at least, developers should rely on Ms.Ext.DI.

I would expect developers to use IRequest and IRequestHandler instead.

Isn't behavior of the IRequest (or we could say IRequest<void> / IRequest<Unit>) and it's IRequestHandler will be the same as with INotification and it's handler? Except for exceptions handling probably, INotification should work as fire-and-forget. Anyway, the #819 was about resolving a handler. I haven't done any changes to IRequestHandler resolution with MS.Ext.DI, but I assume there will be the same issue with contravariance.

Isn't behavior of the IRequest (or we could say IRequest<void> / IRequest<Unit>) and it's IRequestHandler will be the same as with INotification and it's handler? Except for exceptions handling probably, INotification should work as fire-and-forget.

No, it's not. There is an expectation of a single handler for a single request, and 0-n handlers for a single notification. This is enforced by how we use the container, it will throw an exception without a request handler, and do nothing without a notification handler.
Generic variance makes sense in the context of notification handlers and behaviors. NOT in request handlers. Those should be closed generics, never open.

So, I did the benchmarks by isolating the path that resolve the INotificationHandler<T> (so just before we create the wrappers) to measure the impact of your changes.
TL;DR: it is 10x slower and consumes also 10x more memory.
|      Method |       Mean |    Error |   StdDev | Ratio | RatioSD |   Gen0 | Allocated | Alloc Ratio |
|------------ |-----------:|---------:|---------:|------:|--------:|-------:|----------:|------------:|
| BeforePR855 |   156.5 ns |  1.79 ns |  1.68 ns |  1.00 |    0.00 | 0.0162 |     136 B |        1.00 |
|  AfterPR855 | 1,575.5 ns | 14.58 ns | 13.64 ns | 10.07 |    0.13 | 0.1659 |    1392 B |       10.24 | <--- This PR

The benchmark code can be found here.
While I completely agree that you probably don't need to worry about MediatR's performance, in my opinion this PR is a big no no. I'd like your opinion about this @jbogard  if you're keen.
If I had to summarize:

10x slower and 10x more allocations due to reflection
Usage of SelectMany() on IServiceProvider.GetServices which frankly seems odd
Handler order disrupted. Usually IServiceProvider.GetServices takes care of returning services in their registration order, but here we interfere with that.


DO NOT MERGE! Fast Forward stable to 66c0993cb647f98a03bdbeb2978ad52ab3680e01.

This PR will be merged automatically once checks are successful.

This change is

Jenkins job mixer/postsubmit passed


add those links

Paper about mobile device, embedded, efficient key words:
http://www.arxiv-sanity.com/search?q=mobile+device+efficient+embedded
csarron/emdl: Embedded and mobile deep learning research resources
https://github.com/csarron/emdl

added


Update additional-review.md

make readable code format

@hareom284 I think there's a misunderstanding here. This form should be rendered as a form so people can use it, not as readable code.
Thank you!


completely rebuild map

work in progress

make main path more approaches/fluent, make difficulty gradient for smooth to the sides, push some hard challenges further down the way

work in progress


[FEATURE REQ] - Introduce a way to instrument ARM calls from APIOps with a specific user-agent header

Please describe the feature.
We could set the User-Agent header to include "APIOps" (ideally at the start of the header, so the queries are more performant) and potentially even the version of APIOps. The User-Agent header is logged as a column in Kusto for all ARM logs, so we could then easily filter by it. We just need to watch out to make sure that SDKs, or whatever we are using for API calls, don't override this header.
Lets make sure we include a disclaimer following the guidance discussed here. Alternatively, we could just mention that we set this header to better understand the use of APIOps, but there's no personal data (PII) logged or captured - it would only allow us to see which API Management services use APIOps.

We now have telemetry to better understand usage of various APIOps versions.


[Bug]: Jest 27 - useFakeTimers erases performance.mark and friends

Version
27.3.1
Steps to reproduce
window.performance.mark = () => {}
jest.useFakeTimers()
assert(typeof window.performance.mark === 'undefined')

jest.useRealTimers()
assert(typeof window.performance.mark === 'function')

jest.useFakeTimers("legacy")
assert(typeof window.performance.mark === 'function')

Expected behavior
jest.useFakeTimers() to not override global polyfills/mocks of performance.mark
Actual behavior
jest.useFakeTimers() overrides global polyfills/mocks of performance.mark, and sets it to undefined.
Additional context
I found a similar issue in @sinonjs/fake-timers https://github.com/sinonjs/fake-timers/issues/136. Don't know if it's related to jest though.
Environment
System:
    OS: macOS 12.0.1
    CPU: (10) arm64 Apple M1 Pro
Binaries:
    Node: 16.13.0 - /usr/local/bin/node
    Yarn: 3.0.0-rc.9 - /opt/homebrew/bin/yarn
    npm: 8.1.0 - /usr/local/bin/npm

Not clear why this is an issue? Could you describe some use case, please.
Or you try to say that Mock Timers documentation should also mention performance similarly to the docs of @sinonjs/fake-timers?

The issue is that there is no way for a developer currently to mock performance.mark() because useFakeTimers overrides the mock set up in a test setup file, and at the same time it doesn't provide its own mock for the method.
Let me know if more clarification is needed

Is it only an issue for modern timers? If so, this needs to be fixed upstream in sinon I think

If I got it right, Sinon is adding performance to clock.performance.now(). See: https://github.com/sinonjs/fake-timers#clockperformancenow
It seems like window.performance is erased, because this is a read only prop. And then they make it available on clock object.
At least two options: either to expose clock.performance somehow in Jest; or to not erase it. But this also would mean that it will not be mocked. That’s why I was curious about the use case. How to test this? What is the need?
All this is only modern timers related. Legacy timers leave performance. If that works as expected, then perhaps modern timers also should not touch performance. Hm.. It sounds better to expose clock.performance somehow.

They have some node 16 issues, might be related https://github.com/sinonjs/fake-timers/issues/394

The use-case is pretty straight-forward:

I have some code that does performance.mark() somewhere deeply in its implementation
I want to test some parts of this code and I don't care about performance.mark() and I'm fine with it just being a noop.
I put a "global" stub for performance.mark() in my setupFile.js
I expect that performance.mark() always will be around so my tests don't crash.

I'd be ok with any way of making it happen, even things like jest.useFakeTimers({ ignore: [window.performance] }) or whatever

Sorry just realized my code snippet in the description was incorrect and confusing, updated it now.

I think this issue could be fixed pretty easily be just making avoiding to fake performance. This is already possible by telling @sinonjs/fake-timers, but we have just forgot to update our docs ... 🙈  Ref https://github.com/sinonjs/fake-timers/issues/409, https://github.com/DefinitelyTyped/DefinitelyTyped/pull/56961#pullrequestreview-797296492 and https://github.com/sinonjs/fake-timers/issues/374.
@SimenB  I am not to well versed in Jest's use of fake timers, but I do not see any way of passing options to customise its timer faking. If we could exclude performance this issue would go away.
p.s. Thanks for the ☕

https://github.com/facebook/jest/releases/tag/v28.0.0-alpha.8

@benjaminhr - please take a look


Upgrade securevault version

$subject

PR builder started
Link: https://github.com/wso2/product-is/actions/runs/11814222778

PR builder completed
Link: https://github.com/wso2/product-is/actions/runs/11814222778
Status: success


Issues converting d-specs to free form

Converting Procedure Interfaces (PI), it is not converting correctly.
Dcl-PI jrn_fix_02  extpgm('JRN_FIX_02');
Lib_to_Check   Char(10a);
New_Jrn        Char(10a);
New_Jrn_Lib    Char(10a);
End-PI;
It correctly changes to CHAR() but the a should be removed.
Old Code:
d wfield 10a
The other issue I found is it randomly adds End-PR:

@a4ginatl I cannot reproduce this.
Which version of RPGLE Free are you using?
Please provide the before source so I can try exactly what you have.

We are on v7.3.
I cleaned up the program and do not have an example right now.
I’ll try to find an existing program that I can show you the issue. That will be in a few days. I have some deadlines to meet.
Thank you.
Darryl Freinkel
A4G
Telephone:<PHONE_NUMBER> Mobile:<PHONE_NUMBER>
From: Brian Garland @.>
Sent: Thursday, November 17, 2022 8:41 AM
To: BrianGarland/vscode-rpgfree @.>
Cc: a4ginatl @.>; Mention @.>
Subject: Re: [BrianGarland/vscode-rpgfree] Issues converting d-specs to free form (Issue #51)
@a4ginatl https://github.com/a4ginatl  I cannot reproduce this.
Which version of RPGLE Free are you using?
Please provide the before source so I can try exactly what you have.
—
Reply to this email directly, view it on GitHub https://github.com/BrianGarland/vscode-rpgfree/issues/51#issuecomment-1318655030 , or unsubscribe https://github.com/notifications/unsubscribe-auth/A4EMVLYRMS4JWI23IM7M5XTWIYYU7ANCNFSM6AAAAAAR53RSNY .
You are receiving this because you were mentioned.  https://github.com/notifications/beacon/A4EMVL5WTHC5E6MOK5NMK43WIYYU7A5CNFSM6AAAAAAR53RSN2WGG33NNVSW45C7OR4XAZNMJFZXG5LFINXW23LFNZ2KUY3PNVWWK3TUL5UWJTSOTEKDM.gif Message ID: @.*** @.***> >

Not hearing anything I'm going to close this issue.
Please make sure you are on the latest version of VS Code and the extensions.
If you can reproduce it while using the latest please open a new issue with the sample code before and after the conversion.

Please close.
Thanks
Darryl Freinkel
A4G
Telephone:<PHONE_NUMBER> Mobile:<PHONE_NUMBER>
From: Brian Garland @.>
Sent: Wednesday, February 15, 2023 2:28 PM
To: BrianGarland/vscode-rpgfree @.>
Cc: a4ginatl @.>; Mention @.>
Subject: Re: [BrianGarland/vscode-rpgfree] Issues converting d-specs to free form (Issue #51)
Not hearing anything I'm going to close this issue.
Please make sure you are on the latest version of VS Code and the extensions.
If you can reproduce it while using the latest please open a new issue with the sample code before and after the conversion.
—
Reply to this email directly, view it on GitHub https://github.com/BrianGarland/vscode-rpgfree/issues/51#issuecomment-1431901441 , or unsubscribe https://github.com/notifications/unsubscribe-auth/A4EMVL4PBLRK3ZBNV4FJABLWXUU4TANCNFSM6AAAAAAR53RSNY .
You are receiving this because you were mentioned.  https://github.com/notifications/beacon/A4EMVL2OMQLNMK4MRIEPLYLWXUU4TA5CNFSM6AAAAAAR53RSN2WGG33NNVSW45C7OR4XAZNMJFZXG5LFINXW23LFNZ2KUY3PNVWWK3TUL5UWJTSVLEKQC.gif Message ID: @.*** @.***> >


Tracker: remaining PCS urls

Custom_UnsignedAction

[x] add projectSchema
[x] add commPkgUrlId
[x] add swcrUrlId

Custom_UnsignedCommissioningPackageTask

[x] CommPkgUrlId

Custom_WorkorderChecklist

[x] WorkorderUrlId
[x] ChecklistUrlId

Custom_LoopSidesheetChecklists

[x] ProjectSchema

These are the ones I could find for now

@BenSvend please consolidate this

@ken-mellem
All columns have been added to FAM and are in Production. Except swcrUrlId in UnsignedAction which is currently in Dev.
Side note: In the views CommPkgUrlId = CommissioningPackageUrlId.
@Gustav-Eikaas Can you confirm that the issue has been correctly solved?

Poke  @Gustav-Eikaas

LGTM, well done @BenSvend!

@Gustav-Eikaas How do I test/verify this?

@AreWiv Good question, maybe we should have a teams call discussing how these things should be tested. Some of the issues/pr's cant really be tested themselves before they are exposed in frontend

Changes have been in production since June 22nd. Probably safe to assume that we can close this issue? @Gustav-Eikaas

Yeah as far as I can tell it's resolved


电影全部跳过 无法抓取字幕

你使用的 chinesesubfinder 是什么版本，什么环境？
ChineseSubFinder Version: 0.19.6
环境: docker
你遇到什么问题了？
设定目录内的电影被全部跳过，没有抓取
你的问题弄重现嘛？
能够重现
你期望的结果
没有字幕的电影能够启动抓取不要跳过
可以提供遇到问题时候，电影、连续剧的信息吗？
###部分文件截图（按名称排序，可以看到没有字幕文件）

###举例：《德州巴黎》Jellyfin截图，显示没有字幕

###附件：《德州巴黎》nfo文件
Paris.Texas (1984) - REMASTERED.1080p.BluRay.x265-RARBG.nfo.txt
配置文件内容如下
UseProxy: true
HttpProxy: http://<IP_ADDRESS>:1081
EveryTime: 9h
Threads: 1
RunAtStartup: true
MovieFolder: /media/Movie
SeriesFolder: /media/Show
SubNameFormatter: 1
###配置文件下载：
config.yaml.txt
出错的日志信息可以复制一些出来吗？
log文件：
ChineseSubFinder--2022Len02040000--.log.txt
下载最新配置示例文件...,
文件 "/config/config.yaml" 已存在，使用该文件作为配置...,
启动ChineseSubFinder，生成的日志以及setting.db和config.yaml保存在一起...,
[INFO]: 2022-02-04 11:01:26 - ChineseSubFinder Version: 0.19.6,
[INFO]: 2022-02-04 11:01:26 - UseProxy: http://<IP_ADDRESS>:1081,
[INFO]: 2022-02-04 11:01:28 - ProxyTest Target Site http://google.com Speed: 1610 Status: 200,
[INFO]: 2022-02-04 11:01:28 - MovieFolder: /media/Movie,
[INFO]: 2022-02-04 11:01:28 - SeriesFolder: /media/Show,
[INFO]: 2022-02-04 11:01:28 - HotFix Start, wait ...,
[INFO]: 2022-02-04 11:01:28 - If you have a lot of videos, it may take a lot of time, just wait...,
,
2022/02/04 11:01:28 /build/internal/dao/init.go:74 SLOW SQL >= 200ms,
[318.359ms] [rows:0] CREATE TABLE hot_fixes (id integer,created_at datetime,updated_at datetime,deleted_at datetime,key text,done numeric,PRIMARY KEY (id)),
,
2022/02/04 11:01:29 /build/internal/pkg/sqlite/migrator.go:242 SLOW SQL >= 200ms,
[422.070ms] [rows:0] CREATE INDEX idx_hot_fixes_deleted_at ON hot_fixes(deleted_at),
,
2022/02/04 11:01:30 /build/internal/dao/init.go:74 SLOW SQL >= 200ms,
[757.043ms] [rows:0] CREATE TABLE sub_format_recs (id integer,created_at datetime,updated_at datetime,deleted_at datetime,format_name integer,done numeric,PRIMARY KEY (id)),
,
2022/02/04 11:01:30 /build/internal/pkg/sqlite/migrator.go:242 SLOW SQL >= 200ms,
[611.239ms] [rows:0] CREATE INDEX idx_sub_format_recs_deleted_at ON sub_format_recs(deleted_at),
[INFO]: 2022-02-04 11:01:30 - hotfix 001 start...,
[INFO]: 2022-02-04 11:06:01 - hotfix 001 is done,
[INFO]: 2022-02-04 11:06:01 - HotFix End,
[INFO]: 2022-02-04 11:06:01 - Change Sub Name Format Start...,
[INFO]: 2022-02-04 11:06:01 - If you have a lot of videos, it may take a lot of time, just wait...,
,
2022/02/04 11:06:01 /build/internal/pkg/hot_fix/hot_fix_hub.go:58 SLOW SQL >= 200ms,
[328.660ms] [rows:1] INSERT INTO hot_fixes (created_at,updated_at,deleted_at,key,done) VALUES ("2022-02-04 11:06:00.734","2022-02-04 11:06:00.734",NULL,"001",true),
,
2022/02/04 11:06:01 /build/internal/pkg/sub_formatter/sub_format_changer.go:222 record not found,
[1.191ms] [rows:0] SELECT * FROM sub_format_recs WHERE sub_format_recs.deleted_at IS NULL ORDER BY sub_format_recs.id LIMIT 1,
[INFO]: 2022-02-04 11:10:27 - Change Sub Name Format End,
[INFO]: 2022-02-04 11:10:27 - First Time Download Start,
,
2022/02/04 11:10:27 /build/internal/pkg/sub_formatter/sub_format_changer.go:234 SLOW SQL >= 200ms,
[291.134ms] [rows:1] INSERT INTO sub_format_recs (created_at,updated_at,deleted_at,format_name,done) VALUES ("2022-02-04 11:10:27.652","2022-02-04 11:10:27.652",NULL,1,true),
[INFO]: 2022-02-04 11:10:27 - Download One Started...,
[INFO]: 2022-02-04 11:10:27 - Download Movie Sub Started...,
[INFO]: 2022-02-04 11:10:41 - Skip Amadeus (1984) - Directors.Cut.BD-1080p.X264.AAC-99Mp4.mp4 Sub Download, because movie has sub and published more than 2 years,
[INFO]: 2022-02-04 11:10:53 - Skip Another.Round (2020) - 1080p.BluRay.H264.AAC-VXT.mp4 Sub Download, because movie has sub and downloaded or aired more than 3 months,
[INFO]: 2022-02-04 11:11:06 - Skip Asako.I.&.II (2018) - BD1080P.X264.AAC.Japanese.CHS.Mp4Ba.mp4 Sub Download, because movie has sub and published more than 2 years,
[INFO]: 2022-02-04 11:11:18 - Skip Bad Luck Banging or Loony Porn (2021).mp4 Sub Download, because movie has sub and downloaded or aired more than 3 months,
[INFO]: 2022-02-04 11:11:31 - Skip Confessions (2010) - JAPANESE.1080p.BluRay.H264.AAC-VXT.mp4 Sub Download, because movie has sub and published more than 2 years,
[INFO]: 2022-02-04 11:11:44 - Skip Devils on the Doorstep(2000) - 1080p.DVD-CnSCG.mkv Sub Download, because movie has sub and published more than 2 years,
[INFO]: 2022-02-04 11:11:57 - Skip Dont.Look.Up (2021) - 1080p.WEBRip.x264-RARBG.mp4 Sub Download, because movie has sub and downloaded or aired more than 3 months,
[INFO]: 2022-02-04 11:12:09 - Skip Drive.My.Car (2021) - JAPANESE.1080p.HC.WEBSCR.DD2.0.x264.mkv Sub Download, because movie has sub and downloaded or aired more than 3 months,
[INFO]: 2022-02-04 11:12:21 - Skip Eros (2004) - 1080p.WEBRip.x264-RARBG.mp4 Sub Download, because movie has sub and published more than 2 years,
[INFO]: 2022-02-04 11:12:33 - Skip Fellini.Satyricon (1969) - ITALIAN.1080p.BluRay.H264.AAC-VXT.mp4 Sub Download, because movie has sub and published more than 2 years,
[INFO]: 2022-02-04 11:12:45 - Skip From.Up.on.Poppy.Hill (2011) - PROPER.JAPANESE.1080p.BluRay.H264.AAC-VXT.mp4 Sub Download, because movie has sub and published more than 2 years,
[INFO]: 2022-02-04 11:12:56 - Skip Funny Face (1957) - 1080p.BluRay.H264.AAC-RARBG.mp4 Sub Download, because movie has sub and published more than 2 years,
[INFO]: 2022-02-04 11:13:09 - Skip Hokusai (2020).mp4 Sub Download, because movie has sub and downloaded or aired more than 3 months,
[INFO]: 2022-02-04 11:13:20 - Skip It.Feels.So.Good(2019) - BD1080P.X264.AAC.Japanese.CHS.BdRar.mp4 Sub Download, because movie has sub and published more than 2 years,
[INFO]: 2022-02-04 11:13:32 - Skip La.Belle.Epoque (2019) - BluRay.1080p.DTS-HD.MA.5.1.x265.10bit.mkv Sub Download, because movie has sub and published more than 2 years,
[INFO]: 2022-02-04 11:13:44 - Skip Lolita (1962) - 1080p.BluRay.x265-RARBG.mp4 Sub Download, because movie has sub and published more than 2 years,
[INFO]: 2022-02-04 11:13:56 - Skip Paprika (2006) - JAPANESE.1080p.BluRay.x264.DTS-FGT.mkv Sub Download, because movie has sub and published more than 2 years,
[INFO]: 2022-02-04 11:14:08 - Skip Paris.Texas (1984) - REMASTERED.1080p.BluRay.x265-RARBG.mp4 Sub Download, because movie has sub and published more than 2 years,
[INFO]: 2022-02-04 11:14:20 - Skip Rose Island (2020).mp4 Sub Download, because movie has sub and downloaded or aired more than 3 months,
[INFO]: 2022-02-04 11:14:32 - Skip Solaris (1972) - BDRip1080p.mkv Sub Download, because movie has sub and published more than 2 years,
[INFO]: 2022-02-04 11:14:44 - Skip Taxi.Driver (1976) - 4K.Remastered.BluRay.1080p.DTS-HD.MA.5.1.x265.10bit-BBQDDQ.mkv Sub Download, because movie has sub and published more than 2 years,
[INFO]: 2022-02-04 11:14:57 - Skip The Year Earth Changed (2021) - 1080p.BD.mp4 Sub Download, because movie has sub and downloaded or aired more than 3 months,
[INFO]: 2022-02-04 11:15:10 - Skip The.Adventures.of.Picasso (1978) - SWEDISH.1080p.WEBRip.x264-VXT.mp4 Sub Download, because movie has sub and published more than 2 years,
[INFO]: 2022-02-04 11:15:23 - Skip The.Fabulous.Baker.Boys (1989) - 1080p.BluRay.H264.AAC-RARBG.mp4 Sub Download, because movie has sub and published more than 2 years,
[INFO]: 2022-02-04 11:15:35 - Skip The.French.Dispatch (2021) - 1080p.BluRay.DTS.x265-10bit-fh.mkv Sub Download, because movie has sub and downloaded or aired more than 3 months,
[INFO]: 2022-02-04 11:15:47 - Skip The.Great.Buddha.Plus (2017) - CHINESE.1080p.BluRay.H264.AAC-VXT.mp4 Sub Download, because movie has sub and published more than 2 years,
[INFO]: 2022-02-04 11:15:59 - Skip The.Scent.Of.Green.Papaya (1993) - VIETNAMESE.1080p.BluRay.x265-VXT.mp4 Sub Download, because movie has sub and published more than 2 years,
[INFO]: 2022-02-04 11:16:12 - Skip The.Unbearable.Lightness.of.Being (1988) - 1080p.WEBRip.x265-RARBG.mp4 Sub Download, because movie has sub and published more than 2 years,
[INFO]: 2022-02-04 11:16:24 - Skip Wheel.of.Fortune.and.Fantasy (2021) - JAPANESE.1080p.BluRay.H264.AAC-VXT.mp4 Sub Download, because movie has sub and downloaded or aired more than 3 months,
[INFO]: 2022-02-04 11:16:25 - Download Movie Sub End...,
[INFO]: 2022-02-04 11:16:25 - Download Series Sub Started...,
[INFO]: 2022-02-04 11:16:25 - Download Series Sub End...,
[INFO]: 2022-02-04 11:16:25 - Download One End...,
[INFO]: 2022-02-04 11:16:25 - First Time Download End,

去看主页的电影和连续剧目录结构要求，（手机不方便给出）

去看主页的电影和连续剧目录结构要求，（手机不方便给出）
好的，看首页写的是推荐结构，实际是必需的？

去看主页的电影和连续剧目录结构要求，（手机不方便给出）

好的，看首页写的是推荐结构，实际是必需的？

理论上最差可以用影片的名称搜索，但是我没有测试过全部放在一个文件夹的情况。代码逻辑上优先读取 imdb 信息，但是，全部在一起可能会有误识别问题，也就无法触发文件名搜索。

去看主页的电影和连续剧目录结构要求，（手机不方便给出）

好的，看首页写的是推荐结构，实际是必需的？

理论上最差可以用影片的名称搜索，但是我没有测试过全部放在一个文件夹的情况。代码逻辑上优先读取 imdb 信息，但是，全部在一起可能会有误识别问题，也就无法触发文件名搜索。

明白了，谢谢！


Allow defining Swift functions to be called from Python

Motivation
We would like to create Swift functions that can be called from Python. Specifically, we want to use optuna to optimise some parameters going into one of our Swift functions, so it makes sense for Optuna to call our Swift callback to avoid workarounds.
We had a look at PythonLambda (see other open PR) and found it didn't meet our needs in terms of being able to raise Python exceptions from Swift functions. I also found it difficult to add the desired functionality to their codebase (multiple targets / repos, > 1000 lines of code, lots of repetition).
Implementation
I created the current self-contained PR to allow the same functionality in much less code, including support for exceptions and variadic parameters, and without exposing any PythonKit internals.
Here is a contrived example using optuna, integrating example code from their website:
let optuna = Python.import("optuna")

let objective = PythonFunction { trial in
    let x = Float(trial.suggest_uniform("x", -5, 5))! // Optuna suggests a float in the given range

    // Doesn't actually make sense, just showing the possibility to throw a Python exception:
    if x > 3 || x < -3 {
        // If we were doing expensive work, ignoring this trial would save a lot of time
        throw optuna.TrialPruned()
    }

    return pow(x - 2, 2.0) // this will be optimised to be as close to 0.0 as possible
}

let study = optuna.create_study(
    pruner: optuna.pruners.MedianPruner(),
    sampler: optuna.samplers.TPESampler()
)

study.optimize(objective, n_trials: 100)
print(study.best_params) // x: ~2.00

At the moment there I have only implemented explicit support for functions that take a single argument:
PythonFunction { (x: PythonObject) in x + 1 }
or a variadic array of arguments (0, 1 or many):
PythonFunction { (x: [PythonObject]) in print(x); return x }
I did this deliberately to reduce API surface area to start with, but am open to feedback here as much as anywhere.
Open issues
Python Garbage Collection
I did run across some issues with memory management: unless we opt out of Python's garbage collection, the functions we create seem to get swept away as soon as gc.collect() is run manually or automatically. This causes BAD_ACCESS exceptions. There is a workaround in place that works but I'd appreciate feedback here from someone more experienced than me – maybe there is a better way.
Swift Memory Management
I also ran into an issue with Swift memory management: we currently pass a retained reference to the PythonFunction instance when creating the function - this is not ideal because it means that the Swift class will also never reach a reference count of zero. If we don't do this, we get a segmentation fault when the Python function's PyReference is cleaned up. I'm not actually sure exactly why this happens - I think Python is trying to access our selfPointer after Swift has freed it.
Summary
As it stands there are minor memory leaks involved in using this functionality. The question is: can we fix them with PythonKit's current memory management (without getting deeper into making custom C types "GC-ready"), and does it really matter for a first implementation? (The tiny memory overhead of a single swift / python object leaking may not affect most users).
I'm looking forward to any feedback you might have

The tests are failing because we cannot import Python on Ubuntu. This is fair enough.
I only need to import Python to use the PyMethodDef struct. I can work around this by adding a dummy C target and then importing the system Python headers manually. Or I can define a dummy struct in Swift with the same shape and hope that it aligns properly with the C type.
Assuming you are interested in adding this functionality at all, how would we best work around this issue in your opinion?

@liuliu thanks for your additions- I have commented on the commit. It would make sense to merge your work into mine, I am just not sure about the GC_UnTrack thing at the moment. I could of course only merge this if our code continued to work

@liuliu thanks for your additions- I have commented on the commit. It would make sense to merge your work into mine, I am just not sure about the GC_UnTrack thing at the moment. I could of course only merge this if our code continued to work

Yeah, feel free to verify if you still have the GC_UnTrack (and if still have that, how to reprod it so I can dig). Afterwards, will be happy to open a PR against your repo.

Yeah, feel free to verify if you still have the GC_UnTrack (and if still have that, how to reprod it so I can dig). Afterwards, will be happy to open a PR against your repo.

@ephemer my latest changes should fix all memory related issues and make this PR mergable for @pvieito : https://github.com/liuliu/PythonKit/commit/af00ca39bc5366aff3a30d8841220128d1eec221

Previously, the self pointer is not a PyObject, which can be problematic because: https://github.com/python/cpython/blob/master/Objects/methodobject.c#L109 It doesn't only because we get lucky and the closure block header matches python object header in ways not damaging;
It is easier if we are not going to support 3.0 and below Python to use the new PyCapsule interface https://docs.python.org/3/c-api/capsule.html that wraps native objects, in this way, Python manages the memory of the function object and will release them at the end of capsule usage.

@ephemer my latest changes should fix all memory related issues and make this PR mergable for @pvieito : https://github.com/liuliu/PythonKit/commit/af00ca39bc5366aff3a30d8841220128d1eec221

Previously, the self pointer is not a PyObject, which can be problematic because: https://github.com/python/cpython/blob/master/Objects/methodobject.c#L109 It doesn't only because we get lucky and the closure block header matches python object header in ways not damaging;
It is easier if we are not going to support 3.0 and below Python to use the new PyCapsule interface https://docs.python.org/3/c-api/capsule.html that wraps native objects, in this way, Python manages the memory of the function object and will release them at the end of capsule usage.

@ephemer @pvieito let me know what do you think of my changes. I can put up a separate PR if that can be easier, or @ephemer can integrate into this existing PR since most of these are his ideas :)

@ephemer @liuliu @pvieito I think calling Swift functions from Python is both doable and necessary for the future of Python interop. I did it successfully on Swift-Colab (by communicating through a C-interface dynamic library) to allow subclassing of the Jupyter kernel, a Python object. We need to support function attributes like @tf.function, otherwise we will severely limit the functionality of interop with certain Python libraries.
I don't have the time to finish this PR now, but I imagine I'll be the one doing most of the work if it ever goes through.

@philipturner I've been maintaining my own fork of PythonKit: https://github.com/liuliu/PythonKit/commits/main This version of Python calling into Swift function has been working well for me (I mainly use these for some of lambdas for Pandas e.g.).

@ephemer @pvieito let me know what do you think of my changes. I can put up a separate PR if that can be easier, or @ephemer can integrate into this existing PR since most of these are his ideas :)

@pvieito could you please merge this PR?

@philipturner Hi! The main issue with this PR is that it seems to create a leak each time a PythonFunction is created.
I think this should be resolved before merging the PR.

I implemented something like that in Swift-Colab by making a Python object that wraps the function. When it calls __deinit__, it calls into C-interfaced Swift code that then deallocates the function pointer. Is anything like that possible to resolve this leak?

@philipturner Hi! The main issue with this PR is that it seems to create a leak each time a PythonFunction is created.
I think this should be resolved before merging the PR.

Hi, @pvieito , if that is the concern, my commits in my fork has fixed these a while ago and has been running good on my end for couple of months:
https://github.com/liuliu/PythonKit/commit/d7d8d4398de158294eeb840b6ba6b988b9d78df7
https://github.com/liuliu/PythonKit/commit/049f699d206c71c35ceac35e4c62d8552d7949bb
It wraps our OpaquePointer with PyCapsule, which supports deallocation callback. Because PyCapsule is only available in Python 3.1 and above, with these two commits, we cannot support Python 2 any more.
This PR as it is have 2 other issues also fixed in above two commits:

the second parameter to PyFunction_New expects a Python object pointer, whereas we passed a raw function pointer directly: https://docs.python.org/3/c-api/function.html#c.PyFunction_New
To get a stable static string pointer, we should assume the string literal is StaticString and use utf8Start instead: https://github.com/liuliu/PythonKit/commit/d7d8d4398de158294eeb840b6ba6b988b9d78df7#diff-271e10211ef0b04c649337c9bb6490e494806134bb410abd4efff3109b23e5b4R1593

If Python 2 compatibility is an issue, can’t you just disable or avoid PythonFunctions if using Python 2?

@liuliu That's great! If you could create a new PR with those additions I would be glad to merge them. As Python 2 is EOL there is no problem in only supporting Python 3 with this new feature.

@pvieito could you close this thread?


NCBI taxonomy DB out of date

Hi,
I'm finding a lot of taxon IDs from NCBI's taxonomy database are missing from taxoniq. This looks to be because they were added in 2022 and the latest taxoniq copy of the DB is from late 2021. Is there a way to generate more up to date versions of the database?
Best wishes,
Nicole

I'd like to second this! I'm using the scientific_name parameter, and even for strings that appear identically in the database when I use the online GUI search, produce key errors when using taxoniq.
For example, 'Pseudomonas fluorescens Pf-5' comes up with an identical match online but generates a KeyError with the following code:
import taxoniq

t1 = taxoniq.TAxon(scientific_name='Pseudomonas fluorescens Pf-5')

The error:
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/mnt/home/lotrecks/anaconda3/envs/graphs/lib/python3.10/site-packages/taxoniq/__init__.py", line 209, in __init__
    self.tax_id = self._get_db("sn2t")[scientific_name][0][0]
  File "marisa-trie/src/marisa_trie.pyx", line 577, in marisa_trie.BytesTrie.__getitem__
KeyError: 'Pseudomonas fluorescens Pf-5'

Hi all, thanks for your patience and apologies for the stale data. I ran into issues the last few times I tried to rebuild and publish the database built into Taxoniq (by reindexing Genbank). I will try again shortly and update here as soon as my other obligations allow.

Hi! Just wanted to bump to keep the issue from going stale. I am also running into issues where a bunch of newer taxIDs are missing

Hello.
I would also like to raise my concerns on this issue. Updating this data would be very useful.

Hi all, unfortunately I can't rebuild the taxoniq indexes and the taxonomy DB, because NCBI has corrupted the files for the genbank databases. Starting at some point in early 2022 and ongoing, NCBI's own documented tool for extracting data from genbank files, blastdbcmd -db DB_NAME -entry all, now consistently produces errors of the form "Error: [blastdbcmd] error while reading seqid".
I got in touch with NCBI about this issue, but my message was ignored.
Even though maintaining this package comes at a considerable cost and I receive no funding for it, I have been regularly looking into performing the updates - but this error completely blocks me, as this is the only method I know of to extract all records from a BLAST database.

OK, it looks like I've found a way to work around the error - hoping to make an updated release today or next weekend.

OK, thanks everyone for your patience, I've updated the indexes to the data available as of 2023-11-04, and everything appears to be working correctly after this 2-year hiatus. I'm making improvements to the indexing and release automation to make updating the indexes easier and more affordable in the future (this update cost hundreds of dollars in compute and storage).

'Pseudomonas fluorescens Pf-5' comes up with an identical match online

That is not an identical match, and updating the index won't solve that problem. This works though:
>>> taxoniq.Taxon(scientific_name='Pseudomonas protegens Pf-5')
taxoniq.Taxon(220664)


Support protocols as parameter types and type arguments [phab:D1128]

Created by salguarnieri at 2016-10-28 18:03:37:
Protocols can be used as parameter types, but it is done differently than
for regular classes. Protocols are specified as modifying id like this:
-(int)foo:(id<P1, P2>)param
We handle this notation by creating a new builtin called TypeUnion that is used
in type applications to join one or more types.
Type arguments are like generics. We handle them using standard type
application.

salguarnieri wrote at 2016-10-28 23:00:25:

Add link

zarko commented inline (context lost during migration) at 2016-10-31 20:17:13:
The \param name doesn't match the parameter; in addition, it should probably be 'BoundedTypeId', since types (not terms) participate in bounded/ relationships.

zarko commented inline (context lost during migration) at 2016-10-31 20:17:13:
ID is spelled 'Id' elsewhere in this file; is there a reason for this to be inconsistent?

zarko commented inline (context lost during migration) at 2016-10-31 20:17:13:
///?

zarko commented inline (context lost during migration) at 2016-10-31 20:17:13:
CHECK(T->getInterface() == nullptr);?

zarko commented inline (context lost during migration) at 2016-10-31 20:17:13:
I'm suspicious about the use of file offsets to identify semantic objects here. For C++ templates, there's an alternate version of BuildNodeIdForDecl that takes a subobject index (IndexerASTHooks.h:~394); when visiting things that could be templates, both a node ID for the body and a node ID for the abs are maintained (see e.g. IndexerASTVisitor::VisitVarDecl, BodyDeclNode and DeclNode). Is there a reason you've done this differently here?

zarko commented inline (context lost during migration) at 2016-10-31 20:17:13:
These might be better as facts on the absvar ('foo.variance contravariant' and so on).

zarko commented inline (context lost during migration) at 2016-10-31 20:17:13:
Why not std::set<GraphObserver::NodeId>? (Why are the names important?)

salguarnieri wrote at 2016-11-01 00:49:27:

Address comments

salguarnieri commented inline (context lost during migration) at 2016-11-01 00:49:39:
Names are important so the ordering is predictable to write down in the verification test.

salguarnieri commented inline (context lost during migration) at 2016-11-01 00:49:39:
I believe I fixed this up.

salguarnieri wrote at 2016-11-01 00:52:39:

formatted

salguarnieri wrote at 2016-11-01 16:06:19:

rebased

zarko commented inline (context lost during migration) at 2016-11-02 02:12:40:
See the comments in the tests about this; it's rare that you should need to name the node underneath the abs from outside the abs.

zarko commented inline (context lost during migration) at 2016-11-02 02:12:40:
This should be documented in the schema.

zarko commented inline (context lost during migration) at 2016-11-02 02:12:40:
I'm a little concerned about this. completes is a syntactic relationship between an anchor and the target of some other declaration. Here, the other declaration is BoxAbs (it's what the defines/binding edge points to above). BoxDecl has no syntactic anchor.

zarko commented inline (context lost during migration) at 2016-11-02 02:12:40:
Stale comment?

zarko commented inline (context lost during migration) at 2016-11-02 02:12:40:
Is there a reason to emit both of these here (and not just BoxImpl)?

zarko commented inline (context lost during migration) at 2016-11-02 02:12:40:
This doesn't kind-check for me because BoxImpl doesn't appear to be abstracted over any type variables. Do implementations have the same type bindings as their interfaces?

zarko commented inline (context lost during migration) at 2016-11-02 02:12:40:
There's not enough information here to link directly to FuncImpl? Alternatively, are we OK that we're calling FuncDecl and not tapp(FuncDecl, tapp(ptr, ChildImpl))?

zarko commented inline (context lost during migration) at 2016-11-03 00:00:38:
Followup: there should be just one ref here and it should point to BoxImpl if you can get that information from the compiler or BoxDecl if you can't.

zarko commented inline (context lost during migration) at 2016-11-03 00:00:38:
Followup: we are in fact OK not calling a tapp here; use FuncImpl if you can get that information from the compiler and FuncDecl otherwise.

salguarnieri wrote at 2016-11-03 18:31:13:

Update schema doc
Remove stale comment
Don't point to decl node for generic class
Lookup the method impl
Format
clean comment
spelling

salguarnieri commented inline (context lost during migration) at 2016-11-03 19:34:29:
It doesn't quite match up, I agree. Would it be better to link to the decl because that is where the type arguments are? Or should it stay as the Impl because that is the definition site (which is what we usually prefer)?

salguarnieri commented inline (context lost during migration) at 2016-11-03 19:34:29:
I updated the comment in the code. To summarize:
We find the interface of the receiver, then we see if we have an impl for that interface, then we look for the method in that impl.

salguarnieri wrote at 2016-11-03 19:49:59:

rebased

zarko accepted this revision at 2016-11-03 20:09:45

salguarnieri closed via commit 78dd3c1564305d309631d4caba88b90fb39ad349 at 2016-11-03 20:15:22


chore: api-backend to 0.0.9

chore: Promote api-backend to version 0.0.9

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by:
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approvers:
If they are not already assigned, you can assign the PR to them by writing /assign  in a comment when ready.
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment


Add utility functions for MOF->CIF/VASP/Atoms

Make it easy to render our MOF data into a specific format.

Added utility functions for conversions awhile back


No module named PIL?

It goes wrong when running trackgesture.py. something I didn't install?

PIL is PILLOW package. Did you install it ?
try
pip install pillow

@Alpslee did you try installing pillow ? I am closing this issue. Feel free to reopen in case you still see same error msg.


.NET 8, Function Decorators Not Recognized

I am attempting to migrate an Azure Function from .NET 6 to .NET 8 and from in-process to isolated.
I followed the instructions outlined here: https://github.com/Azure/azure-functions-openapi-extension/blob/main/docs/enable-open-api-endpoints-out-of-proc.md
Including updating the the csproj and performing dotnet restore:

However, it appears that the decorators are still not recognized?

Visual Studio Code indicates: 'OpenApiOperation' is not an attribute class
I have tried dotnet clean, I have tried completely exiting VS Code and restarting.

You need to add these usings instead:
using Microsoft.Azure.WebJobs.Extensions.OpenApi.Core.Attributes;
using Microsoft.Azure.WebJobs.Extensions.OpenApi.Core.Enums;
Best regards,
Remi

But then next problem will come I think. Please confirm if you get the same as me when trying to build your project:

.NET 8 doesn't support in-process worker as of now. More precisely, .NET 8 "delayed" supports the in-process worker. Therefore, you can't use .NET 8 for your in-process worker.
Please refer to this page:

.NET 8 will at first only be available on the isolated worker model. There will be an initial preview soon, and we remain committed to full support on the isolated worker model the same day that .NET 8 becomes generally available. However, there will not be an in-process equivalent at the same time. That will come in a subsequent release in early 2024, and we will be doing what we can to make it available as soon as possible. So, if you are looking to use .NET 8 right when it comes out, you will have an option with the isolated worker model. But if you wish to continue using the in-process model, there will be a slight delay.

.NET 8 doesn't support in-process worker as of now. More precisely, .NET 8 "delayed" supports the in-process worker. Therefore, you can't use .NET 8 for your in-process worker.
Please refer to this page:

.NET 8 will at first only be available on the isolated worker model. There will be an initial preview soon, and we remain committed to full support on the isolated worker model the same day that .NET 8 becomes generally available. However, there will not be an in-process equivalent at the same time. That will come in a subsequent release in early 2024, and we will be doing what we can to make it available as soon as possible. So, if you are looking to use .NET 8 right when it comes out, you will have an option with the isolated worker model. But if you wish to continue using the in-process model, there will be a slight delay.

He is using .NET 8 isolated process:) -> "I am attempting to migrate an Azure Function from .NET 6 to .NET 8 and from in-process to isolated."

.NET 8 doesn't support in-process worker as of now. More precisely, .NET 8 "delayed" supports the in-process worker. Therefore, you can't use .NET 8 for your in-process worker.
Please refer to this page:

.NET 8 will at first only be available on the isolated worker model. There will be an initial preview soon, and we remain committed to full support on the isolated worker model the same day that .NET 8 becomes generally available. However, there will not be an in-process equivalent at the same time. That will come in a subsequent release in early 2024, and we will be doing what we can to make it available as soon as possible. So, if you are looking to use .NET 8 right when it comes out, you will have an option with the isolated worker model. But if you wish to continue using the in-process model, there will be a slight delay.

He is using .NET 8 isolated process:) -> "I am attempting to migrate an Azure Function from .NET 6 to .NET 8 and from in-process to isolated."

Oops! My bad! I should have read more carefully.
@eat-sleep-code @remitonning Would you please be able to provide me with a sample repo so that I can repro on my end?

@justinyoo and @remitonning
I was able to get it to build with the following usings:
using Microsoft.AspNetCore.Http;
using Microsoft.AspNetCore.Mvc;
using Microsoft.Azure.Functions.Worker;
using Microsoft.Azure.WebJobs.Extensions.OpenApi.Core.Attributes;
using Microsoft.Azure.WebJobs.Extensions.OpenApi.Core.Enums;
using Microsoft.Extensions.Logging;
using Microsoft.OpenApi.Models;
using Newtonsoft.Json;
using System.Collections.Generic;
using System.IO;
using System.Threading.Tasks;
using System;

and the following decoration:
[Function("Get")]
[OpenApiOperation(operationId: "Run", tags: new[] { "name" })]
[OpenApiSecurity("function_key", SecuritySchemeType.ApiKey, Name = "code", In = OpenApiSecurityLocationType.Query)]
[OpenApiParameter(name: "email", In = ParameterLocation.Query, Required = true, Type = typeof(string), Description = "The email address of the lead you wish to retrieve")]
[OpenApiResponseWithBody(statusCode: System.Net.HttpStatusCode.OK, contentType: "application/json", bodyType: typeof(string), Description = "The OK response")]

But this is a bit off from the decoration that says to remove any Microsoft.Azure.WebJobs usings. I took this to mean to remove the using Microsoft.Azure.WebJobs.Extensions.OpenApi.Core.Attributes; and using Microsoft.Azure.WebJobs.Extensions.OpenApi.Core.Enums;
The documentation likely needs to be updated to clarify that.
Now can someone tell me why I get the following when trying to launch the Swagger UI in dev? Why is it trying to launch the swagger json on a different port than the app?

try to add this .ConfigureFunctionsWorkerDefaults(worker => worker.UseNewtonsoftJson()) in Program.cs

@viktor11prykhidko doing that results in your aforementioned never-ending loading scenario.

@justinyoo and @remitonning
I was able to get it to build with the following usings:
using Microsoft.AspNetCore.Http;
using Microsoft.AspNetCore.Mvc;
using Microsoft.Azure.Functions.Worker;
using Microsoft.Azure.WebJobs.Extensions.OpenApi.Core.Attributes;
using Microsoft.Azure.WebJobs.Extensions.OpenApi.Core.Enums;
using Microsoft.Extensions.Logging;
using Microsoft.OpenApi.Models;
using Newtonsoft.Json;
using System.Collections.Generic;
using System.IO;
using System.Threading.Tasks;
using System;

and the following decoration:
[Function("Get")]
[OpenApiOperation(operationId: "Run", tags: new[] { "name" })]
[OpenApiSecurity("function_key", SecuritySchemeType.ApiKey, Name = "code", In = OpenApiSecurityLocationType.Query)]
[OpenApiParameter(name: "email", In = ParameterLocation.Query, Required = true, Type = typeof(string), Description = "The email address of the lead you wish to retrieve")]
[OpenApiResponseWithBody(statusCode: System.Net.HttpStatusCode.OK, contentType: "application/json", bodyType: typeof(string), Description = "The OK response")]

But this is a bit off from the decoration that says to remove any Microsoft.Azure.WebJobs usings. I took this to mean to remove the using Microsoft.Azure.WebJobs.Extensions.OpenApi.Core.Attributes; and using Microsoft.Azure.WebJobs.Extensions.OpenApi.Core.Enums;
The documentation likely needs to be updated to clarify that.
Now can someone tell me why I get the following when trying to launch the Swagger UI in dev? Why is it trying to launch the swagger json on a different port than the app?

I have the exact same issue and was trying to figure out if the this openapi nuget is compatible with aspnet integration, can't find any info on this topic..
Still wating on someone to confirm..
#628

But then next problem will come I think. Please confirm if you get the same as me when trying to build your project:

It error says: "The type or namespace name 'Azure' does not exist in the namespace '[your namespace]' (are you missing an assembly reference?)

Are you still seeing this error when using the latest version of Microsoft.Azure.Functions.Worker.Sdk package?

@viktor11prykhidko doing that results in your aforementioned never-ending loading scenario.

I was able to get it to work by doing the following in the configuration which isn't pretty but at least locally it now runs.
options.Servers.Clear();
options.Servers.Add(new OpenApiServer()
{
    Url = $"http://{hostContext.Configuration["WEBSITE_HOSTNAME"]}/api"
});

options.IncludeRequestingHostName = false;

For what it's worth to people, if you are adding the worker AspNetCore extensions, you can use a wrapped ForwardedHeadersMiddleware to patch the HttpRequest before the OpenApi function endpoint processes the request
// Program.cs
var host = new HostBuilder()
    .ConfigureFunctionsWebApplication(app =>
    {
        // get current request, and switch from http to https(request scheme), apply the options per request
        app.UseMiddleware<FunctionApp1.Middleware.ForwardedHeadersMiddleware>();
    })
    .ConfigureServices(services =>
    {
        services.Configure<ForwardedHeadersOptions>(options =>
        {
            options.ForwardedHeaders = ForwardedHeaders.XForwardedFor | ForwardedHeaders.XForwardedProto | ForwardedHeaders.XForwardedHost;
            options.KnownNetworks.Clear();
            options.KnownProxies.Clear();
        });
    })
    ...

// ForwardedHeadersMiddleware.cs
namespace FunctionApp1.Middleware
{
    public class ForwardedHeadersMiddleware : Microsoft.AspNetCore.HttpOverrides.ForwardedHeadersMiddleware, IFunctionsWorkerMiddleware
    {
        public ForwardedHeadersMiddleware(ILoggerFactory loggerFactory, IOptions<ForwardedHeadersOptions> options)
            : base( ctx => Task.CompletedTask, loggerFactory, options)
        {

        }
        public Task Invoke(FunctionContext context, FunctionExecutionDelegate next)
        {
            var httpContext = context.GetHttpContext();

            if (httpContext != default)
                this.ApplyForwarders(httpContext);

            return next(context);
        }
    }
}

But then next problem will come I think. Please confirm if you get the same as me when trying to build your project:

It error says: "The type or namespace name 'Azure' does not exist in the namespace '[your namespace]' (are you missing an assembly reference?)

@remitonning Are you still seeing this error when using the latest version of Microsoft.Azure.Functions.Worker.Sdk package?

Hi @kshyju
I have fixed the error.  I had an old file lying in the projekt that was not doing anything. But it had a wrong name on the namespace. I did not know that swagger was so fussy about the naming of the namespaces. But it´s a good thing though. I could get rid of the unused class.
Best regards,
Remi

But then next problem will come I think. Please confirm if you get the same as me when trying to build your project:

It error says: "The type or namespace name 'Azure' does not exist in the namespace '[your namespace]' (are you missing an assembly reference?)

@remitonning Are you still seeing this error when using the latest version of Microsoft.Azure.Functions.Worker.Sdk package?

Hi @kshyju I have fixed the error.  I had an old file lying in the projekt that was not doing anything. But it had a wrong name on the namespace. I did not know that swagger was so fussy about the naming of the namespaces. But it´s a good thing though. I could get rid of the unused class.
Best regards, Remi

Hi again @kshyju
...and that was a really confusing answer from me. I mixed this post with another. I am sorry.
I managed to solve the problem by also updating the Azure function within Visual Studio. Tools ->  Options -> Project and Solutions -> Azure Functions -> Check for updates. See image below. I hope that helps for anyone out there:)

@justinyoo and @remitonning
I was able to get it to build with the following usings:
using Microsoft.AspNetCore.Http;
using Microsoft.AspNetCore.Mvc;
using Microsoft.Azure.Functions.Worker;
using Microsoft.Azure.WebJobs.Extensions.OpenApi.Core.Attributes;
using Microsoft.Azure.WebJobs.Extensions.OpenApi.Core.Enums;
using Microsoft.Extensions.Logging;
using Microsoft.OpenApi.Models;
using Newtonsoft.Json;
using System.Collections.Generic;
using System.IO;
using System.Threading.Tasks;
using System;

and the following decoration:
[Function("Get")]
[OpenApiOperation(operationId: "Run", tags: new[] { "name" })]
[OpenApiSecurity("function_key", SecuritySchemeType.ApiKey, Name = "code", In = OpenApiSecurityLocationType.Query)]
[OpenApiParameter(name: "email", In = ParameterLocation.Query, Required = true, Type = typeof(string), Description = "The email address of the lead you wish to retrieve")]
[OpenApiResponseWithBody(statusCode: System.Net.HttpStatusCode.OK, contentType: "application/json", bodyType: typeof(string), Description = "The OK response")]

But this is a bit off from the decoration that says to remove any Microsoft.Azure.WebJobs usings. I took this to mean to remove the using Microsoft.Azure.WebJobs.Extensions.OpenApi.Core.Attributes; and using Microsoft.Azure.WebJobs.Extensions.OpenApi.Core.Enums;
The documentation likely needs to be updated to clarify that.
Now can someone tell me why I get the following when trying to launch the Swagger UI in dev? Why is it trying to launch the swagger json on a different port than the app?

I have the exact same issue and was trying to figure out if the this openapi nuget is compatible with aspnet integration, can't find any info on this topic..
Still wating on someone to confirm.. #628

Hi @eat-sleep-code and @davidpetric
When you are using Azure Functions isolated (out-of-process) you should not use the using directives
using Microsoft.AspNetCore.Http;
using Microsoft.AspNetCore.Mvc;
and not use HttpRequest in your function(endpoint) as parameter. Instead use HttpRequestData
and not use IActionResult as result. Instead use HttpResponseData
Se image below for an example.
using directives:

Azure function endpoint:

...and make sure you have
.ConfigureFunctionsWorkerDefaults(worker => worker.UseNewtonsoftJson()
in Program.cs as @viktor11prykhidko mentioned.
Best regards,
Remi

@justinyoo and @remitonning
I was able to get it to build with the following usings:
using Microsoft.AspNetCore.Http;
using Microsoft.AspNetCore.Mvc;
using Microsoft.Azure.Functions.Worker;
using Microsoft.Azure.WebJobs.Extensions.OpenApi.Core.Attributes;
using Microsoft.Azure.WebJobs.Extensions.OpenApi.Core.Enums;
using Microsoft.Extensions.Logging;
using Microsoft.OpenApi.Models;
using Newtonsoft.Json;
using System.Collections.Generic;
using System.IO;
using System.Threading.Tasks;
using System;

and the following decoration:
[Function("Get")]
[OpenApiOperation(operationId: "Run", tags: new[] { "name" })]
[OpenApiSecurity("function_key", SecuritySchemeType.ApiKey, Name = "code", In = OpenApiSecurityLocationType.Query)]
[OpenApiParameter(name: "email", In = ParameterLocation.Query, Required = true, Type = typeof(string), Description = "The email address of the lead you wish to retrieve")]
[OpenApiResponseWithBody(statusCode: System.Net.HttpStatusCode.OK, contentType: "application/json", bodyType: typeof(string), Description = "The OK response")]

But this is a bit off from the decoration that says to remove any Microsoft.Azure.WebJobs usings. I took this to mean to remove the using Microsoft.Azure.WebJobs.Extensions.OpenApi.Core.Attributes; and using Microsoft.Azure.WebJobs.Extensions.OpenApi.Core.Enums;
The documentation likely needs to be updated to clarify that.
Now can someone tell me why I get the following when trying to launch the Swagger UI in dev? Why is it trying to launch the swagger json on a different port than the app?

I have the exact same issue and was trying to figure out if the this openapi nuget is compatible with aspnet integration, can't find any info on this topic..
Still wating on someone to confirm.. #628

Hi @eat-sleep-code and @davidpetric
When you are using Azure Functions isolated (out-of-process) you should not use the using directives using Microsoft.AspNetCore.Http; using Microsoft.AspNetCore.Mvc;
and not use HttpRequest in your function(endpoint) as parameter. Instead use HttpRequestData and not use IActionResult as result. Instead use HttpResponseData.
In you function you can than write for example: HttpResponseData response = req.CreateResponse(HttpStatusCode.OK); await response.WriteStringAsync(result); // Or for example .WriteAsJsonAsync return response;
Se image below for an example.
using directives:
Azure function endpoint:
Nuget-packages:
...and make sure you have .ConfigureFunctionsWorkerDefaults(worker => worker.UseNewtonsoftJson() in Program.cs as @viktor11prykhidko mentioned.
Best regards, Remi

Hi thanks for getting back,
Problem is I am using AspNetCore integration which mostly works with OpenAPI(I had opened a issue to have a official answer but no one from team responded)
The problem is that the swagger will use a different PORT than the azure function port for some reason..

@viktor11prykhidko doing that results in your aforementioned never-ending loading scenario.

I was able to get it to work by doing the following in the configuration which isn't pretty but at least locally it now runs.
services.AddSingleton(_ =>
{
  var options = new OpenApiConfigurationOptions
  {
    // Your normal options
  };

  options.IncludeRequestingHostName = false;
  options.Servers.Clear();

  if (hostContext.Configuration["WEBSITE_HOSTNAME"].Contains("localhost", StringComparison.InvariantCultureIgnoreCase))
  {
      options.Servers.Add(new OpenApiServer()
      {
          Url = $"http://{hostContext.Configuration["WEBSITE_HOSTNAME"]}/api"
      });
  }
  else
  {
      options.Servers.Add(new OpenApiServer()
      {
          Url = $"https://{hostContext.Configuration["WEBSITE_HOSTNAME"]}/api"
      });
  }

  return options;
});```

Where do you get the hostContext from?

Take a look at what I did to get it to work above #629 (comment)

Where do you get the hostContext from?

When you are using Azure Functions isolated (out-of-process) you should not use the using directives
using Microsoft.AspNetCore.Http;
using Microsoft.AspNetCore.Mvc;

Why is that @remitonning?
Microsoft is in the documentation for upgrading to isolated process using exactly these usings.

Take a look at what I did to get it to work above #629 (comment)

@joeliir Where do you get the hostContext from?

it's from ConfigureServices extension method:

Take a look at what I did to get it to work above #629 (comment)

@joeliir Where do you get the hostContext from?

This is a program.cs that works. You can see where I'm getting the hostContext from below:
using System;
using Microsoft.Azure.Functions.Worker;
using Microsoft.Azure.WebJobs.Extensions.OpenApi.Core.Abstractions;
using Microsoft.Azure.WebJobs.Extensions.OpenApi.Core.Configurations;
using Microsoft.Azure.WebJobs.Extensions.OpenApi.Core.Enums;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;
using Microsoft.OpenApi.Models;

var host = new HostBuilder()
           .ConfigureFunctionsWebApplication()
           .ConfigureServices((hostContext, services) => {
               services.AddApplicationInsightsTelemetryWorkerService();
               services.ConfigureFunctionsApplicationInsights();

               services.AddSingleton<IOpenApiConfigurationOptions>(_ =>
               {
                   var options = new OpenApiConfigurationOptions
                   {
                       Info = new OpenApiInfo
                       {
                           Version = OpenApiVersionType.V3.ToString(),
                           Title = "",
                           Description = "",
                           TermsOfService = new Uri(""),
                           Contact = new OpenApiContact
                           {
                               Name = "",
                               Email = ""
                           }
                       },
                       Servers = DefaultOpenApiConfigurationOptions.GetHostNames(),
                       OpenApiVersion = OpenApiVersionType.V3,
                       IncludeRequestingHostName = DefaultOpenApiConfigurationOptions.IsFunctionsRuntimeEnvironmentDevelopment(),
                       ForceHttps = DefaultOpenApiConfigurationOptions.IsHttpsForced(),
                       ForceHttp = DefaultOpenApiConfigurationOptions.IsHttpForced(),
                   };

                   options.Servers.Clear();
                   options.Servers.Add(new OpenApiServer()
                   {
                       Url = $"http://{hostContext.Configuration["WEBSITE_HOSTNAME"]}/api"
                   });

                   if (hostContext.HostingEnvironment.IsDevelopment())
                   {
                       options.IncludeRequestingHostName = false;
                   }

                   return options;
               });
           }).Build();

host.Run();

Thanks to @joeliir for the example shared above! It worked for me locally, but it failed when published because the function in Azure was running under https. I first tried using hostContext.HostingEnvironment to determine whether the URL should be http or https, but then I discovered it wasn't necessary. From the documentation and inspecting the source of the DefaultOpenApiConfigurationOptions class, I noticed you can use a server-relative path instead (i.e., /api. This Program.cs works for me in both local development and deployed to Azure Functions:
using Microsoft.Azure.Functions.Worker;
using Microsoft.Azure.Functions.Worker.Extensions.OpenApi.Extensions;
using Microsoft.Azure.WebJobs.Extensions.OpenApi.Core.Abstractions;
using Microsoft.Azure.WebJobs.Extensions.OpenApi.Core.Configurations;
using Microsoft.Azure.WebJobs.Extensions.OpenApi.Core.Enums;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;
using Microsoft.OpenApi.Models;

var host = new HostBuilder()
    .ConfigureFunctionsWebApplication(builder => builder.UseNewtonsoftJson())
    .ConfigureServices(services =>
    {
        services.AddApplicationInsightsTelemetryWorkerService();
        services.ConfigureFunctionsApplicationInsights();

        services.AddSingleton<IOpenApiConfigurationOptions>(_ => new DefaultOpenApiConfigurationOptions
        {
            // The important parts:
            IncludeRequestingHostName = false,
            Servers = [new OpenApiServer { Url = "/api" }],

            // Optional settings:
            Info =
            {
                Version = "1.0.0", // Version of your API
                Title = "",
                Description = "",
                Contact = new OpenApiContact
                {
                    Name = "",
                    Email = ""
                }
            },
            OpenApiVersion = OpenApiVersionType.V3
        });
    })
    .Build();

host.Run();

@viktor11prykhidko @joeliir @eat-sleep-code
This appears to have been fixed in Microsoft.Azure.Functions.Worker.Extensions.Http.AspNetCore 1.2.0. I've confirmed it working on my machine locally.

I think this was solved by PR https://github.com/Azure/azure-functions-dotnet-worker/pull/2149 for this issue https://github.com/Azure/azure-functions-dotnet-worker/issues/2071

I can't follow this thread. Did we indeed determine if these libraries are the correct ones to reference to use the OpenApi attributes?
using Microsoft.Azure.WebJobs.Extensions.OpenApi.Core.Attributes;
using Microsoft.Azure.WebJobs.Extensions.OpenApi.Core.Enums;
Because, as mentioned before, the documentation to migrate to the isolated model from in-proc clearly states in multiple places that references to the Microsoft.Azure.WebJobs.* namespaces should be removed.
https://learn.microsoft.com/en-us/azure/azure-functions/migrate-dotnet-to-isolated-model?tabs=net8#package-references

I can't follow this thread. Did we indeed determine if these libraries are the correct ones to reference to use the OpenApi attributes?
using Microsoft.Azure.WebJobs.Extensions.OpenApi.Core.Attributes; using Microsoft.Azure.WebJobs.Extensions.OpenApi.Core.Enums;
Because, as mentioned before, the documentation to migrate to the isolated model from in-proc clearly states in multiple places that references to the Microsoft.Azure.WebJobs.* namespaces should be removed.
https://learn.microsoft.com/en-us/azure/azure-functions/migrate-dotnet-to-isolated-model?tabs=net8#package-references

Those two Microsoft.Azure.WebJobs.Extensions.OpenApi.Core.* namespaces are in package Microsoft.Azure.WebJobs.Extensions.OpenApi.Core. That package targets .NET Standard 2.0 which makes it usable by both Azure Functions in-process and isolated process assemblies. There was no need to rewrite them with a Microsoft.Azure.Functions.Worker.* namespace. The difference is indeed a little confusing.

Hi, I'm trying to integrate Azure Functions with Swagger. I've successfully done it for .NET 6.0, but now I'm working on a proof of concept (POC) using .NET 8.0. Could someone please provide a step-by-step guide on how to integrate Swagger with Azure Functions in .NET 8?


Make factories compatible with ZF3

ZF3 Upgrade

All factories have to implement the __invoke() function. We have a lot of factories looking like:
use Zend\ServiceManager\FactoryInterface;
use Zend\ServiceManager\ServiceLocatorInterface;

class FooFactory implements FactoryInterface
{
    public function createService(ServiceLocatorInterface $services)
    {
        return new Foo($services->get(Bar::class));
    }
}

these have to be changed to look like
use Interop\Container\ContainerInterface;
use Zend\ServiceManager\FactoryInterface;
use Zend\ServiceManager\ServiceLocatorInterface;

class FooFactory implements FactoryInterface
{
    public function __invoke(ContainerInterface $container, $requestedName, array $options = null)
    {
        return new Foo($container->get(Bar::class));
    }

    public function createService(ServiceLocatorInterface $services)
    {
        return $this($services, Foo::class);
    }
}

most factories where updated. remaining problems are handled in
https://github.com/cross-solution/YAWIK/issues/341


"Malu" is listed in the drop-down menu

Description:
"Malu" is listed in the drop-down menu alongside its corresponding county and commune, according to the law
Precondition:
The website is up an running.

Step 1
Type "Malu" in the search bar.

Expected results
The location is listed in the drop-down menu as "Malu, Argeș (Bârla)".


fix(ui): add control adapters to canvas coherence pass

What type of PR is this? (check all applicable)

[ ] Refactor
[ ] Feature
[x] Bug Fix
[ ] Optimization
[ ] Documentation Update
[ ] Community Node Submission

Description
fix(ui): add control adapters to canvas coherence pass
Related Tickets & Documents

Closes #4619
Closes #4589

QA Instructions, Screenshots, Recordings
I cannot figure out how to get the CLIP Vision model installed but I can confirm that the graph is correct, because I get a Model Not Found error that references this model, when invoking with IP adapter enabled..

@psychedelicious

I cannot figure out how to get the CLIP Vision model installed but I can confirm that the graph is correct, because I get a Model Not Found error that references this model, when invoking with IP adapter enabled..

clip_vision models go in invoke_root>models>any>clip_vision>ip_adapter_sd_image_encoder folder. This folder will have two files - config.json and model.safetensors that you can get from here: https://huggingface.co/InvokeAI/ip_adapter_sd_image_encoder/tree/main
sdxl clip_vision goes in another aptly named folder. More details and download links here: https://github.com/invoke-ai/InvokeAI/pull/4429

@blessedcoolant sweet thanks got it


How do I remove this double border from my split?

Info

Operating System: Endeavouros
Shell: zsh
Terminal: Alacritty
nvim --version: 0.8.1
fzf --version: 0.35

require('fzf-lua').setup({
	winopts = {
		split = "belowright new | resize -10",
	},
	files = {
		previewer = false,
		cmd = "fd --type f --exclude '!{.git,node_modules,target}/'",
	},
	grep = {
		previewer = false,
		cmd = "rg --color=always --smart-case -g '!{.git,node_modules}/'",
	},
	oldfiles = {
		previewer = false,
	},
})

Description
Hello, first of all, I want to thank you for the effort you put into maintaining this awesome plugin. I recently switched back from Telescope and noticed the following when opening a split. As you can see in the image I'm attaching, there's another border above Fzf's split, causing some sort of double border thing. I wonder if one can get rid of it to make it look more accurate? I would appreciate any further suggestion. Thanks in advance.

Ty for the kind words @luisvgs.
This is actually a new feature introduced with fzf 0.35, to disable:
require('fzf-lua').setup({
  fzf_opts = { ['--no-separator'] = '' },
  …
})


Bug in file transfers from previous calculations due to URI path

Details about the quacc environment

quacc version: latest
Python version: 3.11

What is the issue?
When following the example for copying files like WAVECAR from previous directories, I get " Cannot find file" warnings. I believe this is because the file paths in results['dir_name'] are URI that contain the host (e.g. myhost:/my/file/path) which shutil.copy does not understand.
How can we easily reproduce the issue?
Example from quacc docs:
from pathlib import Path
from ase.build import bulk
from quacc.recipes.vasp.core import relax_job, static_job

atoms = bulk("Cu")
results1 = relax_job(atoms)
static_job(results1["atoms"], copy_files=[Path(results1["dir_name"], "WAVECAR")])

After some more digging, I think the problem is that the summary for Vasp jobs come from a vasp TaskDoc, which has the dir_name as a URI.
The TaskDoc is made here:
https://github.com/Quantum-Accelerators/quacc/blob/main/src/quacc/schemas/vasp.py#L88
When the dicts are merged at https://github.com/Quantum-Accelerators/quacc/blob/main/src/quacc/schemas/vasp.py#L124, the dir_name in vasp_task_doc wins and overwrites the one from the default Summary.

This seems to be a sufficient fix (prioritize fields in the base_task_doc over the vasp_task_doc):
unsorted_task_doc = vasp_task_doc  | base_task_doc | additional_fields
https://github.com/Quantum-Accelerators/quacc/blob/main/src/quacc/schemas/vasp.py#L124


expand classpath of pathing jars in scala_legacy command

I think it still be best to delete the command, but we should ensure it works if we're going to ship it
this should have been part of #21121

think it still be best to delete the command,
I agree but it was a collective decision by Scala Core.


Pulse re-runs finished jobs after a process restart

Description
I have a Pulse job that's scheduled to run every midnight on week days (repeatInterval: "0 0 * * 1-5"). When it's finished, Pulse leaves lockedAt: null and lastRunAt with the last run date in the job document (inside pulseJobs MongoDB collection).
Once I restart a Node.js process, Pulse picks up this job again no matter the actual time (the job should run only at midnight). And it happens with every restart, even though the job gets completed, it's not interrupted.
Code example
Pulse client configuration
const config: PulseConfig = {
    processEvery: '30 seconds',
    resumeOnRestart: true,
    defaultLockLimit: 1,
    defaultLockLifetime: 180_000,
};

export const initPulseClient = (db: Db): void => {
    pulse = new Pulse({
        mongo: db,
        ...config,
    });
};

The job configuration
pulse.define(
        'MY_PULSE_JOB',
        async (_job, done) => {
            try {
                await myPulseJob(); // could be any function
                done();
            } catch (error) {
                const msg = 'Failed during publishing signals';
                logger.error(error, msg);
                done(new Error(msg));
            }
        },
        {
            concurrency: 1,
            shouldSaveResult: true,
            lockLifetime: 180_000,
            attempts: 3,
            backoff: {
                type: 'exponential',
                delay: 3_000,
            },
        },
    );

pulse.every('0 0 * * 1-5', 'MY_PULSE_JOB');

Pulse debug logs
pulse:db_init init database collection using name [pulseJobs] +0ms
  pulse:resumeOnRestart Pulse.resumeOnRestart() +0ms
  pulse:db_init attempting index creation +1ms
  pulse:define job [MY_PULSE_JOB] defined with following options:
  pulse:define {
  pulse:define   fn: [AsyncFunction (anonymous)],
  pulse:define   concurrency: 1,
  pulse:define   lockLimit: 1,
  pulse:define   priority: 0,
  pulse:define   lockLifetime: 180000,
  pulse:define   running: 0,
  pulse:define   locked: 0,
  pulse:define   shouldSaveResult: true,
  pulse:define   attempts: 3,
  pulse:define   backoff: { type: 'exponential', delay: 3000 }
  pulse:define } +0ms
  pulse:resumeOnRestart resuming unfinished 1 jobs(2024-10-07T11:47:13.294Z) +98ms
  pulse:db_init index creation success +575ms
  pulse:start Pulse.start called, creating interval to call processJobs every [30000ms] +0ms
  pulse:every Pulse.every(0 0 * * 1-5, 'MY_PULSE_JOB', undefined) +0ms
  pulse:create Pulse.create(MY_PULSE_JOB, [Object]) +0ms
  pulse:job [MY_PULSE_JOB:undefined] computing next run via interval [0 0 * * 1-5] +0ms
  pulse:job [MY_PULSE_JOB:undefined] nextRunAt set to [2024-10-07T20:00:00.000Z] +9ms
  pulse:saveJob attempting to save a job into Pulse instance +0ms
  pulse:saveJob [job undefined] set job props:
  pulse:saveJob {
  pulse:saveJob   name: 'MY_PULSE_JOB',
  pulse:saveJob   data: {},
  pulse:saveJob   priority: 0,
  pulse:saveJob   shouldSaveResult: true,
  pulse:saveJob   attempts: 3,
  pulse:saveJob   backoff: { type: 'exponential', delay: 3000 },
  pulse:saveJob   type: 'single',
  pulse:saveJob   nextRunAt: 2024-10-07T20:00:00.000Z,
  pulse:saveJob   repeatInterval: '0 0 * * 1-5',
  pulse:saveJob   repeatTimezone: null,
  pulse:saveJob   startDate: null,
  pulse:saveJob   endDate: null,
  pulse:saveJob   skipDays: null,
  pulse:saveJob   lastModifiedBy: undefined
  pulse:saveJob } +1ms
  pulse:saveJob current time stored as 2024-10-07T11:47:13.882Z +0ms
  pulse:saveJob job with type of "single" found +0ms
  pulse:saveJob calling findOneAndUpdate() with job name and type of "single" as query +0ms
  pulse:internal:processJobs starting to process jobs: [unknownName:unknownId] +0ms
  pulse:internal:processJobs queuing up job to process: [MY_PULSE_JOB] +0ms
  pulse:internal:processJobs jobQueueFilling: MY_PULSE_JOB isJobQueueFilling: false +0ms
  pulse:internal:processJobs job [MY_PULSE_JOB] lock status: shouldLock = true +0ms
  pulse:internal:_findAndLockNextJob _findAndLockNextJob(MY_PULSE_JOB, [Function]) +0ms
  pulse:internal:_findAndLockNextJob found a job available to lock, creating a new job on Pulse with id [6703c2b8f3d5afdf65e69436] +76ms
  pulse:internal:processJobs job [MY_PULSE_JOB] lock status: shouldLock = true +78ms
  pulse:internal:processJobs [MY_PULSE_JOB:6703c2b8f3d5afdf65e69436] job locked while filling queue +0ms
  pulse:internal:processJobs jobQueueFilling: MY_PULSE_JOB isJobQueueFilling: true +0ms
  pulse:internal:processJobs job [MY_PULSE_JOB] lock status: shouldLock = false +0ms
  pulse:internal:processJobs lock limit reached in queue filling for [MY_PULSE_JOB] +0ms
  pulse:internal:processJobs [MY_PULSE_JOB:6703c2b8f3d5afdf65e69436] about to process job +0ms
  pulse:internal:processJobs [MY_PULSE_JOB:6703c2b8f3d5afdf65e69436] nextRunAt is in the past, run the job immediately +0ms

# Custom logs to see the `job` object
# src/utils/process-jobs.ts:246
# {
#  'job.attrs.nextRunAt': 2024-10-07T11:47:13.294Z,
#  now: 2024-10-07T11:47:13.961Z
# }
# ^ at this point, Pulse overrides `nextRunAt`, so it replaces the midnight rule to `now`

# job.attrs
# {
#     _id: new ObjectId('6703c2b8f3d5afdf65e69436'),
#     name: 'MY_PULSE_JOB',
#     attempts: 3,
#     backoff: { type: 'exponential', delay: 3000 },
#     data: {},
#     endDate: null,
#     priority: 0,
#     repeatInterval: '0 0 * * 1-5',
#     repeatTimezone: null,
#     shouldSaveResult: true,
#     skipDays: null,
#     startDate: null,
#     finishedCount: 4,
#     lastFinishedAt: 2024-10-07T11:46:41.174Z,
#     runCount: 4,
#     lockedAt: 2024-10-07T11:47:13.884Z,
#     type: 'single',
#     nextRunAt: 2024-10-07T11:47:13.294Z
#   }
  pulse:internal:processJobs [MY_PULSE_JOB:6703c2b8f3d5afdf65e69436] processing job +2ms
  pulse:job [MY_PULSE_JOB:6703c2b8f3d5afdf65e69436] setting lastRunAt to: 2024-10-07T11:47:13.964Z +0ms
  pulse:job [MY_PULSE_JOB:6703c2b8f3d5afdf65e69436] computing next run via interval [0 0 * * 1-5] +83ms
  pulse:job [MY_PULSE_JOB:6703c2b8f3d5afdf65e69436] nextRunAt set to [2024-10-07T20:00:00.000Z] +1ms
  pulse:saveJob attempting to save a job into Pulse instance +83ms
  pulse:saveJob [job 6703c2b8f3d5afdf65e69436] set job props:
  pulse:saveJob {
  pulse:saveJob   name: 'MY_PULSE_JOB',
  pulse:saveJob   attempts: 3,
  pulse:saveJob   backoff: { type: 'exponential', delay: 3000 },
  pulse:saveJob   data: {},
  pulse:saveJob   endDate: null,
  pulse:saveJob   priority: 0,
  pulse:saveJob   repeatInterval: '0 0 * * 1-5',
  pulse:saveJob   repeatTimezone: null,
  pulse:saveJob   shouldSaveResult: true,
  pulse:saveJob   skipDays: null,
  pulse:saveJob   startDate: null,
  pulse:saveJob   finishedCount: 4,
  pulse:saveJob   lastFinishedAt: 2024-10-07T11:46:41.174Z,
  pulse:saveJob   runCount: 4,
  pulse:saveJob   lockedAt: 2024-10-07T11:47:13.884Z,
  pulse:saveJob   type: 'single',
  pulse:saveJob   nextRunAt: 2024-10-07T20:00:00.000Z,
  pulse:saveJob   lastRunAt: 2024-10-07T11:47:13.964Z,
  pulse:saveJob   lastModifiedBy: undefined
  pulse:saveJob } +0ms
  pulse:saveJob current time stored as 2024-10-07T11:47:13.965Z +0ms
  pulse:saveJob job already has _id, calling findOneAndUpdate() using _id as query +0ms
  pulse:saveJob processDbResult() called with success, checking whether to process job immediately or not +2ms
  pulse:saveJob processDbResult() called with success, checking whether to process job immediately or not +73ms
  pulse:job [MY_PULSE_JOB:6703c2b8f3d5afdf65e69436] starting job +76ms
  pulse:job [MY_PULSE_JOB:6703c2b8f3d5afdf65e69436] process function being called +0ms

Additional context
Node.js version: 20.12.2 (also happens on 22.x).
MongoDB version: 7.0.
MongoDB Node driver: 6.5.0.

We tested it under the same conditions as yours, but it could not be reproduced."

I'm experiencing the same issue. Using pulse 1.6.6, Node 20.18.0, Mongoose 8.8.2.
const pulse = new Pulse({
  mongo: db.connection.db,
  collection: 'jobs',
  defaultConcurrency: 1,
  maxConcurrency: 2,
  processEvery: '2 minutes',
  resumeOnRestart: true,
});

For example, I have the job below in my database. This job is complete but on every server restart, it gets executed again.
{
  "_id": {
    "$oid": "67406ee23a0dbbb08e3b9fd8"
  },
  "name": "process-upon",
  "data": {
    "documentId": {
      "$oid": "67406ea53a0dbbb08e3b9fcf"
    }
  },
  "priority": 0,
  "shouldSaveResult": false,
  "attempts": 0,
  "backoff": null,
  "type": "normal",
  "nextRunAt": null,
  "runCount": 2,
  "finishedCount": 2,
  "lastFinishedAt": {
    "$date": "2024-11-22T11:46:40.328Z"
  },
  "lockedAt": null,
  "lastModifiedBy": null,
  "lastRunAt": {
    "$date": "2024-11-22T11:46:37.106Z"
  }
}

I wonder if the behavior is coming from using mongo: db.connection.db, which uses an existing database connection, instead of using db: { address: dbUri, collection: 'jobs' } as per docs. Perhaps others with this issue are also doing something similar...?


missing style sheet?

Hi,
Love the theme!
I have tried to install the theme on different platforms...digital ocean/exoscale/ghost pro.
On all platforms I just a get a page that appears to be missing style sheets.
Am i missing something?

Did you build it with webpack? you must build this project before apply this theme on your Ghost.
How?
Just clone this repository into theme path, and build.
Follow commands :
$ git clone https://github.com/if-Team/origami && cd origami
$ npm install
$ npm run build --production

There is the other simple solution that download releases in this repository built, but, however,  it's not up-to-date.

`mrneilypops@nejulinux:/var/www/nejulinux.org/content/themes/origami$ sudo npm install
mrneilypops@nejulinux:/var/www/nejulinux.org/content/themes/origami$ sudo npm run build --production

<EMAIL_ADDRESS>build /var/www/nejulinux.org/content/themes/origami
webpack

loaderUtils.parseQuery() received a non-string value which can be problematic, see https://github.com/webpack/loader-utils/issues/56
parseQuery() will be replaced with getOptions() in the next major version of loader-utils.
Killed
npm ERR! Linux 4.4.0-64-generic
npm ERR! argv "/usr/bin/nodejs" "/usr/bin/npm" "run" "build" "--production"
npm ERR! node v4.7.3
npm ERR! npm  v2.15.11
npm ERR! code ELIFECYCLE
npm ERR<EMAIL_ADDRESS>build: webpack
npm ERR! Exit status 137
npm ERR!
npm ERR! Failed at the<EMAIL_ADDRESS>build script 'webpack'.
npm ERR! This is most likely a problem with the origami package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     webpack
npm ERR! You can get information on how to open an issue for this project with:
npm ERR!     npm bugs origami
npm ERR! Or if that isn't available, you can get their info via:
npm ERR!
npm ERR!     npm owner ls origami
npm ERR! There is likely additional logging output above.
npm ERR! Please include the following file with any support request:
npm ERR!     /var/www/nejulinux.org/content/themes/origami/npm-debug.log
mrneilypops@nejulinux:/var/www/nejulinux.org/content/themes/origami$
`

Sorry, We'll fix ASAP.

No worries...I really like your work...great theme...looking forward to using it!

Thx!
By the way, as you know I'm not a developer of origami. It's @HelloWorld017

loaderUtils.parseQuery() received a non-string value which can be problematic, see webpack/loader-utils#56
parseQuery() will be replaced with getOptions() in the next major version of loader-utils.

are just warnings.
You can skip it.
Or, if you want to use built things, you can download it from releases tab.
(It contains assets directory which contains stylesheets / javascript / fonts)

Hi,
Great work team!
Origami theme up and running at;
https://mrneilypops.io
Thank you.
Neil
On 27/02/17 03:33, Khinenw wrote:

|loaderUtils.parseQuery() received a non-string value which can be
problematic, see webpack/loader-utils#56 parseQuery() will be replaced
with getOptions() in the next major version of loader-utils. |
are just warnings.
You can skip it.
Or, if you want to use built things, you can download it from releases
tab.
(It contains assets directory which contains stylesheets / javascript
/ fonts)
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/if-Team/origami/issues/2#issuecomment-282613617,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABYh0M79ZyDPG0ETZZS-laHRff3xPGGQks5rgjX-gaJpZM4MMd66.

It's glad to hear that theme has been built successfully!
Thank you for using origami!


Add "policy" replacement

Add a similar system to what we do with indexes such that a policy will always back everything.

I've decided this is a bad idea, and it should not be done.

I've decided this is a bad idea, and it should not be done.


tools/build_release.py: fixed port.json files not being sanitized correctly, fixed ports/stop_the_eggspress/port.json file.

This fixes bug #27. :)


Add command history

Adds event handlers for the up and down arrow keys

Codecov Report

Merging #78 (e720fdc) into master (c99bc11) will decrease coverage by 0.77%.
The diff coverage is 0.00%.

@@             Coverage Diff              @@
##             master      #78      +/-   ##
============================================
- Coverage     62.39%   61.62%   -0.78%
  Complexity      460      460
============================================
  Files            90       90
  Lines          1827     1850      +23
  Branches        215      219       +4
============================================
  Hits           1140     1140
- Misses          614      637      +23
  Partials         73       73

Impacted Files
Coverage Δ

src/main/java/safeforhall/ui/CommandBox.java
0.00% <0.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c99bc11...e720fdc. Read the comment docs.


Py-earth installation on Debian giving Deprecation warnings and not identifying version.

I am trying to install using
git clone git://github.com/scikit-learn-contrib/py-earth.git cd py-earth sudo python setup.py install
However, I get stuck at Cloning into 'py-earth' ...
I downloaded the zip folder directly but this leads to version not being identified at the end of installation
Finished processing dependencies for py-earth==0+unknown
and thereby unsuccessful import into Python. How do I fix this? I am using Debian Jessie with Python 2.7.13
Referring to #89, I also removed the package from Downloads folder and unpacked pyearth in site-packages.

The issue was related to multiple versions of Python. A careful check at the installations, and thanks to the nosetests provided, was able to properly get pyearth running.
Thank you and sorry for any trouble caused!


Tool to invert image colours

Description of the new feature / enhancement
A tool to invert the colours in an image, similar to the ctrl+shift+i feature in paint
Scenario when this would be used?
Often times when taking screenshots, they have a bright white background, which clashes with the dark theme in the program i paste it into
Supporting information
No response

What programs do you paste it into?  /needinfo
would love to better understand the end to end scenario

OneNote (for windows 10) is the one i encountered this issue in. I'm sure it exists in other programs too.

i'm gonna bet this would have to paste as invert.  Going to be honest, i doubt we'd implement this feature


How to deploy OVA, if there is no resource pool

https://github.com/vmware/pyvmomi-community-samples/blob/bc14f63065aa360ceca0cca477a8b271d582a090/samples/deploy_ova.py#L158
In my case, I don't have any resource pool. how can I deploy OVA in that case?
Also, I have some other cases, where I don't have a proper data store. How can I deploy OVA by using only host, username, password, cluster, folder?

Hi Team,
Any help with priority is appreciated.

Interested in what a default resourcepool might look like which might be possible to use as workaround.

Adding executed part here:
+ sudo python3 deploy_OVA.py -s HOST.COM -u **** -p **** --ova-path ./ots-rhel-server-7.7_v74.ova -d DATACENTER -ds DATASTORE -n v8-rhel-7.7_beta_1 -m 'v000 - Dist. Test' -f _Templates
Traceback (most recent call last):
  File "deploy_OVA.py", line 517, in <module>
    exit(main())
  File "deploy_OVA.py", line 81, in main
    rp = get_largest_free_rp(si, dc)
  File "deploy_OVA.py", line 204, in get_largest_free_rp
    raise Exception("Failed to find a resource pool in dc %s" % dc.name)
Exception: Failed to find a resource pool in dc DATACENTER

A quick solution would be to Provide the resource pool using the -r option. so that script won't look for the largest RP.

A quick solution would be to Provide the resource pool using the -r option. so that script won't look for the largest RP.

That's when we have a resource pool in the vCenter. How to deploy, if there is no resource pool at all?

I've had success using the --resource-pool flag and passing it "Resources"
pyvmomi-community-samples/samples/deploy_ova.py -s vcsa.contoso.com -u<EMAIL_ADDRESS>-p "abadpassword" -nossl --ova-path 20.04-cloud.ova --datacenter-name Lab --resource-pool Resources --datastore-name 512SSD


Bump google-auth-library-java to 1.0.0 (GA)

Mark anything @BetaApi or @Internal that we want the flexibility to change later (requires adding a dependency on api-common) - only things unused in google-cloud-java and grpc-auth can be marked this way
Remove Guava from the surface
Add back any features removed since 0.4.0 (to maintain compatibility with grpc-auth)
Any other GA cleanup
drop guava-jdk5 support

Upgrade deps:

UTF-8
1.6
1.19.0
4.12
19.0
1.9.34

I don't see any current places where we expose Guava in the public API. Do you know of any?
Do we want to remove the assorted deprecated methods?

Looks like jdk5 is already gone.

GA release happened with https://github.com/googleapis/google-auth-library-java/pull/704
let me know if something is still relevant, but missing


feat: implement i18n for error messages #923 #924

:tada: This PR is included in version 5.4.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


feat(request-templates): API Request Data Mapping

Issue #, if available:
#299
Description of changes:

Add support for VTL request template mapping (matches API Gateway Mapping Template Reference Documentation). The following are supported:

$context
$input
$stageVariables
$util

Request templates must be defined in x-amazon-apigateway-integration in the swagger definition, for example:

...
x-amazon-apigateway-integration:
    uri:
      Fn::Sub: arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${MyFunction.Arn}/invocations
    requestTemplates:
      application/json: "...VTL TEMPLATE HERE..."
...

This is a WIP. Remaining:

[ ] Tests
[ ] Documentation
[ ] Examples

Known caveats/limitations:

$method is not supported.
Java string functions (e.g. replaceAll) are not supported. This means that things like $util.escapeJavaScript($input.json('$.errorMessage')).replaceAll("\\'","'") don't work, but $util.escapeJavaScript($input.json('$.errorMessage')) will.
Kinda unrelated, but something I noticed. Fn::Sub isn't fully implemented so the following will result in the wrong function name being returned:

# parsed function name is CORRECT
Fn::Sub: arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${MyFunction.Arn}/invocations

# parsed function name is INCORRECT (returns "FunctionArn" instead of MyFunction.Arn)
Fn::Sub:
  - arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${FunctionArn}/invocations
  - { FunctionArn: !GetAtt MyFunction.Arn }

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

This pull request will enable VTL in requestTemplates and we have a crucial migration project that's pending this feature. Is it possible to have this in the next release?

@jfuss I find it contradicting with start-api feature since without the VTL that may not work at all due to breakage of APIs which expect a certain event schema. As far as the cloud services, i'm not very familiar but i assume that they do not modify the swagger's hierarchy that often, if at all (so assumption is that requestTemplate mostly stays where it is). That leaves the VTL syntax support, which has been mostly the same for a very long time and this branch only adds a small subset of VTL support as mentioned in the description. Although to add to your concerns, someone could ask for more VTL support and that does drag into the maintenance situation, but the basic can stay as is with minimal maintenance.

Closing due to be inactive and out of date.

@jfuss I would like to restart this PR. I understand the concerns about adding new dependencies.
How about implementing $input.json('$') and $input.params().path['paramname'], for example, instead of trying to support as much expressions as possible? I mean we can choose a few and often used template mappings and implement them without introducing new dependencies. The implementation should look like simple replacements.


Upgrade to Spring Boot 3.4.0

@maciejwalkowiak I already upgraded Boot version in my PR. Can we close this?

Sure!


Unify version output

Summary

Change Type

Check List Before Merging

[ ] This PR passes all pre-commit hook validations.
[ ] This PR is fully tested and regression tests are included.
[ ] Relevant documentation has been updated.

Additional Information

/lgtm


Add section for 2018 program and link to slides

closed with f952fbcebc9e3a5b02ba8a34614d4f2f84040520

current setup on dev branch

@philbowsher

[ ] And add one to github repo.

links added on historical site