Adding objects necessary for mapping Oracle schemas and tables to PostgreSQL databases and tables

Latest improvements:
MSFT employees can try out our new experience at OpenAPI Hub  - one location for using our validation tools and finding your workflow.

Contribution checklist:

[ ] I have reviewed the documentation for the workflow.
[ ] Validation tools were run on swagger spec(s) and have all been fixed in this PR.
[ ] The OpenAPI Hub was used for checking validation status and next steps.

ARM API Review Checklist

[ ] Service team MUST add the "WaitForARMFeedback" label if the management plane API changes fall into one of the below categories.
adding/removing APIs.
adding/removing properties.
adding/removing API-version.
adding a new service in Azure.

Failure to comply may result in delays for manifest application. Note this does not apply to data plane APIs.

[ ] If you are blocked on ARM review and want to get the PR merged urgently, please get the ARM oncall for reviews (RP Manifest Approvers team under Azure Resource Manager service) from IcM and reach out to them.
Please follow the link to find more details on API review process.

Automation for azure-sdk-for-ruby

Automation for azure-sdk-for-python

Can one of the admins verify this patch?

Automation for azure-sdk-for-go

Automation for azure-sdk-for-java

@erich-wang I was told that instead of submitting a new commit for any changes that I should force the new changes onto a single commit. As I am not a git master, do you have guidance on how this is done?

@vchske I think they "squash merge" when landing PRs, so you should be able to just push the additional commit. If you want to cleanup the commits down to 1, you can use a rebase and squash like http://gitready.com/advanced/2009/02/10/squashing-commits-with-rebase.html and then git push -f to force the new single commit as the branch

@vchske , please resolve the comments from @huang91shu  and @hitenjava.

@erich-wang Comments have been resolved and the change is approved so I think we are ready for a merge.

@vchske , I just noticed that you missed the ARM API Review Checklist, so just added WaitForARMFeedback. Of course we could skip this step if @huang91shu is from ARM review board.

@erich-wang We aren't making a breaking change with this. We aren't adding or removing an API endpoint. Based on this, I don't believe an ARM review is necessary. This is essentially the same as https://github.com/Azure/azure-rest-api-specs/pull/5801 and that one also did not require an ARM review.

@vchske , you're adding properties which falls into below checklist:

Service team MUST add the "WaitForARMFeedback" label if the management plane API changes fall into one of the below categories.
adding/removing APIs.
adding/removing properties.
adding/removing API-version.
adding a new service in Azure.

@ravbhatnagar, could you help review this PR from ARM perspective?


Typo in M_erases for multiple heads

There is a typo in M_erases in the case of multiple heads here:
https://github.com/carpedm20/NTM-tensorflow/blob/master/ntm_cell.py#L203-L204
It should be
M_erases.append(tf.ones([self.mem_size, self.mem_dim]) \
                - outer_product(write_w_idx, erase_idx))

@tristandeleu Thanks! I fixed the code


Bug report for Dreams

Flo, please let me know what could work.
A new page which is 'Report bugs' which either leads to a contact sheet (with an email that it sends the bug reports to) or poses an email or links to talk or links to this github.
I would prefer some bug report system that doesn't require sign up. I will see that I re-activate the dreams kiezburn mail address so we can use this one maybe?
It could help us in the early phase of dream to find the stuff we might be missing.

Ideally people should file bugs on this github issues page.
If people don't want to signup then you can also send me an email<EMAIL_ADDRESS>:)

Then add this :)
(Either in a new page saying 'Found a bug?' or 'Bug?' or something on top or in a text box where it is convenient)
"If you find a thing not working on dreams, feel free to file an issue on github or write an email to our rech wizard<EMAIL_ADDRESS>. Thanks <3."


Indent sub-portions of tag fields

This makes the bit breakdown clearer.

Ah, this is a good idea, thanks for this!

Unfortunately due to GitHub Actions flakiness I had to rebase your branch in order to restart CI. Sorry if this caused any inconvenience.

Thanks for the PR!


"DNP" items getting published in BOM

With #37 closed, this plugin shouldn't be including DNP items in either the BOM or the positions file.
However, with the latest 2.3.0 release (which reports itself as 2.2.0), items such as these are still included in the output:

Are you using the latest plugin and KiCad versions?

I'm using 7.0.2:
Application: KiCad PCB Editor arm64 on arm64

Version: 7.0.2-0, release build

Libraries:
	wxWidgets 3.2.2
	FreeType 2.12.1
	HarfBuzz 5.3.1
	FontConfig 2.14.0
	libcurl/7.87.0 (SecureTransport) LibreSSL/3.3.6 zlib/1.2.11 nghttp2/1.51.0

Platform: macOS Ventura Version 13.3.1 (Build 22E261), 64 bit, Little endian, wxMac

Build Info:
	Date: Apr 17 2023 07:41:02
	wxWidgets: 3.2.2 (wchar_t,wx containers)
	Boost: 1.80.0
	OCC: 7.6.3
	Curl: 7.77.0
	ngspice: 38
	Compiler: Clang 13.0.0 with C++ ABI 1002

Build settings:
	KICAD_SPICE=ON

I just re-downloaded the plugin from https://github.com/bennymeg/JLC-Plugin-for-KiCad/releases/tag/2.3.0 and it still reports version 2.2.0, but it does appear as though it's not working.
If I tick Do not populate then it puts a big red X through the component but still includes it in the BOM, positions, and designators files.
This may be an upstream bug, since as you noted there doesn't seem to be any way in pcbnew to know that a component is DNP.

Do not use the tag, use the release, the tag will give you the whore repo, and not the plugin.
it might be an issue with the link between eeschema and pcbnew.

Yep, in particular I downloaded https://github.com/bennymeg/JLC-Plugin-for-KiCad/releases/download/2.3.0/JLC-Plugin-for-KiCad-2.3.0.zip and installed it using "Install from file..."
There are three issues I've run into:

Kicad keeps wanting me to "upgrade" to 2.2.0. I need to remember to ignore the "Update" button because my local copy is newer,
The output path seems different in the newer version -- it's under ~/AppData/Local/Temp/[random-string]/ rather than alongside my Kicad project files, so I was uploading older versions of the board, and
It still adds it to the BOM, it just adds it on its own line with an additional LCSC Part # column set to DNP

If this BOM is fed to JLCPCB it still shows up in the components list with a tickbox next to it and it's ticked by default. Does it make sense to completely omit DNP parts from the BOM and positions files?

This issue is due to JLC linking the components together if it finds a similar Value + Footprint.
I would probably just remove the entrance to BOM + PNP file all together.
Currently I need to edit both manually in a text editor to avoid the components being on

Hi @bennymeg Would you be ok with a PR which just removed the entry to BOM + PNP files if the DNP flag was set?

@Frank-UXV there been some discussion on the matter.
I think the conclusion was that the way it is implemented now is the proper way to go.
Having said that, I'm all for providing flexibility. So if you would be able to add another screen with a checkbox: [ ] ignore DNP part, and a RUN button it would be awesome.
If you would still think these parts should be ignored by default, I would suggest opening a discussion and let the community decide.

@Shackmeister I have added an interface with ignore DNP option. you can use it to remove the entry to BOM + PNP files if it is set.

Thanks Benny! When is there a release scheduled? :)

Released locally in GH.
I will hopefully submit the official PR tomorrow. (I wish I could automate this step, it's quite annoying =/).


Add support for Gemini Structured Outputs

Description
Recently Google released Structured Outputs for Gemini (much like OpenAI).
Should be fairly simple to enable this by updating setup_call for gemini and updating the warning to include gemini as a module that supports strict in the response model schema (although it seems like structured outputs for tools is not quite supported).
See: https://ai.google.dev/gemini-api/docs/json-mode?lang=python#supply-schema-in-config
Only difficulty I can see right now is that the only two supported inputs are either raw type annotations (so no BaseModel) or their genai.protos.Schema type (which will require conversion from BaseModel). Should have most of this implemented already I believe, so still shouldn't be terribly difficult.

hi @willbakst  if this issue's still open, I would love to take this up!

@AbhishekRP2002 it's open! would love for you to take this
if you have any questions let me know! our implementation of OpenAI structured outputs should act as a good reference

sure, will let you know if there's any blocker in any way.

hi @willbakst apologies for no updates since past 1 month, been piled with lots of eng work, so couldn't give this much time.
i was going through the codebase last night and noticed some form of implementation for structured_output in gemini, however i don't see the response_schema param in the config:

i also took a look at the openai implementation,
here is what i am thinking:
adding something like
if response_schema:
        # Convert response_schema to compatible format
        converted_schema = _convert_schema(response_schema)

        call_kwargs['generation_config'] = {
            'response_mime_type': 'application/json',
            'response_schema': converted_schema

i also noticed we can extract structured outputs via function calling in gemini which seemed more close to the openai implementation under _setup_call.py
would be really grateful if u can share your thoughts or help me out here in any way

Ah I can see the confusion here. Hopefully this explanation will clear things up.
There are three types of structured outputs:

Tools (which we attempt to force to be called and then extract / convert into the originally provided model)
JSON mode (same thing as tools except we use JSON to power the structure)
Provider-specific structured outputs (i.e. OpenAI).

We've implemented (1) and (2) across all currently supported providers, but we've only implemented (3) for OpenAI (see here), but Gemini now also supports strict structured outputs.
The idea with this issue would be to add the same support for Gemini that we have for OpenAI (by setting model_config = ResponseModelConfigDict(strict=True)).


Fix output schema generated by CommonSubExprEliminate

Which issue does this PR close?
Closes #3635.
Rationale for this change
The optimization rule CommonSubexprEliminate produces a wrong output schema in many situations. If a logical plan depends on the output schema, it will simply be broken.
What changes are included in this PR?

In the ExprIdentifierVisitor, which is used to determine the datatype of every sub-expression, the datatype is now determined based on the schema of the logical plan for every sub-expression individually instead of determining the datatype of the overall expression and wrongly assigning it to every sub-expression.
The ExprIdentifierVisitor now, accordingly, does not have a datatype attribute but, instead, input_schema and all_schemas.
The latter is necessary for a fall-back logic, where if the datatype of a sub-expression cannot be determined using the input schema, we merge the schemas from all nodes of the overall logical plan and try again.
A test is added to verify the correct datatype of the schema post optimization in the case of optimizing a Filter logical plan.

Are there any user-facing changes?
No.

I will review it tomorrow

Github actions is having issues I think -- the CI failures are not related to changes in this PR

I plan on reviewing this today

Sorry for the late reply, I plan to review it later this day.

I would like to merge this PR -- I'll plan on doing so tomorrow if @waynexia and/or @andygrove  haven't had a chance by then

Thanks again @alex-natzka and @waynexia


add windows habitat plan

This will replace the plan that is currently maintained in core-plans. Unlike the bash plan, this plan does not build from source but grabs the specified version from the packages site instead.

This ignores any logic that may be in the ChefDK installer. If we’re aware of that then I’m fine with it.
Would the strategy be to update this plan only with released/stable versions? Or do we need another way of bumping the versions the same way we do for builds of ChefDK?

@mwrock you'll probably want to implement a similar thing to what was done in the Linux side of things to auto fetch the version from the VERSION file. Otherwise we'll need to get expeditor setup to handle the bump.
https://github.com/chef/chef-dk/blob/master/habitat/plan.sh#L32

the problem with grabbing version from the file is that this is that this plan simply repackages what is on the download site. So if a version does not exist on the public downloads site, the package build will fail.


CON-2691 iphoneX adaptions

Coverage decreased (-0.2%) to 23.712% when pulling 632e028b5a8abd547f11494b2a95013f3845f628 on CON-2691-iphoneX into 43f46233eb05a870fe36d4c07fb2c7e5220c02cc on master.

Coverage decreased (-0.2%) to 23.712% when pulling a37ebb9198c2b3fcc18d971413576681e4a5691e on CON-2691-iphoneX into 120e7762a463c980f8145254c8e65192f1774b07 on master.

Coverage decreased (-0.2%) to 24.278% when pulling 89314915a66c855cfb1b1297cd249b67f2ea333f on CON-2691-iphoneX into 86e66b4bc36b44d7c8f54da82fe7baa306c9c191 on master.

Coverage decreased (-0.006%) to 24.434% when pulling 93e4c172784717bffb5277a53df416eb2e7bd9c3 on CON-2691-iphoneX into 3e34e21551e62031544b3c321180ff1a48352fe4 on master.


Add contribution instructions to search-collector README.

Related Issue:  open-cluster-management/backlog#12714
Description of changes
Add instructions on new PROW based contribution flow. Fork/merge instead of branch/merge

/hold

Thanks for pulling me in @jlpadilla I am taking a look now

@dockerymick @jlpadilla
I made edits based on comments Mikela left. Thanks again for the review!

/unhold


When custom audio renderers are added, app would crash if not removed before disconnecting

This relates to the use of custom audio renderers:
    nonisolated
    func participant(_ participant: RemoteParticipant, didSubscribeTrack publication: RemoteTrackPublication) {
        if let audioTrack = publication.track as? RemoteAudioTrack {
            audioTrack.add(audioRenderer: audioRenderer)
        }
    }

When this is added and the session is disconnected, this would crash fairly frequently.

I'm able to work around this in the majority case by removing the custom renderer before disconnect happens, but sometimes there's a race I can't avoid and we end up crashing. This is quickly becoming one of our top crashers. Crash is easy to reproduce in the simple case at least. Let me know if I can provide more info here.

I have a fix here, but it needs more testing. Also it will expose AVAudioPCMBuffer instead of CMSampleBuffer. If you still need CMSampleBuffer I might write a helper func in the SDK.
https://github.com/webrtc-sdk/webrtc/pull/144

Great!! AVAudioPCMBuffer is easier for me to work with anyways 😄


fix and test some memory leaks

I was working on a new implementation of #75 / #102. I found what I believe are memory leaks, mostly associated with key normalization. For example, _key_normalized_obj was not consistent (across its different conditional branches) as to whether it created a new reference or not. More generally, if a PyObject * key can be overwritten by a normalized or prefixed key, any owned (as opposed to borrowed) references to the original key must be cleaned up.
I've added tests for these issues. The tests pass in this branch, but do not pass in master: https://gist.github.com/slingamn/3b6cec10c1f12f0dbe1b
I think it's possible that there are more leaks:

This testing approach can't identify leaks of intermediate values, only in arguments to public functions
Most of the error recovery code is not covered

but I think this should cover the leaks that the typical user is most likely to encounter.
Thanks for your time!

I'm just a user of pylibmc, but thanks for this. I discovered the memory leak in _key_normalized_obj as well. Hope this can get merged soon.

Exciting stuff, very nice. I'm a little anxious about the amount of changes, so let's hope your approach is solid. It seems better to test ref counts than not though. But extensive testing will be needed. The way I've isolated memory leaks previously is to just run code in a loop and see if the process starts eating memory.

Any ETA for a new release containing this fix?

Thanks!
Yeah, testing this is tricky. I think in order to see a blowup in memory consumption from the issues in this patch, it would be necessary to use distinct keys for each iteration of the loop --- otherwise, only an O(1) amount of objects will become uncollectable.

ping, any update on when this fix will come out?

Well, the originating branch has been closed so I'm not sure how I can move forward with this.

@lericson Not sure what you mean. It looks like it's merged into master. I believe @jkchin was asking when a build would be created with this change. It appears it is in 1.5.1 released 21 March though I didn't see anything in the changelog.


Behavior when hide_project_names and exclude are both true

In my .wakatime.cfg right now, I have both exclude = true to exclude all projects (along with a list of specific folders to include under the include key) and hide_project_names = true. I would expect that any folders not in the include list would be ignored, but right now anytime I open any folder in vscode and edit a file, a .wakatime-project file is created and times for the project appear on my dashboard. Is this intended behavior? I didn't see anything about this behavior in the docs.

@ArtofBugs exclude accepts a list of regular expressions and does not accept "true" as global variable to ignore everything.
https://github.com/wakatime/wakatime-cli/blob/develop/USAGE.md#settings-section

Ohh ok, that would explain things. I saw that bools were accepted in the example config file so I assumed true worked too. Thanks!

Confirmed it works correctly with regex instead of true; closing :) Thanks for the quick reply!

Implemented by https://github.com/wakatime/wakatime-cli/pull/1130 and will be released soon.

Released on v1.105.2


[sqlite , sqlcipher and OpenCL-HPP ] adding to port

it can be good if you can add
OpenCLTM API C++ bindings (cl.hpp and cl2.hpp)
https://github.com/KhronosGroup/OpenCL-CLHPP
sqlcipher
https://github.com/sqlcipher/sqlcipher
sqlite
https://github.com/sqlite/sqlite
FOR sqlcipher how to windows:
https://tribal1012.tistory.com/267

If you wish to add this port in the future, you can reopen this issue, but we're closing it for now.


[IR-7382] to dev - Create endpoint to create new projects using cloudformation

Check PR comments.


Update override_xcodellm.sh to add nvram check

See https://github.com/Kyle-Ye/XcodeLLMEligible/commit/5a1069c9674933c67ae620a99a0e3e06c80ee3d5

Closed by d4b4cbb6ca3a4dc9005ef5e65016fb2efab27bc4


mkl-dnn BatchNorm appears to be broken as used by Conformer

Conformer (see #211) models train and test fine on CUDA, but if I switch to the CPU backend, I get memory corruption in mkl-dnn. Note that I'm primarily using and reproing this on the v0.2 branch of wav2letter/flashlight, but I don't see any relevant changes in BatchNorm.cpp that aren't also in v0.2, so I think this holds as a current flashlight bug.
Repro 1:

Train a conformer model using CUDA
Run asr_test with the newly trained model and notice it seems to work fine.
Build flashlight with a CPU backend and try running asr_test with the Conformer model.
The model will either crash or output gibberish.

Repro 2:

Build mkl-dnn with -fsanitize=address. This can be added by editing mkl-dnn/CMakeLists.txt and adding these lines near the top:

add_definitions(-fsanitize=address -ggdb)
link_libraries(-fsanitize=address)

Build flashlight with the CPU backend and -fsanitize=address (using a similar CMakeLists.txt edit to mkl-dnn)
Run asr_test on a Conformer model.
 If I remove the axes.size() block it stops tripping ASAN.

Conformer is landed on master, planning to run benchmarks as you pointed on time slow down and OOM. One of the fixes which now slowdown a bit training is fix of the bug in Layer Norm, but this should not influence so much as you observed. About CPU batchnorm @jacobkahn @vineelpratap  any comments on this?

@lunixbochs — the MKL-DNN batch norm implementation has been buggy for some time unfortunately and no one has worked on it for almost a year. We're currently in the process of writing an OpenCL backend that will also run on the CPU which will be much more performant (@avidov is working on this) as ArrayFire doesn't thread operations in the CPU backend — only the OpenCL backend has any sort of parallelism.
Sorry that's not the answer you were hoping to hear. I'll consider updating the CPU backend to a newer version of MKL-DNN this week.

@lunixbochs — I'm in the process of updating the CPU backend to use a new version of OneDNN and am adding an RNN primitive. I'll comment here when that stuff is committed. Should be done this week.

@lunixbochs would you mind trying again on top of master? I've just added integration with OneDNN v2.0 and it should hopefully fix some issues that we had with several of those primitives. Let me know if/when you have a chance to try that if it improves things.

Have a good repro. Looking into it.

I haven't been able to test this yet, as it's very slow for me to train a conformer model on flashlight master and the v0.2 models aren't compatible. I was using a batch size of 24 before and I get OOM even with a batch size as small as 4 on master.

Have a fix for everything, including the memory corruption in Conformer, etc. There's been a bad bug in CPU batchnorm for quite a while that's fixed by a chance that should be out shortly. This should also dramatically improve convergence and fix training and inference on the CPU backend throughout.

To be clear I'm training on GPU, that's a separate issue. Glad you found this issue though!

Ah — unsure as to what the issue might be on GPU. Do you have an architecture you might be able to share?

moving back to the #211 thread for that issue which has a bit more context

Just confirmed the batchnorm fix, thanks so much! Conformer inference is working for me on CPU now.


SQL Generation Bug - Select null-able navigation property return empty list

I have two entities, their relationship is 0 or 1 to many.
When I use select to query the parent entity like the following, the result is an empty list:
.Select(s=>new DtoModel{
Id = s.Id,
ChildId = s.ChildId
ChildFullName =(s.Child != null) ?( s.Child.FirstName + " " + s.Child.LastName):string.Empty
}).ToList();
ChildFullName is a string property in the Dto model. In the domain class, ChildId is nullable and thus the entity can have one or no child entity.
I used SQL Server Profiler to capture the sql generated by the above select query, I found that the generated sql uses INNER JOIN to join the child entity. However in this case, a LEFT Join is correct.
I copied the generated sql to management studio, only change INNER to LEFT, it returned the result I wanted.
My EF7 version is 7.0.0-rc1-Final.
If there is any work around at the moment, please let me know.
Thank you very much.

dupe of #3186

Thanks, tried to search the problem before posting, but didn't find any. I obviously need to improve my search skills:)

@rexebin no problem... I look at every issue that comes in and I still spend a lot of searching around thinking "I'm sure we have another issue tracking this somewhere" :smile:

The related issue's title was not clear enough to make it appear in search. I changed it too. :)

You guys are great! Thank you very much for your hard working!
Sent from Outlook Mobile
On Wed, Jan 20, 2016 at 9:53 AM -0800, "Smit Patel"<EMAIL_ADDRESS>wrote:
The related issue's title was not clear enough to make it appear in search. I changed it too. :)
—
Reply to this email directly or view it on GitHub.


feat(ethers-core): OP Bedrock Upgrade

Overview
Sets Optimism Mainnet / Goerli as an EIP-1559 supporting chain in chain.rs.
Motivation
Optimism Mainnet / Goerli both support EIP-1559 after the Bedrock upgrade.
PR Checklist

[ ] Added Tests
[x] Added Documentation
[x] Breaking changes

🎉🎉


DataStream api need support close incremental snapshot

Is your feature request related to a problem? Please describe.
When i use Datastream Api to Collect Mysql Binlog to Kafka , I find it will error when some table don't have primary key.
Error message is below:
Incremental snapshot for tables requires primary key, but table ssb.customer doesn't have primary key

Describe the solution you'd like
flink 1.13.6  , flinkcdc 2.2.1
my code :
  MySqlSource<String> mySqlSource = MySqlSource.<String>builder()
                .hostname("xxx")
                .port(3306)
                .databaseList("ssb") // set captured database
                .tableList("ssb.*") // set captured table
                .username("xxx")
                .password("xxx")
                .startupOptions(StartupOptions.initial())
                .deserializer(new JsonDebeziumDeserializationSchema())
                .build();

How can i close  Incremental snapshot  when i use datastream api?
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

I also encountered this problem. Which God can solve it

How did you solve the problem

hellp

Closing this issue because it was created before version 2.3.0 (2022-11-10). Please try the latest version of Flink CDC to see if the issue has been resolved. If the issue is still valid, kindly report it on Apache Jira under project Flink with component tag Flink CDC. Thank you!


Show payment summary more often

[ ] If an event (attendance event) has a payment relation, show the payment summary tab (currently it requires at least one attendee before it shows).
[ ] Show the payment summary tab when navigating from event to dashboard(admin -> open in dashboard), don't require the user to visit the event from inside the dashboard (dashboard -> upcoming events -> event).

This should be good with the re-design of the dashboard.


Support of LaTeX in GitHub Pages added

As suggested: https://stackoverflow.com/a/46765337/6551233
Example of the result: https://kdorichev.github.io/deeplearning/core

Thanks for the suggestion! I've actually added KaTeX to nbdev_template now since I've found that's a lot faster than mathjax.


doc: Add missing import for Control.FullScreen.css

Without this import. The svg button is not loaded and yo end up with a "blank button"

Any thought on this @BePo65 ?

I'll take a look at it on the next weekend - quite busy at the moment :-)

Basically we could give some hints on how the "Include Control.FullScreen.css in your page" from the "How?" section can be done in esm based frameworks. But the PR is for react only.
So perhaps it would be better to add a 3rd point, which could look like:
3. Add `leaflet.fullscreen/Control.FullScreen.css` from the node_modules directory to your project. In React you have to import this file and in Angular you have to add './node_modules/leaflet.fullscreen/Control.FullScreen.css' to your angular.json file.

@danidee10 could you update your PR as suggested by @BePo65 ?


A small step toward building a toolchain on M1 Mac

Minor change removing impediment to building Swift toolchain on M1 Mac
Motivation:
Would like to be able to work on the Swift compiler on an M1 Mac.
Modifications:
Very small change for when swift/utils/build-toolchain is run on M1 Mac.
Result:
Will not be prevented from building toolchain on M1 Mac.

@swift-ci please smoke test

@johnno1962, could you change this to match what swift-driver does for now and we'll get this in?

Hi @buttaface. Sorry, but as this was outstanding for while I deleted my swifpm fork and am unable to update or test it properly. Could I leave this to you please?

Alright, just close this pull then, we'll get a fix in along the lines sketched out by Anders and Mishal above eventually.

OK, cheers, Closing now. Do you know if there is any progress on being able to work in the compiler on an M1 (the ticket I mentioned)?

That ticket says you got it working though, right? No idea if anyone is working on upstreaming the remaining fixes for M1.

I hacked it until building a toolchain ran to completion more like. The resulting toolchain had it's issues but it was enough for my purposes in pursuing this PR. It really needs someone with clang and build-script knowledge to come up with the right solution. I'm baffled I seem to be the only person trying to get it working, M1s are sooo much faster.


Force push "main" branch with "main-with-history-without-large-files"

Now that the transition crunch time is over, I took few minutes to rewrite the history so that contribution history is maintained, here is the updated branch:
https://github.com/jcfr/slicer_download/commits/main-with-history-without-large-files
@mhalle Would be great if you could:

Push force main accordingly
Update your GitHub settings adding git[at]m.halle.us as an email

Last to ensure the repository remains small, I also rewrote the history removing the following "large" files:
61MiB,30MiB,f3d7349129298556cee9bf327ed0a2946154f7af,etc/geoip/db/GeoLite2-City.mmdb
25MiB,17MiB,04a7cc14e76f50c4ad704c64cfb38d481173b93f,apps/parse-slicer-logs/data/2012-03-06/GeoLiteCity.dat
18MiB,12MiB,20e84f3d0575ea6f950ab861e68051450696ea6a,apps/parse-slicer-logs/data/GeoLiteCity.dat
1.9MiB,1.9MiB,581c8c85e1f08ae1af592502baf41172f544d801,apps/build-slicer-json/node_modules/sqlite3/deps/sqlite-autoconf-3080500.tar.gz
963KiB,428KiB,9e88b71d53ab733b666be5af4cc7cc97fb272ea5,apps/build-slicer-json/node_modules/sqlite3/lib/binding/node-v11-darwin-x64/node_sqlite3.node

@mhalle
Note that this branch has been updated to include the latest commit you pushed on the current main, it is ready to be forced push onto the current main.
The only difference is an indentation fix:
git diff origin/main..jcfr/main-with-history-without-large-files
diff --git a/slicer_download_server/templates/download.html b/slicer_download_server/templates/download.html
index f8de03e..66344f8 100644
--- a/slicer_download_server/templates/download.html
+++ b/slicer_download_server/templates/download.html
@@ -65,7 +65,7 @@

-              Preview Release
+                Preview Release

                         version {{R.win.nightly.version}}

revision {{R.win.nightly.revision}}

Closing. main branch  has been updated.


Access AccountService, error: org.knowm.xchange.bitstamp.dto.BitstampException: API key not found (HTTP status code: 403)

Hi,
I used Example 2: Private Account Info to access AccountService, I got the error: org.knowm.xchange.bitstamp.dto.BitstampException: API key not found (HTTP status code: 403).
When I enter my Kraken account, I can see my account information, everything is good.
What is the problem? How can I resolve it?
Thank you so much!
Sky

Hi,
When I execute to accountService.getAccountInfo() every time, I get the error.
Why getAccountInfo() in AccountService does not work.
Thank you so much!
Sky

Hi,
I use the dependencies:

org.knowm.xchange
xchange-core
4.3.0

org.knowm.xchange
xchange-examples
4.3.0

org.knowm.xchange
xchange-XYZ
4.3.0

Does someone know the reason? or need I set anything else?
Thank you so much!

bitstamp != kraken

Hi Tim,
I am using the code now from your GitHub repository:
ExchangeSpecification exSpec = new
BitstampExchange().getDefaultExchangeSpecification();
exSpec.setUserName("34387");
exSpec.setApiKey("a4SDmpl9s6xWJS5fkKRT6yn41vXuY0AM");
exSpec.setSecretKey("sisJixU6Xd0d1yr6w02EHCb9UwYzTNuj");Exchange
bitstamp = ExchangeFactory.INSTANCE.createExchange(exSpec);
// Get the account informationAccountService accountService =
bitstamp.getAccountService();AccountInfo accountInfo =
accountService.getAccountInfo();System.out.println(accountInfo.toString());
Then how can I get my Kraken and Poloniex account information using XChange?
Thank you for your response!
Best regards,
Sky
On Fri, Nov 17, 2017 at 10:21 AM, Tim Molter<EMAIL_ADDRESS>wrote:

bitstamp != kraken
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/timmolter/XChange/issues/1891#issuecomment-345272515,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AZFyxaU3P1huw4HltNgjad6jv0-3yhBQks5s3aRigaJpZM4Qg1ax
.

Hi Tim,
I use Exchange bitstamp = ExchangeFactory.INSTANCE.createExchange(PoloniexExchange.class.getName(),userName,password,apiKey,secretKey) now.
I can get the account information now.
Thank you!


Nikon reboot needed to restart USB preview

When USB preview is stopped because USB cable is unplugged then USB preview won't start again and reboot is needed. Seems it is issue with Nikon 810 and 850.
Typically: everything switch on, connect, focus via magnifier in camera menu requires disconnect USB cable, when reconnect then no more USB preview, reboot

Switch off camera when running USB preview
2024-07-14 15:38:03,447: root: preview: DEBUG: camera_file.get_data_and_size()
2024-07-14 15:38:03,447: root: preview: DEBUG: capture_lock.release() 4823
2024-07-14 15:38:03,548: root: preview: DEBUG: capture_lock.acquire() 4823
2024-07-14 15:38:03,548: root: preview: DEBUG: gp.gp_camera_capture_preview()
2024-07-14 15:38:03,909: root: preview: ERROR: Failed to capture preview: -7: I/O problem
2024-07-14 15:38:03,909: root: preview: DEBUG: Reinitializing: usb:002,011|Nikon DSC D850
2024-07-14 15:38:03,910: root: preview: DEBUG: gp.Camera.autodetect()
2024-07-14 15:38:04,008: root: preview: DEBUG: Reinit #0
2024-07-14 15:38:04,209: root: preview: DEBUG: gp.Camera.autodetect()
2024-07-14 15:38:04,255: root: preview: DEBUG: Reinit #1
2024-07-14 15:38:04,656: root: preview: DEBUG: gp.Camera.autodetect()
2024-07-14 15:38:04,699: root: preview: DEBUG: Reinit #2
2024-07-14 15:38:05,299: root: preview: DEBUG: gp.Camera.autodetect()
2024-07-14 15:38:05,343: root: preview: DEBUG: Reinit #3
2024-07-14 15:38:06,143: root: preview: DEBUG: gp.Camera.autodetect()
2024-07-14 15:38:06,187: root: preview: DEBUG: Reinit #4
2024-07-14 15:38:07,187: root: preview: DEBUG: gp.Camera.autodetect()
2024-07-14 15:38:07,231: root: preview: DEBUG: Reinit #5
2024-07-14 15:38:08,432: root: preview: DEBUG: gp.Camera.autodetect()
2024-07-14 15:38:08,475: root: preview: DEBUG: Reinit #6
2024-07-14 15:38:09,876: root: preview: DEBUG: gp.Camera.autodetect()
2024-07-14 15:38:09,951: root: preview: DEBUG: Reinit #7
2024-07-14 15:38:11,551: root: preview: DEBUG: gp.Camera.autodetect()
2024-07-14 15:38:11,595: root: preview: DEBUG: Reinit #8
2024-07-14 15:38:13,395: root: preview: DEBUG: gp.Camera.autodetect()
2024-07-14 15:38:13,439: root: preview: DEBUG: Reinit #9
2024-07-14 15:38:15,439: root: preview: DEBUG: gp.Camera.autodetect()
2024-07-14 15:38:15,536: root: preview: DEBUG: Reinit #10
2024-07-14 15:38:15,536: root: preview: DEBUG: capture_lock.release() 4823
2024-07-14 15:38:15,537: root: preview: DEBUG: camera exit
Exception in thread preview:
Traceback (most recent call last):
File "/home/pi/work/digie35-python/digie35/./digie35_server.py", line 631, in _do_preview
self._reinit_camera()
File "/home/pi/work/digie35-python/digie35/./digie35_server.py", line 227, in _reinit_camera
raise CameraControlError(f"Cannot reinit camera: %s" % model)
CameraControlError: Cannot reinit camera: Nikon DSC D850
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.11/threading.py", line 1038, in _bootstrap_inner
self.run()
File "/usr/lib/python3.11/threading.py", line 975, in run
self._target(*self._args, **self._kwargs)
File "/home/pi/work/digie35-python/digie35/./digie35_server.py", line 671, in _do_preview
self._close_camera()
File "/home/pi/work/digie35-python/digie35/./digie35_server.py", line 292, in _close_camera
gp.check_result(gp.gp_camera_exit(self._camera))
File "/home/pi/.local/lib/python3.11/site-packages/gphoto2/result.py", line 52, in check_result
raise GPhoto2Error(error)
gphoto2.GPhoto2Error: [-52] Could not find the requested device on the USB port


Whole pretrained embedding

Thanks to public your codes. And I rerun some of your experiments, I found that 'emb_matrix' in your released link was quite useful, do you have a link for the full embedding?

Hi,
Sorry for the late response... for some reason I didn't get a notification from github !!
The link to the full fasttext embeddings + code to generate are under the section Generate LS embeddings
in readme


Can't set page margin to 0

Hi,
Sounds impossible to set page margin to zero; I always get a ~1cm margin in every border of the page.
I tried :
@page {
    margin: 0 !important;
    padding: 0 !important;
}
body{
    margin:0 !important;
    padding: 0 !important;
}
html{
    margin:0 !important;
    padding: 0 !important;
}
*{
    margin:0 !important;
    padding:0 !important;
}

Any idea ?

OK my mistake, just need to link the correct css stylesheet... shame on me
Thanks for your plugin btw !


Implement numpy.ones_like

Return an array of ones with the same shape and type as a given array.
https://docs.scipy.org/doc/numpy-1.15.1/reference/generated/numpy.ones_like.html

Done and close.


Question Regarding the Baseline Performance of CMPNN in Your Paper

May I kindly ask if you would be able to share the random seeds used in your replication of the CMPNN model? I have been following the procedure you mentioned—"We perform three independent runs on three random-seeded scaffold splitting for all datasets"—but in my replication, the results tend to hover around a 3 repeats score of 0.94436 ± 0.01183, whereas the results in your paper show 92.7 ± 1.7. I am wondering if you might be able to share the baseline random seeds you used for your experiments.
Additionally, I wanted to express my admiration for your prior work, such as KANO and KCL. Thank you very much for your time and assistance.

🤣🤣🤣🤣🤣🤣🤣🤣🤣🤣  Thank you very much for your prompt and detailed response! I will try it later！

I came across something quite interesting that I wanted to get your thoughts on. In "From molecules to scaffolds to functional groups: building context-dependent molecular representation via multi-channel learning," on page 15, I noticed that KANO’s performance on the BBBP dataset is reported as 69.9 ± 1.9. I’m curious to know how you view this result and if there might be specific factors contributing to this level of performance.

Hi, thank you very much for pointing that out! I read the original text and noticed how they described their data splitting method.

I think the difference in results might be due to using different data splits.

Okk! I see! Thank you very much for your detailed response!


Support for Swift 5.1

How could I contribute to add support for Swift 5.1?

has Swift 5.1 came out? I don't see any 5.1 listed in Swift releases for download at https://swift.org/download/#releases

Sorry, I meant the development snapshot. Is it possible to specify it already?

Yes, it should be possible. As these snapshots move relatively fast and frequent, swiftenv will hit swiftenv-api to get a list (which crawls the list of versions every hour from swift download page -- https://github.com/kylef/swiftenv-api).
You can use the following to get a list:
$ swiftenv install --list-snapshots

There are some recent ones:
$ swiftenv install --list-snapshots | grep 2019-08
DEVELOPMENT-SNAPSHOT-2019-08-01-a
DEVELOPMENT-SNAPSHOT-2019-08-02-a
DEVELOPMENT-SNAPSHOT-2019-08-03-a

Along with some 5.1 specific snapshots (I don't recall what the difference is between these branches):
$ swiftenv install --list-snapshots | grep '^5.1'
5.1-DEVELOPMENT-SNAPSHOT-2019-04-16-a
5.1-DEVELOPMENT-SNAPSHOT-2019-04-17-a
...
5.1-DEVELOPMENT-SNAPSHOT-2019-07-23-a
5.1-DEVELOPMENT-SNAPSHOT-2019-07-24-a
5.1-DEVELOPMENT-SNAPSHOT-2019-07-25-a

You can pass these identifiers directly to swiftenv install 5.1-DEVELOPMENT-SNAPSHOT-2019-07-25-a, you may be able pass a URL to a particular snapshot tarball/pkg (platform dependent) although I am not sure how relaible that works, you can easily copy binary URL for wrong platform and it may not work.

Great! Thank you very much Kyle.


Fix problems for master branch and Github secrets from NPPO AWS

Solution is to use following command when syncing repository state:
POL_AWS_LOAD_SECRETS=0 APPLICATION_MODE=production invoke aws.sync-repository-state


Add File Upload Retry

More like file continue (it should continue where it left off on failure)

#99


Switch to single Scene instance per problem

Replace std::mapstd::string,exotica::Scene_ptr with just exotica::Scene_ptr.
This variable and corresponding getters/setters appear in a lot of depending source files - refactor accordingly.

Fixed in #36


Set linkage against the wrong scopesym

@jacob-carlborg @rainers @WalterBright
Noticed this in #4321 post merge.

LGTM.


Extra '' characters causing issue in localized pages.

Near the bottom of the page there are a couple of '' characters that should be removed.   They are causing an issue in the machine translated versions which results in the text after them being left in English.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: d4271558-62a9-db3c-078c-13b5e5fe99da
Version Independent ID: 813d7002-20a2-b1be-2771-975a0b1cfd48
Content: Use network upload to import your organization's PST files - Microsoft 365 Compliance
Content Source: microsoft-365/compliance/use-network-upload-to-import-pst-files.md
Service: o365-seccomp
GitHub Login: @markjjo
Microsoft Alias: markjjo

@TinaMcN fixed; fix was to add a third tick (`) because this is text block. Revised topic will be live shortly.

@markjjo thanks for the speedy fix!


Error occurred when executing Joy_caption:

Error occurred when executing Joy_caption:
rope_scaling must be a dictionary with with two fields, type and factor, got {'factor': 8.0, 'high_freq_factor': 4.0, 'low_freq_factor': 1.0, 'original_max_position_embeddings': 8192, 'rope_type': 'llama3'}

update transforms version


Clarify URL for running docker locally on Linux.

Updated docs/local_development_with_docker.md to clarify URL for accessing a locally running docker instance on Linux.

Thanks dude! I added you as a collaborator, so you can do stuff like assign yourself to issues now. Good to meet you, and look forward to working w/you in the future. :)


introduce MUST, SHOULD, MAY to allow better reporting

@michielbdejong @ylebre @edwardsph
Each test SHOULD be specified as a MUST, SHOULD, MAY so as to know the specification level required. An html report could be generated with the results  (report: work to be done)
I tested a quite easy solution around wrapping jest functions with an itIs function :

test name will always begin with SKIP, MUST, SHOULD or MAY example itIs('MAY') (this allow for further reporting)
env variables allow to skip MUST, SHOULD and/or MAY
default is MUST : itIs()

export function itIs(arg = 'MUST') {
  if (arg === 'SKIP') return (name, runner) => { it.skip(`${arg} ${name}`, runner); }
  if (arg === 'MUST' && process.env.SKIP_MUST) return (name, runner) => { it.skip(`${arg} ${name}`, runner); }
  if (arg === 'SHOULD' && process.env.SKIP_SHOULD) return (name, runner) => { it.skip(`${arg} ${name}`, runner); }
  if (arg === 'MAY' && process.env.SKIP_MAY) return (name, runner) => { it.skip(`${arg} ${name}`, runner); }
  return (name, runner) => { it(`${arg} ${name}`, runner); }
}

The test would look like this :
  describe("Get RDFa", () => {
    describe("As JSON-LD", () => {
....
      itIs('MAY')("Triples", async () => {
        const triples = await asTriples(
          jsonText,
          `${testFolderUrl}example.html`,
          "application/ld+json"
        );
        expect(triples).toIncludeAllMembers(triplesFromHtml);
      });
    });
    describe("As Turtle", () => {
....
      itIs()("Triples", async () => {
        const triples = await asTriples(
          text,
          `${testFolderUrl}example.html`,
          "text/turtle"
        );
        expect(triples).toIncludeAllMembers(triplesFromHtml);
      });
    });
  });
  describe("GET Turtle", () => {
    describe("As JSON-LD", () => {
...
      itIs('MUST')("Triples", async () => {
        const triples = await asTriples(
          jsonText,
          `${testFolderUrl}example.ttl`,
          "application/ld+json"
        );
        expect(triples).toEqual(triplesFromTurtle);
      });
    });

The result for SKIP_MAY=1 npm run jest conneg.test.ts -- --verbose
the use of verbose is not needed here but allow to display each test result when there is a default folder of tests.

I really like this idea!
A few considerations I have (maybe for future versions, because I think just this change would be a great improvement already)

Should we keep a list of the rules we have, together with a link to where in which spec the rule came from?
Should we have some kind of versioning in the rules, so we can run the test suite against a specific spec version?
I think we need to be able to single out MAY rules, so a server can choose to run the test for the one they implement but not the rest, to that the applicable MAY rules can be run in CI as well.

I think we need to be able to single out MAY rules, so a server can choose to run the test for the one they implement but not the rest, to that the applicable MAY rules can be run in CI as well.

May be this could be done with an options parameter in function itIs (arg  = 'MUST', options) or multiple MAY depending on the complexity

I think it would be best to require the level for itIs - it will require a bit of work behorehand, but will be more explicit from there on.
Also, it might be a good idea to have an ID for every test so that you can explicitly toggle each one.
And just to put it out there: maybe we should make 'MUST' and 'SHOULD' required by default, so you'd have to opt-in to skip them.
For 'MAY', the tests are skipped by default and you'd have to opt-in to include them;
It would end up something like this:
export function itIs(level, id) {
  switch (level) {
    case 'SKIP':
      return (name, runner) => { it.skip(`${level} ${name}`, runner); }
    break;
    case 'MUST':
      if (
        (process.env.SKIP_MUST) ||
        (process.env_SKIP_MUST_' + id])
      ) {{
          return (name, runner) => { it.skip(`${level} ${name}`, runner); }
      } else {
          return (name, runner) => { it(`${level} ${name}`, runner); }
      }
    break;
    case 'SHOULD':
      if (
        (process.env.SKIP_SHOULD) ||
        (process.env['SKIP_SHOULD_' + id])
      ) {
        return (name, runner) => { it.skip(`${level} ${name}`, runner); }
      } else {
        return (name, runner) => { it(`${level} ${name}`, runner); }
      }
    break;
    case 'MAY':
      if (
        (process.env.INCLUDE_MAY) ||
        (process.env['SKIP_SHOULD_' + id])
      ) {
        return (name, runner) => { it(`${level} ${name}`, runner); }
      } else {
        return (name, runner) => { it.skip(`${level} ${name}`, runner); }
      }
    break;
  }
  return (name, runner) => { it(`${level} ${name}`, runner); }
}

If the ID option is retained, if defined the ID shall be displayed explicitly in the test name
it(${level} ${Id} ${name}, runner)
I agree to require a level.
For practical reason it may be better to keep the ID optional until the tests and spec are stabilised. Changes imply a review of the CI.
The ID could be the link with the specification.


Change file name

Change file name contains „search” to „find”. This change should touch fn names

changed files, functions and intext in files (readme)


Usage

Hi
Thanks for your nice work!
I'm new to this repo, and there is not a general knowledge share on the page on how to do the gpu share on Comfy!
Please be paitaient if i'm asking noob questions.
1- i have installed a ComfyUI on a Cloud based ubuntu image which has a T4 GPU. I also have two 8Gb VRAM Gpus on other ubuntu PC. How can i configure the Comfy to make use of the PC GPUs?
2- Do i need to install Comfy on Both Cloud and PC? ( Cloud already has Comfy installed)
3- Does it work over SSH? and it works for ControlNet?
4- Can you please explain steps on how to set it up in both devices?
5- Does it works on Cloud + Windows?
Thanks
Best regards

Hey! This was originally a random one-weekend project so the documentation (and the code for that matter) is pretty shoddy lol. There's also a rewrite branch, which is a bit different and might be more stable. I don't think I have an image of that but here's one I quickly threw together:

1- i have installed a ComfyUI on a Cloud based ubuntu image which has a T4 GPU. I also have two 8Gb VRAM Gpus on other ubuntu PC. How can i configure the Comfy to make use of the PC GPUs?

You need to pick a machine that will be your "main" PC - this is where you run the webui you actually want to access. The other ones are only used over the API, but you still need a way to reach them.
So for instance, let's say you have two GPUs in your ubuntu PC. You would start your instance as normal, this would be accessible on <IP_ADDRESS>:8188. Then you start a second one, but force it to use the second GPU (put CUDA_VISIBLE_DEVICES=1 in front of your command). This second one will have to be on a different port, let's say <IP_ADDRESS>:8288. Now you can use the queue_on_remote node to start the workflow on the second GPU instance while also running on your main one. I recommend you get this working before trying to add the cloud one, because debugging it locally should be easier.

2- Do i need to install Comfy on Both Cloud and PC? ( Cloud already has Comfy installed)

Yes, but it's also a bit more annoying since all your checkpoints/LoRAs will have to have the exact same name to work. (This is why there is also a windows/posix switch in the node - not even the path separator can differ). The whole script relies on just sending the current workflow to the remote UI's API as if it were a normal workflow, so it's pretty fragile in general.

3- Does it work over SSH?

Yes! look up "SSH port forwarding". You could forward the cloud instance to be available on <IP_ADDRESS>:8388, or do the reverse and make your two local UIs accessible on the cloud machine's localhost.

and it works for ControlNet?

I think it'd only work if you run it locally, or somehow sync the image folders between your local/remote instance. (When you load up a controlnet image, it gets saved to disk so the workflow can load it). You could try rsync or even something simple like sharing the entire ComfyUI/input folder between the two by attaching it via sshfs on the client machines.

4- Can you please explain steps on how to set it up in both devices?

Install ComfyUI on both machines, also download any models you might need on both.

Pick a machine as your main one, I recommend the local one. Make sure it can access the remote one - either via regular or ssh port forwarding.

Set up the remote nodes. You should have one node to start the workflow on the remote machine, while the rest of the workflow will run as normal on the local one. See the image above or in the readme for an example.

5- Does it works on Cloud + Windows?

It should, just set the OS correctly in the node to avoid the path issues mentioned above.
Hope this helps trying to get my jank to work. Just ask if something breaks/sounds confusing/etc.

@city96
Thank you so much for the detailed guide, i'll try that
I'm a liitle bit confused about the computations. I'm not sure which parts of my workflow could get into another Machine(port forwarded to the main PC)?
My workflow for a Single GPU is something like this:
1- The input directory is loaded as path for images folder
2- Each image goes into some preprocessing ControlNet nodes
3- Preprocessed images then goes into the Controlnet load and apply model
4- ControlNet output gets into the Ksampler and then to the decoder and save image
5- This process repeats for all images into that directory
P.S one of the Controlnets have TemporalNet model!
I'm not sure how can i distribute these Nodes onto multiple GPU/Comfy instances!?
So instead of your Empty latent, I give {input image + VAE} from the Load Checkpoint to the {VAE Encode} and then the conncets it to latent image
and
instead of Positive prompt connect directly to the ksampler Positive,  The Positive prompt connects to CN apply and then to the ksampler Positive!
Seed and Prompt is always Fix. just Input image and ControlNet[in regards] gets different!
I have a laptop with very Low-end GPU[2GB], and i have a T4 GPU [16GB] on cloud, Two 1080 on a ubuntu PC, two 2080 GPU on Win10 PC!
Can you please guide me how to utilize all of them?
Thanks
Best regards

Seed and Prompt is always Fix. just Input image and ControlNet[in regards] gets different!

Hmm, that is indeed a different usecase. So if I understand it right you have a "fixed" workflow (fixed seed and everything) but need the input image to change for some kind of batch processing. Are you using it for animations or something else?
I've experimented with this kind of distributed batch processing too but I never published that code anywhere. If you can wait until the weekend then I might be able fix up and publish some of that old code for you!

you have a "fixed" workflow (fixed seed and everything)

Yes, I need to run the same workflow for multiple images[Batch], but thats not the first thing to try.
Inside each workflow, i also have multiple preprocessing on the input image, including CN preprocessings and BBox detection and cropping which takes time and also needs to get distributed into multiple GPUs!
In other word the Priority goes with distributing preprocessing on multiple GPUs for a single input image. and then Sure i would be glad to wait for a week to distribute the batch processing!
As mentioned above:

My workflow for a Single GPU is something like this:
1- The input directory is loaded as path for images folder
2- Each image goes into some preprocessing ControlNet nodes
3- Preprocessed images then goes into the Controlnet load and apply model
4- ControlNet output gets into the Ksampler and then to the decoder and save image
5- This process repeats for all images into that directory

This file contain a simple workflow

Thanks in advance

@p0mad
Hey. I uploaded the distribution script. I cleaned it up a bit but it's pretty rudimentary: https://github.com/city96/ComfyUI_NetDist/tree/main/mass-process
You need to edit the example config to match what you need - it should have comments on what does what.
I think it should work with the preprocessor as long as you have it installed on all the machines you're going to use for this.
You can just launch it with server.py --conf your_job.yaml and it should connect to all the machines in the config. (Don't accidentally launch the wrong "server.py" lol, I did that once while writing it.)
Ping me if it doesn't work. I can write a better explanation of how to run it tomorrow!

Here is a basic NetDist workflow.
This is the 'primary' workflow and no workfow or UI is required for the 'secondary', located at port 8189 in my case.
The server just need to be started. The secondary server will use the same workflow as the primary, but with a different seed.


Codeforces: don't update the submissions when contest is ongoing

Need to fix the submission view links too


Move emerging applications section up adjacent to major applications section

This PR just moves text without modifying it, to make diffs easier to review.
This is the second step mentioned in the description of PR #153
Signed-off-by: Dave Thaler<EMAIL_ADDRESS>

I think this should wait once we have split out projects page into two pages, meaning, applications vs developer tooling as otherwise the more crucial tooling projects are hidden at the end whereas from the list the emerging applications might have the smallest prio (from a reader PoV).

The foundation website has moved to https://github.com/ebpffoundation/ebpf.foundation
This pull request likely needs to be opened there.

The projects page is going to be split in two so I'll close this for now #194 thanks for the suggestion though


Make the install directory configurable via an environment variable

As described in the title. Haven't been able to test on a Windows machine as I don't have access to one.

I should probably update the docs on this one also as the ENV variable needs to be added to shell *rc file, hold off merging for the moment.

I should probably update the docs on this one also as the ENV variable needs to be added to shell *rc file, hold off merging for the moment.


-su: syntax error near unexpected token `('

I can import any sub libray,
like from pytube import YouTube OR from pytube import Playlist
But when trying any command with that like :
pl = Playlist("https://www.youtube.com/watch?v=Edpy1szoG80&list=PL153hDY-y1E00uQtCVCVC8xJ25TYX8yPU")
-su: syntax error near unexpected token `('
OR
YouTube('http://youtube.com/watch?v=9bZkp7q19f0').streams.first().download()
-su: syntax error near unexpected token `'http://youtube.com/watch?v=9bZkp7q19f0''
any idea pls?

Hmm...I just tried your playlist link through the IDLE, and all three videos downloaded fine on my end (Python 3.7). It could be that you're using the wrong style of quotation marks, like you're mixing both single and double quotes.
Code I used:
from pytube import Playlist
pl = Playlist("https://www.youtube.com/watch?v=Edpy1szoG80&list=PL153hDY-y1E00uQtCVCVC8xJ25TYX8yPU")
pl.download_all(download_path="C:/Users/test/Desktop")
I would try copying and pasting what I have above to see if the case really is a syntax error. If the error persists, try upgrading your installation of pytube. If neither work, would you kindly post a snapshot or a copy-paste of the code you're using? Also, please submit what version of Python you are using as to make sure it's not a compatibility issue.


Reformat new samples to fix the build.

I've been having this problem today of seeing failed checks, like "Build / JDK 17 (push)" above, but then seeing no information when I click through, only "This job failed"....


cleaner unittest with mock

Some of the unittests can be improved with unittest.mock (https://docs.python.org/3/library/unittest.mock-examples.html)?

That looks useful! If there are specific tests that you think should be using this, can you create new issue(s) with those details? I'll mark this issue as a question and close it.

not at the moment, I'll create specific issues when I see them. just
brought this to the spotlight.
On Mon, Feb 29, 2016 at 8:32 AM, Jai Ram Rideout<EMAIL_ADDRESS>wrote:

That looks useful! If there are specific tests that you think should be
using this, can you create new issue(s) with those details? I'll mark this
issue as a question and close it.
—
Reply to this email directly or view it on GitHub
https://github.com/biocore/scikit-bio/issues/1294#issuecomment-190278611
.

Awesome, thanks!

@gregcaporaso and I made use of unittest.mock for mocking parts of the QIIME 2 plugin system in unit tests (PR here). It's incredibly handy, thanks for pointing out this package @RNAer!

no problem! I have been using it in micronota - it is very handy.
On Mon, Apr 11, 2016 at 4:48 PM, Jai Ram Rideout<EMAIL_ADDRESS>wrote:

@gregcaporaso https://github.com/gregcaporaso and I made use of
unittest.mock for mocking parts of the QIIME 2 plugin system in unit
tests (PR here https://github.com/biocore/qiime2/pull/42). It's
incredibly handy, thanks for pointing out this package @RNAer
https://github.com/RNAer!
—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/biocore/scikit-bio/issues/1294#issuecomment-208624160


ol.layer.Heatmap does not support gradient in webgl rendering mode

Using webgl renderer, there's no color (black) on the ol.layer.heatmap, the gradient is not taken into account.

ol.layer.Heatmap only works with canvas renderer. This should be enforced by the code

Ok, so this is a won't fix? Because I had the feeling that my animated heatmap was more stable on webgl than on canvas.

Not a won't fix but a patch welcome !

@ponceta  By "stable on webgl" you mean "faster on webgl" ?
The heatmap computation could probably be speeded up by used webworkers

Since I'm adding many features dynamically, the computation seems to be faster on webgl.
I use text/event stream to add/remove these elements progressively with time.

I really need this feature as well. Mobile devices can't handle HeatMap well except with WebGL.

I've tried to work this into the WebGL renderer, but I haven't gotten very good results. Basically, I added a second texture to be a color palette that's refreshed when the gradient changes, and I added a fragment shader looks at it for the color value.
However, a heatmap really needs to be rendered to it's own buffer, and then the result is what needs to be paletted. The reason that my results are bad are that each point in the heatmap renders the canvas generated image (a blurred circle), and only that is colored. Only the final composted image should be paletted, so I need to render the vectors to a texture, and then use that texture as input to the result that will be composted. I know how to do that with WebGL, but it seems that OpenLayers 3 only has a way to render single pass graphics. Is there a "right way" to render to an FBO and composite it (or several textures) to the map?

Closing due to lack of activity.

I really would like this issue fixed. I had to work around it at the time. The canvas renderer was way too slow, and the WebGL renderer with my patches did work after some optimization. What I have described here isn't hard to reproduce. I don't have access to that source anymore, but this issue may come up for me again at some point soon.


Difficult to select version of Kubernetes plugin to use

Currently, we can install our Kubernetes integration using the --kubernetes option on calicoctl node.
However, the calico-kubernetes plugin version is hard-coded in calicoctl with no way to configure it.  This is inflexible and leads to difficulty upgrading the plugin, which makes it hard to deploy bugfixes to users (a new release of calicoctl is required)
A few options here:

calicoctl node --kubernetes=version

problem: docopt would require a value for version, so we couldn't set a default of latest.

calicoctl node --kubernetes --plugin-version=version or something like that
We could eliminate the --kubernetes flag altogether, use @Symmetric's work to containerize the kubernetes plugin and use DockerHub for distribution.
... ???

For libnetwork we have a --libnetwork option to turn it on and --libnetwork-image= to specify the image.  This seems preferable since in the normal use case you don't need to specify the image.

@robbrockbank - yes, I'm leaning towards that option.
I'd propose:
calicoctl node --kubernetes --kube-plugin-version=<version>

The default for --kube-plugin-version would have to be hard-coded in calicoctl for now since there is no way to easily get the "latest" GitHub release (unlike for docker images).  If we containerize the plugin then we could change this behavior down the road.
I'm pretty keen to do this soon (and will do so myself if we agree on the syntax) - it slows us down significantly and makes it very difficult to release bugfixes. @Symmetric - thoughts?

@caseydavenport This ties in with the proposal for streamlining the release process.  In that I suggested we implement a PHP script accessible on projectcalico.org that get return the latest build of the binaries scraped from the CircleCI artifacts.
On the master branch of calico-docker, we really would refer to the latest (bleeding edge) build of the plugins rather than referring to the latest release.

Please also add option to pull the plugin from an internal repo for those of us behind corporate firewalls.


Update v19.2.0.md

addressing issues in the following:
https://github.com/cockroachdb/docs/issues/5835
https://github.com/cockroachdb/docs/issues/5837

This change is

LGTM. Thanks, @dbist!
For future reference, you can auto-close issues from a PR if you use Fixes #<issue number> or Closes #<issue number>.

Online preview: http://cockroach-docs-review.s3-website-us-east-1.amazonaws.com/4bdc0203162654878db97cf24002ce8234aed9a7/
Edited pages:

releases/v19.2.0.md


sanity_var check is failing with correctly set variables

This issue only happens in the latest tag, 3.0 works correctly. I have my environment variables defined in .env which is in the same directory as docker-compose.yml, everything is defined correctly, however, upon container start I get the error:
[cont-init.d] 10-cloudflare-companion: executing...
cloudflare     | [ERROR] ** [nginx-proxy-cloudflare-companion] No 'Cloudflare Email' Entered! - Set<EMAIL_ADDRESS>
Where<EMAIL_ADDRESS>is my correct e-mail address.

Thanks for submitting. I'll review this and fix up latest.

A new latest or 3.1.1 will resolve this issue. I've also removed the requirement to have CF_EMAIL as there are some cases that you can get by with it being blank using specific tokens. 3.1.2 runs on Alpine 3.12 now as well, but won't make a difference in the operation of the image.

Just pulled latest and confirmed working, thanks for the quick fix!


Add "manyof" syntax to simplify AST representation

I think the feature itself is done.
I'm moving the sxproto2textproto to a different repo, so it's best to close this out and follow up elsewhere as needed.


Allow to limit the objects that will be geocoded by rake task

That's a proposal to limit the objects that will be geocoded by the rake task, using an environment variable (LIMIT).
It could be useful if you want to geocode thousands of objects and don't want to reach the per-day limits of the geocoding services (for example if you are running the rake task in production and you are using the service for other porpoises.
It won't use batches when LIMIT is used, because limit and find_each don't work together.
Comments and/or corrections are very welcome :)

This looks good to me. @alexreisner ?

Thanks! Sorry for the delay.

 Thanks for merging it 👍


It is needed to motivate cluster nodes to do their job

In the current state of the project cluster nodes do not have a reason to stay online and submit transactions.

@yegor256/z please, pay attention to this issue

@vizotov yes, you are right. I believe, the best way is to just charge a fixed fee for each transaction processing, just like Bitcoin does.

If the fee is fixed, there is a possibility that it may not be enough. This will depend on the exchange rate. In the Bitcoin, the calculation of transaction fees is a partially automatic process, and is partially determined by the market. But here, since the currency is just a debt issuance of the issuer, then he must control the fee in order to keep the currency alive.


rawHeaders are unusued in KafkaMessageDrivenChannelAdapter.toMessage()

I mean that this code snippet in line 197 is dead.
Map<String, Object> rawHeaders = kafkaMessageHeaders.getRawHeaders();
rawHeaders.put(KafkaHeaders.MESSAGE_KEY, key);
rawHeaders.put(KafkaHeaders.TOPIC, metadata.getPartition().getTopic());
rawHeaders.put(KafkaHeaders.PARTITION_ID, metadata.getPartition().getId());
rawHeaders.put(KafkaHeaders.OFFSET, metadata.getOffset());
rawHeaders.put(KafkaHeaders.NEXT_OFFSET, metadata.getNextOffset());

if (!this.autoCommitOffset) {
	rawHeaders.put(KafkaHeaders.ACKNOWLEDGMENT, acknowledgment);
}

I don't understand what you mean. This is shortcut for kafkaMessageHeaders.getRawHeaders(), which is used to set the headers of the sent message.
If you wish to understand how this works, I will suggest running one of the tests that exercise header propagation, such as org.springframework.integration.kafka.config.xml.KafkaMessageDrivenChannelAdapterParserTests.
If you have an actual case where this doesn't work, than feel free to open a new one with a more detailed description.

Hi, sorry if I'm missing something but the point is that "rawHeaders" are not used during Message creation, only kafkaMessageHeaders are used.
if (this.useMessageBuilderFactory) {
			return getMessageBuilderFactory()
					.withPayload(payload)
					.copyHeaders(kafkaMessageHeaders)
					.build();
		}
		else {
			return MessageBuilder.createMessage(payload, kafkaMessageHeaders);
		}

You're right tests are working as expected.

Even if it (rawHeaders) is local variable it is a reference to the real object. That is how Java works.
See the KafkaMessageHeaders source code. it is a Map anyway. and it is an immutable wrapper around rawHeaders Map.
You might be correct that we will have the same result even with the rawHeaders here, but it is fully minor and I don't see reason to rework.


References to class_name when extending Reference within the same script causes object leaks

Godot version:
3.2 branch commit 3de89f78e51eab7b2e3299098a34c9a9d0764005
OS/device including version:
Arch Linux
Issue description:
The following script causes a cyclic reference which causes an object leak:
class_name CyclicReference
extends Reference

func test(_p_cyclic : CyclicReference):
        pass

Steps to reproduce:
Open and close the Editor on the attached project.
Minimal reproduction project:
CyclicReferenceDependency.zip

Seems related to #40330


Sequence grouped(by:) and keyed(by:)

Pitched here: https://forums.swift.org/t/pitch-sequence-grouped-by-and-keyed-by/65511
Sample implementation: https://github.com/apple/swift-algorithms/pull/197

We're going to leave this just in swift-algorithms for the time being, so I'm closing out the evolution PR.


What to replace APIWeaver.Swagger with?

Hey, we were using AddOpenApiDocument and MapSwaggerUi. I tried upgrading to your latest NuGet (2.7.0) and noticed that this feature is gone. What should we replace it with?

Ah found it in the nuget package description (didn't see it because i used a version of the bundle package 'APIWeaver').
But i decided to go for Swashbuckle.AspNetCore.SwaggerUI.
Thanks anyways!


Are there any plans for v4?

Hey there, yes there are some, but will leave v4 for later, when we have it stable I think, but as far as I know, it should work, I mean I see no reason why they won't work as is now.
Have you tried it?

Hi and thanks for sharing your hard work with us! Honestly I haven't found some spare time to actually play with the alpha, but I'll try to test run it this week and I will come up with some actual results


Create basis data for a specific energy supplier within a certain grid area

Synopsis
As an energy supplier
I want basis data per grid area per batch (to be created)
So that I can eventually get/verify the data
Acceptance Criteria

[ ] Basis data is created for each energy supplier for each grid area in a batch (anticipated testing: developer provides the files to PO/SME)

Task List

[ ] Create basis data for energy suppliers @BjarkeMeier
[ ] Move migration function helpers inside the migration script @BjarkeMeier
[ ] Move basis data contract tests from test_calculator_job to test_basis_data_writer

Tasks Completed

[x] Update existing code according to newly defined structure @energinet-ajw @BjarkeMeier
[x] Define folder structure for basis data files in Data Lake @energinet-ajw @BjarkeMeier
See new folder structure in tech notes

Tech. Notes
   calculation-output
      batch_id=
         basis_data
            time_series_quarter
               grouping=total_ga
                  grid_area={partitioned}
                     part-xx.csv

               grouping=es_ga
                     grid_area={partitioned}
                        gln={partitioned}
                           part-xx.csv

            time_series_hour
               grouping=total_ga
                  grid_area={partitioned}
                     part-xx.csv

               grouping=es_ga
                     grid_area={partitioned}
                        gln={partitioned}
                           part-xx.csv

            master_basis_data
               grouping=total_ga
                  grid_area={partitioned}
                     part-xx.csv

               grouping=es_ga
                     grid_area={partitioned}
                        gln={partitioned}
                           part-xx.csv

@MadsBloendAndersen , @khatozen what are the expected columns in the basis data files as compared to the ones we create for the energinet user per grid area?


calico-kube-controller 3.26 high cpu usage frequent ipam syncs

We updated calico in one of our larger clusters from 3.24.5 to 3.26.1 and since then the kube-controller has a comparably high cpu usage and seems to perform many ipam syncs.
3.24 in an cluster of about 200 nodes and 10000 pods uses about 200m cpu while 3.26.1 uses a full cpu.
The underlying cause appears to be a change probably added in 3.24.6 which made the allocationIsValid function actually use the cache which reduced the cache sync time from a few minutes to a couple seconds.
So essentially a problem was fixed but maybe the high cpu usage is a fixable issue or the sync frequency is too high.
a pprofile showed it spends lots of time in defaultWorkloadEndpointConverter in checkAllocations
         0     0%     0%    159.53s 67.54%  github.com/projectcalico/calico/kube-controllers/pkg/controllers/node.(*ipamController).acceptScheduleRequests
     0.64s  0.27%  0.27%    153.88s 65.15%  github.com/projectcalico/calico/kube-controllers/pkg/controllers/node.(*ipamController).checkAllocations
         0     0%  0.27%    153.88s 65.15%  github.com/projectcalico/calico/kube-controllers/pkg/controllers/node.(*ipamController).syncIPAM
     1.44s  0.61%  0.88%    129.67s 54.90%  github.com/projectcalico/calico/kube-controllers/pkg/controllers/node.(*ipamController).allocationIsValid
         0     0%  0.88%     99.52s 42.13%  github.com/projectcalico/calico/libcalico-go/lib/backend/k8s/conversion.defaultWorkloadEndpointConverter.PodToWorkloadEndpoints
     8.49s  3.59%  4.48%     99.52s 42.13%  github.com/projectcalico/calico/libcalico-go/lib/backend/k8s/conversion.defaultWorkloadEndpointConverter.podToDefaultWorkloadEndpoint

Current Behavior
grep -E "Synchronizing IPAM|sync com|Triggering sync after batch of updates "
calico-kube-controllers-7cc79d76fd-n5526 calico-kube-controllers 2023-07-05 15:46:54.246 [DEBUG][1] ipam.go 968: Synchronizing IPAM data
calico-kube-controllers-7cc79d76fd-n5526 calico-kube-controllers 2023-07-05 15:46:57.021 [DEBUG][1] ipam.go 1009: IPAM sync completed
calico-kube-controllers-7cc79d76fd-n5526 calico-kube-controllers 2023-07-05 15:46:57.026 [DEBUG][1] ipam.go 285: Triggered IPAM sync complete
calico-kube-controllers-7cc79d76fd-n5526 calico-kube-controllers 2023-07-05 15:47:08.677 [DEBUG][1] ipam.go 262: Triggering sync after batch of updates batchSize=1
calico-kube-controllers-7cc79d76fd-n5526 calico-kube-controllers 2023-07-05 15:47:08.677 [DEBUG][1] ipam.go 968: Synchronizing IPAM data
calico-kube-controllers-7cc79d76fd-n5526 calico-kube-controllers 2023-07-05 15:47:11.352 [DEBUG][1] ipam.go 1009: IPAM sync completed
calico-kube-controllers-7cc79d76fd-n5526 calico-kube-controllers 2023-07-05 15:47:11.357 [DEBUG][1] ipam.go 285: Triggered IPAM sync complete
calico-kube-controllers-7cc79d76fd-n5526 calico-kube-controllers 2023-07-05 15:47:11.357 [DEBUG][1] ipam.go 262: Triggering sync after batch of updates batchSize=13
calico-kube-controllers-7cc79d76fd-n5526 calico-kube-controllers 2023-07-05 15:47:11.357 [DEBUG][1] ipam.go 968: Synchronizing IPAM data
calico-kube-controllers-7cc79d76fd-n5526 calico-kube-controllers 2023-07-05 15:47:13.949 [DEBUG][1] ipam.go 1009: IPAM sync completed
calico-kube-controllers-7cc79d76fd-n5526 calico-kube-controllers 2023-07-05 15:47:13.954 [DEBUG][1] ipam.go 285: Triggered IPAM sync complete
calico-kube-controllers-7cc79d76fd-n5526 calico-kube-controllers 2023-07-05 15:47:13.954 [DEBUG][1] ipam.go 262: Triggering sync after batch of updates batchSize=13
calico-kube-controllers-7cc79d76fd-n5526 calico-kube-controllers 2023-07-05 15:47:13.954 [DEBUG][1] ipam.go 968: Synchronizing IPAM data
calico-kube-controllers-7cc79d76fd-n5526 calico-kube-controllers 2023-07-05 15:47:48.794 [DEBUG][1] ipam.go 1009: IPAM sync completed
calico-kube-controllers-7cc79d76fd-n5526 calico-kube-controllers 2023-07-05 15:47:48.801 [DEBUG][1] ipam.go 285: Triggered IPAM sync complete
calico-kube-controllers-7cc79d76fd-n5526 calico-kube-controllers 2023-07-05 15:47:48.801 [DEBUG][1] ipam.go 262: Triggering sync after batch of updates batchSize=14
calico-kube-controllers-7cc79d76fd-n5526 calico-kube-controllers 2023-07-05 15:47:48.801 [DEBUG][1] ipam.go 968: Synchronizing IPAM data
calico-kube-controllers-7cc79d76fd-n5526 calico-kube-controllers 2023-07-05 15:47:51.709 [DEBUG][1] ipam.go 1009: IPAM sync completed
calico-kube-controllers-7cc79d76fd-n5526 calico-kube-controllers 2023-07-05 15:47:51.716 [DEBUG][1] ipam.go 285: Triggered IPAM sync complete
calico-kube-controllers-7cc79d76fd-n5526 calico-kube-controllers 2023-07-05 15:47:51.716 [DEBUG][1] ipam.go 262: Triggering sync after batch of updates batchSize=14
calico-kube-controllers-7cc79d76fd-n5526 calico-kube-controllers 2023-07-05 15:47:51.716 [DEBUG][1] ipam.go 968: Synchronizing IPAM data
calico-kube-controllers-7cc79d76fd-n5526 calico-kube-controllers 2023-07-05 15:47:54.466 [DEBUG][1] ipam.go 1009: IPAM sync completed
calico-kube-controllers-7cc79d76fd-n5526 calico-kube-controllers 2023-07-05 15:47:54.471 [DEBUG][1] ipam.go 285: Triggered IPAM sync complete
calico-kube-controllers-7cc79d76fd-n5526 calico-kube-controllers 2023-07-05 15:47:54.471 [DEBUG][1] ipam.go 262: Triggering sync after batch of updates batchSize=13
calico-kube-controllers-7cc79d76fd-n5526 calico-kube-controllers 2023-07-05 15:47:54.471 [DEBUG][1] ipam.go 968: Synchronizing IPAM data
calico-kube-controllers-7cc79d76fd-n5526 calico-kube-controllers 2023-07-05 15:48:28.954 [DEBUG][1] ipam.go 1009: IPAM sync completed
calico-kube-controllers-7cc79d76fd-n5526 calico-kube-controllers 2023-07-05 15:48:28.960 [DEBUG][1] ipam.go 285: Triggered IPAM sync complete
calico-kube-controllers-7cc79d76fd-n5526 calico-kube-controllers 2023-07-05 15:48:28.960 [DEBUG][1] ipam.go 262: Triggering sync after batch of updates batchSize=14
calico-kube-controllers-7cc79d76fd-n5526 calico-kube-controllers 2023-07-05 15:48:28.960 [DEBUG][1] ipam.go 968: Synchronizing IPAM data

The difference to 3.24 seems to be that it uses the podcache which was changed in 3.24.6 https://github.com/projectcalico/calico/pull/7503
3.24.5 instead logs the queries for each of the 9000 ipamhandles which takes several minutes:
calico-kube-controllers-bc987cddc-vr4gk calico-kube-controllers 2023-07-05 16:20:28.566 [DEBUG][1] ipam.go 1008: Synchronizing IPAM data
calico-kube-controllers-bc987cddc-vr4gk calico-kube-controllers 2023-07-05 16:20:28.572 [DEBUG][1] ipam.go 920: Querying Kubernetes API for pod handle="k8s-pod-network.9...
.

So the cache is now effective which reduces the load on the apiserver.

Calico version 3.26.1
Orchestrator version (e.g. kubernetes, mesos, rkt): kubernetes
Operating System and version: flatcar linux

Sounds like we pushed one bottleneck somewhere else :sweat_smile:
perhaps we need some better internal ratelimiting / batching to make sure we're not eating up too much resource at once.

Any progress here? This issue was fixed?
We hit same issue with high cpu usage on 3.26.4

I think this TODO is essentially the problem here: https://github.com/projectcalico/calico/blob/a1273bd2bf0341082152719eed19c4df8cd67acf/kube-controllers/pkg/controllers/node/ipam.go#L717
We previously had unintentional rate limiting, but now we're iterating everything for every sync. We do some batching, but it's clearly not enough.
I think we want to introduce "dirty" tracking here, so that we can reduce this loop to only checking the allocations that we know have changed since the last sync.

This is a candidate PR that might reduce the CPU usage: https://github.com/projectcalico/calico/pull/9654


Update ammonite-compiler_2.13.7, ... to 2.5.4-26-9cd15abe

Updates

com.lihaoyi:ammonite-compiler_2.13.7
com.lihaoyi:ammonite-compiler_2.13.8
com.lihaoyi:ammonite-compiler_3.1.0
com.lihaoyi:ammonite-compiler_3.1.1
com.lihaoyi:ammonite-compiler_3.1.2
com.lihaoyi:ammonite-compiler_3.1.3
com.lihaoyi:ammonite-cross-3.1.0-repl-api_2.13.7
com.lihaoyi:ammonite-cross-3.1.0-repl_2.13.7
com.lihaoyi:ammonite-cross-3.1.1-repl-api_2.13.7
com.lihaoyi:ammonite-cross-3.1.1-repl_2.13.7
com.lihaoyi:ammonite-cross-3.1.2-repl-api_2.13.7
com.lihaoyi:ammonite-cross-3.1.2-repl_2.13.7
com.lihaoyi:ammonite-cross-3.1.3-repl-api_2.13.7
com.lihaoyi:ammonite-cross-3.1.3-repl_2.13.7
com.lihaoyi:ammonite-repl-api_2.13.6
com.lihaoyi:ammonite-repl-api_2.13.7
com.lihaoyi:ammonite-repl-api_2.13.8
com.lihaoyi:ammonite-repl_2.13.6
com.lihaoyi:ammonite-repl_2.13.7
com.lihaoyi:ammonite-repl_2.13.8

from 2.5.4-22-4a9e6989 to 2.5.4-26-9cd15abe.
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Adjust future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "com.lihaoyi" } ]

Or, add this to slow down future updates of this dependency:
dependencyOverrides = [{
  pullRequests = { frequency = "@monthly" },
  dependency = { groupId = "com.lihaoyi" }
}]

labels: library-update, early-semver-pre-release, semver-spec-pre-release, commit-count:1

Superseded by https://github.com/almond-sh/almond/pull/991.


blurry results from stage1 when using img clip

I succeed to inference your work with text clip in stage1, but I saw blurry results when using img clip

I tried below code
from ldm.modules.encoders.modules import ClipImageProjector
from torchvision import transforms

version="./../pretrain_models/clip-vit-large-patch14"
clip_model2 = ClipImageProjector(version=version).to(device)
tform = transforms.ToTensor()

# text = tokenizer(text_description,truncation=True, max_length=77, return_length=True,
#             return_overflowing_tokens=False, padding="max_length", return_tensors="pt")

# text_features = clip_model(text["input_ids"].cuda(non_blocking=True))
# text_features = text_features.last_hidden_state # torch.Size([1, 77, 768])

garment_condition_path = os.path.join("./Sample_data/Cloth_White_Background", file_name[0])
garment_condition = tform(Image.open(garment_condition_path).convert("RGB"))

garment_condition = garment_condition * 2. - 1.
garment_condition = clip_model2.preprocess(garment_condition.unsqueeze(0)) # got similar results with / without preprocessing
text_features = clip_model2(garment_condition.cuda(non_blocking=True))

c = [concat_feature,text_features]
sampler.sample(S=opt.ddim_steps,
               conditioning=c,
               ...)

Could you please help me to use img clip?

Below code also doesn't work.. (makes shape error in sampling)
garment_condition = tform(Image.open(garment_condition_path).resize((224,224), Image.LANCZOS).convert("RGB"))
text_features = clip_model3(garment_condition.unsqueeze(0).cuda(non_blocking=True))
text_features = text_features.last_hidden_state

current, I just release the version using text to train stage 1, if you want to use img clip, I think you can train a model.


Fix typo in documentation

I can't make GitHub's UI not add that newline.  I'll clone and try again


Adds solution for 128. Longest consecutive sequence and 150. Evaluate Reverse Polish notation in Kotlin

File(s) Modified: 128-Longest-Consecutive-Sequence.kt,150-Evaluate-Reverse-Polish-Notation.kt
Language(s) Used: Kotlin
Submission URL: https://leetcode.com/submissions/detail/761280821/ , https://leetcode.com/submissions/detail/761296106/

Thanks, @t3chkid!


Canary: Colored text

The colored text option in Core Settings doesn't function properly after the recent Canary updates. I have cycled the option, reinstalled BBD, and there have been a few updates since the problem first showed up. I waited a while to report as some issues resolve themselves in future updates but this time it seems to be persistent.

Oh, and emote conversion isn't happening either.

fixed as of a1b44b49b2b6b0ecade39f5e7b885624f22358ee

Problem with colored text returned.


EfficientPose conversion

There seems to be something wrong with the EfficientPose models (or at least EfficientPoseII, didnt test the others yet)
The output is very small and fuzzy, not at all like the expected keypoint heatmap. For comparison I converted the current model myself with tf2onnx, and that one works as expected.
moreover, I wanted to compare the .pb from your repo with the original .pb, but wasn't able to load/run your version..
I see the repo also provides torch models, did you use those for conversion to ONNX? Or was it an older, buggy version? Either way, I think the efficientpose-folder might need an update :)
Windows10, x86_64
example model EfficientPoseII
onnx 1.8.1, onnxruntime 1.5.2 with DirectML
If you want to test, following pre-processing should do the trick:
sess = rt.InferenceSession("model_float32_opt.onnx")
input_name = sess.get_inputs()[0].name

# resize    (test_frame is the result of cv2 videocapture)
f = min(368 / test_frame.shape[0], 368 / test_frame.shape[1])
test_frame = cv2.resize(test_frame, (0, 0), fx=f, fy=f)

test_frame = test_frame.astype('float32') / 255  # scale to range 0..1
test_frame = test_frame[..., ::-1]  # assumes channels last! ->  x = x[::-1, ...]  # BGR->RGB

# pad
blob = np.zeros((368,368,3), dtype='float32')  # channels last
d1 = int((368-test_frame.shape[0])/2)
d2 = int((368-test_frame.shape[1])/2)
blob[d1:368-d1,d2:368-d2] = test_frame

mean=[0.485, 0.456, 0.406]
std=[0.229, 0.224, 0.225]
for i in range(3):
    blob[..., i] -= mean[i]
    blob[..., i] /= std[i]

out = sess.run(None, {input_name: [blob]})

Thank you for pointing this out to me.
Two months ago I did not use an automatic conversion tool, I did the conversion work manually. So it is very likely that I made a mistake in the process of converting the model.
This time, I used the automatic conversion tool openvino2tensorflow to convert the data automatically. The .pb was a file downloaded from the original repository.
$ pip install openvino2tensorflow --upgrade

$ pb_to_saved_model \
--pb_file_path saved_model/EfficientPoseII.pb \
--inputs input_res1:0 \
--outputs upscaled_confs/BiasAdd:0

$ saved_model_to_tflite \
--saved_model_dir_path saved_model \
--model_output_dir_path saved_model \
--input_shapes [1,368,368,3] \
--output_no_quant_float32_tflite True \
--output_onnx True

I will try to verify the results using the test code you provided when I have time. Once I have confirmation that it works correctly, I will update all the committed models. Just in case, I'll share the ONNX and tflite that I automatically converted using the tool.
saved_model.tar.gz
Once again, thank you.

Special procedure using inversion from tflite. tflite2tensorflow GitHub
$ pip install tflite2tensorflow --upgrade

$ tflite2tensorflow \
  --model_path EfficientPoseII.tflite \
  --flatc_path ./flatc \
  --schema_path schema.fbs \
  --output_pb True

$ tflite2tensorflow \
  --model_path EfficientPoseII.tflite \
  --flatc_path ./flatc \
  --schema_path schema.fbs \
  --output_no_quant_float32_tflite True \
  --output_weight_quant_tflite True \
  --output_tfjs True \
  --output_coreml True \
  --output_tftrt True \
  --output_onnx True \
  --onnx_opset 13

https://drive.google.com/file/d/1kIMvvRGfQ6mv4IzQ72Iv-1S57PzpxMn1/view?usp=sharing

The onnx in the first folder gives me
INVALID_GRAPH : Load model from ../models/PINTO/084_EfficientPose/saved_model/model_float32.onnx failed:This is an invalid model. Error in Node:upscaled_deconv2/stack_Unsqueeze__1175 : Node (upscaled_deconv2/stack_Unsqueeze__1175) has input size 1 not in range [min=2, max=2].

The second looks better, but there seems to be still some differences compared to the original model

I think this needs to be investigated a bit more detailed to see what exactly is going on, it seems to be a bit a special case model

wait, let me check again, it seems the order of outputs is differnt with the tflite-based model

So, just fyi:
pass3_detection2_confs/BiasAdd:0 and pass2_detection1_confs/BiasAdd:0 output look correct, e.g. detection2:

but the upscaled_confs/BiasAdd:0 is empty

Hmmm. The tflite used as the source of the conversion is a float16, so the accuracy may be slightly worse than the weights extracted from a float32 model, but it is strange that the results are all empty.
I will need to investigate in detail the differences between the original model and the model I transformed in various patterns to identify bugs in the transformation.
Thank you for cooperating with me so many times.

no worries, thank you for your work!

It turns out that the TransposeConv weights are not transcribing correctly. We will check other parts of the code later to see if there are any problems.

original

After conversion

Thanks to you I was able to notice a bug in the tool and did the conversion again and found that the TransposeConv was converted correctly. Thanks!
https://drive.google.com/file/d/1Lcgj-fLfiwLJ867nbAvrZRA952V9W7Zh/view?usp=sharing

All models have been updated. The URL from which they are downloaded is the same as before the update, but the resources downloaded have been updated.
https://github.com/PINTO0309/PINTO_model_zoo/tree/main/084_EfficientPose

Thanks, the onnx look good!
A bit unexpectedly, they don't perform better on my (admittedly, integrated) GPU, they actually perform a bit worse sometimes. Moreover, even when using fp16, inference time is exactly the same... With other models I got better performance with GPU usage and especially fp16+GPU.
To compare I wanted to run the OpenVINO model, to see if the model itself is the problem, but I can't load it with cv2.dnn.readNet. Can you?

It is probably different from the OpenCV backend you are using, but I will share my test results.

EfficientPoseII - saved_model_openvino_myriad/openvino/FP16

import cv2
import time

print('opencv version:', cv2.__version__)

xml = 'saved_model.xml'
bin = 'saved_model.bin'
image = cv2.imread('test.png')
blob = cv2.dnn.blobFromImage(image, 1, (368, 368), (104, 117, 123))

print("[INFO] DNN_BACKEND_INFERENCE_ENGINE loading model...")
net = cv2.dnn.readNetFromModelOptimizer(xml, bin)
net.setPreferableBackend(cv2.dnn.DNN_BACKEND_INFERENCE_ENGINE)
net.setPreferableTarget(cv2.dnn.DNN_BACKEND_INFERENCE_ENGINE)
net.setInput(blob)
start = time.time()
preds = net.forward()
end = time.time()
print("[INFO] DNN_BACKEND_INFERENCE_ENGINE inference took " + str((end - start) * 1000) + " ms")

print("[INFO] DNN_TARGET_OPENCL_FP16 loading model...")
net = cv2.dnn.readNetFromModelOptimizer(xml, bin)
net.setPreferableBackend(cv2.dnn.DNN_BACKEND_INFERENCE_ENGINE)
net.setPreferableTarget(cv2.dnn.DNN_TARGET_OPENCL_FP16)
net.setInput(blob)
start = time.time()
preds = net.forward()
end = time.time()
print("[INFO] DNN_TARGET_OPENCL_FP16 inference took " + str((end - start) * 1000) + " ms")

print("[INFO] DNN_TARGET_CPU loading model...")
net = cv2.dnn.readNetFromModelOptimizer(xml, bin)
net.setPreferableBackend(cv2.dnn.DNN_BACKEND_INFERENCE_ENGINE)
net.setPreferableTarget(cv2.dnn.DNN_TARGET_CPU)
net.setInput(blob)
start = time.time()
preds = net.forward()
end = time.time()
print("[INFO] DNN_TARGET_CPU inference took " + str((end - start) * 1000) + " ms")

print("[INFO] DNN_TARGET_MYRIAD loading model...")
net = cv2.dnn.readNetFromModelOptimizer(xml, bin)
net.setPreferableBackend(cv2.dnn.DNN_BACKEND_INFERENCE_ENGINE)
net.setPreferableTarget(cv2.dnn.DNN_TARGET_MYRIAD)
net.setInput(blob)
start = time.time()
preds = net.forward()
end = time.time()
print("[INFO] DNN_TARGET_MYRIAD inference took " + str((end - start) * 1000) + " ms")

$ python3 test.py
opencv version: 4.5.1-openvino
[INFO] DNN_BACKEND_INFERENCE_ENGINE loading model...
[INFO] DNN_BACKEND_INFERENCE_ENGINE inference took 48395.20835876465 ms
[INFO] DNN_TARGET_OPENCL_FP16 loading model...
[INFO] DNN_TARGET_OPENCL_FP16 inference took 47927.51097679138 ms
[INFO] DNN_TARGET_CPU loading model...
[INFO] DNN_TARGET_CPU inference took<PHONE_NUMBER>239502 ms
[INFO] DNN_TARGET_MYRIAD loading model...
[INFO] DNN_TARGET_MYRIAD inference took 6135.213851928711 ms

hmm ok thanks, seems to be an issue with my setup, not the model then.
on my end python crashes upon trying to read the net, before even setting target/backend. I'll try a few more things to see if i can figure it out.
The original issue sure is solved, thanks :)

FYI: works after upgrading my openvino from 2020.4 to 2020.2 :) also GPU inference seems plausible now


Add TheWebToolbox.cc

Add a new section for more tool collections and add The Web Toolbox to this list. All tools hosted there are free and most don't require any sort of account.
I'm hoping everyone is happy with adding a new section which leads to further tool collections? Might be useful to link to some of the Awesome GitHub collections people have created, or in the future merge TWT and this list together?

Any chance I could get some clarification on why this was closed please? Is this tool not allowed, or just in the wrong section?