Running out of memory

Hi there,
I am trying to get pciSeq  0.0.51 to run on our Merfish data, but I constantly run out of memory on a 150 GB Ram, 30 cpu job.
[2023-07-22 09:28:33] INFO    (app.py:204)  output_path is set to ['./pciseq_results']
[2023-07-22 09:28:33] INFO    (app.py:204)  Inefficiency is set to 0.2
[2023-07-22 09:28:33] INFO    (app.py:204)  InsideCellBonus is set to 2
[2023-07-22 09:28:33] INFO    (app.py:204)  dtype is set to <class 'numpy.float16'>
[2023-07-22 09:28:33] INFO    (app.py:204)  save_data is set to True
[2023-07-22 09:28:33] INFO    (app.py:204)  launch_viewer is set to True
[2023-07-22 09:28:33] INFO    (app.py:118)  Preprocessing data
[2023-07-22 09:28:33] INFO    (spot_labels.py: 66)  The labels in the label image do not seem to be a sequence of successive integers. Relabelling the label image.
[2023-07-22 09:29:03] INFO    (spot_labels.py: 69)  Number of spots passed-in: 1002301
[2023-07-22 09:29:04] INFO    (spot_labels.py: 70)  Number of segmented cells: 65527
[2023-07-22 09:29:04] INFO    (spot_labels.py: 71)  Segmentation array implies that image has width: 14485px and height: 10422px
slurmstepd: error: Detected 1 oom-kill event(s) in StepId=37082961.batch. Some of your processes may have been killed by the cgroup out-of-memory handler.

This is on a pretty small 2D measurement and the single cell reference is also small, dataframe 29046 rows × 11416 columns with 11 cell types. I already set dtype to float16, is there anything else I can optimize?

I believe it is the segmentation array that causes the out-of-memory error but I think I had fixed this in a previous version. Anyway your image doesnt look that big anyway, it shouldnt happen.
Can you share the data so I can have a look?

@acycliq I uploaded the files here https://rwth-aachen.sciebo.de/s/oBShlsFJOr0njp8
Thank you for your help

I will send you a fix later but are you ok with your segmentation, do you really have more than 65,000 cells on that slice? I would say it looks too oversegmented to me but I guess you know your data better . Note also that the max label in your coo_matrix is 65535 which equals the max value for uint8.  What I am trying to say is that the cell segmentation algorithm overflows the uint8 range. It reaches label 65535 then it runs out of integers to use and loops over from the start again, ie using label 1, 2 etc again. I dont really know, thats something to highlight so you can have it in mind.

Thank you great point on the uint8.
We are still settling on a segmentation strategy right now. This is the default that the machine outputs and I am trying to see how pciseq / baysor as a follow-up might improve that.
Do you have any suggestions as far as segmentation goes?

Can you try this dev version: pip install pciSeq==0.0.57.dev1. I tried your data and didnt get any errors (on a simple desktop with 4cores, 64GB RAM). The error was due to the function that calcs the centroids and area of the cells. I think it is now been handled better, doesn't drop errors but it takes long to finish, around 40mins on my desktop.
You will need to massage a little bit your data. Some of the spots have to be removed because there are no data for them in the reference library and the columns, index of the refData dataframe need to set again, see below
refData = pd.read_pickle("sc-reference.pkl")

cols = refData.columns.to_list()
idx = refData.index.to_list()
sc_ref = pd.DataFrame(refData.values, index=idx, columns=cols)

drop_spots = ['AL391650.1', 'HOTAIR', 'MHRT', 'PTPRC-exon4-6-introns', 'RGS5', 'eGFP', 'mCherry2', 'tdToma']
idx = ([i for i, v in enumerate(transcripts_pciseq.Gene) if v not in drop_spots])
transcripts_pciseq = transcripts_pciseq.iloc[idx]

opts = {'save_data': True, 'launch_diagnostics': True, 'launch_viewer': True}
cellData, geneData = fit(spots=transcripts_pciseq, coo=matrix, scRNAseq=sc_ref, opts=opts)

This will produce a lot of Zero-classed cells, around 66%, I dont know if that makes sense or you have to tweak a bit the inefficiency maybe....
With regards to your question about segmentation, we tend to use cellpose, but when I was looking at it (around version 1,0 or just before it) I think it would also overflow the 16bit ints especially in 3d. I am not 100% sure though and I dont know if that is still the case.
Hope that helps, let me know if you have any problems. Out of interest, may I ask which lab are you from?

Thank you so much, it runs now!
I am working at the University of Aachen in Germany :)
For my understanding, a Zero class cell represents an entity which Pciseq thinks is a cell but which it cannot assign a celltype from the reference?
And the inefficency value represents how closely I suspect that spatial and single cell data match on a scale from 0-1?
So increasing this value tends to produce more Zero class cells, as the algorithm needs more convincing that a celltype from the reference applies?
What about the other parameters. Should I increase InsideCellBonus if I ever get to the point where I am very confident abou the supplied coomatrix?

Zero class is literally an extra column that is added at the end of your reference data with counts very close to zero. It acts as a "none from the above" class. However pciSeq is not a segmentation tool. It doesn't try to answer a question like "this is my image, tell me what shapes you can see". Segmentation is an input. Hence instead of "..thinks as a cell...." that you mention above, I would say that the point pattern around that area doesn't fit any of the classes in the ref data and the cell is assigned the Zero class.
Yes, inefficiency represents how closely your data match the single cell data but it can be any value greater that zero, not just between 0 and 1. Its main role is to bring the counts on the spatial-data side and single-cell data side at about the same level so pciSeq can pick up a class from the single cell data. Something similar to having miles vs kilometers, or EUR vs USD. It has to do with how sensitive a method is and the single cell data.  There is a python notebook about some diagnostics on the Readme of the this repo, have a look there, maybe it will help. It needs redis to run, I hope it works for you.
pciSeq reads the segmentation and calculates the mean radius of the cells. Then given this radius, it fits a bivariate normal distribution (with diagonal correlation matrix, ie no dependence between x and y)  to assess how likely it is a spot to "belong" to a cell. The InsideCellBonus  adds an extra bias to push that probability even higher if the spot is within the boundaries. If you are really confident about your segmentation then yes, push that hyperparameter even higher.
There is however a branch of the code that is very relaxed about segmentation. It doesn't care about the cell boundaries at all. It starts from the cell centroids and then fits again a bivariate normal distribution but now the correlation matrix and the centroid are getting estimated on the fly. The algorithm tries to fit a distribution that fits best the point pattern around that area, something like a Gaussian Mixture Model for example. Let me know if you want me to release that.
PciSeq will not fix/correct the segmentation. It will not merge or splt two shapes and then check if the counts in the new shape make more sense or not. The branch of the code I mentioned two sentences above, maybe can be seen from this perspective (but not exactly), in the sense that it fits a normal distribution and its shape grows or maybe shrinks because the covariance matrix changes based on the points in that area. The same happens for the centroid, it gets shifted.  It cannot be considered as a segmentation mechanism whatsoever.

Thanks you, that is quite helpful.


Memories with non-combinational reads

Update by @rachitnigam:
The memory primitives in Calyx's standard library allow for combinational reads. Different FPGAs and ASIC processes might allow for different kinds of memories, some of which might not allow for combinational reads. We should attempt to support those by either adding a new primitive, or updating the memory primitives to use a read_done signal.

Original issue by @sgpthomas:
Change the memories and Dahlia Futil backend to use the read_done signal to know that a value is ready to be read.

To reup this, our current memories are a lie. Combinational reads from memories don't exist. We should think more carefully about what the correct memory interface is. To elaborate the problem a little, the obvious thing to do is give memories a normal Calyx go/done interface. But it's not entirely obvious what this should look like because there are two distinct actions a memory can perform: reads and writes. When you set go high, which one should the memory perform?
Imagine some interface like:
primitive mem(go: 1, addr: 32, write_data: 32) -> (read_data: 32, done: 1);

and a read
group read {
  mem.go = 1;
  mem.addr = 0x00;
  read_value.in = mem.done ? mem.read_data;
  read[done] = mem.done;
}

How does mem distinguish this use from:
group write {
  mem.go = 1;
  mem.addr = 0x00;
  mem.write_data = 0x10;
  read[done] = mem.done;
}

Ports always have values, so we can't look at whether write_data is written to to decide whether to do a read or a write.
A possible answer is to have some extra "type" port which the component uses to decide, so for example
group A {
  mem.go = 1;
  mem.perform_write = 1; // <- go = 1 & perform_write = 1 means do write
  ...
}

This is a fine solution for now, but it seems like the thing that's really going on here is there are two seperate ways to call this component that each should get their own go/done interface. So this would look something like:
group read {
  mem.read.go = 1;
  mem.addr = 0x00;
  read_value.in = mem.read.done ? mem.read_data;
  read[done] = mem.read.done;
}

and similarly for writes. However, this raises the question: what happens if you use both interfaces at once? Like:
group both {
  mem.read.go = 1;
  mem.write.go = 1;
  ...

Is this valid? For memories, maybe not (though maybe dual ported memories might want an interface like this?), but for an AXI interface this seems very natural (maybe necessary even). There's an idea of simultaneous points of control flow through a single component here that also maybe generalizes to pipelines. Obviously more thought needs to be given to this, but it's an intriguing idea.

Aside from the practical issue of supporting more realistic memories, it's an intriguing idea to make memories more "normal," i.e., to make it so using a memory is more like using any other component. Or to put it another way, what would it take to make it possible to invoke a memory to perform a read or a write?
All of @sgpthomas's suggestions seem like possibilities for how to do that!

primitive mem(go: 1, addr: 32, write_data: 32) -> (read_data: 32, done: 1);

Why not instead an interface for dual-ported memories like this:
component mem_d1(go: 1, read_addr0: 32, write_addr0: 32, write_data: 32, write_en: 1, read_en: 1) -> (read_data: 32, done: 1) {
...
}

If you write it as a Calyx component, you can have the read and write execute in parallel. Same behavior can be executed in SV modules as well. FWIW, I think this is essentially the idea you get at in the latter half of your post.
And then to follow on @sampsyo 's comment, if a user wants to read and write to the memory,
invoke mem_d1(write_en=1'd1, read_en=1'd1, ...)(...);

This is how I model being able to write and search with the TCAM.

@sgpthomas did we end up dealing with this for #492?

This was not dealt with there. You can get brams that have combinational reads and so our memory model is fine. I left this open though because there's a lot of room to explore different kinds of memories / memory interfaces that have different performance characteristics.


Windows Defender marks exe file as virus

Also the virustotal output: https://www.virustotal.com/gui/file/ff41951c3f519138bb0e61038d7155c6c38194d4d8a3304f46c67c4572ee8bec/detection

I submitted this to the Defender team as a false positive and it has been removed:

We have removed the detection.  Please follow the steps below to clear cached detection and obtain the latest malware definitions.
 1. Open command prompt as administrator and change directory to c:\Program Files\Windows Defender
 2. Run “MpCmdRun.exe -removedefinitions -dynamicsignatures”
 3. Run "MpCmdRun.exe -SignatureUpdate"

Alternatively, the latest definition is available for download here: https://www.microsoft.com/en-us/wdsi/definitions

And it worked. Defender no longer removes it.
Thanks!

Glad it worked - thanks for confirming 😃

This is back, see https://www.virustotal.com/gui/file/4e3c8793543b96738e041946ee73118669aaaba20d2fd8310ebf5ffbb6d15928/detection

Yeah......I kinda get why Windows Defender keeps flagging this file, Go loads all APIs dynamically via LoadLibrary / GetProcAddress, from a debugger perspective it looks suuuuuuper shady


Checkmark vs no checkmark selection

Is your feature request related to a problem? Please describe.
There are mixed opinions about whether the menu should include a checkmark to indication a selection.
Describe the solution you'd like
Consult UI designers to find best solution to indicate selection. We will be weighing the different options and finding a more effective solution for our brand.

Add me back when a review is needed.

I came across this from the spectrum picker element.

It would be interesting to consider this option for the check mark. This would also make it much simpler to support a use case of checkmark or no checkmark.

After the UI sync, the consensus was that the checkmark should be on the left. If having the checkmark as an option is not possible, having the checkmark would be preferred.
The checkmark should live on the left side due to readability issues with horizontally longer options. The only change that should be made to menu is spacing and hover color. This will be added to issue #88.
What are the engineering limitations @blackfalcon?

@leeejune there are no limitations per se. It's more of an issue of complexity.
I'd like to see the outcome of #88 before making the next set of decisions.

Unless I am understanding this wrong, the current decision has no work associated with it. So closing this issue.
When the time comes that a non-checkmark UI is requested, we will create a new issue for that specific work.

Reopening this issue.  We want to have an option to optionally turn on or off the checkbox.    In auro-combobox the checkbox doesn't make sense in most use cases.


Support path and query parameters

Add support for routing path params and expose both path params and query params in screen scope.

@evshi
we are adding the screen paths as per
screens: [
    {
      id: "menu",
      name: "Menu",
      content: String(MenuYAML),
    },
]

where, screen name become the route path, so to enhance the route functionality and support the path params, we need to add one more key to this object. something like this,
screens: [
    {
      id: "menu",
      name: "Menu",
      params: ["param1", "param2"]
      content: String(MenuYAML),
    },
]

or we can add one property with name of path, and refer that property as route path
screens: [
    {
      id: "menu",
      name: "Menu",
      path: "menu/:param1/:param2",
      content: String(MenuYAML),
    },
]

need your suggestions on this

@sagardspeed2 let's go with second option of adding optional path property to each screen

okay sure


Gitlab Icon

Adding icon for Gitlab.

Giving this a bump. Hoping to get a review/approval on this.

@schmalzs released in v5.12.0, thank you for the picture and thank you for contributing.


Add javascript equals in 2.0

This function was added on 1.0 but it is missing on 2.0:
https://github.com/Medology/FlexibleMink/pull/292/files

Closing since there is a very similar function already existing 🤷♂


Package Request: SEGGER J-Link

Package Requested

[x] I would like someone else to build the manifest.
[ ] I would like help so I can submit the manifest.
[x] I have performed a search and couldn't find this package.
[ ] I think there is a new version available and I have provided the URL.

Please provide the following information if you can:
Publisher: SEGGER Microcontroller
Publisher Url: https://www.segger.com/
Package Name: SEGGER J-Link
Package Url: https://www.segger.com/downloads/jlink/
Description: Software for use with the J-Link debug probes
Package Version: V7.54b
Installer URL: https://www.segger.com/downloads/jlink/JLink_Windows_V754b_x86_64.exe
For new versions:
Current Package Identifier: V7.54b

Download url leads to Terms of Use

Download url leads to Terms of Use

@KaranKad Does this mean it can't be integrated into winget?

Not right now, but it is possible to download files behind ELUA/TOS using wget, winget could ask to install wget when downloading files behind ELUA/TOS.

@KaranKad Do you use oh-my-posh or something else for your terminal theme?

@KaranKad Do you use oh-my-posh or something else for your terminal theme?

Yeah I use oh-my-posh and terminal themes from https://windowsterminalthemes.dev/

Could I have some help with this package? I believe that winget now supports .exe installers.

@KaranKad FWIW here is the equivalent curl arg:
--data 'accept_license_agreement=accepted&non_emb_ctr=confirmed&submit=Download+software'

(I'm not sure if the last two params are required)
I don't know how winget does its HTTP but it may be possible to include this POST data for a particular package.

Unfortunately, winget does not support custom arguments while downloading packages as of now.


Add transact with

See: https://gist.github.com/josevalim/8130b19eb62706e1ab37

Alternative: https://github.com/michalmuskala/transchange


is it necessary to finetune the t5 model?

the results are not correct when i generate augment examples using t5 chinese model, is it necessary to finetune the t5 model?

Sorry to hear that. We do not fine-tune the T5 model. (It will undoubtedly overfit the training set in the few-shot scenario).
I didn't try the Chinese T5 model, but I have the following suggestions, which might be helpful. Maybe you could judge what kind of situation you are in and then choose the corresponding actions.

The generated sentences are not fluent or even wrong in grammar. It should be solved almost perfectly because the pretrained models are powerful. In this case, (1) you should check whether the fill-in function is correct, and (2) you should check whether the model is good enough. (For example, T5-large is much much better than T5-base.)

The generated sentences look good but with the wrong labels. In this case, the following actions might help.
(1) generate a lot of candidates and then use the classifier to find out correct examples (as we mentioned as self-training). It will still not be totally correct, but some wrong labeled examples will not influence the performance a lot.
(2) Sometimes the task is too difficult. You could try to construct the prompt in another way and make the model predict more easily. (Well, it is flexible according to your problem)
(3) I didn't try this or check the details. I think the idea might help if you still think finetune is necessary. Treat the generating process also a few-shot learning process, and use prompt-based methods to finetune the T5-model (a little like Paper "PromDA: Prompt-based Data Augmentation for Low-Resource NLU Tasks" does.)
(4) The last way that I could think of is to split the data into two parts, i.e., A and B. Finetune part A and predict for part B, and then finetune part B and then predict for part A. (It might mitigate overfitting to some extent.)


🐞 - Dropdown context menu does not close

Which @taiga-ui/* package(s) are the source of the bug?
core
Please provide a link to a minimal reproduction of the bug
https://stackblitz.com/edit/angular-323fzw?file=src%2Fapp%2Fapp.component.ts
Is this issue blocking you?
Non-Blocking
Description
After opening the dropdown context menu, when you click outside to close it, it does not close and goes into a state where you can't select any options from the menu.
Angular version
14.2.12
Taiga UI version
3.15
Which browsers have you used?

[X] Chrome
[X] Firefox
[ ] Safari
[ ] Edge

Which operating systems have you used?

[ ] macOS
[X] Windows
[ ] Linux
[ ] iOS
[ ] Android

Hi! Did you only face the problem on stackblitz example or real project? I can't reproduce it outside of stackblitz 🤔

Hi, yes, I found the bug in my project first.
When I wanted to see if the problem appeared on taiga-ui.dev, there was no bug on the demo, but on the stackblitz only.
The stackblitz I gave is a copy of the original demo.


Custom metadata in file

Add documentation on how to write/read custom metadata to the file. If it's not exposed publicly also needs to be done as well.

Hi,
Does it support .Net core?  Can someone provide samples for struct and map data type?
Thanks
Ramesh S B

Yes it does.. Net standard 1.4 and up! Samples are in docs, the link is on front page)

https://github.com/elastacloud/parquet-dotnet/blob/master/doc/complex.md#maps


The user should not be able to place the order if the user is inactive.

Bagisto Version: Master
Steps To reproduce--
1. Log in as a customer
2. Add any item to the cat.
3. Click on place order.
4. Select the payment method.
5. log in as admin.
6. Inactive the same user.
7. Go to the shop as a customer
8. continue the order process.
Expected result: The user should not be able to place the order if the user is inactive.
Actual result: The user is able to place the order while inactive. Please find the screen recording Here

Write logic in customer middleware and logout the user by killing the session.


Landing page redesign ideas

Is your feature request related to a problem? Please describe.
Landing page needs design update, while it serves its purpose and looks decent, it's not memorable at all.
Describe the solution you'd like
Landing page redesign with newest standards.

https://www.behance.net/gallery/163847835/NFT-Landing-Page-Design?tracking_source=search_projects|landing+page

Closed due of #241


FAQ page need to be fixed

Can you please assign level2 to this issue?

@kashnx  as this is gonna be a very small change so i will keep just gssoc for now, if i see that the level needs to be added, i will add it


grpc: with the last version of grpc we have a compiler error

Building Linux application...
ERROR: lib/api/cln/cln_client.dart:[11](https://github.com/dart-lightning/lndart.clnapp/runs/7815097891?check_suite_focus=true#step:7:12)3:56: Error: No named parameter with the name 'msatoshi'.
ERROR:               grpcRequest: PayRequest(bolt11: invoice, msatoshi: amount));
ERROR:                                                        ^^^^^^^^
ERROR: /opt/hostedtoolcache/flutter/stable-3.0.5-x64/.pub-cache/hosted/pub.dartlang.org/cln_grpc-0.0.1-beta.3/lib/src/generated/node.pb.dart:1[13](https://github.com/dart-lightning/lndart.clnapp/runs/7815097891?check_suite_focus=true#step:7:14)25:11: Context: Found this candidate, but the arguments don't match.
ERROR:   factory PayRequest({
ERROR:           ^
Build process failed

This is because in new version of core lightning the field is change to a amountMsat

Yep! i fixed in https://github.com/dart-lightning/lndart.clnapp/pull/46


node --inspect appears to be broken or unsupported

See https://github.com/facebook/jest/issues/1652. As of this issue being fixed in Jest and Node 8.4, this should be fixed in create-react-app but I can't get it to work.
Is this a bug report?
Yes
Can you also reproduce the problem with npm 4.x?
Yes
Which terms did you search for in User Guide?
inspect
Environment

node -v: v8.4.0
npm -v: 4.6.1 and 5.3.0 (confirmed on both versions)
npm ls react-scripts: 1.0.11
Operating system: macOS
Browser and version: Google Chrome Version 60.0.3112.101 (Official Build) (64-bit)

Steps to Reproduce

Open chrome://inspect/#devices in Chromium.
Enable networking and forward localhost:9229.
Set a debugger breakpoint in a test file.
node --inspect-brk node_modules/.bin/react-scripts test
Inspect the target in Chromium.
Resume script execution.
Wait for tests to complete/break.

Expected Behavior
Chromium debugger stops on debugger breakpoint.
Actual Behavior
Chromium debugger doesn't stop on breakpoint and remains at the beginning of the script.

I'm experiencing the same issue running jest with
node --inspect --inspect-brk node_modules/jest/bin/jest.js --watch
node --version
v8.4.0

One of the following changes worked for me:

Using node --inspect-brk node_modules/.bin/jest --watch
Not using Open dedicated DevTools for Node in chrome://inspect.

Where did you find these instructions? Or are you trying something specific that you think may work?
If these are instructions, they're incorrect -- we have yet to document them.
Please link me so I may comment on them and correct them. 😄
Try this instead:
./node_modules/.bin/react-scripts --inspect-brk test -- --runInBand`

@0xcaff Uunning Jest directly (in other projects) works fine for me. I guess it was related to using the different binary path. Also I didn't use Open dedicated DevTools for Node, but that's interesting too.
@Timer I adapted https://github.com/facebook/jest/issues/1652#issuecomment-322834017 for react-scripts thinking it should work (I can't call jest directly because I would be missing CRA support). Unfortunately if I --runInBand, I get an Invalid testPattern error and Babel stops transpiling JSX, which breaks my test suite.

I don't think you need extra -- there. It's only needed when you npm test. Try removing it in above command.

Sorry @nickmccurdy, this should do it:
./node_modules/.bin/react-scripts --inspect-brk test --env=jsdom --runInBand

Thanks, it works! I can remove the leading ./ and --env=jsdom which my suite doesn't rely on. I still need to leave --runInBand enabled, which is odd since Jest itself seems to support breakpoints out of band. This works fine for now though.
Should we close this and/or document @Timer's command in the User Guide?

We need to add a more ergonomic way of running it that doesn't require specifying --runInBand manually and then document it.

Should we open another issue for that?

That issue already exists :); we can close this. Thanks!

I agree with closing this issue, but where can I find the other issue so I can subscribe?

https://github.com/facebookincubator/create-react-app/issues/594 and https://github.com/facebookincubator/create-react-app/pull/2041


Return error instead of returning nil

We've been getting some issues with not receiving error notifications when we hit an "Unknown Offset" error while starting a consumer.
/cc @bsm

Thanks, well spotted


feat(performance): start transaction for echo middleware/integration

Closes #642
Let me know if I should also edit CHANGELOG.md too.

@aldy505 is there any news regarding the implementation? I could really use this

@aldy505 is there any news regarding the implementation? I could really use this

Michi (@cleptric) said to me a few weeks (or month?) ago over Discord that the Go SDK would need to prioritize to convert every performance section to use OpenTelemetry under the hood first before any other changes. That.. would mean this PR is on hold until the performance section is refactored.
Now, for you, would obviously not a good idea to wait for this PR to be merged. But one thing you can do is to copy and paste everything inside the echo directory (exclude the test, it's fine) to your project as your own "sentryecho" package. Then import your own "sentryecho" package instead of this one. I think that is doable on your part.

Since the perf by otel is on halt, here's a gentle ping @cleptric

I wanted to add starfish properties (or otel properties, really) as seen in #786, but I think it's better for this and #723 to be merged first, and I update #786 to include both echo and fasthttp. Or.. it can be a separate PR.

LGTM, thanks.
Any chance you could add a changelog entry?

@vaind in 3-4 hours from now

LGTM, thanks.
Any chance you could add a changelog entry?

@vaind in 3-4 hours from now

I've noticed this repo doesn't have rules to prepare changelog in advance with PRs. Added #799 to fix that, afterwards we can add the entry in this PR.


update packages

:white_check_mark: Build TheIdServer 4.4.1-fix-dependencies1-0048 completed (commit https://github.com/Aguafrommars/TheIdServer/commit/29a4bd132c by @)


Overrides Message Logging

Ticket Link
Ticket
Test Checklist

[X] Exhibits Clear Code Structure:
[X] Project Unit Tests Passed:
[x] Project Jasmine Tests Passed:
[ ] Executes Expected Functionality:
[ ] Tested For Incorrect/Unexpected Inputs:

Is the PR ready?

Yes, sorry, my unit tests stopped working so I had to fix them before I requested for review

Could you port the PropertiesFileOverrideReporter and its test file from the API to the UI? Thanks!


Slow deployment with already pulled images.

When I'm running kargo with already pulled images I expect super fast deployment. Unfortunately kargo does not check if image is present and always call docker pull (https://github.com/kubernetes-incubator/kargo/blob/3ca11b70c4213d6d92a635ca15cfd720115ef5a3/roles/download/tasks/main.yml#L73). With slow network or slow docker registry it takes lot's of time to just get the confirmation that local images are the same as the remote ones, significantly slowing the deployment.

We could rework with docker-py, but that'd add more underlay requirements @mattymo what do you think?

I believe something as simple as code below would work just fine.
`

name: Download | check if containers are present
shell: "/usr/bin/docker images | grep -c {{download.repo}}:{{download.tag}}"
register: image_present
when: "{{ download.enabled|bool and download.container|bool }}"
ignore_errors: true

#NOTE(bogdando) this brings no docker-py deps for nodes

name: Download containers
`

@lleszczu indeed, thank you! Will you submit the PR?

@dkrzyszczyk Do you want to help with upstreaming this?

@lleszczu can you check for the sha instead of image name+tag

What I pasted is just an example. Generally I agree that sha is safer, though right now kargo is using just name+tag for identifying images, so adding sha matching would require adding same map to identify which name+tag matches which sha.

yes, my bad, I created #735.  Matching with image:tag is good.
I would add maybe a variable to keep the current behavior docker pull, quite like kubernetes 'imagePullPolicy: Always' configuration.

@lleszczu sure, I'll do everything that I can

@ant31 we have such a var now, see https://github.com/kubernetes-incubator/kargo/search?utf8=✓&q=k8s_image_pull_policy

This flag is used by kubernetes, I am not sure if this is good idea to reuse it in here. I believe it calls for new config flag.

@bogdando , I agree with @lleszczu , we should use a separated config flag as the usage is different.
like download_always_pull: no

The usage is different, not sure if the use case is. Would one want containers to be always pulled from k8s apps bootstrap and never be pulled for containerized k8s components infra? And vice versa? If yes, then we want a two separate flags.


Add option for italicised modified filename

The name of a modified file in the tab line is italicised. After the addition of #27, some substitute character for modified_icon, like [+], will instead look like [+], which is arguably not as pleasing to look at (unlike the original character, , which is not affected by italicisation).
This PR adds a new modified_italic option changeable through setup:
require('tabline').setup {
    enable = true,
    options = {
        modified_icon = "[+] ",
+       modified_italic = false,
    },
}

Thanks for the PR! I will test this out later.


Avoid edges to overlap nodes

The layout of the nodes by dagre does not prevent edges overlapping nodes see:

generated by:
diagram "image processing" [direction=lr] {
  generic.user user [label="User"];
  aws.appsync appsync [label="AWS AppSync"];
  aws.s3 bucket [label="AWS S3 Bucket"];
  aws.cognito cognito [label="AWS Cognito"];
  aws.lambdaFunction workflow [label="Start workflow"];
  generic.component resolver [label="Resolver"];

  group backend [label="Backend",style=solid,opacity=0,stroke=red,width=4] {
    aws.stepFunctions stepFunction [label="AWS Step Function"];

    aws.lambdaFunction extraction [label="Extract metadata"];
    aws.lambdaFunction detection [label="Object/Scena detection"];
    aws.lambdaFunction thumbnailing [label="Create thumbnail"];
    aws.lambdaFunction persistence [label="Persist metadata"];

    aws.dynamodb dynamo [label="AWS DynamoDB"];
    aws.rekognition rekognition [label="AWS Rekognition"];

    stepFunction -> extraction;
    stepFunction -> detection;
    stepFunction -> thumbnailing;
    stepFunction -> persistence;

    persistence -> dynamo;
    detection -> rekognition;
  }

  user -> cognito;
  user -> appsync;
  user -> bucket;
  bucket -> workflow;
  workflow -> appsync;
  appsync -> resolver;

  resolver -> stepFunction;
  resolver -> dynamo;
}

This has momentarily been solved in 7ca7d363115228d3b58e5ecc0ead1c8ffaf7c181 by allowing nodes to be dragged so that users can finalise the look of the graph before export.
However this functionality should be disable when embedded via iFrame for example, as the result should be reproducible, and as such the overlap problem requires a solution at the layout algorithm level.


MkDocs Website Finished

Summary:
I created the website for Peasy hosted with mkdocs. This website contains the following:

the motivations behind the P Extension
the features of the P extension, with webpages for each feature
a button to download the P extension in VSCode
provides users a way to contribute to the P extension

It uses HTML, CSS, and mkdocs extension Material by MkDocs to create the website.
Testing:
I tested the website by running the command mkdocs serve to host the website on my local computer and see how it looks.

@suesther Looks good. Can you resolve the merge/rebase conflicts? Thanks.


Problem with update to v0.16 on VSCode 1.90

VS Code Version: 1.90
OS Version: MacOS 14.5
Logs: 2024-06-06 19:34:22.292 [error] Error: Chat response provider uses UNKNOWN vendor github.copilot-chat.

Steps to Reproduce: N/A
https://github.com/microsoft/vscode-copilot-release/assets/46487312/910a37ad-4ebc-4d6b-9ec0-567e967593af

@mdragoss What version of VS Code does this happen with? Also, what other extensions do you have installed? Finally, enable trace logging and share the window log please

VSCode Version 1.90 (also added it to note title)
trace] [LM] registering language model chat github.copilot-chat/1/copilot-gpt-3.5-turbo {"extension":{"value":"GitHub.copilot-chat","_lower":"github.copilot-chat"},"id":"copilot-gpt-3.5-turbo","vendor":"github.copilot-chat","name":"gpt-3.5-turbo","family":"","version":"gpt-3.5-turbo","auth":{"providerLabel":"GitHub Copilot Chat","accountLabel":"mdragoss"}}
2024-06-10 17:01:03.735 [error] Chat response provider uses UNKNOWN vendor github.copilot-chat.: Error: Chat response provider uses UNKNOWN vendor github.copilot-chat.
    at i.registerLanguageModelChat (vscode-file://vscode-app/Applications/Visual%20Studio%20Code.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:1639:20612)
    at i.$registerLanguageModelProvider (vscode-file://vscode-app/Applications/Visual%20Studio%20Code.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:1651:1412)
    at y.S (vscode-file://vscode-app/Applications/Visual%20Studio%20Code.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:1658:14598)
    at y.Q (vscode-file://vscode-app/Applications/Visual%20Studio%20Code.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:1658:14364)
    at y.M (vscode-file://vscode-app/Applications/Visual%20Studio%20Code.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:1658:13397)
    at y.L (vscode-file://vscode-app/Applications/Visual%20Studio%20Code.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:1658:12232)
    at u.value (vscode-file://vscode-app/Applications/Visual%20Studio%20Code.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:1658:10928)
    at h.y (vscode-file://vscode-app/Applications/Visual%20Studio%20Code.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:89:659)
    at h.fire (vscode-file://vscode-app/Applications/Visual%20Studio%20Code.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:89:876)
    at r.fire (vscode-file://vscode-app/Applications/Visual%20Studio%20Code.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:615:14848)
    at U.onmessage (vscode-file://vscode-app/Applications/Visual%20Studio%20Code.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:1673:8154)
2024-06-10 17:01:03.849 [trace] [LM] registering language model chat github.copilot-chat/2/copilot-gpt-4 {"extension":{"value":"GitHub.copilot-chat","_lower":"github.copilot-chat"},"id":"copilot-gpt-4","vendor":"github.copilot-chat","name":"gpt-4","family":"","version":"gpt-4","auth":{"providerLabel":"GitHub Copilot Chat","accountLabel":"mdragoss"}}
2024-06-10 17:01:03.903 [trace] CommandService#executeCommand _setContext
2024-06-10 17:01:04.631 [trace] SearchService#search {"_reason":"startFileSearch","folderQueries":[{"folder":{"$mid":1,"fsPath":"/test","external":"file:///test","path":"/test","scheme":"file"},"excludePattern":{"**/.git":true,"**/.svn":true,"**/.hg":true,"**/CVS":true,"**/.DS_Store":true,"**/Thumbs.db":true,"**/__pycache__":true,"**/.cache":true,"**/.coverage":true,"**/.coverage.*":true,"**/.hypothesis":true,"**/.mypy_cache":true,"**/.nox":true,"**/.pytest_cache":true,"**/.ruff_cache":true,"**/.tox":true},"fileEncoding":"utf8","disregardIgnoreFiles":true,"disregardGlobalIgnoreFiles":true,"disregardParentIgnoreFiles":true,"ignoreSymlinks":false}],"usingSearchPaths":false,"excludePattern":{"**/node_modules/**":true},"includePattern":{"**/package.json":true},"type":1,"shouldGlobMatchFilePattern":true}
2024-06-10 17:01:04.666 [trace] SearchService#search: 34ms
2024-06-10 17:01:04.762 [trace] MainThreadFileSystemEventService#$watch(): request to start watching uncorrelated (extension: ms-python.python, path: file:///Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12, recursive: true, session: 0.06683906689520192)
2024-06-10 17:01:06.245 [trace] CommandService#executeCommand _setContext
2024-06-10 17:01:06.337 [trace] MainThreadFileSystemEventService#$unwatch(): request to stop watching (session: 0.369363579711975)
2024-06-10 17:01:06.338 [trace] MainThreadFileSystemEventService#$watch(): request to start watching uncorrelated (extension: vscode.git, path: file:///test/.git, recursive: false, session: 0.37168217987496277)
2024-06-10 17:01:06.339 [trace] CommandService#executeCommand _setContext
2024-06-10 17:01:06.340 [trace] CommandService#executeCommand _setContext
2024-06-10 17:01:07.162 [debug] User data changed
2024-06-10 17:01:11.629 [trace] [History stack global-default]: notifyNavigation() (editor: extension:github.copilot-chat/extension, event: <none>)
2024-06-10 17:01:11.629 [trace] [History stack global-default]: notifyNavigation() not ignoring (editor: extension:github.copilot-chat/extension, event: <none>)
2024-06-10 17:01:11.629 [trace] [History stack global-default]: onNonSelectionAwareEditorNavigation() (editor: extension:github.copilot-chat/extension, event: <none>)
2024-06-10 17:01:11.630 [trace] [History stack global-default]: add() (editor: extension:github.copilot-chat/extension, event: <none>)
2024-06-10 17:01:11.630 [trace] [History stack global-default]: index: 0, navigating: false
- group: 0, editor: extension:github.copilot-chat/extension, selection: <none>

2024-06-10 17:01:11.679 [debug] Webview(9671b8d1-4594-4e91-906f-da88ca9513f7): will update content
2024-06-10 17:01:11.804 [debug] Webview(9671b8d1-4594-4e91-906f-da88ca9513f7): webview ready
2024-06-10 17:01:12.609 [trace] RunAutomaticTasks: waited some extra time, but no update of tasks configuration
2024-06-10 17:01:14.504 [debug] User data changed
2024-06-10 17:01:14.763 [trace] CommandService#executeCommand workbench.action.toggleMaximizedPanel
2024-06-10 17:01:16.925 [trace] CommandService#executeCommand workbench.extensions.action.showExtensionsWithIds
2024-06-10 17:01:16.928 [trace] writeTextAreaState(reason: focusgain)
2024-06-10 17:01:16.936 [trace] writeTextAreaState(reason: selection changed)
2024-06-10 17:01:16.944 [trace] writeTextAreaState(reason: focusgain)
2024-06-10 17:01:27.230 [trace] writeTextAreaState(reason: focusgain)
2024-06-10 17:01:27.235 [trace] writeTextAreaState(reason: selection changed)
2024-06-10 17:01:27.246 [trace] CommandService#executeCommand _setContext
2024-06-10 17:01:27.426 [trace] writeTextAreaState(reason: selection changed)
2024-06-10 17:01:27.458 [trace] writeTextAreaState(reason: selection changed)
2024-06-10 17:01:27.475 [trace] writeTextAreaState(reason: selection changed)
2024-06-10 17:01:27.491 [trace] writeTextAreaState(reason: selection changed)
2024-06-10 17:01:27.508 [trace] writeTextAreaState(reason: selection changed)

This is really confusing and unexpected. The copilot chat extension uses the copilot vender, this is hardcoded and I see working correctly locally. Also, the data you are receiving has no family property which I have no explanation for.

Can you please try with a new empty profile in which you install the extension fresh

I enabled some extensions for all profiles, but switching profiles updated them successfully. Thanks for the help!


Added keyboard shortcuts.

Hope it'll help.

[x] I have read freeCodeCamp's contribution guidelines.
[x] My pull request has a descriptive title (not a vague title like Update index.md)
[x] My pull request targets the master branch of freeCodeCamp.
[x] None of my changes are plagiarized from another source without proper attribution.
[x] My article does not contain shortened URLs or affiliate links.

If your pull request closes a GitHub issue, replace the XXXXX below with the issue number.
Closes #XXXXX

Hi @Ralphie99, thank you for opening this pull request.
This is a standard message notifying you that we’ve reviewed your pull request and have decided not to merge it. We would welcome future pull requests from you.
Thank you and happy coding.


MeanTargetEncoder

Can you please give more clarification on how  mean target encoder works
I have tried the below code:
"lost" is my target variable
from verstack import MeanTargetEncoder
mean_target_encoder = `MeanTargetEncoder(save_inverse_transform` = True)
X_train= mean_target_encoder.fit_transform(X_train, 'health_center', 'lost')
X_test = mean_target_encoder.transform(X_test)

but keep getting the below error:
AssertionError                            Traceback (most recent call last)
File ~\anaconda3\lib\site-packages\verstack\categoric_encoders\args_validators.py:43, in assert_fit_transform_args(df, colname, targetname)
42 try:
---> 43     assert(targetname in df)
44 except:
AssertionError:
During handling of the above exception, another exception occurred:
KeyError                                  Traceback (most recent call last)
Input In [458], in <cell line: 3>()
      1 from verstack import MeanTargetEncoder
      2 mean_target_encoder = MeanTargetEncoder(save_inverse_transform = True)
----> 3 X_train= mean_target_encoder.fit_transform(X_train, 'health_center', 'lost')
      4 X_test = mean_target_encoder.transform(X_test)

File ~\anaconda3\lib\site-packages\verstack\categoric_encoders\MeanTargetEncoder.py:79, in MeanTargetEncoder.fit_transform(self, df, colname, targetname)
     61 def fit_transform(self, df, colname, targetname):
     62     '''
     63     Fit encoder, transform column in df, save attributes for transform(/inverse_transform().
     64
   (...)
     77         Data with the column transformed.
     78     '''
---> 79     assert_fit_transform_args(df, colname, targetname)
     80     from sklearn.model_selection import KFold
     82     self._colname = colname

File ~\anaconda3\lib\site-packages\verstack\categoric_encoders\args_validators.py:45, in assert_fit_transform_args(df, colname, targetname)
     43     assert(targetname in df)
     44 except:
---> 45     raise KeyError('"targetname" must a valid column name in df')

KeyError: '"targetname" must a valid column name in df'

Yeah, the source of the error is quite simple. MeanTargetEncoder at fit_transform requires the target column to be a part of the dataframe.
So just concatenate your X_train with y_train and run it through MeanTargetEncoder.
So the complete solution would be:
df = pd.concat([X_train, y_train])
mean_target_encoder = MeanTargetEncoder(save_inverse_transform = True)
df_encoded = mean_target_encoder.fit_transform(df, colname = 'health_center', target_name = 'lost')

X_train = df_encoded.drop('lost', axis = 1)
y_train = df_encoded['lost']

X_test = mean_target_encoder.transform(X_test)

Just noticed that after concatenating  X_train and y_train using  df = pd.concat([X_train, y_train]), all values in column y_train are NaN

sorry, the argument must be targetname instead of target_name. I have corrected the original code above.
And regarding the NaN, make sure the indexes are the same in X_train and y_train. You may want to reset their indexes before concatenation:
X_train.reset_index(drop=True, inplace=True)
y_train.reset_index(drop=True, inplace=True)

That is done through the .transform() method. If your test set is available right away, you can do mean_target_encoder.transform(test). If your test set will become available later, you can pickle the mean_target_encoder object to your disk, it will contain all the learned categoric-to-mean-target mapping inside it's class variables. And when your test set arrives, load it back into python and use mean_target_encoder.transform(test).
# after mean_target_encoder.fit_transform(df, colname='health_center', targetname='lost')
import pickle
with open('path_to_save_the_trained_mean_target_encoder', 'wb') as f:
    pickle.dump(mean_target_encoder, f)
# ------------------------------------------------------------------------
# when the test set becomes available
import pickle
with open('path_to_saved_mean_target_encoder', 'rb') as f:
    mean_target_encoder = pickle.load(f)
test_encoded = mean_target_encoder.transform(test)

Thank you very much. I appreciate your assistance

Thanks for the detailed tutorial.
I tried to impute missing values using NaNimputer but it is not imputing all missing values, what may be the cause
Train set NaNs num: 1488
val set NaNs num : 139
Test set NaNs num: 171
df1 set NaNs num : 14445
NaNImputer(conservative = False, n_feats = 10,
           fix_string_nans = True, verbose = True,
           multiprocessing_load = 3, fill_nans_in_pure_text = True,
           drop_empty_cols = True, drop_nan_cols_with_constant = True
           feature_selection = correlation)

Dataset dimensions:
 - rows:         85050
 - columns:      28
 - mb in memory: 18.49
 - NaN cols num: 6
--------------------------

Deploy multiprocessing with 4 parallel proceses

NaNs imputation time: 15.97 minutes
--------------------------------------------------
NaNImputer(conservative = False, n_feats = 10,
           fix_string_nans = True, verbose = True,
           multiprocessing_load = 3, fill_nans_in_pure_text = True,
           drop_empty_cols = True, drop_nan_cols_with_constant = True
           feature_selection = correlation)

Dataset dimensions:
 - rows:         9450
 - columns:      28
 - mb in memory: 2.05
 - NaN cols num: 6
--------------------------

Deploy multiprocessing with 4 parallel proceses

NaNs imputation time: 9.81 minutes
--------------------------------------------------
NaNImputer(conservative = False, n_feats = 10,
           fix_string_nans = True, verbose = True,
           multiprocessing_load = 3, fill_nans_in_pure_text = True,
           drop_empty_cols = True, drop_nan_cols_with_constant = True
           feature_selection = correlation)

Dataset dimensions:
 - rows:         10500
 - columns:      28
 - mb in memory: 2.28
 - NaN cols num: 6
--------------------------

Deploy multiprocessing with 4 parallel proceses

NaNs imputation time: 1.18 minutes
--------------------------------------------------
NaNImputer(conservative = False, n_feats = 10,
           fix_string_nans = True, verbose = True,
           multiprocessing_load = 3, fill_nans_in_pure_text = True,
           drop_empty_cols = True, drop_nan_cols_with_constant = True
           feature_selection = correlation)

Dataset dimensions:
 - rows:         45000
 - columns:      28
 - mb in memory: 9.44
 - NaN cols num: 8
--------------------------

Deploy multiprocessing with 4 parallel proceses

NaNs imputation time: 2.57 minutes
--------------------------------------------------
Train set NaNs num: 582
val set NaNs num : 54
Test set NaNs num: 69
df1 set NaNs num : 324
CPU times: total: 7.77 s
Wall time: 29min 33s

Could you please provide some more details.
The log you've provided - is this the truncated console print from a few different attempts of NaNImputer application on different datasets? The reason I am asking is that this seems to be not the full log that is produced upon each .impute() operation.
Please provide the code you are using starting from the data definition, NaNImputer initialisation and usage.

%%time
# IMPUTE NANs by XGBoost
print(f'Train set NaNs num: {sum(X_train.isnull().sum())}')
print(f'val set NaNs num : {sum(X_val.isnull().sum())}')
print(f'Test set NaNs num: {sum(X_test.isnull().sum())}')
print(f'df1 set NaNs num : {sum(df1.isnull().sum())}')

imputer = NaNImputer()
train = imputer.impute(X_train)
imputer = NaNImputer()
val = imputer.impute(X_val)
imputer = NaNImputer()
test = imputer.impute(X_test)
imputer = NaNImputer()
df1 = imputer.impute(df1)
print(f'Train set NaNs num: {sum(X_train.isnull().sum())}')
print(f'val set NaNs num : {sum(X_val.isnull().sum())}')
print(f'Test set NaNs num: {sum(X_test.isnull().sum())}')
print(f'df1 set NaNs num : {sum(df1.isnull().sum())}')

shared the data on linkedin

Okay, I think I see what the problem might be...
After imputation when you print the missing values, you are using the original X_train/X_val/X_test instead of the imputed train/val/test

just checked, that's not the issue because there are others columns where missing values were imputed correctly ,only few columns were not, which somehow weird.
Maybe I can share my notebook if its ok with you

Okay, the problem was internal. Updated version uploaded to pypi.
pip install --upgrade verstack
You should get version 3.3.4
By the end of the weekend I will upload the entirely new&reworked version of NaNImputer that works much faster. I have been testing it for quite some time now and looks like it is ready for open-source.

Good , I love all Verstack tools ,even if some of them seem a bite complicated for me as a newbie data scientist but keep improving everyday ,thanks to you.

I have just rolled out the new version of NaNImputer that is much faster and more reliable. The previous version has been kept since it has more configurability under the name NaNImputerLegacy. It is all reflected in the documentation.

I have pickled the stacker but getting the below error
# when the test set becomes available
import pickle
with open('path_to_save_stacker_tra.sav', 'rb') as f:
    stacker = pickle.load(f)
X_test = stacker.transform(X_test)

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-100-5273fd77575f> in <module>
      3 with open('path_to_save_the_trained_tra.sav', 'rb') as f:
      4     stacker = pickle.load(f)
----> 5 X_test = stacker.transform(X_test)

AttributeError: 'str' object has no attribute 'transform'

yet the X_test has only numerical values
<class 'pandas.core.frame.DataFrame'>
Int64Index: 132 entries, 418 to 254
Data columns (total 18 columns):
 #   Column                         Non-Null Count  Dtype
---  ------                         --------------  -----
 0   health_center                  132 non-null    float64
 1   enrollment_year                132 non-null    float64
 2   enrollment_quarter             132 non-null    float64
 3   enrollment_day                 132 non-null    float64
 4   age                            132 non-null    float64
 5   number_of_programs             132 non-null    float64
 6   days_taken_to_record_the_data  132 non-null    float64
 7   hospital_district_kayonza      132 non-null    float64
 8   hospital_district_kirehe       132 non-null    float64
 9   enrollment_week_day_Monday     132 non-null    float64
 10  enrollment_week_day_Saturday   132 non-null    float64
 11  enrollment_week_day_Sunday     132 non-null    float64
 12  enrollment_week_day_Thursday   132 non-null    float64
 13  enrollment_week_day_Tuesday    132 non-null    float64
 14  enrollment_week_day_Wednesday  132 non-null    float64
 15  gender_M                       132 non-null    float64
 16  patient_district_kayonza       132 non-null    float64
 17  patient_district_kirehe        132 non-null    float64
dtypes: float64(18)
memory usage: 23.7 KB

Yeah. Since Stacker includes tf/keras models, they cannot be just pickled, they have their own methods for saving fitted neural networks. So Stacker, that holds tf/keras models cannot be picked.
In this regard Stacker has its own save/load method. Please refer to the article closing remarks. There is an explanation and code example to save / load fitted Stacker instances: https://medium.com/@danilzherebtsov/meet-stacker-automated-ensemble-creation-library-abcad0648a5d


[Translation it] VR

Italian translation of VR.
Please review by @carlaofarrell
@sotayamashita & @chikathreesix  please analyze the CLA tag
Check out a review request at GitLocalize

CLA: no might be a reason because of mt-gitlocalize account but you do not have to worry

Integrated @carlaofarrell 's review and updated wf_updated_on.

Up

:tada: This has been pushed live to https://developers.google.com/web/


The support for SB 2 is hopeless

I think we should stop buying this product as the support does not exist and so does the documentation. These guys are may be misleading every one.  I am preparing some legal action on the company and also on trade malpractices.

Urm, you know its Free right, and the Pro version isn't on GitHub...

您好，您的邮件我已经收到，我会尽快回复！  谢谢！

I am a licensed own of the pro version and i am talking of the pro version!

I am a licensed own of the pro version and i am talking of the pro version!

Well if you look in the README.md file for the pro version, you'll see that you need to raise an issue on https://github.com/StartBootstrap/pro-feedback/issues

您好，您的邮件我已经收到，我会尽快回复！  谢谢！

 Is this for Xi JIn Ping?

Decided to throw it away. I advised all other users in our ASEAN forum to do the same. You proved me right. My last post was on Nov 8, 2022 and you replied today, Jan 8, 2023, 2 months later!

To be clear, I am not part of the StartBootstrap team.  Is this for Xi JIn Ping?

I had to Google Translate it, it's their email auto-responder... GitHub is sending them an email that is has been updated, and it's sending back that. It basically says 'I've received your email'

@methecooldude
I am so sorry i thought you were part of the team. I got very frustrated with developers who dont response. They are just for the money but forgot quality service.
Please accept my sincere apology for the misunderstanding.
With regards to Chinese message, should kick them out if they cannot communicate in English especially when they respond to our posts. For all you know the person could be a "wolf-worrier" from the CCP.

@methecooldude
I am so sorry i thought you were part of the team. I got very frustrated with the developers who dont response. They are just for the money but forgot quality service.
Please accept my sincere apology for the misunderstanding.
With regards to Chinese message, should kick them out if they cannot communicate in English especially when they respond to our posts. For all you know the person could be a "wolf-worrier" from the CCP. I may be paranoid but you do not what the CCP bastards have been doing ever since the Great Rift!

It's all good man, don't worry about it :)


Make a ddp broadcast clone that handles main-process-only

Checkpointing with DDP uses a torch.distributed broadcast to communicate the folder for saving to all procs. However, this broadcast causes a freeze if checkpointing is called in a main-process-only section of the code. This PR addresses this by adding a ddp_broadcast function that should correctly handle the main-process-only limitation.

See issue #2401 for more info

Hi @pplantinga , I am the author of the issue. Could you explain the reason of the broadcasting during checkpointing? I mean, it's unclear to me why do we have to broadcast ckpt_directory.

Hi @pplantinga , I am the author of the issue. Could you explain the reason of the broadcasting during checkpointing? I mean, it's unclear to me why do we have to broadcast ckpt_directory.

Of course, I'm happy to explain. Basically there's some cases where checkpointing needs to be run on multiple processes, for example with FSDP there are shards of the model on each device that need to be combined before saving. In these cases, every process needs to know where the checkpointing directory will be. Given our current setup where the default checkpointing directory name is based on the time, this can't be determined ahead of time, so we must rely on inter-process communication.
This same strategy is followed by pytorch_lightning if i remember correctly

Looking for the broadcast in pytorch lightning, it looks like its done in a special fsdp class, not sure if there are other places where this is done:
https://github.com/Lightning-AI/pytorch-lightning/blob/47c8f4cba089a78fa3fe31dcac6a43416bc13820/src/lightning/fabric/strategies/fsdp.py#L447

Thank you @pplantinga. I contacted @Adel-Moumen privately and he confirmed that it is fine with that. We can merge it


Adding a spec for

See #372

Looks good but make sure to name it portal.md and not portal-spec.md


bytes 32 input parameter

https://github.com/hashgraph/hedera-sdk-js/blob/master/src/contract/ContractFunctionParams.ts
Please adjust function for bytes32 as it is incorrectly mapping as a Dynamic parameter:
addBytes32(value) {
    if (value.length !== 32) {
        throw new Error(`addBytes32 expected array to be of length 32, but received ${value.length}`);
    }
    this._selector.addBytes32();
    return this._addParam(value, false);
}

Thank your mentioning this. This should now be fixed on master https://github.com/hashgraph/hedera-sdk-js/commit/5a599683d440b0d5a0300f1a0ce5079305097530


Remove deprecated Credit parameters

From #6005
Remove the text, imageUrl and link parameters in Credit

is there a way to contribute with this project?

Hi @Jahrkor!  Thanks for your interest in contributing!
Yes, there are lots of ways to contribute. See our contributing documentation
Beginner issues are always a great place to start.
If you want to take a stab at this issue, we want to remove the deprecated parameters from Credit  before the next release on January 2nd.  We want to remove the imageUrl and link parameters and keep the options parameter instead.


s390x: Remove uses of copy_reg

After the switch to providing SSA input to RA2, these uses of copy_reg should no longer be necessary. Additionally, as RA2-0.7.0 has removed support for program moves, these moves will no longer be optimized away.
@uweigand, could you take a look as well? These uses of copy_reg all looked like they'd be fine to remove, but I could be missing something.

@elliottt I think these copy_reg uses are all remnants from back when register pairs were using non-allocatable registers (%r0 / %r1), and it wasn't safe to use those in anything but a copy_reg.   But this has long since been changed, and regpairs now are just a pair of normal SSA registers, so I agree these copy_regs can all go away.  Thanks!


[WIP] fix: require a gravity user ID for submissions when contact information is not provided

This PR addresses an issue where admins were able to manually create submissions without a user in the admin UI. This caused the submission list page (/submissions) to crash when it tried to render submissions with neither a gravity_user_id nor any contact information (name, email, or phone).
This was resolved by re-introducing validation for the presence of a gravity_user_id, on the condition that it is only required when contact information is not provided.

This PR is a WIP because I am still able to create a manual submission without a user in the admin screens. The submissions created in this way are no longer crashing the submission list page (/submissions) but I expected the form to fail validation. Instead, a user with the following contact information was assigned to the submission: user<EMAIL_ADDRESS>555-5555.


[v1.11] Reinitialize IPsec on device changes

A custom fix for v1.11 to reload IPsec when the network devices change. This is required when IPsec is enabled
and used with IPAM ENI mode where new network devices may be added at runtime.
Since v1.11 does not contain the "runtime-device-detection" feature, this fixes the issue by subscribing to
link updates directly in loader.

/test-backport-1.11

This fix was tested to be working on a EKS cluster with IPsec and ENI mode. The new ensX interface added at runtime was verified to have the bpf_network program loaded onto it.

Closing in favor of backporting to v1.11 from https://github.com/cilium/cilium/pull/25744


Support script with arguments

E.g. this is how I use it:
analysis-runner --dataset tob-wgs --access-level test --output-dir "gs://cpg-tob-wgs-temporary/joint-calling" --description "joint calling" "scripts/batch_workflow.py --callset tob-wgs --version v0 --qc-csv-fname R_210315_BINKAN1_1K1KDNA_M001.csv --keep-scratch --billing-project tob-wgs"

That's odd, I thought this is already working -- at least I've used arguments before.
@illusional Doesn't click capture all arguments as a list?
Doesn't it just work if you don't use quotes, like this:
analysis-runner --dataset tob-wgs --access-level test --output-dir "gs://cpg-tob-wgs-temporary/joint-calling" --description "joint calling" scripts/batch_workflow.py --callset tob-wgs --version v0 --qc-csv-fname R_210315_BINKAN1_1K1KDNA_M001.csv --keep-scratch --billing-project tob-wgs

Yep, for nargs=-1, click captures all arguments as a list: https://github.com/populationgenomics/analysis-runner/blob/6ef1de0544af015b69f7dc74a17ea7648dbbf857/analysis_runner/cli.py#L72

I think for me Click tries to interpret all options first, before building the nargs=-1 list. So it attempts to parse options to your script like they were options to the analysis-runner cli:

Usage: analysis-runner [OPTIONS] [SCRIPT]...
Try 'analysis-runner --help' for help.

Error: no such option: --callset

It works though if your script only has positional arguments, but not if there are options that start with dash.
Did it work with options for you too? It's click==7.1.2 in my environment.


Build on a ARM host

[This is a question]Dear zephyr community, hope you can help me on this. Stuck here for a while and any suggestions will be appreciated. Thanks a lot.
Describe the bug
I'm trying to setup zephyr build environment on raspberry pi, so the host will be raspberry pi and the target will be stm32f4 MCU. It seems that the SDK binaries provided by zephyr is for host x86-64 only. Is there any other way to compile zephyr for cortex-m mcu on an ARM host with/without zephyr SDK?
To Reproduce
Steps to reproduce the behavior:

west build -b reel_board samples/hello_world
cmake report error when trying to run i586-zephyr-elf-gcc because it's x86-64 binary

Expected behavior
Successfully built hello_world on pi.
Impact
Screenshots or console output
Environment (please complete the following information):

OS: (Linux on PI)
Toolchain (Zephyr SDK)

Additional context

If there's arm embedded build of GCC with the distro you are running on the RPI than you could try the instructions for using a generic toolchain here:
https://docs.zephyrproject.org/latest/getting_started/toolchain_other_x_compilers.html

@galak  Thanks, I will try that

@codingspirit Were you able to build zephyr on an RPi? If yes, which environment variables did you need to set to which values? I'm trying ZEPHYR_TOOLCHAIN_VARIANT="cross-compile" and CROSS_COMPILE="/usr/bin", but I get the error that Zephyr was unable to find the toolchain, showing those two values.

Have you solved the problem?
I had the same issue.

I just installed arm-none-eabi-gcc on the RPi by following the instructions on https://github.com/vanbwodonk/gcc-arm-embedded-build-armhf (tested on commit<PHONE_NUMBER>e81ac9ab205ffb1d82db55f32d50fe). With that toolchain, compilation works perfectly normal and fine.


Create initial version of module

@rich-ehrhardt
we have created terraform module  "open vpn server"  by creating  azure linux vm with "open vpn ssl server" market place image. But right now we have not re-used existing vm module as suggested by you the reason being is in our existing linux vm module, we need to set one extra parameter "plan" as below, which is right now not part of our existing vm module.
plan {
name      = var.openvpn_image_sku
product   = var.openvpn_image_offer
publisher = var.openvpn_image_publisher
}
If we want re-using existing vm module then we need to update this module to accommodate above plan parameter.
this module test case is passed. Please verify this approach and let us know your thoughts.

@rich-ehrhardt
We have implemented below changes in this module:

Existing vm module reuse to create open-vpn-server module. (ubuntu 18.x/20.x supported)
boot strap script(custom data)  re-written to install open-vpn server and configure into vm.
script file created to download client.ovpn file and check vpn connection through cmd line open vpn client.
Integrated this script file to execute post module creation.(local-exec)

Please review.


Namer changed metadata URL

Hi,
When I started using Namer ( https://github.com/ThePornDatabase/namer ) it inserted data into .nfo files that were combined by Jellyfin / this plugin to a valid URL. e.g. in the field referencing a scene in metadataapi there was just the ID of the scene. So the plugin could fetch missing or updated metadata.
A new version of namer now prefixes the id (it's even a different id) with scene/ or movie/. Thus the resulting URL is broken and Jellyfin (more exactly this plugin) can't find the corresponding entry in metadataapi.net .
Could you add an if to either use the old URL scheme or use the new one with scene/ / movie/?

maybe swap to this id format, need to think about it

Thanks!
I can imagine, lots of libraries still use the old one in the metadata. So if you switch, maybe it would be good to have some sort of automatic transition. Like, if scene or movie is missing, fetch the data and rewrite the metadata.

in dev build


#1777, #1034 Enhanced SpecificDeserializerImpl to use actual type of data class

... when creating new instance of that data class for deserialization.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.Serge Matsevilo seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.


Missing file: all_loci_score_matrix.txt

Hi, I've been working through the training notebook and encountered an issue during the training section 4.2. It seems that the "all_loci_score_matrix.txt" file is not available in either the provided folder or on the Zenodo page (https://zenodo.org/records/8095914). Could you please share this text file? Your assistance would be greatly appreciated.
Thank you!

Hi, thanks for your interest in running our code and analyses! You're right, I hadn't uploaded that file yet to Zenodo. I now provided an updated version of the record with the all_loci_score_matrix.txt included! Here it is: https://zenodo.org/records/11061100. Just an FYI: you only need this file when you want to recreate our exact evaluation on our exact datasets. If you just care about inference or training a model on your own data, this file is not of use :).
If you have any more questions, feel free to ask.
Dimi

Many thanks, problem solved!


Use postgres db for S3 Integration local development

Is your feature request related to a problem? Please describe.
The S3 Integration (service and example application) use h2 database locally.
Describe the solution you'd like
We should use the same database vendor (postgres) in all environments and therefore use locally a postgres db too.
Additional context

Add a new postgres db container to the docker compose stack
Configure S3 Database Environments to use the local postgres db
Remove h2 configurations from application.ymls (and remove application-local.yml from the s3-integration-service).
For the example application we may also need to setup flyway

How should we handle multiple (identical) database for integration example and service applications in local development?
I can think of these scenarios:

example and service use the same postgres database
example and service have seperate postgres databases (2 postgres containers)
example uses h2 and the service uses a postgresql database


dworker does not work on Windows 7

dworker does not work on Windows 7 using Python 2.7 (no other versions tested).
The process created by the nanny using multiprocessing.Process fails to execute due to an issue with how multiprocess handles forking in Windows.  The error message is "ImportError: No module named dworker".  I don't know the best way to address this.  A workaround is to copy "dworker" to a filename that ends in ".py"--let's say "run_dworker.py"--and then run "python run_dworker.py".
Also, the dworker process just hung.

I think I may have found a solution.  Simply change the name of "dcenter" to "dcenter.py" and update the name in the scripts field of setup.py.  In Windows, this installs both "dcenter" and "dcenter.py" into the scripts bin directory and the dcenter command works as expected.

Hrm, we could presumably handle this logic within setup.py.  Do other projects have this issue?

I believe that this is now resolved in master.  Closing.  Please re-raise if the issue persists.


Compress symlinks?

Is there a way to compress the symlink as-is instead of copying the content of where it's originally pointing to?

+1

+1


Internal network exception.

Hi, I'm trying send 1 bot to server and bot connect, but bot instantly disconnect. Disconnect reason is "Internal network exception.",
My logs: https://pastebin.com/Mpzmuhkx

I wanna add when I connect (bot forwards packets to me, and I to him) = same situation

java.lang.AbstractMethodError there indicates that something is wrong with the netty inbound handler... if you haven't done anything with netty then you should check what version MCProtocolLib is using and what version you are providing on runtime.

fixed thanks.


Allow dateutil below 3.0.0

The dateutil dependency was tightened in #1519, in order to match botocore. However, it looks like botocore is working on loosening that dependency for Python 2.7 and above: https://github.com/boto/botocore/pull/1433. Therefore it makes sense that this project should also loosen its dependencies, and since we don't support Python 2.6 or below, we can do so more simply.

Coverage decreased (-0.08%) to 92.651% when pulling 04b36b4488b19e516ea6e06525e385a358be7727 on singingwolfboy:latest-dateutil into 11e3fa3200b0ce3299975af119ba746c874a5de0 on spulec:master.

Looks good; thanks!

Did this change get reverted?
https://github.com/spulec/moto/blob/1.3.3/setup.py#L21 looks like it's still requiring <2.7.0.

Correction, can we get a release with this change in it? It looks like it didn't make it out with 1.3.3.

@spulec Thanks -- do you anticipate another release to bring this in?

Yes, we'll try to get a release out in the next few days.

The commit message was a little premature of me, issue hasn't been fixed yet

Disappointing this missed the 1.3.4 release.


updateed profileCard.js

Assignee Tasks

[ ] added unit tests (or none needed)
[ ] written relevant docs (or none needed)
[ ] referenced any relevant issues (or none exist)

Guidelines
Please add a description of this Pull Request's motivation, scope, outstanding issues or potential alternatives, reasoning behind the current solution, and any other relevant information for posterity.

Your PR Notes Here

closing until useState handled correctly


Trying to start dev

I wanted to give it a try but can't set it up correctly..
I've got
Couldn't prefetch kirbySite KQL query:  (404 Not Found (http://blablabla:8888/api/kql))
when trying to connect to http://localhost:3000
I set up my .env file like this  :
NUXT_PUBLIC_SITE_URL=http://blablabla:8888/
KIRBY_BASE_URL=http://blablabla:8888/
KIRBY_API_TOKEN=6883351828
what did I miss ?

Unfortunately, that's too little information. You haven't set up the Kirby Headless Starter correctly, I guess. Please read into #1 and then come back if your issue persists. Thanks.

Hi there!
First of all, thanks for the starterkit(s), I am new to Nuxt but it all looks promising and I’m really looking forward to use it at its best. I'm writing here because it looks like I’m having a similar issue to some other people.
I tried to follow all the recommendations:

Clone both kirby-nuxt-starter and kirby-headless-starter
Make sure .env files are looking good on both sides (nuxt and headless)
Use Valet to serve Kirby locally, configured to use HTTPS
Fetch data successfully with Insomnia via basicAuth and Bearer token.

After the setup, I tried for two days to fetch data from my https://headless-nuxt.test without success.
I did a lot of tests with useKql(), useFetch(), fetch(), I even tried the IPv6 trick mentioned in the #1, but no luck.

Then I did this test, where I run in parallel useKql() and useFetch(), using the same params:

<template>

BaseURL: {{ KIRBY_BASE_URL }}

With useKql():
{{ resultWithKql }}

With useFetch():
{{ resultWithFetch }}

</template>

✅ Case 1: if I set KIRBY_BASE_URL to your https://kirby-headless-starter.jhnn.dev, both methods work:

❌ Case 2: if I set KIRBY_BASE_URL to my https://headless-nuxt.test, only useFetch() method is successful:

Do you have an clue of why would that happen?
Thanks in advance!

@Gianlucamonaco Thanks for the thorough investigation. It may be related to usage of HTTPS. Under the hood, Nuxt KQL uses useAsyncData alongside $fetch, which is the same Nuxt uses to compose useFetch.
Unfortunately, I'm not sure how to debug this, since I don't have access to the same setup you use. Personally, I use Valet as well. Just not in HTTP context for the Kirby backend. Can you try disabling HTTPS on your local machine?
I'm not sure your issue is related to Nuxt KQL. Could you provide a minimum reproduction?

I've bumped into the same problem unfortunally. On my Imac it works as expected. On my macbook the useKql gives the same error. However with valet it does seem to work :) I'll try to look into the different configurations and will come back to it if I've found it.

Closing this, as it is not an issue of KQL or Kirby itself, but rather how Nuxt and HTTPS has to be set up.
If you need further help, feel free to book some consulting from me. 🙂


Update X (Twitter) fact sheet

Remove dead link to query builder
Update API information
Add link to "navigating X/Twitter" article

Link to the corresponding issue for this PR: https://github.com/QUT-Digital-Observatory/australian_digital_observatory_site/issues/60


markereditable is not working after closing continuedrawing

this is the code
this.map.pm.enableDraw('Marker', {
continueDrawing: false,
})
I set options like this, expecting to draw one point at a time, and being editable at once.
But when I set continueDrawing 'false', the marker cannot be editable
why?

Hi Jamie,
I cannot reproduce this error that you're getting. Here's what I tried:
//
var map = L.map('map', {}).setView([40, -80, -0.09], 13)

map.pm.addControls({
 position: 'topleft',
 drawControls: true,
 editControls: true,
 optionsControls: false,
 drawMarker: false,
 customControls: true,
 oneBlock: false,
});

var OpenStreetMap_HOT = new L.tileLayer('https://{s}.tile.openstreetmap.fr/hot/{z}/{x}/{y}.png', {
  maxZoom: 19,
  attribution: '© OpenStreetMap contributors, Tiles style by Humanitarian OpenStreetMap Team hosted by OpenStreetMap France',
});
map.addLayer(OpenStreetMap_HOT)

// set enableDraw right when you load the map.
map.pm.enableDraw('Marker',{
  continueDrawing: false
})

Then: I added a marker and it did not continue drawing, as expected. I then went to edit mode, and it was editable.
Perhaps there is some other interfering logic?

To enable a single Marker directly after drawing you can use this:
map.on('pm:create',(e)=>{
    setTimeout(()=>{
        if(e.layer instanceof L.Marker){
            e.layer.pm.enable();
        }
    },10);
});


Feature/fix retry handling

This PR fixes an issue where the plugin causes non-ratelimit retrys on the API when a user uses an invalid API token. To do this it removes non-cacheable parts of the API reply.
According to the unit test this should be ok.

replaced by https://github.com/jkahrs/cert-manager-webhook-hostingde/pull/11


jank on dynamic infinite lists

I've taken the infinte scrolling example and made it dynamic height based on the index. The list jumps down whenever new rows have been fetched.
https://codesandbox.io/s/eager-williamson-2ydyo?file=/src/index.js

Were you able to find a solution? I'm experiencing this issue using React Virtual v2.4.0 on an infinite scrolling list using dynamic measurement rendering.

From what I can tell, this issue occurs when a new page of data is loaded, triggering a change in the size argument passed to the useVirtual hook. When this happens, the useIsomorphicLayoutEffect hook is triggered, clearing the measure cache:
https://github.com/tannerlinsley/react-virtual/blob/7ad46288d5362ce94d600357f8cba087e6adbe63/src/index.js#L134-L139
The sizes of elements that were already measured dynamically and previously rendered to the screen are cleared from the cache, which causes calculateRange to return different start and end values than the range that was shown on screen prior to the page load.

@mattmikolay thanks! I've spent days trying to find out what is happening with scroll.
@tannerlinsley what's the point of this effect? I've deleted this hook from source code and  infinite list with dynamic size row now works fine.

@nctay No problem!

what's the point of this effect?

Tanner could confirm, but I'm willing to bet it's to handle the case where an entirely new list is being rendered. On the first render, React Virtual might render an array of items [A, B, C]. On the next render, the array could change to [C, D, E, F]. While the array's size only increased by one, its elements changed entirely, so we should not preserve the existing measurements.
I've had success using a custom fork of React Virtual that removes the aforementioned useIsomorphicLayoutEffect and introduces measurement caching by ID rather than an item's position in the list. I added a keyExtractor option of type (index: number) => number | string to useVirtual that is meant to return a unique ID for any item in the list, given its index.
I'd be happy to open a PR if Tanner feels this approach is the right solution. I haven't tested my changes since the addition of the measure function in #55, though.

@mattmikolay wow, your fork works like a charm! Thank's a lot! Hope Tanner will use your solution.

If you can integrate the latest changes I would love to see a PR for unique key tracking

@tannerlinsley Great! I'll have a PR for you shortly. Thanks.

@tannerlinsley #107 is ready for review, at your convenience. 🙏

:tada: This issue has been resolved in version 2.6.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


\moderncvcolor not respected in 2.4.1

In version 2.4.1, the option \moderncvcolor is not followed.  No matter what color is provided, the section and subsection titles are black.  Using pdfLaTeX on a Mac via TeXShop.
Here is a minimum working example:
\documentclass[letterpaper, sans, 10pt]{moderncv}
\moderncvstyle{classic}
\moderncvcolor{red}

\name{First}{Last}

\begin{document}

\makecvtitle
\section{Education}

\end{document}

I reviewed another submitted issue that had been closed, and I see that \moderncvcolor needs to be before \moderncvstyle.  This is a change from previous behavior.

Hi @smithjh-lafayette ,
thanks for your own research! Yes, unfortunately we had to introduce a breaking change for color adjustments.
I will mark this as (kind of) duplicate of #198 and hope your cv is working again. If not just feel free to comment below.


JSON.encode throws error when encoding a Map with non-String keys, even when specifying a toEncodable method

This issue was originally filed by<EMAIL_ADDRESS>
What steps will reproduce the problem?

Use the sample code at https://dartpad.dartlang.org/f0e04824b5e77d83cd03

What is the expected output? What do you see instead?
The expected outcome is for the input Map to be converted using the provided toEncodable method. Instead, an exception is thrown.
In DartPad, the throw error is "Uncaught TypeError: t1.get$length is not a function".
On my local machine (running on the Dart VM) the following output is seen:
"C:\Program Files\dart\dart-sdk\bin\dart.exe" --ignore-unrecognized-flags --checked --package-root=C:\code\test\packages --enable-vm-service:53078 --trace_service_pause_events C:\code\test\main.dart
Observatory listening on http://<IP_ADDRESS>:53078
{"0":"zero","1":"one"}
Unhandled exception:
type 'int' is not a subtype of type 'String' of 'key'.
#0      _JsonStringifier.writeMap.<anonymous closure> (dart:convert/json.dart:721)
#1      _CompactLinkedHashMap.forEach (dart:collection-patch/compact_hash.dart:289)
#2      _JsonStringifier.writeMap (dart:convert/json.dart:721)
#3      _JsonStringifier.writeJsonValue (dart:convert/json.dart:696)
#4      _JsonStringifier.writeObject (dart:convert/json.dart:651)
#5      _JsonStringStringifier.printOn (dart:convert/json.dart:831)
#6      _JsonStringStringifier.stringify (dart:convert/json.dart:813)
#7      JsonEncoder.convert (dart:convert/json.dart:243)
#8      JsonCodec.encode (dart:convert/json.dart:142)
#9      main (file:///C:/code/test/main.dart:23:14)
#10     _startIsolate.<anonymous closure> (dart:isolate-patch/isolate_patch.dart:255)
#11     _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:142)
Process finished with exit code 255
What version of the product are you using?
Dart VM version: 1.10.0 (Fri Apr 24 06:02:24 2015) on "windows_x64"
On what operating system?
Windows 7 64 bit
What browser (if applicable)?
Running as a command line application.

Thanks for the response on this. What version(s) of Dart will I be able to start seeing this fix in?

The fix should be in version 1.11.


Frequent missing publishedAt element for versions where default = true

We have observed many responses the default version does not have a publishedAt element present.
Eg:
curl https://api.deps.dev/v3alpha/systems/maven/packages/org.hdrhistogram:HdrHistogram/versions/2.2.1 | jq .
Shows publishedAt for all versions, except the 2.2.1 which is the default version.

Hello, thanks for letting us know.
There are some challenges collecting Maven timestamps. We currently collect what we can from the Maven central index files (https://repo.maven.apache.org/maven2/.index/), but those are only updated once a week so it can take a while for the timestamps to fill in and I am aware of a couple of other issues which could slow them down even further.
I do have some work in progress to try and improve the situation by also collecting some other sources of timestamps, so hopefully the situation will improve over the next few months.


WanAddress is still used

This is just a reminder that we need to checkup on the ramifications of removing the WanAddress from PublicSystemInfo.
The WanAddress is used in https://github.com/jellyfin/jellyfin-apiclient-javascript/blob/master/src/connectionmanager.js#L66 despite being removed from the response in https://github.com/jellyfin/jellyfin/pull/1770
This line did not exist in the version of the apiclient that jf-web uses, so it was assumed not to be in use.
It probably relates to falling back to a remote connection if a local connection could not be established.

From what I've gathered in the few explorations I'm doing, this API Client is more or less the same thing, but has additions compared to what's in jellyfin-web.
I'm writing tests to check if everything works as the web client expects and I've already hit some failures, so things will need to be adjusted :)

This is fixed with 1.1.0.


Docker install question - Plex Assistant not showing up in integrations

Hi, I am installing via HA Docker image on a Synology NAS. HA runs fine, but Plex Assistant does not seem to be available under integrations. Plex media server integration is installed correctly and working properly.
I tried the HACS install method, but that only showed the Plex Assistant IFTTT integration and not the actual Plex Assistant integration. I then wiped the HA docker image, redownloaded it, and used the manual install method into custom_components/plex_assistant, and it seems that HA loads this properly since I was using an old install method that included editing the YAML config file, and Plex Assistant popped up a message telling me to delete the portions in the YAML and created a pycache directory, so I am assuming that it is running properly? I again could not find the Plex Assistant integration component by searching, so there is no way to add it and no initial configuration is triggered.
Could you please tell me if there is some way to fix this? Or is this something specific to the docker install?

Have you tried clicking add button on the integrations page, then finding Plex Assistant?


Added Zeep wrapper (this still needs a lot of work)

This PR adds the Zeep-wrapper base client. Due to the complex nature of the MMS payload structure, each request will require multiple XML-conversions (one for the payload and one for the envelope, plus whatever has to be done for attachments). Since Zeep requires awareness of the WSDL file, and these WSDL files only go up to the "envelope" level, we'll have to do custom XML conversion on top. Boy, isn't this just a bucket of fun!?

Task linked: CU-8693z3etc MMS API library MVP complete


Class 'Mailgun\Mailgun' not found

I have cloned the plugin in app/Plugin/Mailgun
Also loaded the plugin with CakePlugin::load('Mailgun'); in bootstrap.
I also added the configuration you told in email class in app/Config
But when i try to send the email it gives error Class 'Mailgun\Mailgun' not found
Any help please?

As per the issue closed #1
I removed the line use Mailgun\Mailgun
Now i get the error: Class 'Mailgun' not found
I have manually installed the plugin. Is anything need to do ?

Usage of Composer for installing is strongly recommended. Otherwise, if you have manually copied the file, besides removing that line you will have to include Mailgun class in your code (using php require function).

The below is the structure after copying manually.

I have also added the require('Mailgun\Mailgun.php'); instead of use 'Mailgun/Mailgun'
But still i get missing class 'Mailgun' error in app\Plugin\Mailgun\Lib\Network\Email\MailgunTransport.php
Any thing else i need to do?

I had a similar problem.  The root issue was that although my composer file had the Mailgun/Mailgun-php reference in it, I did NOT have the dependencies it needs.  See this page for more details:
https://github.com/mailgun/mailgun-php
The three additional references I had to add to my composer.json file, plus Mailgun, are:
"Mailgun/Mailgun-php": "^1.0",
"php-http/curl-client": "^1.0",
"guzzlehttp/psr7": "^1.0",
"php-http/message": "^1.0",

Hope that helps people avoid the issues I did.
-j


Updated the welcome message.

Fixes #41 🔐
Are you a part of GSSOC' 21 ❓

[x] Yes ✔
[ ] No ✖

Describe the changes you've made 🏆
A clear and concise description of what you have done to successfully close your assigned issue. Have you added any new files? Any changes that you think are relevant to be added. Feel free to let us know! 😊

Reduced the emoticons.
Corrected the spelling mistake.

Type of change 🖇
Please delete options that are not relevant.

[ ] Bug fixing (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Code style update (formatting, local variables)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[x] This change requires a documentation update

Checklist: 📋

[x] My code follows the style guidelines of this project.
[ ] I have performed a self-review of my own code.
[ ] I have commented my code, particularly in hard-to-understand areas.
[x] I have made corresponding changes to the documentation.
[ ] My changes generate no new warnings.
[ ] I have added tests that prove my fix is effective or that my feature works.
[ ] New and existing unit tests pass locally with my changes.
[ ] Any dependent changes have been merged and published in downstream modules.

Screenshots (if required) 📷
Add screenshots of the changes you have made.

Original
Updated

original screenshot
updated screenshot

Please change the link to CODE_OF_CONDUCT.md and CONTRIBUTING.md to their respective permalinks. These links don't work in the issues or PR tab.

Sir, I have made the changes. Please do check it.

Yeah, @ManasUniyal I always squash and merge but it's still better if they learn to squash the commits into a single message.

Nice @krayong 👍

Yeah, @ManasUniyal I always squash and merge but it's still better if they learn to squash the commits into a single message.

Nice @krayong 👍


Security Vulnerability

Describe the bug
The latest graph data science client (1.8) depends on pyarrow >= 4.0, < 15.0 which includes the vulnerable versions 0.14.0 to 14.0.0 (severity: critical) as described here:

https://arrow.apache.org/security/
https://www.cve.org/CVERecord?id=CVE-2023-47248

Proposed mitigation is to make sure to use pyarrow version 14.0.1 or greater.
Would be great if you can update the requirements, as we currently cannot install package due to policy violations because of this vulnerability.

Hello @stetelepta,
we did shrink the available versions now, but not as narrow as you suggested, as we might have customers that use older versions of pyarrow.
Pip will automatically install the newest version of pyarrow unless you specify otherwise. Therefore it's not clear to us why your policy doesn't allow you to use a library that lets you use a secure version of pyarrow.

Hi Sören,
Thank you, I am also not completely sure, but when it failed the scanner (Nexus) mentioned the security vulnerability. Just tried again and package is now installed successfully!


GMUDefaultClusterRendererTest fails

Was trying to understand why my code works not as expected and found issue with unit tests.
It seems to be issue with OCMock
Return value cannot be used for method; method signature declares '{?={?=dd}{?=dd}{?=dd}{?=dd}}' but value is '{?={CLLocationCoordinate2D=dd}{CLLocationCoordinate2D=dd}{CLLocationCoordinate2D=dd}{CLLocationCoordinate2D=dd}}

Which version of OCMock do you have? I cannot reproduce the issue on my end.

OCMock (3.3.1)

It's interesting I also had 3.3.1. Which line in the GMUDefaultClusterRendererTest is that? Was it a compile error?

No, it occur during execution when code attempt to access visibleBounds of projection. Can give exact log later today

4 окт. 2016 г., в 9:23, Son Nguyen<EMAIL_ADDRESS>написал(а):
It's interesting I also had 3.3.1. Which line in the GMUDefaultClusterRendererTest is that? Was it a compile error?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.

Here is full log http://pastebin.com/24L90vF1

Hi there, I'm also facing the same issue. All the tests within GMUDefaultClusterRendererTest fail except for testShouldRenderAsClusterAtZoom. I do have the latest XCode and OCMock.

Any news on the issue? I'm sure issue is not on our side as I try on few other machines, result is always same. We use latest Xcode and latest SDK

I have been able to reproduce the issue with XCode 8 and OCMock 3.3.1. It seems OCMock does not allow stub method to return a struct in XCode 8.
If these tests are blocking your development on XCode 8, you can comment them out until we find a solution.

Hello, I'm having a similar issue, also using XCode 8 and OCMock 3.3.1. But I found it curious that the test passes when I use the simulator on iOS 10, but the problem with the stub method appears when I use iOS 9.3.
Any updates on the issue?


Python won't use 3.7 and is stuck on 2.7.

Environment data

VS Code version: 1.30(latest)
Extension version (available under the Extensions sidebar): 2018.12.1
OS and version: MAC OS Mojave
Python version (& distribution if applicable, e.g. Anaconda): 3.7 but only uses 2.7
Type of virtual environment used (N/A | venv | virtualenv | conda | ...): na
Relevant/affected Python packages and their versions: 3.7 and 2.7

Expected behaviour
I want to be using python version 3.7.
Actual behaviour
Doesnt let me use python 3.7 only uses 2.7
Steps to reproduce:
Not sure you can reproduce this. I followed all the instructions online on how to make python 3.7 the right interpreter but it wont work.
Logs
Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)
Nothings wrong with the output i just cannot code using 3.7 it only allows for 2.7 to be used .

Output from Console under the Developer Tools panel (toggle Developer Tools on under Help; turn on source maps to make any tracebacks be useful by running Enable source map support for extension debugging)
XXX

I am having this issue as well. When using Select Interpreter, it sees all of my conda environments. However, when I am using the integrated terminal, even after using Python: Create Terminal, it still uses the macOS default 2.7 version.

@mferraro23 and @MattMorgis, is this strictly a problem with conda?  Which version of conda are you using?  Do you still have this problem if you use the latest release of the extension?

@MattMorgis
Please could you try making the following change and letting me know whether it works:
"terminal.integrated.env.osx": {
		"PATH": ""
}

That seems to have fixed the issue

Great!


QSL depends on a very wide range of minecraft versions by default

This is due to QMJ's versions field of a depends block interpreting prefix-less versions (like 1.20.1) as being prefixed with ^, which means it will be loaded in any version of minecraft after 1.20 and before 2.0. I think the fix for this is to always prefix the minecraft version with = here: https://github.com/QuiltMC/quilt-standard-libraries/blob/3e912ab0bfc3272b99ab6121d80f3e65ef9c6933/build-logic/src/main/java/qsl/internal/json/QmjBuilder.java#L51-L59
(This doesn't affect snapshots)
I found this out from quilt-loader 0.19.2-beta.2, which logs a warning when encountering unnecessary version constraints:
Warnings for quilt.mod.json at: '/qfapi-7.0.3_qsl-6.0.3_fapi-0.83.1_mc-1.20.1.jar!/META-INF/jars/block_content_registry-6.0.3+1.20.1.jar!/quilt.mod.json'

Unnecessary dependency range 1.20.1 at line 31 column 7 path $.quilt_loader.depends[1].versions[0] (resolves as [1.20.1, 2)), since it doesn't affect the result [1.20, 2)

(FWIW I think the decision to default to ^ is an odd one, and probably causes a lot of confusion. :/ Perhaps in future QMJ specs we can remove it? Either way that doesn't help us now)

About the default being ^, I believe it was a decision made in order to match common SemVer implementations, and it had regular dependencies in mind rather than the least-semver-following component ever; I'd rather have it stay ^ (with the clarification being provided through the in-dev-hell JSON schema) than use = because of Minecraft
Anyway, huh, looks like we did miss a = here, and we didn't mind that much due to many of our versions supporting one version only, with nothing else other than adding an extra version being done for cases like this one

Fixed on QSL v6.0.4, which adopts the new method introduced by Quilt Loader 0.19.2


Link broken : https://frontside.com/interactors/docs/existing-interactors#material-ui

Hi @dkoul! Thanks for taking the time to report this. What page did you find this broken link on?

Sorry about not sharing the url earlier, this page, https://frontside.com/interactors/mui/api/index.html, last link

@dkoul Thanks, we'll get that fixed. In the mean time, this is the url it should go to https://frontside.com/interactors/docs/library#material-ui


All the countries in a region organised by largest population to smallest.

Unable to complete due to project time constraints


How to get win10 app icon

How to get win10 app icon

https://github.com/timsneath/win32/discussions/505#discussioncomment-2980147

Closing based on answer given by @Far-Se. Please add more information to the bug if this doesn't answer your question. Thanks!