Add option to colour wires in diagrams with frames

Now we can draw wires with colours. the diagram has frames will be rendered with colours by default,

Wire colours be turned off using flag color_wires=False.
Change the thickness by passing a float value as wires_width=1.5 in draw(). by default it uses 1.25 units
Mat_backend and Tikz_backend colouring changes

Code
Mat_Backend
reader = DisCoCircReader()
reader.text2circuit(
    "Thunder dere 's a bump . Vy don'd you drive unk Sharlie . Drive yourself if you think you can do any better .").draw(
    figsize=(12, 8), foliated=True, wires_width=1.5, color_wires=True)

Tikz_backend
reader.text2circuit("Thunder dere 's a bump . Vy don'd you drive unk Sharlie . Drive yourself if you think you can do any better .").draw( foliated=False, to_tikz=True, color_boxes=True, color_wires=True, use_tikzstyles=True)

Screenshots

Tikz drawing

@Ragunath1729 a few questions/remarks:

The colored wires start with different colors in between diagrams - this is not intended, right?
The wires inside frames don't match the dom and cod wires of the frame - maybe it's better to leave them as black?

I'll create a more thorough review later.

@Ragunath1729 a few questions/remarks:

The colored wires start with different colors in between diagrams - this is not intended, right?

 We using a class level shared variable for the noun ids, so once object instantiated, all the wires get unique IDs

The wires inside frames don't match the dom and cod wires of the frame - maybe it's better to leave them as black?

yes, now all wires inside the frames are drawn as black

I'll create a more thorough review later.

@Ragunath1729 @neiljdo In one of the last commits, Ragunath added thickness to the wires, which looks good if the diagram is not very big. It is worth however to check how it looks in the really big diagrams with many nouns, probably not that good. Perhaps we should try to reduce a bit the thickness, keeping the wires thicker than before though?

@Ragunath1729 I've left some comments, please take a look.
@neiljdo ,  I have updated the code as per the review comments. kindly review again


package sync with pipeline

Package updates to match pipeline and CP4D 4.0/python 3.8 definitions.

I did this in another PR


How to approach implementing Escrow Buddy with pre-existing FileVault2 enforcement profile in Jamf Pro

Our Jamf Pro instance has already rolled out FV2 encryption enablement/enforcement configuration profiles and is set to escrow the PRK to Jamf Pro. How do I approach using Escrow Buddy to force encrypted endpoints with an invalid/unknown individual recovery key validation to escrow a new PRK? These endpoints with invalid/unknown recovery keys already have the existing FV2 config profile, and I'm worried that scoping an additional FV2 profile with the same settings will cause more issues.

Hi @jybarbo79 - No need to scope an additional profile; just make sure your existing profile is scoped to all Macs. Then you can proceed with steps 2 and 3 of the deployment instructions.

Perfect! Thanks for the quick reply!


Feature/Update Return Response

Some users would like to have more info from API response:

Wow thanks! That was quick :)
Will this be released in the new version?


gobbleStringLiteral small improvement.

gobbleStringLiteral small improvement.
evaluate a string as bellow
`openCompositeScreenInNewWindow('http://localhost:9089/verAaProductLine_Aas(\'DEPOSITS\')?t24Intent=View');

every char after '' is ignored

@soney What do you think about this? Why there is a special case for those kind of characters at this moment?


V10 Compatibility

is possible to get the compatability to v10 foundry?

v1.3 of the module is compatible with v10. What issues are you having?

sorry for delay, it simple doesn't work (yes api is correctly in)

yes

I have removed the Screenshot as it showed your API Key. Are you sure you copied the WHOLE api key? that one looked a bit weird compared to mine.

yes i'm sure, don't worry about delete i have already changed it (thx for fast support too)

don't worry about the image, i change it again after posting this answer

did you add the :fx part at the end? it seems as if that was missing?
Otherwise, please send me a JSON of the Journal that you are trying to translate.

yes, how can i export the json?

Right-Click the Journal in the right sidebar and click on Export Data

I suppose I have found the problem, if the paragraph to be translated has too many characters (i don't find the exact number of characters) the module does not work, on the contrary if I divide it into several pages it works correctly, do you think it is possible to fix it?

I somehow assumed that was the issue.
I'll see if I can build a fix in the next few days!
Am Do., 27. Okt. 2022 um 16:46 Uhr schrieb Arka @.***>:

I suppose I have found the problem, if the paragraph to be translated has
too many characters (i don't find the exact number of characters) the
module does not work, on the contrary if I divide it into several pages it
works correctly, do you think it is possible to fix it?
—
Reply to this email directly, view it on GitHub
https://github.com/Meneltamar/fvtt-comprehend-languages/issues/8#issuecomment-1293637541,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AE4ZVBX7H3IG2TFZ6KNNYIDWFKIVTANCNFSM6AAAAAARJMEH4I
.
You are receiving this because you commented.Message ID:
@.***>

@arkangel85 I just pushed a new version (v1.4) that should fix the issue. At least it worked with your example without issue.


IndexError on Scales with Zero

Overview
Warning for when using invalid scale range that includes zero which is easy to accidentally do when setting up scales dynamically and would be nice to fail gracefully.
Bug and Steps to Reproduce
When running an invalid scale range (that includes zero), pywt throws an unrelated IndexError
time, sst_data = pywt.data.nino()
scales = [0, 1, 2]
wavelet_coeffs, freqs = pywt.cwt(data=sst_data,
                                 scales=scales,
                                 wavelet="morl")

Throws:
pywt/_cwt.py:154, in cwt(data, scales, wavelet, sampling_period, method, axis)
    152 if j[-1] >= int_psi.size:
    153     j = np.extract(j < int_psi.size, j)
--> 154 int_psi_scale = int_psi[j][::-1]
    156 if method == 'conv':
    157     if data.ndim == 1:

IndexError: index -9223372036854775808 is out of bounds for axis 0 with size 1024

This error actually originates on pywt/_cwt.py:150 when attempting to divide by the scale. If the scale range includes zero then it will attempt to divide by zero. However, this error is hidden by casting with astype.
For example, the example below should throw a divide by zero issue, but doesn't. It instead returns an empty array. As a result, trying to index by the empty array on line 154 will throw IndexError: index -9223372036854775808 is out of bounds for axis 0 with size 1024
For example:
j = np.arange(0 * (28-0)) / (0 * 1)
j = j.astype(int)
print(j[::-1])
--> Returns: []

Solution
Throw relevant ValueError when attempting to use a scale that includes zero

With the unconditional conversion for scales
scales = np.asarray(scales)

I added a ValueError if the scale was not given as a array_like (list/np.array), which enforces the array_like expected unit type from the docs
# convert array_like scales to a np.array
if not isinstance(scales, list) and not isinstance(scales, np.ndarray):
    raise ValueError("scales should be an array_like, list or np.array")
scales = np.asarray(scales)

I added a ValueError if the scale was not given as a array_like (list/np.array), which enforces the array_like expected unit type from the docs

Note that array-like includes scalars (and anything else that np.asarray can convert to a numpy array), so this last change is not correct and would be a backwards-compatibility break.
The problem from the second commit was:
FAILED ..\tests\test_cwt_wavelets.py::test_cwt_small_scales - TypeError: iteration over a 0-d array
FAILED ..\tests\test_matlab_compatibility_cwt.py::test_accuracy_precomputed_cwt - TypeError: iteration over a 0-d array

So probably the right solution is to use np.atleast_1d(scales).

I pushed a rebase with that suggestion - let's see if we can get this merged:)


Integration with Uncertainties

It would be absolutely great to be able to propagate unit-aware arrays with uncertainties through xarray, but it's unclear to me to what extent pint is currently integrated with the Uncertainties package.
I also haven't thought much about whether this would present any additional challenges on the xarray side.

Right now, Pint does not simultaneously support NumPy and Uncertainties, so this integration would likely have to be on hold until they would become mutually supported.
xref https://github.com/pydata/xarray/issues/3509

@jthielen Do you happen to know if there are any plans to support integrating both packages at once?

As far as I'm aware, it's a desired feature but has no roadmap or timeline to implementation. Though, I'd encourage you to ask directly on the Pint issue, as the Pint maintainers themselves may have a better idea!

Thanks for your response. Somehow I didn't notice that this issue was actually on a different repo!

With regards to uncertainties, I think one difficulty with the existing uncertainties package is that you have to hand-write the propagation rules for each operation, which is definitely not feasible. If you used an auto-differentiation library like JAX, you could imagine trying to have that implement array_function for all numpy functions. I tried, and was a little successful with getting something like this working here, but there's a ton of issues with the implementation that I don't think can be resolved with the hacky way I implemented it.
Probably the best way to do it would be to re-implement it in xarray. Would there be any interest in this? I would definitely be interesting in contributing towards getting something like this to work.

@varchasgopalaswamy if you can get uncertainty support working in xarray in a general and interoperable way, people would love that! I would expect it to be a pretty big project though.

re-implement it in xarray

Can you expand on what you mean by this? Making uncertainties work in a way that doesn't prevent you from using xarray as your top-level object, and also using different array types underneath (e.g. dask, cupy), would be critical for widespread adoption.
There are also some discussions about potentially wrapping Jax in xarray ( I can find links if you want).

Agreed, I don't expect it to be easy - but I would find it useful for my research. We're thinking of adopting xarray throughout our code, which requires support for uncertainty propagation, so I have a vested interest in getting this to work!

Can you expand on what you mean by this?

I am still new to xarray and I need to get more familiar with the data model and terminology, so I might not be saying this right.
Looking through this repo, it looks like how pint integration is being done is by either converting the value stored in a DataArray to a Quantity, or by having a "units" entry in attrs. Let's say you have two DataArrays that you add together. How does pint enter the picture here? Do you quantify the dataset values, add the two Quantity arrays together, and then create a new DataArray with the resulting Quantity array?

We're thinking of adopting xarray throughout our code, which requires support for uncertainty propagation, so I have a vested interest in getting this to work!

This is how every cool feature gets made :)

Looking through this repo, it looks like how pint integration is being done is by either converting the value stored in a DataArray to a Quantity, or by having a "units" entry in attrs.

Xarray objects are wrappers of numpy arrays, or of arrays that can be treated as if they were numpy arrays (so-called "duck-typed arrays"). Xarray expects that the wrapped array exposes certain methods and attributes (e.g. .shape, .mean(), __add__()), which act in the same way. The rules for what exactly counts as a numpy-like array has recently become a lot more formalized (thankfully) via the python array API standard.
Xarray organises and aligns the wrapped data, but the actual computation is always performed by the underlying array type. So da + da will ultimately call ndarray.__add__ if each da is wrapping a numpy array. You can see what particular array object the DataArray is wrapping by calling .data.
So in the case of pint, xarray is wrapping a pint.Quantity, and all operations are delegated to the pint.Quantity object. (pint.Quantity happens to further wrap numpy.ndarray, but this isn't a requirement from xarray's perspective, it's just an implementation detail of pint.)

Let's say you have two DataArrays that you add together. How does pint enter the picture here? Do you quantify the dataset values, add the two Quantity arrays together, and then create a new DataArray with the resulting Quantity array?

Basically yes, but if you start with a Dataset with quantified values, the other steps are handled automatically by xarray delegating to the wrapped pint.Quantity objects.


Updated Footer

Issue Number: #41
fixes #41
Thanks for giving me this opportunity.
Describe the changes you've made
A clear and concise description of what you have done to successfully close your assigned issue. Any new files? or anything you feel to let us know!
Describe if there is any unusual behaviour of your code(Write NA if there isn't)
A clear and concise description of it.
Additional context (OPTIONAL)
Add any other context or screenshots about the feature request here.
Test plan (OPTIONAL)
A good test plan should give instructions that someone else can easily follow.
How someone can test your code?
Checklist

[ ] I have added my name in the contributors list at the end of README.md file.
[ ] My code follows the code style of this project.
[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.
[ ] I have added tests to cover my changes.
[ ] All new and existing tests passed.
[ ] The title of my pull request is a short description of the requested changes.
[ ] Open Source Program names(OPTIONAL: If you participated in any open-source program then please mention the program name here)

Resolve conflicts.
Make another PR if the issue persists.

I have made another PR

Resolve conflicts.
First, fetch upstream, make only necessary changes then make PR.
Try to make a new PR it will resolve the conflicts.


In windows console, Chinese file name can not be ls correctly.

Please use the following bug reporting template to help produce actionable and reproducible issues:

A brief description
When I cd to /mnt/d and ls the files in my windows D: , Thers are a lots of directoies with chinese name and long name, I found they can not show right in the default windows console. Just it can show half of chinese word.
Expected results
Get the answer how to set

Actual results (with terminal output if applicable)
Your Windows build number
Steps / commands required to reproduce the error
Strace of the failing command
Required packages and commands to install

See our contributing instructions for assistance.

hi xuzhen0107,
The same problem happens to both Japanese console and xterm on Xming/VcXsrv. I guess this problem is language independent.
IMHO, the source cause seems the very buggy terminal emulation where escape sequence are not properly handled.
Check if you have alias lines, for instance, alias ls='ls --color=auto'.
Temporary workaround would be life with white and black console (Joke):
opt-1) comment out alias commands in ~/.bashrc where "--color=auto" is specified for cosmetic reason
opt-2) run ls with "--color=none" option
opt-3) run /bin/ls
opt-4) redirect output of ls to cat : ls /mnt/d | cat
I have checked with the script command that the output of ls itself is error free.
Anyway, the real solution seems missing. I also wish for project team's efforts.
/Skinny

one more comment on the gargled characters,
Change Keyboard settings for Chinese with less gargled text screen.
$ LANG=en_US.UTF-8 sudo dpkg-reconfigure keyboard-configuration
/Skinny

Thanks for the discussion.
Closing this issue since:

This issue is very old and has likely been fixed in recent Win10 builds
This is the WSL issues repo, but this is an issue in Console which has its own Console GitHub Repo
GitHub doesn't allow issues to be moved between repos, preserving posters' identity :(

If you have further asks/issues, please file new issues on our Console GitHub Repo.


Wallpapers stop updating

Hi,
Has anyone had the issue where wallpapers stop changing on there Magic Mirror?
Getting my feed from Chromecast,  seems to change and work as expected for a bit and then all together stops changing - and stays on one static wallpaper.

I've never been able to reproduce this behavior myself, but it sounds like maybe you're having the same issue as #42.  Can you try to follow the instructions in that thread and see if you have any failing requests?

I have the same issue but only on the target not in the webserver. Can someone help to solve this problem?

I've experienced this too and think it is related to network hiccups (temporary wifi or Internet connection dropouts and in #42 perhaps temporary name resolution issues). I find if the network drops out, even after thet network recovers then the images fail to update. Could be MM not handling the network hiccup or the Wallpaper module (I only run the wallpaper module) but it seems to die and not recover.
I'd like to test with an Ethernet connection but i'm not in a position to do that at the moment.

Can you try latest master?  I've made a change to use setInterval instead of setTimeout for the image load timer, at the very least, that should retry each slideInterval to load the next image, and shouldn't get stuck.

Sorry but how?

cd ~/MagicMirror/modules/MMM-Wallpaper
git pull

I've updated and i'll keep an eye on it :)

Might be worse... it's hung already. Anything you'd like me to grab whilst it is hung?

Ok, pull latest again, I think I've finally figured it out.

I've updated... see how we go :)

Thank you, will try  updating today.
Get BlueMail for Android
On 23 Jan 2022, 17:22, at 17:22, Jon Kolb @.***> wrote:

cd ~/MagicMirror/modules/MMM-Wallpaper
git pull

--
Reply to this email directly or view it on GitHub:
https://github.com/kolbyjack/MMM-Wallpaper/issues/58#issuecomment-1019529500
You are receiving this because you authored the thread.
Message ID: @.***>

I have updated it. I will monitor it and let you know.

Since you were all very prompt in reporting failures, I'll assume that this is fixed

Haha no issues here so far 👍🏼

It is solved.

Looking good, thanks for resolving the issue. Really good module to have running.


fix typo in classname | cms-breadcrumb container

1. Why is this change necessary?
fixed a typo in classname definition in cms-breadcrumb container
2. What does this change do, exactly?
3. Describe each step to reproduce the issue or behaviour.
4. Please link to the relevant issues (if any).
5. Checklist

[ ] I have written tests and verified that they fail without my change
[x] I have squashed any insignificant commits
[ ] I have written or adjusted the documentation according to my changes
[ ] This change has comments for package types, values, functions, and non-obvious lines of code
[x] I have read the contribution requirements and fulfil them.

Hi @lietzi,
sorry for my late response. Thank you very much for changing this to a deprecation. I have imported your PR and rebased it to 6.1 so it should be in the next patch release.
Best regards
Tobias


Where can I get the supplementary material?

Hi, Thanks for your work. I am very interested in your work and would like to learn more about your experimental results from the supplementary material. Where can I get the supplementary material?

Hi, thanks for your interest. In our arxiv version  we have attached the supplementary material to the end. I can also update the project website to include it, but it may take a while...
Best,
Congyue


[DO NOT MERGE] Test new docker images

Summary
Test new docker images in ci
Additional Details
@cerner/terra

+1 once the toolkit images are used


Nuget install issues

In VS 2015 I get this reported trying to install-package imageprocessor
Given its core to Umbraco, I'm guessing its local issue, so I reinstalled VS 2015, but still no luck. Any ideas? I have tried all types of .net framework targets...
Interestingly I can install anything up to and including version 2.3.2

install-package : Could not install package 'ImageProcessor 2.3.3'. You are trying to install this package into a project that targets '.NETFramework,Version=v4.5.1', but the package does not contain any assembly references or content files that are
compatible with that framework. For more information, contact the package author.

Very odd... I have heard of this issue once before though.
I'll have to double check what version I am running. I might have switched to 4.5.2 as 4.5.1 has now reached end of life.
https://blogs.msdn.microsoft.com/dotnet/2015/12/09/support-ending-for-the-net-framework-4-4-5-and-4-5-1/
Have you tried switching your project to that?

Yeah tried all the variants. Ended up going the manual route for Umbraco. DIsappointing but had to get on. Certainly something amiss in my local env. just no idea what. I did a full reinstall of VS but still the same. Will try fixing frameworks

I wonder if it is caused by different Nuget versions? I always keep my version up to date.
I have set no constraints but you never know with Nuget.

Fingers crossed this issue does not affect the new release.

Yes that works. The lurgy has now jumped to Markdown 1.14.4.
Anything in particular was changed to fix that?

Excellent!
I updated all the individual projects to use the latest Visual Studio build version. Other than that I've no idea what could have caused it unless there was a bug in the version of Nuget I was using to pack it.
I'm working from a fresh Windows install now.

well a forced install of Markdown + Automapper specific versions before Umbraco-cms has worked.
i had been installing umbraco first via VS 2013, then switching...actually less painful. i really dont like getting involved with nuget. its just supposed to work right?

That's the idea yeah... Never had any positively identifiable issues.


a

Could we add (when you get a chance low priority) a model field .....Institutional Email
Right now we will add email in comments section

@alexanderkgonzalez this is for use in the initial registration process?

Added to BRP version 1.0.5


add new subscription via fever api

is there a way to do that? is it just not supported yet by miniflux?

Not yet supported.

would be a great feature. Hope this will come!

Unfortunately, that doesn't seems to be supported by the Fever API itself:

The public beta of the API does not provide a way to add, edit or delete feeds or groups [...]

See: http://www.feedafever.com/api


Ask about quaternion_matrix function

When using the quaternion_matrix function I notice that the result is not as expected. For example,  running quaternion_matrix([0.99810947, 0.06146124, 0, 0]) I got:
[[ 1.          0.          0.          0.        ]
[ 0.          0.99244503 -0.12269009  0.        ]
[ 0.          0.12269009  0.99244503  0.        ]
[ 0.          0.          0.          1.        ]]
but in fact, the rotation matrix should be
[  0.9924451,  0.1226901,  0.0000000;
0.1226901, -0.9924451,  0.0000000;
0.0000000,  0.0000000, -1.0000000 ]
Is the function incorrect or I understand it wrong?

@TrinhTUHH  Hey, did you figure out why?

Yes, the arguments in quaternion_matrix() function is: x,y,z,w but I thought it is w,x,y,z. That's why.


Designate a special character \ for all prefixes and commands

Currently, the / character is used for commands which may exist in some names. We make a decision to change the special character to the much more seldom used \ character to minimize confusion.

can use | also


Missing traffic signs in Mapillary traffic sign layer

iD Editor is not showing all the traffic signs that Mapillary has identified in given area.
Here you can see the same location on Mapillary, JOSM, and iD Editor.
Mapillary

JOSM

iD Editor

@ptpt has looked into the issue and summarised the following:

There are more than 1000 features in the bbox, so the rest features are truncated. I can see iD tries to paginate by page=1,2,3,4 but it’s wrong; it should follow the next url in the link header.
ELVA is using the vector tile paramter. When you zoom in you get smaller bbox in the request, so you see all features (edited)
Here is API call in ELVA: https://a.mapillary.com/v3/map_features?tile=14/8783/5133&client_id=MkJKbDA0bnZuZlcxeTJHTmFqN3g1dzo1YTM0NjRkM2EyZGU5MzBh&layers=trafficsigns&per_page=1000

I think this might be related to a variable in the iD code called limit. I am not sure exactly what this function is doing: https://github.com/openstreetmap/iD/blob/master/modules/services/mapillary.js#L233-L244
I see it's used here:
signs: function(projection) {
        var limit = 5;
        return searchLimited(limit, projection, _mlyCache.map_features.rtree);
    },

That aside, I think it's just that the pagination is wrong. I think @nickplesha may have added that originally somewhere here but not sure where it is being used now: https://github.com/openstreetmap/iD/blob/master/modules/services/mapillary.js#L202
I have some sample code for Mapillary pagination here:
function getNext(next_url) {
  $.ajax({
      dataType: "json",
      url: next_url,
      async: false,
      success: function(next_points, status, xhr) {
        console.log('fetching next page: ' + next_url);
        if (typeof next_points !== 'undefined' && next_points['features'].length > 0) {
          var next_features = next_points.features;
          for (f=0;f<next_features.length;f++) {
            output['features'].push(next_features[f]);
            report.innerHTML = 'Retrieving items...';
          }
          if (next_features.length == 1000) {
            paginate(xhr);
            report.innerHTML = 'Retrieving items... (' + output['features'].length + ' so far)';
          }
        }
      }
  });
}

function paginate(xhr) {
  var link = xhr.getResponseHeader('Link')
  let arrLink = link.split("link:");
  link = arrLink.length == 2? arrLink[1]: link;
  let parsed_link = {}
  arrLink= link.split(",");;
  for (l of arrLink){
      linkInfo = /<([^>]+)>;\s+rel="([^"]+)"/ig.exec(l)
      parsed_link[linkInfo[2]]=linkInfo[1]
  }
  if (typeof parsed_link['next'] !== 'undefined' && output['features'].length < limit) {
    var next_url = parsed_link['next'];
    getNext(next_url);
  }
  else if (typeof parsed_link['next'] == 'undefined') {
    console.log('no more detections');
  }
  else if (output['features'].length > limit) {
    alert('try a smaller area, too many detections');
  }
}

Closing as duplicate of #6510


CocoaPods Library integration

Instead of including source code, switch to linking a prebuilt library and including the library headers.

include j2objc-generated-library.a
add j2objc headers folder to HEADER_SEARCH_PATH
update PROJECT-bridgingheader.h to reference the all the headers within Swift, as defined by an "export interface" sourceSet

@confile - make the changes to output the correct file and then reassign this to...
@advayDev1 - so he can modify the file with the correct output

@brunobowden Shouldn't  j2objc-generated-library.a be something like  j2objc-generated-projectName-library.a?
to 3): There is no ``PROJECT-bridgingheader.h`. With pods we can only have one bridging header per target and this one should not be in a pod.

@brunobowden By

j2objc headers

you mean ${project.buildDir}/j2objcright?

@brunobowden

j2objc headers folder

means ${project.buildDir}/j2objc right?

@Confile - talking with @advayDev1, we settled on "projectName-j2objc.a", for the headers, we may yet move them around. So don't worry about further changes for now. Thanks for what you've fixed already.


fix: watch more than one

storage.watch({
  x:()=>console.log('x1'),
  y:()=>console.log('y1'),
})

storage.watch({
  x:()=>console.log('x2'),
  y:()=>console.log('y2'),
})

storage.set('y',2); // should console y1 and y2

Hmm.. @yiminghe are you sure this is needed? The original idea of the return is that if we already initiated a callback prior, then subsequent callback is not needed. Since right after that call, we loop through the set in the internal event listener to invoke all the callback:
https://github.com/PlasmoHQ/storage/pull/35/files#diff-a2a171449d862fe29692ce031981047d7ab755ae7f84c707aef80701b3ea0c80R270-R278
I.e, we shouldn't need to initiate another internal watcher as the original one would loop through the set

Hmm.. @yiminghe are you sure this is needed? The original idea of the return is that if we already initiated a callback prior, then subsequent callback is not needed. Since right after that call, we loop through the set in the internal event listener to invoke all the callback:
https://github.com/PlasmoHQ/storage/pull/35/files#diff-a2a171449d862fe29692ce031981047d7ab755ae7f84c707aef80701b3ea0c80R270-R278
I.e, we shouldn't need to initiate another internal watcher as the original one would loop through the set

You should write a demo and tests it.
It's inside a for loop, the  internal event listener is used to handle single nsKey(cbKey), not all callbackMap keys.

Ahhh whoops @yiminghe thanks so much for pointing that out 🙏


Ant doc

Add a document for Ant plugin https://github.com/launchableinc/cli/pull/202

@awilkes I wrote a draft doc. Please review and revise it!


Stop using deprecated ContextLocator and ContextBuilder.

Thanks for your contribution! Please replace this text with a description of what this PR is changing or adding and why, list any relevant issues, and review the contribution guidelines below.

[x] I’ve reviewed the contributor guide and applied the relevant portions to this PR.

Contribution guidelines:

See our contributor guide for general expectations for PRs.
Larger or significant changes should be discussed in an issue before creating a PR.
Contributions to our repos should follow the Dart style guide and use dart format.
Most changes should add an entry to the changelog and may need to rev the pubspec package version.
Changes to packages require corresponding tests.

Note that many Dart repos have a weekly cadence for reviewing PRs - please allow for some latency before initial review feedback.

Hmm, the version resolution here seems complicated:
https://github.com/dart-lang/tools/actions/runs/9963959255/job/27531185159?pr=285

(we should see if we can get the CI here green before landing)


No support for list indexes

If I want to get an item from a list, I can't.
For example, I can't do list[2].
Please add this feature.

Tryout https://github.com/radon-project/radon for more advance features.

In this repo it uses the / operator to retrieve the element at an index of a list.


Collection::slice() method should have a 3rd parameter to preserve keys

According to the documentation for the slice method, there is a 3rd parameter for preserving the keys when using the slice method (https://laravel.com/docs/5.2/collections#method-slice)
However this is not the case and Collection::slice passes true by default to array_slice. Shouldn't there  be a 3rd parameter to pass on to array_slice?

I think this issue can be closed.


Allow the Emulator user to specify a User ID

Fixes #1344
Description
This PR allows the Emulator users to specify a User ID which will be used within conversations.
Changes made
Functionality changes
To implement this feature we made the following changes:

Add an input to the Emulator Settings so that the user can configure a GUID
Change the startNewConversation method to choose between a randomly generated UserId, the UserId set in the props or the UserId set by the user.'
When the Use custom id checkbox is set to false, the input is disabled and cleaned. When true, it is a required field and it gets enabled.
The User ID field has a validation so that the user is not able to enter anything other than a GUID.
Restarting a conversation with the same userId will choose between the one set by the user or the one set by default (if the first one is empty).
Restarting a conversation with a new userId will generate a random ID the same way it was doing it until now.
Remove Restart Conversation Split Button
Add two new buttons Restart with Same User Id and Restart with New User Id to replace Restart Conversation Split Button.

UI Changes
Before

After

Testing
Manual testing

Automated testing
Added automated tests for the new settings options.

Pull Request Test Coverage Report for Build 2326

12 of 15 (80.0%)  changed or added relevant lines in 2 files are covered.
17 unchanged lines in 1 file lost coverage.
Overall coverage decreased (-0.2%) to 55.251%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

packages/app/client/src/ui/editor/emulator/emulator.tsx
6
9
66.67%

Files with Coverage Reduction
New Missed Lines
%

packages/app/client/src/ui/editor/emulator/emulator.tsx
17
60.63%

Totals

Change from base Build 2323:
-0.2%

Covered Lines:
4884

Relevant Lines:
8326

💛  - Coveralls

Great job! 🎉

Thanks! 🎉

Hi @Aliandi and @denscollo:
Justin's in-flight work in https://github.com/Microsoft/BotFramework-Emulator/pull/1400 had problems reconciling with some of the changes made in this diff, so we reverted it in the hopes that when his branch lands @Aliandi can reissue the pull request against master again.

@denscollo @Aliandi this is ready to be re-opened against master. There has been a refactor to some of the code-paths involved so some changes will be necessary.


EGL extensions not found in FreeBSD 12 (nvidia-driver 390.87_1)

I have a quick question on some EGL extensions for the FreeBSD driver that can’t seem to be found. I’m posting here because the Nvidia forums for FreeBSD don’t seem to get any answers from developers and I found this issue trying to run this repository. If there is a better place for me to ask this question please let me know.
I am trying to write a small wayland compositor on FreeBSD, and while testing the eglstream example the FreeBSD driver reports that It can’t find certain extensions that patchnotes say are included.
Replication on FreeBSD 12.0-RELEASE amd64 on a GTX 1070 with nvidia-driver 390.87_1
(pkg install gmake …)
git clone https://github.com/aritger/eglstreams-kms-example
cd eglstreams-kms-example
gmake       <——— Compilation with gmake works fine
running ./eglstreams-kms-example results in:
ERROR: eglGetProcAddress(eglGetOutputLayersEXT) failed

Some reading found that the extensions are searched for in the “utils.c” file. The 346 driver notes say:
Added support for the following EGL extensions:
EGL_EXT_device_base
EGL_EXT_platform_device
EGL_EXT_output_base        <— It looks like this is the one missing
I don’t see EGL_EXT_output_base in the nvidia-settings app either. Is this repository a good way to get an OpenGL context without X11 on FreeBSD? And are these extensions still supported?
Please let me know if there is any more information I can provide, I am more than happy to help in any way to get this working, as it is a feature I am extremely interested in.
Thank you so much for your time!
Austin Shafer

we don't support Vulkan on FreeBSD

If you don't mind random strangers interjecting, are there currently any plans to support Vulkan on FreeBSD in the future?

We don't have current plans for Vulkan support on FreeBSD.  But, knowing it is important to users helps us prioritize future work.

Admittedly, I've only seen about a dozen or so complaints (like this one), which is not much considering it's been 3 years. What really annoys me is zero communication regarding supported features in Linux/Solaris/FreeBSD driver — even a simple comparison tables posted somewhere on devtalk.nvidia.com would be nice.

Hi @aritger,
Back with some more questions, One is a bug I think I've stumbled upon and the other is about gem page faulting.
Also would like to second adding vulkan support. I'd much rather be interacting with Vulkan than EGL, as Vulkan definitely seems to be the future. Out of curiosity is Vulkan dependent on nvidia-drm? (in the linux driver at least)
Nvidia-modeset Acquiring Duplicate Locks
Kernel Panic: nvidia-modeset tries to acquire os.lock_sx @ nvidia_os.c:651 despite currently holding the lock
Steps to replicate - Just run kldload nvidia-modeset on FreeBSD-CURRENT or any version with witness debugging enabled.
Can be encountered on a clean version of 415.25. It's pretty trivial to replicate, although the result can be different depending on how you trigger it. Behavior can range from warnings/kernel panic during module loading to crashing the system later while trying to start x11. This is in the closed source portion of the driver so there's nothing I can do. It'd be great if you have a FreeBSD CURRENT test system that you can install 415.25 on see.
While debugging I enabled most of the kernel debugging options available for FreeBSD. The strict lock checking from the witness debugger notices locks being acquired while they are already held. I am on 12-RELEASE, but FreeBSD 13-CURRENT should have the same results, with more strict checking.
Normally on a release branch this isn't a problem at all, it just gets ignored. However the CURRENT branch of FreeBSD (the development branch) always has the strict lock checking enabled. It's my guess that the port maintainer for the nvidia driver encountered this on their development system, assumed it was also broken on 12.0, and won't pull 415+ until it works on CURRENT.
Feb  7 12:26:23 wolfgang kernel: acquiring duplicate lock of same type: "os.lock_sx"
Feb  7 12:26:23 wolfgang kernel:  1st os.lock_sx @ nvidia_os.c:651
Feb  7 12:26:23 wolfgang kernel:  2nd os.lock_sx @ nvidia_os.c:651
Feb  7 12:26:23 wolfgang kernel: stack backtrace:
Feb  7 12:26:23 wolfgang kernel: #0 0xffffffff80be1b93 at witnes
Feb  7 12:26:23 wolfgang kernel: s_debugger+0x73
Feb  7 12:26:23 wolfgang kernel: #1 0xffffffff80be18e3 at witness_checkorder+0xab3
Feb  7 12:26:23 wolfgang kernel: #2 0xffffffff80b82a98 at _sx_xlock+0x68
Feb  7 12:26:23 wolfgang kernel: #3 0xffffffff83be8522 at os_acquire_mutex+0x32
Feb  7 12:26:23 wolfgang kernel: #4 0xffffffff83acbac6 at _nv034611rm+0x16
Feb  7 12:27:19 wolfgang kernel: nvidia_open_dev:
Feb  7 12:27:20 wolfgang kernel: acquiring duplicate lock of same type: "os.lock_mtx"
Feb  7 12:27:20 wolfgang kernel:  1st os.lock_mtx @ nvidia_os.c:886
Feb  7 12:27:20 wolfgang kernel:  2nd os.lock_mtx @ nvidia_os.c:886
Feb  7 12:27:20 wolfgang kernel: stack backtrace:
Feb  7 12:27:20 wolfgang kernel: #0 0xffffffff80be1b93 at witness_debugger+0x73
Feb  7 12:27:20 wolfgang kernel: #1 0xffffffff80be18e3 at witness_checkorder+0xab3
Feb  7 12:27:20 wolfgang kernel: #2 0xffffffff80b58c33 at __mtx_lock_flags+0x93
Feb  7 12:27:20 wolfgang kernel: #3 0xffffffff83be899b at os_acquire_spinlock+0x1b
Feb  7 12:27:20 wolfgang kernel: #4 0xffffffff83ac70ac at _nv033903rm+0xc

Just for reference, here are the exact debug options I added to my kernel config (/usr/src/sys/amd64/conf/GENERIC):
Index: sys/amd64/conf/GENERIC
===================================================================
--- sys/amd64/conf/GENERIC      (revision 343025)
+++ sys/amd64/conf/GENERIC      (working copy)
@@ -88,6 +88,18 @@
 # Debugging support.  Always need this:
 options        KDB                     # Enable kernel debugger support.
 options        KDB_TRACE               # Print a stack trace for a panic.
+# For full debugger support use (turn off in stable branch):
+options        BUF_TRACKING            # Track buffer history
+options        DDB                     # Support DDB.
+options        FULL_BUF_TRACKING       # Track more buffer history
+options        GDB                     # Support remote GDB.
+options        DEADLKRES               # Enable the deadlock resolver
+options        INVARIANTS              # Enable calls of extra sanity checking
+options        INVARIANT_SUPPORT       # Extra sanity checks of internal structures, required by INVARIANTS
+options        WITNESS                 # Enable checks to detect deadlocks and cycles
+options        WITNESS_SKIPSPIN        # Don't run witness on spinlocks for speed
+options        MALLOC_DEBUG_MAXZONES=8 # Separate malloc(9) zones
+options VERBOSE_SYSINIT=0 # Support debug.verbose_sysinit, off by default

 # Kernel dump features.
 options        EKCD                    # Support for encrypted kernel dumps

Please let me know if there is anything you need me to test or do for this.
How does __nv_drm_vma_fault work? (This is my real problem)
Update - Porting nvidia-drm is going relatively well, I can load the module, open /dev/dri/card0, and generally get pretty far before things start going wrong. Thanks again for all your advice along the way!
Problem - I can't seem to process the GEM nvkms page faults correctly. I don't find the correct vm_page_t to add in the FreeBSD portion, and best case I cause the userland EGL application to segfault. Most of my problems result from me not having done this before. I've written virtual memory/page fault handling for XINU, but have had to learn the Linux and FreeBSD virtual memory systems on the fly for this.
I have read up on lwn.net about GEM and fault handling in Linux, and I found this which appears to be the Intel equivalent from the FreeBSD linuxkpi-based drm tree.
The relevant code is here
static int __nv_drm_vma_fault(struct vm_area_struct *vma,
                              struct vm_fault *vmf)
{
    unsigned long address = nv_page_fault_va(vmf);
    struct drm_gem_object *gem = vma->vm_private_data;
    struct nv_drm_gem_nvkms_memory *nv_nvkms_memory = to_nv_nvkms_memory(
        to_nv_gem_object(gem));
    unsigned long page_offset, pfn;
    int ret = -EINVAL;

    pfn = (unsigned long)(uintptr_t)nv_nvkms_memory->pPhysicalAddress;
    pfn >>= PAGE_SHIFT;

    page_offset = vmf->pgoff - drm_vma_node_start(&gem->vma_node);
#ifdef __linux__
#if defined(NV_VMF_INSERT_PFN_PRESENT)
    (void)ret;
    return vmf_insert_pfn(vma, address, pfn + page_offset);
#else
... (leaving out the related vm_insert_pfn section for clarity)

Request* - I'd love if you could give me some explanation as to what the __nv_drm_vma_fault handler does, or point me to some resources I can use to learn. I can follow it roughly, but I am confused on things like:
Is this the page fault handler for video memory?

Is the GPU video memory handle mapped at nv_nvkms_memory->pPhysicalAddress? If not, what is this address?
If not video memory, does this handler just back a gem buffer with normal anonymous memory?

What is the significance of drm_vma_node_start(&gem->vma_node)? It's unclear to me why this offset is used. I assume its just the offset into the virtual memory area where the gem memory should be stored?
I think most of my confusion can be summed up by "How do you find the location from which to back the faulted page?"

Just for reference, and to others reading this, here is my (broken) FreeBSD fault handling equivalent:
    page_offset = address + drm_vma_node_start(&gem->vma_node);
    vm_page_t page = PHYS_TO_VM_PAGE(IDX_TO_OFF(pfn + page_offset)); <--- can't find the right page
    vm_object_t obj = vma->vm_obj;
    vm_pindex_t pidx = OFF_TO_IDX(address);

    if (!page || !obj) {
	    NV_DRM_LOG_INFO("__nv_drm_vma_fault: page was busy, probably got the wrong one");
	    return VM_FAULT_OOM;
    }

    if (vm_page_busied(page)) {
	    NV_DRM_LOG_INFO("__nv_drm_vma_fault: page was busy, probably got the wrong one");
	    return VM_FAULT_OOM;
    }
    if (vm_page_insert(page, obj, pidx)) {
	    NV_DRM_LOG_INFO("__nv_drm_vma_fault: Could not insert the page");
	    return VM_FAULT_OOM;
    }

    page->valid = VM_PAGE_BITS_ALL;
    vm_page_xbusy(page);

    ret = VM_FAULT_NOPAGE;
    vma->vm_pfn_count++;
    return ret;

If any FreeBSD people see this and know what I've done wrong please feel free to comment. My work is stored in the git repo linked earlier.
As always please let me know if there is anything I can do differently to get this to work.
Thanks again for all of your time and your help!

Oh cool, I assume thats what your repository is for right? When you say it works is that with or without x11 running?

Oh cool, I assume that's what your repository is for right?

Not quite, my repo contains a crude glibc shim allowing loading Linux libGL.so (& friends) into a regular FreeBSD process without Linuxulator.

When you say it works is that with or without x11 running?

With X11.

Hi Austin.  Yes, I've taken note of the FreeBSD Vulkan requests.
nvidia-drm is not required for Vulkan, either within X11 or for VK_KHR_display.
I've filed NVIDIA internal bug 2507077 for the duplicate lock issue (I don't believe you can access the bug report, but you can at least use that number as a handle if asking about it in the future).
Virtual memory management is not my forte, but here is my understanding (looking at the Linux version of nvidia-drm-gen-nvkms-memory.c):

nv_drm_dumb_create() calls nvKms->allocateMemory() to allocate video memory.

nv_drm_dumb_create() calls nvKms->mapMemory() to map that video memory to the CPU.

nv_nvkms_memory->pPhysicalAddress is the physical address where the video memory is mapped.  The GPU has several BARs (Base Address Registers) which are hardware-specific regions of address space.  The system firmware and/or OS assign a physical address to each BAR.  The BARs are smaller than all of video memory.  So, mapping video memory to the CPU involves programming a portion of the GPU's BAR to point to a particular region of video memory.  The physical address of the video memory is then the physical address of the BAR, plus the offset into the BAR where that video memory is mapped.

The physical mapping is linear: i.e., it occupies the physical addresses from
nv_nvkms_memory->pPhysicalAddress
to
nv_nvkms_memory->pPhysicalAddress + args->size

Hopefully that helps with some context.
What sorts of values do you get for page_offset if you compute it like this?
page_offset = vmf->pgoff - drm_vma_node_start(&gem->vma_node);

are those small values, less than PAGE_SIZE?
Looking at the FreeBSD linuxkpi-based drm tree you pointed to, does just this give you something sane?
vm_page_t page = PHYS_TO_VM_PAGE(nv_nvkms_memory->pPhysicalAddress);

It might also help to sprinkle some printouts in the Linux version of __nv_drm_vma_fault() to get a better understanding of the values that it computes.
I hope that helps.

Hi @aritger
Thanks! I'm also happy to test anything regarding that bug if needed.
Also thanks for the explanation, its very helpful. I'll take a look at finding the correct physical location this weekend, but I'll follow up with a few things first.

The physical address of the video memory is then the physical address of the BAR, plus the offset into the BAR where that video memory is mapped.

so drm_vma_node_start(&gem->vma_node) is the offset into the BAR where the gem buffer is located? (and the location I should use to handle the fault?) Just wanting to make sure I'm not misunderstanding what you're saying.

What sorts of values do you get for page_offset if you compute it like this?

I get something like 0xfffffffff0000 or other such bad values. I do get reasonable values for the drm_vma_node_start(&gem->vma_node) bit. In theory this line should work, however it appears FreeBSD doesn't have an equivalent field to vm->pgoff in its vm system, so vm->pgoff will always be zero (causing int wraparound to the value listed earlier). The virtual_address field is filled with the offset calculated from the index into the vm_object corresponding to the page fault, so thats what it looks like my code should pass to vm_page_insert. The index value for FreeBSD serves as both the physical offset and the index into the internal radix tree, which is pretty cool. The index values I get don't seem to be the problem, selecting the right page to back the request with is where I seem to go wrong.

Looking at the FreeBSD linuxkpi-based drm tree you pointed to, does just this give you something sane?

This does, although the vm_page_t is busy, I assume this is because it was already mapped/in use from nvidia.ko. I'm still learning the ins and outs of FreeBSD vm so I'm exploring around to see what is and isn't normal. I can also get valid vm_page_t structs (that may or may not be busy) with the page_offset stuff that is currently in my code.
Thanks!

@ashaferian, if you haven't seen it, this looks like a helpful resource for what you're investigating:
https://www.kernel.org/doc/html/v4.15/gpu/drm-mm.html
I'm not too familiar with drm_vma_node_start(), but hopefully the above document helps.

Thanks! I had actually read the first part about the concepts and then got sidetracked by trying to fix things. Looking at it again made me notice the function descriptions, including drm_vma_node_start. my guess is the note about shifting by PAGE_SHIFT bits to get an offset is the mistake I've been making.

Hi @aritger
I've made some progress, and have run into an issue with EGL (that I assume is coming from a drm initialization problem). I don't have source for libEGL_nvidia so I was hoping you could point me in the right direction.
Where I am at
I solved the page faulting problem from my previous messages. (For anyone reading this in the future, it came from me not properly setting the vma area's pfn_first and pfn_count fields.) The repo I linked earlier has the changes pushed, the commit messages have more details.
Running this eglstream example repo I can make it all the way through drm setup and mode setting! (kms.c) The screen goes black as the vt sets itself to graphical mode, but EGL is not initialized (egl.c) so there is no data to display
Current Problem
Here's the bit in egl.c where it breaks:
ret = pEglGetOutputLayersEXT(eglDpy, layerAttribs, &eglLayer, 1, &n);

    if (!ret || !n) {
        Fatal("Unable to get EGLOutputLayer for plane 0x%08x\n", planeID);
    }

pEglGetOutputLayersEXT is a valid pointer, but n returns as 0. ret is 1 (EGL_TRUE It appears) which means that the EGL call itself succeeded, and there just was no output layer for the plane. The plane id is 0x1e, if its relevant.
I've tried dtracing around as its been really helpful with the kernel part of this port, but I can't seem to get anywhere without the source. There are no calls to the nvidia-drm module during this EGL line. Is there some way that I need to register an output layer for a drm plane that I am not doing currently? Any help would be appreciated.
EGL linking question
Whats the proper way (simplest case) to link with EGL using the 415.25 driver? I see that there is a libEGL.so in the glvnd folder, and a libgEGL_nvidia.so seems to be installed. The 410* driver series is currently being brought into the FreeBSD port tree (see bugzilla ), so glvnd isn't yet available through a package manager. I assume Nvidia has some docs or tutorials on getting started with glvnd, but I can't seem to find anything about a simple EGL example. (Also I assume glvnd is used and not linking with libEGL directly)
Thank you so much!
Austin

@ashaferian, nice debugging.
I stepped through things on the EGL side on Linux, and it looks like there is a good chunk of the code that interfaces EGL and DRM that is only built into the Linux EGL binary, but not the FreeBSD EGL binary.  Sigh.  Sorry :(
I think that explains why your output layer query is returning n==0.
I'll see what I can do to correct the FreeBSD build of EGL for a future NVIDIA driver release.  In the meantime: is running the Linux EGL driver through the linuxulator viable to help test your nvidia-drm implementation?  Other DRM-KMS tests might include the xf86-video-modesetting X11 driver.
For EGL linking, there is some stale, Linux-centric documentation about packaging glvnd here:
https://devtalk.nvidia.com/default/topic/915640/unix-graphics-announcements-and-news/multiple-glx-client-libraries-in-the-nvidia-linux-driver-installer-package/
The intent is that libEGL.so should be provided by glvnd, and that NVIDIA just provides a libEGL_nvidia.so that gets loaded by glvnd.  On Linux, we bundle a glvnd libEGL with the driver .run file, and at install time detect if the distro has a distro-provided glvnd libEGL; if not, we install the one in the driver package.  I forget what we do on FreeBSD, but I suspect we unconditionally install the driver-packaged glvnd libEGL.so.  In any case, applications should just be able to link against libEGL.so, independent of any particular driver implementation.

@aritger I'm sorry to hear that :( If you manage to get it included at some point please let me know! Would introducing the DRM changes to FreeBSD's EGL mess with EGL users that aren't using DRM?
I can't seem to get the linuxlator working with EGL, although I think it's within the linuxlators ability. I had some trouble linking with the libraries from the Nvidia driver because they are 32-bit, and I need to go track down a libdrm package for things to work. I can try this at some point in the future when I have a little more free time.
Also thanks for the link, Its definitely helpful even if it's a little out dated. Maybe I'll help the committers finally get glvnd included in ports. And from a quick look I think you are right, and Nvidia just automatically makes a link for libEGL.so to the libEGL_nvidia. I vaguely remember hearing this causes problems if you install mesa-libs too, but I could be wrong.
Somewhat unrelated, but even though the EGL problems spell doom for this port, I did have a lot of fun learning how to do it. I'm looking for an internship before I go to grad school this fall, and if you all are still hiring interns for this summer I'd love to keep working on stuff like this.
Thank you so much for all of your help!
Austin

I did have a lot of fun learning how to do it

I've got a suspicion you might be interested in looking at CUDA stuff. Specifically a few stubs in nvidia.ko: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=224358#c15.

I can't seem to get the linuxulator working with EGL, although I think it's within the linuxulators ability. I had some trouble linking with the libraries from the Nvidia driver because they are 32-bit

If you have installed nvidia-driver port, you should also have 64-bit Linux libraries.

Nvidia just automatically makes a link for libEGL.so to the libEGL_nvidia.

Not really, libEGL.so is one of the glvnd wrapper libraries.


MODFQMMGR-217 Fix read/write split bug

This commit fixes a bug where Spring wasn't properly configured to ever use the read replica. It also fixes some minor sonar warnings related to Spring dependency injection

I think I'm just going to cherry-pick this over. So closing the PR


added a simple end-to-end pipeline test

Codecov Report

Merging #92 (8f53024) into main (db0d557) will increase coverage by 3.61%.
The diff coverage is 60.00%.

@@            Coverage Diff             @@
##             main      #92      +/-   ##
==========================================
+ Coverage   54.34%   57.96%   +3.61%
==========================================
  Files          35       35
  Lines        2107     2110       +3
==========================================
+ Hits         1145     1223      +78
+ Misses        889      815      -74
+ Partials       73       72       -1

Flag
Coverage Δ

unittests
57.96% <60.00%> (+3.61%)
:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

pkg/pipeline/ingest/ingest_file.go
0.00% <0.00%> (ø)

pkg/pipeline/decode/decode_json.go
90.47% <100.00%> (+0.47%)
:arrow_up:

pkg/pipeline/write/write.go
100.00% <100.00%> (ø)

pkg/pipeline/utils/exit.go
100.00% <0.00%> (+17.39%)
:arrow_up:

pkg/pipeline/pipeline.go
70.87% <0.00%> (+70.87%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update db0d557...8f53024. Read the comment docs.


Custom Save Image node does not visialize or save images produced

Hello and thank you for your work.
Currently, the Custom Save Image node does not save the generated images or display them.
I am using the latest ComfyUI commit available on windows 10.
Thanks for the support.

Please attach your screenshots or provide a clearer description of your working environment. Alternatively, include your terminal report. Thank you! I will review the code to verify whether it can correctly locate the folder on Windows.

L:\ComfyUI_FLUX_1.0>.\python_embeded\python.exe -s ComfyUI\main.py --windows-standalone-build
[START] Security scan
[DONE] Security scan
## ComfyUI-Manager: installing dependencies done.
** ComfyUI startup time: 2024-11-18 07:07:57.537919
** Platform: Windows
** Python version: 3.12.7 (tags/v3.12.7:0b05ead, Oct  1 2024, 03:06:41) [MSC v.1941 64 bit (AMD64)]
** Python executable: L:\ComfyUI_FLUX_1.0\python_embeded\python.exe
** ComfyUI Path: L:\ComfyUI_FLUX_1.0\ComfyUI
** Log path: L:\ComfyUI_FLUX_1.0\comfyui.log

Prestartup times for custom nodes:
   0.0 seconds: L:\ComfyUI_FLUX_1.0\ComfyUI\custom_nodes\rgthree-comfy
   0.0 seconds: L:\ComfyUI_FLUX_1.0\ComfyUI\custom_nodes\ComfyUI-Easy-Use
   1.7 seconds: L:\ComfyUI_FLUX_1.0\ComfyUI\custom_nodes\ComfyUI-Manager

Total VRAM 24575 MB, total RAM 65467 MB
pytorch version: 2.5.1+cu124
Set vram state to: NORMAL_VRAM
Device: cuda:0 NVIDIA GeForce RTX 3090 : cudaMallocAsync
Using pytorch cross attention
[Prompt Server] web root: L:\ComfyUI_FLUX_1.0\ComfyUI\web
[ComfyUI-Easy-Use] server: v1.2.5 Loaded
[ComfyUI-Easy-Use] web root: L:\ComfyUI_FLUX_1.0\ComfyUI\custom_nodes\ComfyUI-Easy-Use\web_version/v2 Loaded
### Loading: ComfyUI-Impact-Pack (V7.11.3)
### Loading: ComfyUI-Impact-Pack (Subpack: V0.8)
[WARN] ComfyUI-Impact-Pack: custom_wildcards path not found: L:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Impact-Pack\custom_wildcards. Using default path.
[Impact Pack] Wildcards loading done.
Total VRAM 24575 MB, total RAM 65467 MB
pytorch version: 2.5.1+cu124
Set vram state to: NORMAL_VRAM
Device: cuda:0 NVIDIA GeForce RTX 3090 : cudaMallocAsync
### Loading: ComfyUI-Manager (V2.51.9)
### ComfyUI Revision: 2835 [d9f90965] | Released on '2024-11-17'
L:\ComfyUI_FLUX_1.0
############################################
L:\ComfyUI_FLUX_1.0\custom_nodes\ComfyUI-Universal-Styler\CSV
############################################
[]
############################################
[ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/alter-list.json[ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/model-list.json[ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/github-stats.json
[ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/extension-node-map.json
[ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/custom-node-list.json

[rgthree-comfy] Loaded 42 magnificent nodes. 🎉

WAS Node Suite: OpenCV Python FFMPEG support is enabled
WAS Node Suite Warning: `ffmpeg_bin_path` is not set in `L:\ComfyUI_FLUX_1.0\ComfyUI\custom_nodes\was-node-suite-comfyui\was_suite_config.json` config file. Will attempt to use system ffmpeg binaries if available.
WAS Node Suite: Finished. Loaded 218 nodes successfully.

        "Believe you can and you're halfway there." - Theodore Roosevelt

Import times for custom nodes:
   0.0 seconds: L:\ComfyUI_FLUX_1.0\ComfyUI\custom_nodes\websocket_image_save.py
   0.0 seconds: L:\ComfyUI_FLUX_1.0\ComfyUI\custom_nodes\ComfyUI-Miaoshouai-Tagger
   0.0 seconds: L:\ComfyUI_FLUX_1.0\ComfyUI\custom_nodes\ComfyUI-Universal-Styler
   0.0 seconds: L:\ComfyUI_FLUX_1.0\ComfyUI\custom_nodes\ComfyUI-Custom-Scripts
   0.0 seconds: L:\ComfyUI_FLUX_1.0\ComfyUI\custom_nodes\ComfyUI-GGUF
   0.0 seconds: L:\ComfyUI_FLUX_1.0\ComfyUI\custom_nodes\ComfyUI_essentials
   0.0 seconds: L:\ComfyUI_FLUX_1.0\ComfyUI\custom_nodes\rgthree-comfy
   0.0 seconds: L:\ComfyUI_FLUX_1.0\ComfyUI\custom_nodes\ComfyUI-SaveImageWithMetaData
   0.0 seconds: L:\ComfyUI_FLUX_1.0\ComfyUI\custom_nodes\OneButtonPrompt
   0.0 seconds: L:\ComfyUI_FLUX_1.0\ComfyUI\custom_nodes\ComfyUI-ImageMetadataExtension
   0.0 seconds: L:\ComfyUI_FLUX_1.0\ComfyUI\custom_nodes\ComfyUI-KJNodes
   0.1 seconds: L:\ComfyUI_FLUX_1.0\ComfyUI\custom_nodes\ComfyUI-Fluxpromptenhancer
   0.1 seconds: L:\ComfyUI_FLUX_1.0\ComfyUI\custom_nodes\KayTool
   0.3 seconds: L:\ComfyUI_FLUX_1.0\ComfyUI\custom_nodes\ComfyUI-Impact-Pack
   0.4 seconds: L:\ComfyUI_FLUX_1.0\ComfyUI\custom_nodes\ComfyUI-Manager
   0.5 seconds: L:\ComfyUI_FLUX_1.0\ComfyUI\custom_nodes\ComfyUI-Detail-Daemon
   0.7 seconds: L:\ComfyUI_FLUX_1.0\ComfyUI\custom_nodes\ComfyUI-Easy-Use
   1.0 seconds: L:\ComfyUI_FLUX_1.0\ComfyUI\custom_nodes\comfyui-dynamicprompts
   1.9 seconds: L:\ComfyUI_FLUX_1.0\ComfyUI\custom_nodes\was-node-suite-comfyui

Starting server

To see the GUI go to: http://<IP_ADDRESS>:8188
FETCH DATA from: L:\ComfyUI_FLUX_1.0\ComfyUI\custom_nodes\ComfyUI-Manager\extension-node-map.json [DONE]
got prompt
Using pytorch attention in VAE
Using pytorch attention in VAE
New prompt: Professional photography, 35mm film, sharp lens, full shot photo of a man. Daytime outdoor location. Professional lighting and color grading for a clean sharp film look. Extreme details like in a professional macro photography.
L:\ComfyUI_FLUX_1.0\ComfyUI\custom_nodes\ComfyUI-GGUF\nodes.py:79: UserWarning: The given NumPy array is not writable, and PyTorch does not support non-writable tensors. This means writing to this tensor will result in undefined behavior. You may want to copy the array to protect its data or make it writable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\torch\csrc\utils\tensor_numpy.cpp:212.)
  torch_tensor = torch.from_numpy(tensor.data) # mmap

ggml_sd_loader:
 12                            161
 0                              50
 13                              7
 14                              1

ggml_sd_loader:
 0                             471
 12                            304
 1                               5
model weight dtype torch.bfloat16, manual cast: None
model_type FLUX
Number of sigmas: 17, Number of schedule steps: 16
Adjusting sigma at index 0 with multiplier 0.0
Adjusting sigma at index 1 with multiplier 0.0
Adjusting sigma at index 2 with multiplier 0.020000000000000004
Adjusting sigma at index 3 with multiplier 0.020000000000000004
Adjusting sigma at index 4 with multiplier 0.020000000000000004
Adjusting sigma at index 5 with multiplier 0.020000000000000004
Adjusting sigma at index 6 with multiplier 0.020000000000000004
Adjusting sigma at index 7 with multiplier 0.020000000000000004
Adjusting sigma at index 8 with multiplier 0.020000000000000004
Adjusting sigma at index 9 with multiplier 0.020000000000000004
Adjusting sigma at index 10 with multiplier 0.020000000000000004
Adjusting sigma at index 11 with multiplier 0.020000000000000004
Adjusting sigma at index 12 with multiplier 0.020000000000000004
Adjusting sigma at index 13 with multiplier 0.020000000000000004
Adjusting sigma at index 14 with multiplier 0.020000000000000004
Adjusting sigma at index 15 with multiplier 0.0
Requested to load FluxClipModel_
Loading 1 new model
loaded completely 0.0 2847.30810546875 True
Requested to load Flux
Loading 1 new model
loaded completely 0.0 6490.570556640625 True
100%|██████████████████████████████████████████████████████████████████████████████████| 16/16 [00:40<00:00,  2.54s/it]
Requested to load AutoencodingEngine
Loading 1 new model
loaded completely 0.0<PHONE_NUMBER>7282715 True
Prompt executed in 63.77 seconds
FETCH DATA from: L:\ComfyUI_FLUX_1.0\ComfyUI\custom_nodes\ComfyUI-Manager\.cache\1742899825_extension-node-map.json [DONE]
FETCH DATA from: L:\ComfyUI_FLUX_1.0\ComfyUI\custom_nodes\ComfyUI-Manager\.cache\1514988643_custom-node-list.json [DONE]FETCH DATA from: L:\ComfyUI_FLUX_1.0\ComfyUI\custom_nodes\ComfyUI-Manager\.cache\746607195_github-stats.json [DONE]

Please provide me with a screenshot showing the connection setup of your nodes.

Hello,
Thank you for your feedback on the Custom Save Image node. I'm sorry to hear that it's not working as expected. Let's troubleshoot the issue:
File Permissions : Make sure that the user running ComfyUI has the necessary permissions to write to the output directory. Check if the directory L:\ComfyUI_FLUX_1.0\output\Custom_Save_Image exists and that you can write to it. If not, try creating it manually or ensure your account has write access.
Check for Errors : Look for any error messages in the console or logs (L:\ComfyUI_FLUX_1.0\comfyui.log) that might indicate why the images aren't being saved. There might be an exception or error occurring during the save process.
Path Issues : Ensure that the paths to ICC profiles (resources/sRGB Profile.icc and resources/AdobeRGB1998.icc) are correct relative to where ComfyUI is installed. If these files are missing or not in the expected location, the conversion to Adobe RGB might fail.
Node Configuration : Verify that your node is properly configured in the workflow. Check that the inputs for the images, filename_prefix, format, etc., are correctly connected and populated.
ComfyUI Version : You're using the latest commit, which is good. However, ensure there haven't been any recent changes or known issues with custom nodes in this version.
Python and Dependencies : Confirm that all necessary dependencies are installed correctly, especially Pillow for image processing. You can try reinstalling ComfyUI or updating its dependencies to ensure everything is up-to-date.
Testing with Different Inputs : Try saving images with different parameters, like changing the format to JPG or altering the filename prefix to see if it makes any difference.
If you still face issues after checking these points, please provide:
Any error messages or logs related to the Custom Save Image node.
A screenshot or description of your workflow configuration involving this node.
The content of the comfyui.log

Hello,
Thank you for your feedback on the Custom Save Image node. I'm sorry to hear that it's not working as expected. Let's troubleshoot the issue:

File Permissions: Make sure that the user running ComfyUI has the necessary permissions to write to the output directory. Check if the directory L:\ComfyUI_FLUX_1.0\output\Custom_Save_Image exists and that you can write to it. If not, try creating it manually or ensure your account has write access.

Check for Errors: Look for any error messages in the console or logs (L:\ComfyUI_FLUX_1.0\comfyui.log) that might indicate why the images aren't being saved. There might be an exception or error occurring during the save process.

Path Issues: Ensure that the paths to ICC profiles (resources/sRGB Profile.icc and resources/AdobeRGB1998.icc) are correct relative to where ComfyUI is installed. If these files are missing or not in the expected location, the conversion to Adobe RGB might fail.

Node Configuration: Verify that your node is properly configured in the workflow. Check that the inputs for the images, filename_prefix, format, etc., are correctly connected and populated.

ComfyUI Version: You're using the latest commit, which is good. However, ensure there haven't been any recent changes or known issues with custom nodes in this version.

Python and Dependencies: Confirm that all necessary dependencies are installed correctly, especially Pillow for image processing. You can try reinstalling ComfyUI or updating its dependencies to ensure everything is up-to-date.

Testing with Different Inputs: Try saving images with different parameters, like changing the format to JPG or altering the filename prefix to see if it makes any difference.

If you still face issues after checking these points, please provide:

Any error messages or logs related to the Custom Save Image node.
A screenshot or description of your workflow configuration involving this node.
The content of the comfyui.log file after attempting to save an image.

This information will help in further diagnosing the problem. Thank you for your patience, and I look forward to helping you resolve this issue.

I did some further checking, apparently the problem I have is related to the preview image not being displayed, I just now realized that the save works.
So the problem is that it saves without giving me the preview on Comfy.
I'm sorry for that.

Custom Save Image.json

I finally figured out what's going on, the saving of images happens in the main Comfy Protable folder. In patica it does this:
C:\comfyUI_portable\output\Custom_Save_Image
Instead of the way it should be:
C:\comfyUI_portable\ComfyUI\output\Custom_Save_Image
I also noticed that you cannot use dynamic tags to save, such as: %date:yyyy-MM-dd%
In this case the error is: [Errno 22] Invalid argument.

Thank you very much for your feedback. I will take the time to test and iterate on the version. In the meantime, you can use the Preview Image node as a temporary workaround for the preview issue. If the save function is working correctly (please verify if the image parameters are set as expected), you can use it.

Hey friend, after testing, image previews are working properly in the latest version of ComfyUI on both Windows and Linux. Let me know if you can give it another test to confirm everything runs smoothly on your end.


New Section Error in google API!

According to the install and Travis fix in PR #14, the Google API is missing a key in the dictionary, which needs to be fixed.
Otherwise, I think the Travis tests will run now.

Need to move across to pytest and actions.


Supervised test of skipping bad samples

Following https://github.com/DiamondLightSource/mx-bluesky/issues/470 we should now be in a position where we can run a full collection in hyperion that will skip collecting on samples that do not diffract. We should try and prove this before the end of the run.
Things to test:

We can run a single collection with and w/o skipping
We can run a native (chi changing) collection with and w/o skipping
We can run a S+C with and w/o skipping
We can run at different energies (https://github.com/DiamondLightSource/mx-bluesky/issues/277)

Acceptance Criteria

The above is tested on test pins to @npat113's satisfaction
We run for some length of time (~.5 days) on user samples with the above whilst hyperion is being supervised
Any problems are spun into new issues

For S+C removing the wait for analysis results will make this better, discuss with irakli

Began testing on 2024-12-05 but ran into some issues notably #703 but see also various other things from the same testing session #697


[WIP] Add azure cloud provisioner for e2e test

Draft PR for azure Provisioner. Not yet up for review need to test and add some more changes.

#696 has slightly updated the provisioner related code. Could you rebase your patch?
https://github.com/confidential-containers/cloud-api-adaptor/pull/696/files#diff-627592112a06de6bfd545b8585d174910ddfc1b6b3fb03b55b480805121d258f

#696 has slightly updated the provisioner related code. Could you rebase your patch?
https://github.com/confidential-containers/cloud-api-adaptor/pull/696/files#diff-627592112a06de6bfd545b8585d174910ddfc1b6b3fb03b55b480805121d258f

Sure, will handle it as part of next patch. Thanks

Tested as below
make BUILTIN_CLOUD_PROVIDERS="azure" build

export CLOUD_PROVIDER="azure"
export TEST_E2E_PROVISION=yes
export TEST_E2E_PROVISION_FILE=test/provisioner/provision_azure.properties

root@kartik-ThinkPad-X1-Titanium-Gen-1:/home/kartik/gitwork/gitworknew/cloud-api-adaptor# ./cluster-provisioner -action=provision
DEBU[0000] &{ResourceGroup:<nil> CloudProvider:azure SubscriptionID:d1aa957b-94f5-49ef-b29a-0178c58a7132 ClientID:e123c35e-761e-4ab9-9861-1e32ce778c2e ClientSecret:pTK8Q~SwMlvR5zplNljpHeiDMFuhe52CHQC91bQa TenantID:72f988bf-86f1-41af-91ab-2d7cd011db47 ResourceGroupName:e2e-test-rg-final ClusterName:e2e-test-cluster-final Location:eastus SshKeyID:~/.ssh/id_rsa SubnetName:e2e-test-cluster-final_subnet VnetName:e2e-test-cluster-final_vnet SubnetID: ImageID: InstanceSize:Standard_D8as_v5 NodeName:caaaks OsType:Ubuntu ResourceGroupClient:<nil> ManagedclusterClient:<nil> ManagedvnetClient:<nil> ManagedsubnetClient:<nil> ManagedaksClient:<nil> ManagedvmClient:<nil>}
INFO[0000] Creating Resource group e2e-test-rg-final.
INFO[0018] Successfully Created Resource group e2e-test-rg-final.
INFO[0018] waiting for the Resource group to be available before creating vnet...
INFO[0078] Creating  vnet e2e-test-cluster-final_vnet in resource group e2e-test-rg-final with addressPrefix <IP_ADDRESS>/16 subnetAddressPrefix <IP_ADDRESS>/24.
INFO[0085] Successfully Created vnet e2e-test-cluster-final_vnet in resource group e2e-test-rg-final.
INFO[0085] Creating cluster e2e-test-cluster-final.
INFO[0096] Waiting for cluster e2e-test-cluster-final to be available.
INFO[0344] Cluster e2e-test-cluster-final is available.
INFO[0344] Sync cluster kubeconfig with current config context

Tested as follows
make BUILTIN_CLOUD_PROVIDERS="azure" build

export CLOUD_PROVIDER="azure"
export TEST_E2E_PROVISION_FILE="test/provisioner/provision_azure.properties"
export TEST_E2E_PROVISION="yes"

INFO[0000] Creating Resource group e2e-test-rg-final.
INFO[0018] Successfully Created Resource group e2e-test-rg-final.
INFO[0018] waiting for the Resource group to be available before creating vnet...
INFO[0078] Creating  vnet e2e-test-cluster-final_vnet in resource group e2e-test-rg-final with addressPrefix <IP_ADDRESS>/16 subnetAddressPrefix <IP_ADDRESS>/24.
INFO[0085] Successfully Created vnet e2e-test-cluster-final_vnet in resource group e2e-test-rg-final.
INFO[0085] Creating cluster e2e-test-cluster-final.
INFO[0096] Waiting for cluster e2e-test-cluster-final to be available.
INFO[0344] Cluster e2e-test-cluster-final is available.
INFO[0344] Sync cluster kubeconfig with current config context

Thanks @kartikjoshi21 for adding this, I added several comments but I think several are enhancements which need not be addressed in this PR.

Thanks @kartikjoshi21 for adding this, I added several comments but I think several are enhancements which need not be addressed in this PR.

@huoqifeng Addressed all the comments in this PR Itself. Thanks a lot for review.

Thanks @kartikjoshi21 . Let's get this merged first then I adjust #668 . What do you think?

Sure, that will be good too, if this looks good to others too we can merge this one. @huoqifeng can you please take a look again and approve if looks good, Thanks

Tested this one functionality wise:
✔ ~/cloud-api-adaptor [kartikjoshi21/azure-e2e-provisioner L|✚ 1…1]
$ ./cluster-provisioner -action=provision
INFO[0000] Creating Resource group e2e-test-rg.
INFO[0001] Successfully Created Resource group e2e-test-rg.
INFO[0001] waiting for the Resource group to be available before creating vnet...
INFO[0061] Creating  vnet e2e-test-cluster_vnet in resource group e2e-test-rg with addressPrefix <IP_ADDRESS>/16 subnetAddressPrefix <IP_ADDRESS>/24.
INFO[0063] Successfully Created vnet e2e-test-cluster_vnet in resource group e2e-test-rg.
INFO[0063] Creating cluster e2e-test-cluster.
INFO[0077] Waiting for cluster e2e-test-cluster to be available.
INFO[0322] Cluster e2e-test-cluster is available.
INFO[0322] Sync cluster kubeconfig with current config context

✔ ~/cloud-api-adaptor [kartikjoshi21/azure-e2e-provisioner L|✚ 1…1]
$ make test-e2e
go test -v -tags=azure -timeout 20m -count=1 ./test/e2e
open test/provisioner/provision_azure.properties: no such file or directory
FAIL    github.com/confidential-containers/cloud-api-adaptor/test/e2e   0.012s
FAIL
make: *** [Makefile:88: test-e2e] Error 1

✔ ~/cloud-api-adaptor [kartikjoshi21/azure-e2e-provisioner L|✚ 1…1]
$ ll test/provisioner/provision_azure.properties
.rw-rw-r-- 525 surajssd 14 Mar 23:40 test/provisioner/provision_azure.properties

The cluster came up fine, but I could not run the e2e test suite.
Another problem, something is wrong with the cluster I am not able to use it:
➜  ~ kubectl get pods -A
Unable to connect to the server: dial tcp: lookup caa-suraj-caa-cvm0-d1aa95-vdw1je1q.hcp.eastus.azmk8s.io on <IP_ADDRESS>:53: no such host

I tried it on different machines just to be sure the DNS entry hasn't been made, not sure why.
The deprovisioning worked as well:
$ ./cluster-provisioner -action=deprovision
INFO[0000] Deleting Cluster e2e-test-cluster.
INFO[0001] Waiting for cluster e2e-test-cluster to be removed...
INFO[0306] Cluster e2e-test-cluster is removed.
INFO[0306] Deleting vnet e2e-test-cluster_vnet in resource group e2e-test-rg.
INFO[0307] Successfully Deleted vnet e2e-test-cluster_vnet in resource group e2e-test-rg.
INFO[0307] Deleting Resource group e2e-test-rg.
INFO[0308] Successfully Deleted Resource group e2e-test-rg.

Tested this one functionality wise:
✔ ~/cloud-api-adaptor [kartikjoshi21/azure-e2e-provisioner L|✚ 1…1]
$ ./cluster-provisioner -action=provision
INFO[0000] Creating Resource group e2e-test-rg.
INFO[0001] Successfully Created Resource group e2e-test-rg.
INFO[0001] waiting for the Resource group to be available before creating vnet...
INFO[0061] Creating  vnet e2e-test-cluster_vnet in resource group e2e-test-rg with addressPrefix <IP_ADDRESS>/16 subnetAddressPrefix <IP_ADDRESS>/24.
INFO[0063] Successfully Created vnet e2e-test-cluster_vnet in resource group e2e-test-rg.
INFO[0063] Creating cluster e2e-test-cluster.
INFO[0077] Waiting for cluster e2e-test-cluster to be available.
INFO[0322] Cluster e2e-test-cluster is available.
INFO[0322] Sync cluster kubeconfig with current config context

✔ ~/cloud-api-adaptor [kartikjoshi21/azure-e2e-provisioner L|✚ 1…1]
$ make test-e2e
go test -v -tags=azure -timeout 20m -count=1 ./test/e2e
open test/provisioner/provision_azure.properties: no such file or directory
FAIL    github.com/confidential-containers/cloud-api-adaptor/test/e2e   0.012s
FAIL
make: *** [Makefile:88: test-e2e] Error 1

✔ ~/cloud-api-adaptor [kartikjoshi21/azure-e2e-provisioner L|✚ 1…1]
$ ll test/provisioner/provision_azure.properties
.rw-rw-r-- 525 surajssd 14 Mar 23:40 test/provisioner/provision_azure.properties

The cluster came up fine, but I could not run the e2e test suite.
Another problem, something is wrong with the cluster I am not able to use it:
➜  ~ kubectl get pods -A
Unable to connect to the server: dial tcp: lookup caa-suraj-caa-cvm0-d1aa95-vdw1je1q.hcp.eastus.azmk8s.io on <IP_ADDRESS>:53: no such host

I tried it on different machines just to be sure the DNS entry hasn't been made, not sure why.
The deprovisioning worked as well:
$ ./cluster-provisioner -action=deprovision
INFO[0000] Deleting Cluster e2e-test-cluster.
INFO[0001] Waiting for cluster e2e-test-cluster to be removed...
INFO[0306] Cluster e2e-test-cluster is removed.
INFO[0306] Deleting vnet e2e-test-cluster_vnet in resource group e2e-test-rg.
INFO[0307] Successfully Deleted vnet e2e-test-cluster_vnet in resource group e2e-test-rg.
INFO[0307] Deleting Resource group e2e-test-rg.
INFO[0308] Successfully Deleted Resource group e2e-test-rg.

@surajssd yes this PR is supposed to get provisioner working, other than that I will submit a new PR for e2e test to work with this. It requires install overlay side things as well.

@kartikjoshi21 https://github.com/confidential-containers/cloud-api-adaptor/pull/668 just got approved but I didn't merge it yet, if I do so then this PR will need changes. And if we merge this one first, the other way around :)
Mind if we get 668 in first as it is touches the core of the framework and others providers implementations depends on it? I will assist you on any needed changes on this PR.

@kartikjoshi21 #668 just got approved but I didn't merge it yet, if I do so then this PR will need changes. And if we merge this one first, the other way around :)
Mind if we get 668 in first as it is touches the core of the framework and others providers implementations depends on it? I will assist you on any needed changes on this PR.

@wainersm If 668 is ready lets merge it i will make changes to this PR accordingly. Thanks !!

668 is merged now.

The deprovisioning failed for me, twice:
$ ./cluster-provisioner -action=deprovision
INFO[0000] Deleting Cluster e2e-test-cluster.
INFO[0001] Waiting for cluster e2e-test-cluster to be removed...
ERRO[0302] failed to get cluster: containerservice.ManagedClustersClient#Get: Failure responding to request: StatusCode=404 -- Original Error: autorest/azure: Service returned an error. Status=404 Code="ResourceNotFound" Message="The Resource 'Microsoft.ContainerService/managedClusters/e2e-test-cluster' under resource group 'e2e-test-rg' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"
waiting for cluster to be deleted failed to wait for condition: failed to get cluster: containerservice.ManagedClustersClient#Get: Failure responding to request: StatusCode=404 -- Original Error: autorest/azure: Service returned an error. Status=404 Code="ResourceNotFound" Message="The Resource 'Microsoft.ContainerService/managedClusters/e2e-test-cluster' under resource group 'e2e-test-rg' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"

I think we need to add some form of cleanup, this allows leftover resources to stay back. Aggressively try to delete the resource group a couple of times.

The kubeconfig isn't configured properly. To test this, I deleted the kubeconfig file provisioned the cluster and still no kubeconfig file was created.
$ rm -rf ~/.kube/config

$ ./cluster-provisioner -action=provision
INFO[0000] Creating Resource group e2e-test-rg1.
INFO[0006] Successfully Created Resource group e2e-test-rg1.
INFO[0006] waiting for the Resource group to be available before creating vnet...
INFO[0066] Creating  vnet e2e-test-cluster_vnet in resource group e2e-test-rg1 with addressPrefix <IP_ADDRESS>/16 subnetAddressPrefix <IP_ADDRESS>/24.
INFO[0069] Successfully Created vnet e2e-test-cluster_vnet in resource group e2e-test-rg1.
INFO[0069] Creating cluster e2e-test-cluster.
INFO[0081] Waiting for cluster e2e-test-cluster to be available.
INFO[0323] Sync cluster kubeconfig with current config context

$ ls ~/.kube/config
"/home/surajaz/.kube/config": No such file or directory (os error 2)

The deprovisioning failed for me, twice:
$ ./cluster-provisioner -action=deprovision
INFO[0000] Deleting Cluster e2e-test-cluster.
INFO[0001] Waiting for cluster e2e-test-cluster to be removed...
ERRO[0302] failed to get cluster: containerservice.ManagedClustersClient#Get: Failure responding to request: StatusCode=404 -- Original Error: autorest/azure: Service returned an error. Status=404 Code="ResourceNotFound" Message="The Resource 'Microsoft.ContainerService/managedClusters/e2e-test-cluster' under resource group 'e2e-test-rg' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"
waiting for cluster to be deleted failed to wait for condition: failed to get cluster: containerservice.ManagedClustersClient#Get: Failure responding to request: StatusCode=404 -- Original Error: autorest/azure: Service returned an error. Status=404 Code="ResourceNotFound" Message="The Resource 'Microsoft.ContainerService/managedClusters/e2e-test-cluster' under resource group 'e2e-test-rg' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"

I think we need to add some form of cleanup, this allows leftover resources to stay back. Aggressively try to delete the resource group a couple of times.
The kubeconfig isn't configured properly. To test this, I deleted the kubeconfig file provisioned the cluster and still no kubeconfig file was created.
$ rm -rf ~/.kube/config

$ ./cluster-provisioner -action=provision
INFO[0000] Creating Resource group e2e-test-rg1.
INFO[0006] Successfully Created Resource group e2e-test-rg1.
INFO[0006] waiting for the Resource group to be available before creating vnet...
INFO[0066] Creating  vnet e2e-test-cluster_vnet in resource group e2e-test-rg1 with addressPrefix <IP_ADDRESS>/16 subnetAddressPrefix <IP_ADDRESS>/24.
INFO[0069] Successfully Created vnet e2e-test-cluster_vnet in resource group e2e-test-rg1.
INFO[0069] Creating cluster e2e-test-cluster.
INFO[0081] Waiting for cluster e2e-test-cluster to be available.
INFO[0323] Sync cluster kubeconfig with current config context

$ ls ~/.kube/config
"/home/surajaz/.kube/config": No such file or directory (os error 2)

@surajssd yes there was one status code which was not working fine. Resolved all the comments
root@kartik-ThinkPad-X1-Titanium-Gen-1:/home/kartik/gitwork/gitworknew/cloud-api-adaptor# ./cluster-provisioner -action=provision
INFO[0000] Creating Resource group e2e-test-rg.
INFO[0003] Successfully Created Resource group e2e-test-rg.
INFO[0003] waiting for the Resource group to be available before creating vnet...
INFO[0063] Creating  vnet e2e-test-cluster_vnet in resource group e2e-test-rg with addressPrefix <IP_ADDRESS>/16 subnetAddressPrefix <IP_ADDRESS>/24.
INFO[0069] Successfully Created vnet e2e-test-cluster_vnet in resource group e2e-test-rg.
INFO[0069] Creating cluster e2e-test-cluster.
INFO[0081] Waiting for cluster e2e-test-cluster to be available.
DEBU[0081] isAzureClusterReady()
DEBU[0143] isAzureClusterReady()
DEBU[0203] isAzureClusterReady()
DEBU[0263] isAzureClusterReady()
DEBU[0323] isAzureClusterReady()
INFO[0325] Sync cluster kubeconfig with current config context
root@kartik-ThinkPad-X1-Titanium-Gen-1:/home/kartik/gitwork/gitworknew/cloud-api-adaptor# ls -lrt $HOME/.kube/config
-rw------- 1 root docker 9728 Mar 16 15:42 /root/.kube/config
root@kartik-ThinkPad-X1-Titanium-Gen-1:/home/kartik/gitwork/gitworknew/cloud-api-adaptor# date
Thursday 16 March 2023 03:43:45 PM IST

root@kartik-ThinkPad-X1-Titanium-Gen-1:/home/kartik/gitwork/gitworknew/cloud-api-adaptor# ./cluster-provisioner -action=deprovision
INFO[0000] Deleting Cluster e2e-test-cluster.
INFO[0013] Waiting for cluster e2e-test-cluster to be removed...
DEBU[0013] isClusterDeleted()
DEBU[0075] isClusterDeleted()
DEBU[0135] isClusterDeleted()
DEBU[0195] isClusterDeleted()
DEBU[0255] isClusterDeleted()
DEBU[0315] isClusterDeleted()
INFO[0315] Deleting Resource group e2e-test-rg.
INFO[0321] Successfully Deleted Resource group e2e-test-rg.

Provision and deprovision works fine. Regarding running e2e test that will be taken care of in next PR, which will consist of adding install overlay part. Thanks

cluster-provisioner

Thankyou @surajssd for review, made all the mentioned changes.


WIP: Add msg parser using regex/ generate listener with jinja

This improves the msg parser and so we  can scan topics etc easier.
Closes #https://github.com/PX4/Firmware/issues/6607
Tasks

[x] Add parser
[ ] Add jinja2 template for listener.

What about combining this with the existing scripting that already parses the msg files? It seems like the listener code could be generated during the same stage, but placed in a separate module for optional compilation.

Nice! What is the status of the jinja2 template for the listener (open TODO in your list)?

@LorenzMeier The listener is about half way done. I set this up because @mhkabir needed it and I was hoping he would have time to finish it so he could learn the jinja syntax. @mhkabir let me know if you don't have time and I will finish it. Most of the hard part is done, just needs polishing and to do a diff on the output to make sure it matches the old generation.

@jgoppert I don't have the bandwidth to do this right now. Not for 2-3 weeks atleast. Would be great if you could pick it up.

@jgoppert what do you think about generating the listener stubs as part of the existing msg generation? Then it'll line up with the virtual topics. Only the listener module will build the stubs.

Closing as stale. This should be combined with the core message generation.


Feature/market component

I wrote down my and market component for your request.

Vercel check is not successful. Maybe spelling error.


Reverse one-hot logic is now updated to output unique labels

Fixes #341
Proposed Changes

updating reverse one-hot logic

Checklist

[x] I have read the CONTRIBUTING guide
[x] My PR is based from the current GaNDLF master
[x] Non-breaking change (would not break existing functionality): please provide as many details as possible for any breaking change
[x] Function/class source code documentation added/updated
[x] Code has been blacked for style consistency
[x] If applicable, version information has been updated in GANDLF/version.py
[x] If adding a git submodule, add to list of exceptions for black styling in pyproject.toml file
[x] Usage documentation has been updated, if appropriate
[x] History has been updated, if appropriate
[x] Tests added or modified to cover the changes; if coverage is reduced, please give explanation

Cover coverage is reducing because the number of lines of code has gone down. 🚀


Documentation on how to switch between estimators

Done!

URL ?

@nicolaerosia  https://github.com/PX4/Devguide/pull/136

@tumbili thanks!


Moved gmock to Debug section and added LLDB and uniassert

Please make an individual pull request for each suggestion.


Upload to maven central

Have this project uploaded into maven central for easy use.

Maybe we are not talking about the same maven, but I have already uploaded to maven through sonatype,
which you can find here or here.
So you can then easily add Neureka as dependency in the config of your build system of choice:
<dependency>
  <groupId>com.github.gleethos</groupId>
  <artifactId>neureka</artifactId>
  <version>1.0.0</version>
</dependency>

or Gradle
implementation 'com.github.gleethos:neureka:1.0.0'

or Groovy Grape
@Grapes(
  @Grab(group='com.github.gleethos', module='neureka', version='1.0.0')
)

...etc

I found it.


OME teams: add schema.org markup

This PR explores options to improve the structure of the teams page using the schema.org markup.
The YAML front matter is converted into the equivalent schema.org representation (using a combination of https://schema.org/Organization and https://schema.org/Person). Some markup is also included to the standard head.html to markup the structured data as JSON-LD and make it available via <script>
Three options are investigated:

303582a keeps the structured metadata as front matter but includes the relevant schema.org markup
370bd4c migrates the structured metadata as a JSON data file. The new schemadotorg page variable now becomes of pointer to the relevant data file
66daf3b is an alternative to 370bd4c where the structure metadata is stored in YAML format

Looks like something is wrong in the header syntax.

Looks like something is wrong in the header syntax.

Ha thanks, I got tripped by local caching using my browser view-source. The last commit should try to unwrap the markup from the YAML file and convert it into JSON. Locally _site/teams/index.html  now contains
        <script type="application/ld+json">

          {"@context":"https://schema.org","@type":"Organization","member":[{"@type":"Person","givenName":"Jason","familyName":"Swedlow","jobTitle":"Co-Founder","description":"Jason Swedlow earned a BA in Chemistry from Brandeis University in 1982. He performed his PhD in Biophysics with Profs D. A. Agard and J. W. Sedat, finishing in 1994. Dr Swedlow was a postdoctoral fellow at UCSF and then Harvard Medical School from 1994 and 1998, supported by a Damon Runyon Walter Winchell Cancer Research Fund Fellowship from 1995 to 1997. In 1998, Dr Swedlow established his own laboratory at the Wellcome Trust Biocentre, University of Dundee, Scotland as a Principal Investigator and Wellcome Trust Career Development Fellow. He was awarded a Wellcome Trust Senior Fellowship in 2002...

:+1: https://search.google.com/structured-data/testing-tool/u/0/#url=https%3A%2F%2Fsnoopycrimecop.github.io%2Fwww.openmicroscopy.org%2Fteams%2F (deprecated) gives no errors or warnings.

Is there any way to access the json files directly or do they disappear in jekyll?

I agree that JSON -> Hash -> JSON feels unnecessary. My searches for trying access the raw content of the data files  have been unsuccessful so far.


installation issue-ImportError: DLL load failed while importing _pywrap_tensorflow_internal: The specified module could not be found.

Please make sure that this is a build/installation issue. As per our GitHub Policy, we only address code/doc bugs, performance issues, feature requests and build/installation issues on GitHub. tag:build_template
System information

OS Platform and Distribution (e.g., Linux Ubuntu 16.04):windows 10
Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device:NO
TensorFlow installed from (source or binary):source
TensorFlow version:2.0
Python version:3.8.2
Installed using virtualenv? pip? conda?:pip
Bazel version (if compiling from source):
GCC/Compiler version (if compiling from source):
CUDA/cuDNN version:
GPU model and memory:

Describe the problem
Provide the exact sequence of commands / steps that you executed before running into the problem
ImportError: Traceback (most recent call last):
File "C:\Users\Abhishek Sharma\anaconda3\envs\tfp3.8\lib\site-packages\tensorflow\python\pywrap_tensorflow.py", line 58, in
from tensorflow.python.pywrap_tensorflow_internal import *
File "C:\Users\Abhishek Sharma\anaconda3\envs\tfp3.8\lib\site-packages\tensorflow\python\pywrap_tensorflow_internal.py", line 28, in
_pywrap_tensorflow_internal = swig_import_helper()
File "C:\Users\Abhishek Sharma\anaconda3\envs\tfp3.8\lib\site-packages\tensorflow\python\pywrap_tensorflow_internal.py", line 24, in swig_import_helper
_mod = imp.load_module('_pywrap_tensorflow_internal', fp, pathname, description)
File "C:\Users\Abhishek Sharma\anaconda3\envs\tfp3.8\lib\imp.py", line 242, in load_module
return load_dynamic(name, filename, file)
File "C:\Users\Abhishek Sharma\anaconda3\envs\tfp3.8\lib\imp.py", line 342, in load_dynamic
return _load(spec)
ImportError: DLL load failed while importing _pywrap_tensorflow_internal: The specified module could not be found.
Failed to load the native TensorFlow runtime.
See https://www.tensorflow.org/install/errors
for some common reasons and solutions.  Include the entire stack trace
above this error message when asking for help.
Any other info / logs
Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached.

@abhisharma048 ,
#36167 (comment)
#36167 #36151 #36138 #36054 #36045 #36020 #36003 #35988 #35903 #35880 #35865

I have the same problem using PyCharm.
TensorFlow version 2.2.0rc3
I tried reinstall new CUDA driver version and  adding the CUDA, CUPTI, and cuDNN installation directories to the %PATH% environment variable.
I have Python version 3.8.2., maybe I have to install the previous version (3.7)?

@abhisharma048
is this still an issue

I have the same issue with all version > 2.0.0 (not with 2.0.0) for the prebuilt CPU package, python 3.7 and 3.8.

I have the same issue:

Windows 10 1909
Python 3.8.2
Tensorflow-GPU 2.2.0
NVIDIA CUDA Toolkit 10.2
cuDNN 10.2 for Windows 10 x64 v<IP_ADDRESS> (cudnn64_7.dll)

ImportError: DLL load failed while importing _pywrap_tensorflow_internal: The specified module could not be found.


[Snyk] Security upgrade ruby from 3.3.4-alpine3.19 to 3.3.5-alpine3.19

Snyk has created this PR to fix 0 vulnerabilities in the dockerfile dependencies of this project.
Keeping your Docker base image up-to-date means you’ll benefit from security fixes in the latest version of your chosen image.
Snyk changed the following file(s):

Dockerfile

We recommend upgrading to ruby:3.3.5-alpine3.19, as this image has only 0 known vulnerabilities. To do this, merge this pull request, then verify your application still works as expected.

[!IMPORTANT]

Check the changes in this PR to ensure they won't cause issues with your project.
Max score is 1000. Note that the real score may have changed since the PR was raised.
This PR was automatically created by Snyk using the credentials of a real user.

Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.
For more information:
🧐 View latest project report
📜 Customise PR templates
🛠 Adjust project settings
📚 Read about Snyk's upgrade logic

Learn how to fix vulnerabilities with free interactive lessons:
🦉 Learn about vulnerability in an interactive lesson of Snyk Learn.

AAAAAARGH HOW DO I MAKE IT STOP. I REALLY THOUGHT I HAD DISABLED EVERY SNYK SCAN


High contrast theme is ignored

VSCode Version:  1.45.1
OS Version: Windows 10.0.18363

Steps to Reproduce:

Enable High Contrast mode in Windows settings
Choose a "High Contrast White" theme
Visual studio code shows High contrast black theme, ignoring this setting

We only have a dark high contrast theme/


Support for alignment to Azure Verified Module specification

Is there an existing issue for this?

[X] I have searched the existing issues

Description
This is in support of a PR in branch avm-sa, which is tracking alignment to the AVM spec.
New or Affected Resource(s)/Data Source(s)
n/a
Potential Terraform Configuration
No response
References
No response

We understand you eagerness to help here but please leave this to us to continue the work.
There is no SLA but we will will release and publish 0.1.0 of the module when we are ready.

Merged #26


[Improvement] Image asset optimization

As discussed, optipng is no longer used because palettization hurts embeds on some social media platforms. However, the -nc flag is sufficient to make optipng leave palettization alone (that is, non-indexed images will not become indexed). As such, I recommend running all PNGs through optipng -o7 -strip=all -fix -nc.
The Museum of ZZT's logo is currently the largest asset on a new page load, taking up 130KB (or ~25% of the total bandwidth used! D:). As such, it should be special-cased: here's a palettized, quantized version I made which should look identical perceptually but is ~90KB smaller.

Functions that generate images on the site itself (file upload, a staff only tool that lists boards in a file, and a "use the current canvas" staff button for play online) now call optipng after generating the initial image. I've gotten in the habit of optimizing things on my end before upload for other static images used in articles and so on.