controller: upgrade engine automatically to default version

Reduce the amount of manual work users have to do when upgrading Longhorn by automatically upgrade engine images for volumes when they are ok to upgrade. E.g. when we can do engine live upgrade or when volume is in detaching state.
Engine image controller will continuously watch and upgrade volumes' engine image to default engine image
More detail about the implementation at https://github.com/longhorn/longhorn/pull/2172
longhorn/longhorn#2152

Need more manual testing and run E2E test

Need more manual testing and run E2E test

@shuo-wu

What if a user is using the customized/modified engine image? In this case, the version can be the same but the repository may be different. Or the image tag doesn't match the official tag partten.

We only consider the image's tag when comparing images. If the tags don't have valid semantic version format, we do nothing and thus this is a fail-safe approach.

we can simply consider the default engine image as the latest engine image

We allow user do deploy additional images in Longhorn UI. User might deploy newer images. In this case, we cannot constantly force the newer images back to the default image.
I am thinking about a failed-to-uprade case where user rollback Longhorn. In this case, the default image is not the newest one
This feature is design for end user who are not supposed to change Longhorn version (image tags). I think most users are doing that.
One side note, Rancher new UI automatically add 2 more digit into the tag, this doesn't change the correctness of this design.

@shuo-wu

What if a user is using the customized/modified engine image? In this case, the version can be the same but the repository may be different. Or the image tag doesn't match the official tag partten.

We only consider the image's tag when comparing images. If the tags don't have valid semantic version format, we do nothing and thus this is a fail-safe approach.

we can simply consider the default engine image as the latest engine image

We allow user do deploy additional images in Longhorn UI. User might deploy newer images. In this case, we cannot constantly force the newer images back to the default image.
I am thinking about a failed-to-uprade case where user rollback Longhorn. In this case, the default image is not the newest one
This feature is design for end user who are not supposed to change Longhorn version (image tags). I think most users are doing that.
One side note, Rancher new UI automatically add 2 more digit into the tag, this doesn't change the correctness of this design.

@PhanLe1010 I agree with @shuo-wu , I don't think it's necessary on insisting upgrading to a newer version.
Longhorn manager is operating on the assumption that the default engine image is compatible and works the best with the current version of the manager. So users shouldn't just deploy a new engine image by themselves. If they're sure this is the best image for the manager, they should change the default image when starting the manager. If they want to try out new images (which is very rare at the moment), it's easier to disable the automatic upgrade feature until the next formal release. Also, in that case, without disabling the auto-upgrade feature, the engine upgrade will be triggered for every newly created volume, which is a bit weird.
Always upgrade to the newer version have a problem with roll-back during fail to upgrade as well, since based on the assumption that we might roll back the volume with a newer image, it might still don't work since we don't know if the newer version of the engine works with the older version of manager at all. Though if the manager upgrade failed, I don't expect users are able to upgrade engines at all.
So I feel it's simpler to stick with always upgrade to the default version.

@PhanLe1010 I agree with @shuo-wu , I don't think it's necessary on insisting upgrading to a newer version.
Longhorn manager is operating on the assumption that the default engine image is compatible and works the best with the current version of the manager. So users shouldn't just deploy a new engine image by themselves. If they're sure this is the best image for the manager, they should change the default image when starting the manager. If they want to try out new images (which is very rare at the moment), it's easier to disable the automatic upgrade feature until the next formal release. Also, in that case, without disabling the auto-upgrade feature, the engine upgrade will be triggered for every newly created volume, which is a bit weird.
Always upgrade to the newer version have a problem with roll-back during fail to upgrade as well, since based on the assumption that we might roll back the volume with a newer image, it might still don't work since we don't know if the newer version of the engine works with the older version of manager at all. Though if the manager upgrade failed, I don't expect users are able to upgrade engines at all.
So I feel it's simpler to stick with always upgrade to the default version.

Thanks @yasker and @shuo-wu
I updated the code.
Running mall test again and the E2E core tests

Thanks @yasker and @shuo-wu
I updated the code.
Running mall test again and the E2E core tests

I am hitting an issue when updating manually updating engine to a new version and Longhorn immediately upgrade to default version. The latter upgrade is stuck with the error:
 [longhorn-manager-wg8cl] time="2021-01-20T02:31:34Z" level=debug msg="Volume old healthy replica count 0 doesn't match the desired replica count 3" accessMode=rwo controller=longhorn-volume frontend= node=phan-cluster-v40-worker-1 owner=phan-cluster-v40-worker-1 state=attached volume=dr

Investigating the issue.

I am hitting an issue when updating manually updating engine to a new version and Longhorn immediately upgrade to default version. The latter upgrade is stuck with the error:
 [longhorn-manager-wg8cl] time="2021-01-20T02:31:34Z" level=debug msg="Volume old healthy replica count 0 doesn't match the desired replica count 3" accessMode=rwo controller=longhorn-volume frontend= node=phan-cluster-v40-worker-1 owner=phan-cluster-v40-worker-1 state=attached volume=dr

Investigating the issue.

I fixed the above problem
Also disabled engine upgrade API if user turns on Automatically upgrade volumes' engine to the default engine image
Manual tests: passed
E2E core tests: running

I fixed the above problem
Also disabled engine upgrade API if user turns on Automatically upgrade volumes' engine to the default engine image
Manual tests: passed
E2E core tests: running

I will the confused that why the 2nd commit controller: only start upgrading engine if there is exist newly upgraded replica could prevent from starting the 2nd live upgrade before the 1st upgrade is done. IMO, you can avoid the case you mentioned by checking the following before upgrading r.spec.engineImage(this is the 1st step of the live upgrade):

The engine images in all replicas and engines spec are the same
The current images of the engines and replicas matches the engine image in spec.
No upgrade replica map in engine spec/status

I will the confused that why the 2nd commit controller: only start upgrading engine if there is exist newly upgraded replica could prevent from starting the 2nd live upgrade before the 1st upgrade is done. IMO, you can avoid the case you mentioned by checking the following before upgrading r.spec.engineImage(this is the 1st step of the live upgrade):

The engine images in all replicas and engines spec are the same
The current images of the engines and replicas matches the engine image in spec.
No upgrade replica map in engine spec/status

I will the confused that why the 2nd commit controller: only start upgrading engine if there is exist newly upgraded replica could prevent from starting the 2nd live upgrade before the 1st upgrade is done. IMO, you can avoid the case you mentioned by checking the following before upgrading r.spec.engineImage(this is the 1st step of the live upgrade):The engine images in all replicas and engines spec are the same
The current images of the engines and replicas matches the engine image in spec.
No upgrade replica map in engine spec/status

Will schedule a meeting with you to discuss

I will the confused that why the 2nd commit controller: only start upgrading engine if there is exist newly upgraded replica could prevent from starting the 2nd live upgrade before the 1st upgrade is done. IMO, you can avoid the case you mentioned by checking the following before upgrading r.spec.engineImage(this is the 1st step of the live upgrade):The engine images in all replicas and engines spec are the same
The current images of the engines and replicas matches the engine image in spec.
No upgrade replica map in engine spec/status

Will schedule a meeting with you to discuss

@shuo-wu Do you mean the chart in longhorn repo https://github.com/longhorn/longhorn/blob/master/chart/templates/default-setting.yaml ?
I thought we will update during the release process

@shuo-wu Do you mean the chart in longhorn repo https://github.com/longhorn/longhorn/blob/master/chart/templates/default-setting.yaml ?
I thought we will update during the release process

Yeah. I will check the chart before the release cut. But it would be better if we can update it once we implement a new feature.

Yeah. I will check the chart before the release cut. But it would be better if we can update it once we implement a new feature.

Yeah. I will check the chart before the release cut. But it would be better if we can update it once we implement a new feature.

Sure, I will create a PR into Longhorn repo

As for the YAML file of this repo, we need to update it in this PR.

It is already updated.

Yeah. I will check the chart before the release cut. But it would be better if we can update it once we implement a new feature.

Sure, I will create a PR into Longhorn repo

As for the YAML file of this repo, we need to update it in this PR.

It is already updated.


Updated analytical rule for anyalytical validation test

Required items, please complete
Change(s):

See guidance below

Reason for Change(s):

See guidance below

Version Updated:

Required only for Detections/Analytic Rule templates
See guidance below

Testing Completed:

See guidance below

Checked that the validations are passing and have addressed any issues that are present:

See guidance below

Guidance <- remove section before submitting

Before submitting this PR please ensure that you have read the following sections and filled out the changes, reason for change and testing complete sections:
Thank you for your contribution to the Microsoft Sentinel Github repo.

Details of the code changes in your submitted PR.  Providing descriptions for pull requests ensures there is context to changes being made and greatly enhances the code review process.  Providing associated Issues that this resolves also easily connects the reason.

Change(s):

Updated syntax for XYZ.yaml

Reason for Change(s):

New schema used for XYZ.yaml
Resolves ISSUE #1234

Version updated:

Yes
Detections/Analytic Rule templates are required to have the version updated

The code should have been tested in a Microsoft Sentinel environment that does not have any custom parsers, functions or tables, so that you validate no incorrect syntax and execution functions properly.  If your submission requires a custom parser or function, it must be submitted with the PR.

Testing Completed:

Yes/No/Need Help

Note: If updating a detection, you must update the version field.

Before the submission has been made, please look at running the KQL and Yaml Validation Checks locally.
https://github.com/Azure/Azure-Sentinel#run-kql-validation-locally

Checked that the validations are passing and have addressed any issues that are present:

Yes/No/Need Help

Note: Let us know if you have tried fixing the validation error and need help.

References:

Guidance for Detection checks
General contribution guidance
PR validation troubleshooting

Hi @mkchiliveri
Sorry for multiple comments. I just realized that there was an old PR #5138 related to AWS that was merged to the master recently, so all the files in the detections folder moved to AWS solution folder. Can you please make the respective changes in this folder instead of detection folder

Hi @mkchiliveri,
Please let me know if you need any clarity on the above request.
Thanks

Hi @mkchiliveri,
Can you please accommodate the changes requested above. Let me know if you need any help over there.
Thanks

Hi @mkchiliveri,
Can you please accommodate the changes requested above. Let me know if you need any help over there.
Thanks

Hi @mkchiliveri
Can you please help us by accommodating the requested changes, let me know if you need any help over there.
Thanks

Hi @mkchiliveri,
It would be great if you could provide any update on the above request.
Thanks

Hi @mkchiliveri,
Can you please provide update on the requested changes
Thanks

Hi @mkchiliveri,
Can you please address the suggested changes.
Thanks

Hi @mkchiliveri,
Can you please resolve the conflicts.
Thanks

Hi @mkchiliveri,
Can you please resolve conflicts.
Thanks

Hi @mkchiliveri,
It would be great if you could spare sometime on fixing the conflicts.
Thanks

Hi @mkchiliveri
We wanted to check on the status of PR #6393. PR is pending from more than 30 days. Let us know if any assistance is required for this PR. As per our standard operating procedures if no response is received in the next 7 business days, we will close this PR. Thank you for your cooperation.
Thanks

Hi @mkchiliveri,.
Can you please address the changes.
Thanks

Hi @mkchiliveri,
If you need some time to work on this, please move this PR to draft and resume once you get time.
Thanks

Hi @mkchiliveri Since we have not received response from last 7 days, we are closing your PR per our standard operating procedures. If you still need support for this issue you can re-open the PR at any time.
If you do re-open, we simply request that you ensure the PR has response to the last request. Thank you for your cooperation.


Add support for text-based block/item IDs

Should work for the most part, I tried to add all the itemIDs that didn't match the naming convention - didn't add any of the 1.8.x blocks though. The Get* functions could be changed into a string-based cached dictionary.

https://github.com/minecraft-dotnet/Substrate/pull/33 is a much better solution

Were your NBT changes based on real world data or pre-emptive? I haven't seen the name ids used yet.

They're definitely used in 1.8+ worlds.

Can you provide some links and/or data? And feel free to re-submit your nbt changes against the 2.x branch

Which bits are you referring to? Whatever I did I think you did better.


Documentation: Updating tutorials to use TensorFlow Datasets (TFDS).

TensorFlow Datasets (TFDS) is a collection of datasets ready-to-use with TensorFlow. Each dataset is defined as a tfds.core.DatasetBuilder, which encapsulates the logic to download the dataset and construct an input pipeline, as well as contains the dataset documentation (version, splits, number of examples, etc.).
The purpose of this issue is to migrate all tutorials to use TensorFlow Datasets, and to be compliant with the TF 2.0 API. Each migrated tutorial must be eager and distribution compatible, with tests, and all associated engineering artifacts.

I want to contribute.
Is this referring to imports like https://github.com/tensorflow/tensorflow/blob/226398bd70d6369dd97fa9cf4bee94af9a05f1d0/tensorflow/examples/tutorials/mnist/mnist_with_summaries.py#L33 where we would maybe want to import data from Tensorflow Datasets:
import tensorflow_datasets as tfds
ds_train, ds_test = tfds.load(name="mnist", split=["train", "test"])

Hi, @piyush-kgp! Thanks for the offer to help - we appreciate it! 😊
For more information on tf.data.Datasets, check here.
@rsepassi would be the best contact, and is working with @yashk2810 + several other members of the TensorFlow team to get tutorials ready for TF 2.0. Also make sure to follow progress on these two issues for TFDS support and testing: #31, #25367 .
Ryan, Yash - are there any tutorials on GitHub right now that Piyush could help migrate?

That’s great, thanks @piyush-kgp. TFDS is now TF 2.0-ready and can be used in the tutorials. @yashk2810 and his colleagues can coordinate work on the tutorials.


Is it able to run ML Kit in Xcode 8.3

I'm wondering if it's possible to run ML Kit in my Xcode 8.3 project? It's an enterprise application. Thanks,

Firebase dropped support for Xcode 8.3 earlier this summer when Apple started requiring Xcode 9 for building apps to publish to the App store.
I don't know of a reason that Xcode 8.3 wouldn't work right now - but it could break at any time in the future without warning.

@paulb777 Thanks. I migrated my code to Xcode 9 already.


[Requirement] Add python DLPack interface for HPS lookup

Is your feature request related to a problem? Please describe.
In HPS, for GNNs it would be great to have a python lookup in function which could take in indices on the GPU and return the gathered rows also on the GPU.
Describe the solution you'd like
It would be great to have a python function like taking in DLPack objects (https://dmlc.github.io/dlpack/latest/python_spec.html):
def lookup_from_dlpack(indices, out):
""" Gather the rows associated with the indices into the tensor out.
Parameters:
indices : DLPack capsule
  The input indices on the GPU to fetch the corresponding rows.
out : DLPack capsule
  The output memory location the lookup--should be of shape [indices.shape[0], embedding_size].

"""
The inputs don't have to be dlpack objects, but at least should be object convertible to dlpack (e.g., PyTorch Tensors, cuPY tensors, etc.).
Describe alternatives you've considered
The alternative is to create a second library to wrap the C++ interface of HPS and provide the above function there.
Additional context
This would be used both in training and inference of GNNs at large scale.

@yingcanw Thanks for the feedback. Let me give you some more background on how this would be used for GNNs. We would use this both for training and inference, as the input to a network is a subgraph and embeddings for the nodes and/or edges in that subgraph.
Typically, for a given mini-batch, we fetch the input features associated with the input nodes of our subgraph. The number of input nodes usually ranges from 100 thousand to 1 million, and the input dimension is ranges from 64 to 4096. This means the output of lookup would be in the range 64 MB to 16 GB (a few GB would probably be most common).
We also will have 1 python process per GPU. The GIL isn't usually an issue since most work is performed asynchronously, and python is just used to schedule it. The ability to pass in a stream-id when performing a lookup would be useful in allowing computational work to overlap with the lookup, but at this point is not required.

@nv-dlasalle Thanks for your detailed background info.  If my understanding is correct, then I think you will basically not use complex HPS deployment scenarios (just 1 python process per GPU due to the large input batch size), so there will be no issue like cuda initialization for multi-process/multi-thread on multi-GPU.
Therefore, through HPS supporting dlpack format tensors, it will indeed provide more convenient platform compatibility for offline inference with supporting dlpack capsule input . We will support the DLPack interface for HPS lookup in the next release.

The DLpack interface has supported in version 22.07


Use an UTF-8 locale to build the linux native image

Fixes #78

I'd like to email this to you and see if it really solves the problem.  If I email it from my mac, I think it will end up on a cloud somewhere at apple, and send you a like where you can download it.  So, if you give me an email address, I'll send it to you.  Thanks!

Ok, please send it to mynomoto [at] gmail.com
I did build it on the docker image that you use and tested it but I have no problem testing the one you built.


feat: add accountid and arbitrary additional info in usagedata

Description of changes
Previously, we sent accountId along with all usageData reports. This was reverted (https://github.com/aws-amplify/amplify-cli/pull/7092) in an effort to mitigate https://github.com/aws-amplify/amplify-cli/issues/7090.
However, I have re-implemented this change and I am unable to reproduce the customer issue that caused us to revert (again, https://github.com/aws-amplify/amplify-cli/issues/7090)
This PR re-adds the accountId to our usageData statistics. It also adds a new property to our usage data, record that allows us to include arbitrary additional data fields to our payloads.

Description of how you validated changes

Wrote a plugin that uses the new addRecord functionality
Attempted to reproduce the failure described in https://github.com/aws-amplify/amplify-cli/issues/7090, was unable to
Interacted with projects in a sandbox environment, including without any credentials set in ~/.aws/credentials. No errors or unexpected auth prompts.
Interacted with projects in a non-sandbox, authenticated environment. No errors or unexpected auth prompts.

Checklist

[x] PR description included
[x] yarn test passes

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Codecov Report

Merging #8754 (e818201) into master (a6142b8) will increase coverage by 0.00%.
The diff coverage is 71.42%.

@@           Coverage Diff           @@
##           master    #8754   +/-   ##
=======================================
  Coverage   56.88%   56.88%
=======================================
  Files         756      756
  Lines       42238    42243    +5
  Branches     8649     8649
=======================================
+ Hits        24027    24030    +3
- Misses      17390    17392    +2
  Partials      821      821

Impacted Files
Coverage Δ

...lify-cli/src/domain/amplify-usageData/UsageData.ts
94.82% <33.33%> (-3.36%)
:arrow_down:

packages/amplify-cli/src/context-manager.ts
78.12% <100.00%> (+0.70%)
:arrow_up:

...i/src/domain/amplify-usageData/UsageDataPayload.ts
100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a6142b8...e818201. Read the comment docs.


user interface

Again, nice project, this is promising !
The teleportation is very nicely done.
I think it would be a good thing to let the player pick a cube type, and a tool type (eraser, brush for painting existing block.. etc.. ). Also you might want an in-game menu to look for a game server for instance, or basic game parameters like sound.
I actually made a tool for VR user interfaces, check out the examples : https://three-mesh-ui.herokuapp.com/#keyboard
It's got some cool features like MSDF text, CSS-like syntax, a virtual keyboard...
If you're interested, I'm willing to help you code the user interface using this tool. (if you take contributions)

I did already check out your project when you posted it on the threejs forum. It's really nice.
I'll prolly will implement something like that when it's time to get serious with the UI.
But for now, I have enough with my current quick&dirty canvas texture based system.
I just need selecting a couple options, picking colors and simple stuff like that.
I agree my current UI situation could be way better haha.. but now I want to focus my efforts on adding some more new features.

BTW.. this is what I mean by my current canvas texture based system:

My bad, I actually didn't try to click on the menu button behind the hand. I tried again, it works smoothly and this is a smart idea to put it behind the hand. I would suggest making this menu button more obvious, but as you said this is not a priority.
Also the text is very pixelated, I think that's how you would benefit the most from three-mesh-ui, or at least get inspiration to make your own solution. (MSDF text uses a shader to render glyphs on plane geometries, it's both very efficient and very neat, try zooming in the example). The color picker however makes sense in canvas texture.
It looks like both events 'selectstart' and 'selectend' activate buttons, is that intentional ? I found it was not very intuitive.

Yeah. MSDF text seems like the way to go. Altough, the pixelated aliased fonts kinda fit with the art style. I dunno. All of the UI is just a temporary placeholder for developing the functionality.
I will revisit this once I finish the rest of the main stuff (or if the current system starts getting out hand).


Korean input not properly work of TextEdit control on Android system. (+proposal)

Operating system or device - Godot version:
Android - Tested on godot engine 2.1 branch recent.
Issue description:
In Android, when you type Korean characters, only the first consonant of the combined character will be entered and will be passed to next composition without the previous combination result applied. This is the result of receiving only the keystrokes without using the Android composition result. Only the callback of GodotTextInputWrapper will be called without any key events occurring during the compositing.
The final solution I think is that the input system itself is going to get the synthesis result of the OS, but I think the changes are going to be big, so I made it to work with small modifications first.
It clears the latest character including the characters in the composition, and replaces the last two(or one) character(s) with the characters provided by android TextView.OnEditorActionListener.
Following is the change I made.
Https://github.com/bugcaptor/godot/tree/fix-android-korean-input
That is based on master branch, but it can be merged on 2.1 branch because that file is same.
I have not tested this issue on iOS.
Steps to reproduce:
Type Korean characters in TextEdit on Android device.

Please, make a PR, instead of just linking random branches in random issues -- that would help the discussion centered around the code to be seperate from the discussion about the issue itself :smiley:

OK. It looks different from what I intended. I thought it would be easier to see the changes.
I was not sure which branch would be good. But it looks good to make PR on the 2.1 branch which is good to test build rather than on the unstable master branch. The application of the changes will be the problem after that.
Thanks for reply.

closed as #7936 is merged now.


Math on notebook not rendering

Required Prerequisites for filing a bug
You must follow ALL the steps in the troubleshooting guide.  Not doing so may result in automatic closure of the issue.
Required information

Steps to reproduce the problem

Just execute notebook. Output is not rendering. So I hid output and tried to copy and past latex code on markdown cell. This also didnt work.

A link to the notebook or markdown file where the error is occurring
https://gjunqueira-sys.github.io/J-J-Engineering-Blog/jupyter/math/calculus/symbolics/julialang/2021/12/28/EulersMethod.html

If the error is happening in GitHub Actions, a link to the specific error along with how you are able to reproduce this error.  You must provide this in addition to the link to the notebook or markdown file.
no errors are happening.

A screenshot / dump of relevant logs or error messages you are receiving from your local development environment. Instructions of running a local development server is provided in the development guide.

duplicate of https://github.com/fastai/fastpages/issues/577, will try to fix this son


fix: provide a valid useId type signature

🔗 Linked issue

❓ Type of change

[ ] 📖 Documentation (updates to the documentation, readme or JSdoc annotations)
[x] 🐞 Bug fix (a non-breaking change that fixes an issue)
[ ] 👌 Enhancement (improving an existing functionality like performance)
[ ] ✨ New feature (a non-breaking change that adds functionality)
[ ] 🧹 Chore (updates to the build process or auxiliary tools and libraries)
[ ] ⚠ Breaking change (fix or feature that would cause existing functionality to change)

📚 Description

Current implementation gives a TypeScript error when I try to pass in a key as parameter, just to then throw at runtime because the key is not provided 😬
📝 Checklist

[ ] I have linked an issue or discussion.
[ ] I have added tests (if possible).
[ ] I have updated the documentation accordingly.

The key should be provided by the Nuxt compiler, which is why it is disallowed. Can you open an issue with a reproduction instead? 🙏
Is there a reason you want to provide the key instead of using the auto generated one?

I wanted to use this composable with Radix Vue. Unfortunately it throws an error when prefix is not provided.
https://stackblitz.com/edit/nuxt-starter-he5ner?file=nuxt.config.ts%2Cpages%2Fno-prefix.vue%2Capp.vue%2Cpages%2Fprefix.vue

I think you could do something like this:
<template>
  <ConfigProvider :use-id="providedUseId">
    <TooltipProvider>
      <TooltipRoot>
        <TooltipTrigger class="IconButton"> Hover me </TooltipTrigger>
        <TooltipPortal>
          <TooltipContent as-child class="TooltipContent" :side-offset="5">
            Add to library
            <TooltipArrow class="TooltipArrow" size="8" />
          </TooltipContent>
        </TooltipPortal>
      </TooltipRoot>
    </TooltipProvider>
  </ConfigProvider>
</template>

<script setup>
const nuxtApp = useNuxtApp()
const providedUseId = () => nuxtApp.runWithContext(() => useId())
</script>


Search with Line Numbers

Based on issue https://github.com/Microsoft/vscode/issues/39219, we added the line number to the VS Code search tool.
A reupload from pull request https://github.com/Microsoft/vscode/pull/39724 because we switched branches and the other was closed because master had no changes.

Screenshot:

I'll have to think a little more about the UI


Quickstart install Istio problems (PodDisruptionBudget)

I'm trying to use the quickstart install guide and I'm running into istio issues:
> curl -s "https://raw.githubusercontent.com/kserve/kserve/release-0.9/hack/quick_install.sh" | bash

Flag --short has been deprecated, and will be removed in the future. The --short output will become the default.
bash: line 40: [: v4: integer expression expected
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   101  100   101    0     0   1725      0 --:--:-- --:--:-- --:--:--  1771
100  4856  100  4856    0     0  22349      0 --:--:-- --:--:-- --:--:-- 22349

Downloading istio-1.14.0 from https://github.com/istio/istio/releases/download/1.14.0/istio-1.14.0-linux-amd64.tar.gz ...

Istio 1.14.0 Download Complete!

Istio has been successfully downloaded into the istio-1.14.0 folder on your system.

Next Steps:
See https://istio.io/latest/docs/setup/install/ to add Istio to your Kubernetes cluster.

To configure the istioctl client tool for your workstation,
add the /home/tux/dev/kserve-test/istio-1.14.0/bin directory to your environment path variable with:
	 export PATH="$PATH:/home/tux/dev/kserve-test/istio-1.14.0/bin"

Begin the Istio pre-installation check by running:
	 istioctl x precheck

Need more information? Visit https://istio.io/latest/docs/setup/install/
namespace/istio-system created
✔ Istio core installed
- Processing resources for Istiod.                                                                                2022-10-19T09:05:47.341894Z	 error	 installer	 failed to update resource with server-side apply for obj PodDisruptionBudget/istio-system/istiod: no matches for kind "PodDisruptionBudget" in version "policy/v1beta1"
✘ Istiod encountered an error: failed to update resource with server-side apply for obj PodDisruptionBudget/istio-system/istiod: no matches for kind "PodDisruptionBudget" in version "policy/v1beta1"
- Processing resources for Ingress gateways.                                                                                2022-10-19T09:05:47.426293Z	 error	 installer	 failed to update resource with server-side apply for obj PodDisruptionBudget/istio-system/istio-ingressgateway: no matches for kind "PodDisruptionBudget" in version "policy/v1beta1"
✘ Ingress gateways encountered an error: failed to update resource with server-side apply for obj PodDisruptionBudget/istio-system/istio-ingressgateway: no matches for kind "PodDisruptionBudget" in version "policy/v1beta1"
- Pruning removed resources                                                                                2022-10-19T09:05:47.688725Z	 warn	 installer	 retrieving resources to prune type autoscaling/v2beta1, Kind=HorizontalPodAutoscaler: no matches for kind "HorizontalPodAutoscaler" in version "autoscaling/v2beta1"
2022-10-19T09:05:48.328322Z	warn	installer	retrieving resources to prune type policy/v1beta1, Kind=PodDisruptionBudget: no matches for kind "PodDisruptionBudget" in version "policy/v1beta1"
Error: failed to install manifests: errors occurred during operation

@tamis-laan see https://github.com/kserve/kserve/issues/2474

@tamis-laan two options here:

Use Kubernetes version lower than 1.25
Use quick install script from master branch which installs Istio 1.15.


Noisy background using isampler regression

When doing inference after CT-MRI regression using the isampler (niftynet.contrib.regression_weighted_sampler.isample_regression.ISampleRegression), I get noisy backgrounds. This issue does not happen when a use the standard net_regress application. I attach a picture of both cases (isampler left). Training was performed with isampler, but when using net_regress for the final inference I get a smoother output (right).

@wyli @xygorn - Any suggestion as you both contributed to the file that contains this sampler.

net_regress implements a more generic version of ISampleRegression, so please use net_regress in this case


merging JOSS-review branch into master

addressing comments made by @afrubin

[x] removed conda_requirements.txt

[x] moved testing specific requirements (flake8, coverage, pytest) to travis.yml

[x] added DOIs for all refs

[x] added several additional refs, including pyfaidx and vcfpy

[x] streamlined makefile

[x] removed docs/ directory entirely

[x] reformatted refs to include full names

[x] updated ORCID IDs

It's probably not necessary to perform CI testing against OS X. As you're finding this can be a real pain! Also OS X has a lot of differences between versions.

yeah a 'pain' is putting it lightly...the reason I started down that path is because I was noticing some strange behavior on my personal computer (Mac). The install seemed to work with python3.6 but not python3.7+, so I figured I would automate testing against the install on OSX with various python versions

merged PR as this was getting unwieldy. will look to add new features with additional PRs


How to show text or tooltip e.g (bad, good, very good) on each star?

This component is very flexible in what you can use as symbols. You can use inline style objects, CSS classes, react elements or an array of them.
The question is a little bit vague so I am going to assume two options:

If what you want is to show a text somewhere when the symbol is hovered you could just use the onHover callback to populate the label with the desired text:
<Rating
  onHover={(rate) => document.getElementById('label-onrate').innerHTML = rate || ''}
/>

If what you want is to show a tooltip, well, there are many different implementations available. I could name, off the top of my head, using title attribute, using CSS, using javascript, or even react component. Just search for it and pick the one best fits your requirements. Check this example using the title attribute:
const TooltipRating = ({tooltips}) =>
  <Rating
    stop={tooltips.length}
    emptySymbol={}
    fullSymbol={tooltips.map(tooltip => )}
  />;

<TooltipRating tooltips={['bad', 'good', 'very good']} />


⚠ Contemporanea B2B has degraded performance

In f4d313a, Contemporanea B2B ($SITE_CNTMB) experienced degraded performance:

HTTP code: 200
Response time: 8082 ms

Resolved: Contemporanea B2B performance has improved in e50ddab.


Remove Kore.Internal.SideCondition.andCondition

Fixes #1733
Reviewer checklist

[ ] Test coverage: stack test --coverage
[ ] Public API documentation: stack haddock

What I learned from running functions/positive-spec.k in the repl, both before and after removing andCondition:

Both go to node 1 the same way, using Axiom 0.
Then, from node 1:
  <T>
    <k> next true </k>
    <n> N:Int </n>
  </T>
#And
  { N >Int 0 #Equals true }

both use Claim 0:
  #Ceil ( positive ( N ) )
#And
  <T>
    <k> next positive ( N ) </k>
    <n> N:Int </n>
  </T>

→ ◇w
<T>
  <k> end </k>
  <n> N:Int </n>
</T>

But instead of getting node 2 with the configuration
  <T>
    <k> end </k>
    <n> N:Int </n>
 </T>
#And { N >Int 0 #Equals true }

After removing andCondition it gets node 2:
  #Ceil ( positive ( N ) )
#And
  <T>
    <k> end </k>
    <n> N:Int </n>
  </T>
#And { N >Int 0 #Equals true }
#And { true #Equals positive ( N ) }

I think the difference is in applying the rule
rule positive(M) => true
  requires M >Int 0

We should check the call sites of applyEquation to see what SideCondition it is called with.

We should check the call sites of applyEquation to see what SideCondition it is called with.

Sorry, I meant attemptEquation here.

We should check the call sites of applyEquation to see what SideCondition it is called with.

Sorry, I meant attemptEquation here.

Is this the SideCondition we are looking for?
\and{_PREDICATE{}}(
    /* term: */
    /* D Sfa */ \top{_PREDICATE{}}(),
\and{_PREDICATE{}}(
    /* predicate: */
    /* Sfc */
    \ceil{SortBool{}, _PREDICATE{}}(
        /* Fn Sfc */
        Lblpositive'LParUndsRParUnds'FUNCTIONS'Unds'Bool'Unds'Int{}(
            /* Fl Fn D Sfa */ VarN:SortInt{}
        )
    ),
    /* substitution: */
    /* Spa */
    \equals{SortGeneratedCounterCell{}, _PREDICATE{}}(
        /* Fl Fn D Sfa */ VarDotVar0:SortGeneratedCounterCell{},
        /* Fl Fn D Sfa */
        Lbl'-LT-'generatedCounter'-GT-'{}(
            /* Fl Fn D Sfa */ VarDotVar00:SortInt{}
        )
    )
))

It's passed to attemptEquation inside definitionEvaluation

The problem is because we do not pass in a SideCondition when applying a semantic rule. The constraint
true #Equals positive ( N )

originates when we apply the claim next positive(N) => end to the configuration next true. We need to use the top-level SideCondition
N >Int 0 #Equals true

when we simplify the positive constraint. The backend does not yet support re-simplifying the top-level SideCondition with itself, so we have to complete the simplification of positive before we add that constraint to the top-level SideCondition (that is: before we finish applying the claim).

Now there is only one unit test failing.
The reason is that the rule
claim {}
/* Spa */
    \implies{testSort{}}(
        /* Spa */
        \and{testSort{}}(
            /* Spa */
            \equals{testSort{}, testSort{}}(
                /* Fl Fn D Sfa */ x:testSort{},
                /* Fl Fn D Sfa Cl */ a{}()
            ),
            /* Fl Fn D Sfa */ x:testSort{}    <---- this is the only difference
        ),
        /* Spa */
        weakExistsFinally{testSort{}}(
            /* Spa */
            \and{testSort{}}(
                /* Spa */
                \not{testSort{}}(
                    /* Spa */
                    \equals{testSort{}, testSort{}}(
                        /* Fl Fn D Sfa */ x:testSort{},
                        /* Fl Fn D Sfa Cl */ a{}()
                    )
                ),
                /* Fl Fn D Sfa */ x:testSort{}
            )
        )
    )
[]

becomes
    claim {}
    /* Spa */
        \implies{testSort{}}(
            /* Spa */
            \and{testSort{}}(
                /* Spa */
                \equals{testSort{}, testSort{}}(
                    /* Fl Fn D Sfa */ x:testSort{},
                    /* Fl Fn D Sfa Cl */ a{}()
                ),
                /* Fl Fn D Sfa Cl */ a{}()    <---- this is the only difference
            ),
            /* Spa */
            weakExistsFinally{testSort{}}(
                /* Spa */
                \and{testSort{}}(
                    /* Spa */
                    \not{testSort{}}(
                        /* Spa */
                        \equals{testSort{}, testSort{}}(
                            /* Fl Fn D Sfa */ x:testSort{},
                            /* Fl Fn D Sfa Cl */ a{}()
                        )
                    ),
                    /* Fl Fn D Sfa */ x:testSort{}
                )
            )
        )
    []

when given as an argument to runOnePathSteps
I think requires is applied to left.


notify the download failure so as to quickly restart a new request

In state sync stage, if the txhashset download fail, we have to wait a 10 minutes timeout before restarting a new request.
20191204 09:10:01.289 DEBUG gotts_p2p::protocol - handle_payload: txhashset archive for 046b17883ff0 at 2010. size=1048516
20191204 09:10:12.356 DEBUG gotts_p2p::protocol - handle_payload: txhashset archive: 528000/1048516
20191204 09:10:13.367 ERROR gotts_p2p::protocol - handle_payload: txhashset archive save to file fail. err=Connection(Os { code: 11, kind: WouldBlock, message: "Resource temporarily unavailable" })

This PR can notify this failure and cause the sync thread restart a new request immediately.

Another possible improvement is to explicitly move the accepted TCP stream into blocking mode, which is found in https://github.com/mimblewimble/grin/pull/3154 and should be helpful to avoid above failure.
Leave a TODO item here to check what's the perfect way to set this non-blocking / blocking:
	/// Starts a new TCP server and listen to incoming connections.
	pub fn listen(&self) -> Result<(), Error> {
		...
		listener.set_nonblocking(true)?;
		...
		loop {
			...
			match listener.accept() {
				Ok((stream, peer_addr)) => {
					// Explicitly move the accepted TCP stream into blocking mode.
					stream.set_nonblocking(false)?;

The test result is OK:
20191204 10:42:02.130 DEBUG gotts_p2p::peer - Asking xxx:13514 for txhashset archive at 2010 046b17883ff0.
20191204 10:42:03.347 DEBUG gotts_p2p::protocol - handle_payload: txhashset archive for 046b17883ff0 at 2010. size=1048516
20191204 10:42:12.375 ERROR gotts_p2p::protocol - handle_payload: txhashset archive save to file fail. err=Connection(Os { code: 11, kind: WouldBlock, message: "Resource temporarily unavailable" })
         >>>> failed and restarted immediately !
20191204 10:42:12.402 DEBUG gotts_p2p::peer - Asking xxx:13514 for txhashset archive at 2010 046b17883ff0.
20191204 10:42:13.500 DEBUG gotts_p2p::protocol - handle_payload: txhashset archive for 046b17883ff0 at 2010. size=1048516
20191204 10:42:18.195 DEBUG gotts_p2p::protocol - handle_payload: txhashset archive: 1048516/1048516 ... DONE


Numbering incorrect?

I'm not sure if there's already a way to do this and I'm just missing it... in the image below, the number 19 shouldn't be there, right? There's no word that starts there, either down or across.
I'd be glad to take a look and work on this (this is an incredible tool!), I just don't want to duplicate work if there's already a way.

I believe the relevant code is here:
https://github.com/keiranking/Phil/blob/58243dd252dbb84f596819c0028117ca10d94f2c/cross.js#L535-L566

Addressed the above here: https://github.com/keiranking/Phil/pull/19

I still see this issue.  Numbers incorrectly added to all one-cell entries.

Is there any way to force renumbering?


[Release] Release of vizro 0.1.24

Description
Screenshot
Notice

[x] I acknowledge and agree that, by checking this box and clicking "Submit Pull Request":

I submit this contribution under the Apache 2.0 license and represent that I am entitled to do so on behalf of myself, my employer, or relevant third parties, as applicable.
I certify that (a) this contribution is my original creation and / or (b) to the extent it is not my original creation, I am authorized to submit this contribution on behalf of the original creator(s) or their licensees.
I certify that the use of this contribution as authorized by the Apache 2.0 license does not violate the intellectual property rights of anyone else.
I have not referenced individuals, products or companies in any commits, directly or indirectly.
I have not added data or restricted code in any commits, directly or indirectly.

Some of the tests are failing, you might have to restart the release process again tomorrow 😬 I think there are some issues if you open the PR on another day than it is merged in case we don't manage to fix in time

Some of the tests are failing, you might have to restart the release process again tomorrow 😬 I think there are some issues if you open the PR on another day than it is merged in case we don't manage to fix in time

Ah ok sure, will restart the release tmr (probably won't get another approval tonight anyway). Currently checking the CI error, related to one of the vizro-ai dependency bug: https://github.com/langchain-ai/langgraph/issues/1914


[IMPROVEMENT] Create a pipeline to automatically update Longhorn upgrade responder when we release a new version

The upgrade responder server is at: https://github.com/longhorn/upgrade-responder/tree/longhorn
Pipeline could be drone.io or jenkins

cc @yangchiu

Possible helm plugin for drone.io https://github.com/ipedrazas/drone-helm

Moving into the QA board instead. @yangchiu should work with @PhanLe1010 to get this done in the future. Not a priority for now.


Question: loading yolov5/torch pt file in OpenCV DNN API on Windows?

OpenCV 4.2.0/4.4.0/4.5.0
DNN API (readNetFromTorch(model_file))
I have not been able to load a yolov5/torch format pt file (or the torchscript CUDA variant) in OpenCV without an exception being thrown.  Have you tried this?
Thanks,
Rob

Hi @rtrahms, I don't have experience to load models into OpenCV, but I guess maybe some of the layers are not supported in current OpenCV, and I would recommend to use LibTorch instead.

Hi @yasenh, thanks for the advice - yes, I am using libTorch now.  Have built in Visual Studio with help from the following article:
https://expoundai.wordpress.com/2020/10/13/setting-up-a-cpp-project-in-visual-studio-2019-with-libtorch-1-6/
Builds successfully, but throws exception on torch::jit::load() call.   I've tried both the original pt file from training and the modified export version you suggested.  Not happy with the file for some reason, but both versions are readable by netron.
Rob

@rtrahms when you use the gpu model, did you add --gpu flag when running the application? or remove the gpu flag when use CPU version?

@yasenh I don't use your code directly, but using it as a code reference.  In my code I have set torch::Device device = torch::kCPU and torch::kCUDA.  the issue is on the initial load, which may indicate my torchscript converted file is not correct.  I do need that torchscript version for the load, yes?  The original pt file will not work I assume, for either CPU or GPU versions.

@rtrahms  Could you share the issues when using "export.py"? And I notice that you set img_size = 416 here, so did you export the model with image size as 416?
python models/export.py --weights yolov5s.pt --img 416 --batch 1

And I would highly recommend that you use the export.py from the official yolov5 python version.

@yasenh - Yes, I have used 416x416 as the original Darknet Yolo image input size.
I wanted to understand the issues with export.py, so I created what I thought would be a standalone utility to do the same thing.  The code is below.  What I discovered was that the pt file used as input is making references to files in the original yolov5 filestructure, namely models and utils folders.  After copying those over, the code below worked for me.  Besides netron, I don't know of an easy way to edit/modify either the original PT file or the generated torchscript PT file.
import torch
import torch.nn as nn
import torchvision
import os
pt_file = 'yolov5s_ob12.pt'
torchscript_file = "yolov5s_ob12.torchscript-cpu.pt"
#torchscript_file = "yolov5s_ob12.torchscript-cuda.pt"
is_file = os.path.isfile(pt_file)
create dummy input image
batch_size = 1
img_size = 416
print("creating dummy img")
img = torch.rand((batch_size, 3, img_size, img_size))
print("loading network")
model = torch.load(pt_file, map_location=torch.device('cpu'))['model'].float()
model.eval()
model.model[-1].export = True  # set Detect() layer export=True
#model.model[-1].export = False  # modified 10/21 rgt
y = model(img)  # dry run
print("tracing network")
trace (run model with dummy input) - store trace in torchscript module
traced_script_module = torch.jit.trace(model,img)
print("saving script module")
serialize torchscript module
traced_script_module.save(torchscript_file)
print("Complete!  Exiting.")

@rtrahms So there are some difference between your version and the expory.py:
https://github.com/ultralytics/yolov5/blob/c8c5ef36c9a19c7843993ee8d51aebb685467eca/models/experimental.py#L137-L144
https://github.com/ultralytics/yolov5/blob/master/models/export.py#L43-L47
BTW, did you set "model.model[-1].export = False" when export? And I am still not sure why you don't use the "export.py" directly?

Mainly I am using a separate script to understand what is going on.  I have tried setting export to both True and False, no change.  I actually do not know what that flag does anyway.
Making this even more simple, I attempted creating a torchscript file from a natively constructed network (code below).  Generated a torchscript file, but also throws exception when attempting to load in cpp application.
import torch
import torch.nn as nn
import torchvision
import os
torchscript_file = "native_test.torchscript-cpu.pt"
#torchscript_file = "native_test.torchscript-cuda.pt"
class MyModule(torch.nn.Module):
def init(self, N, M):
super(MyModule, self).init()
self.weight = torch.nn.Parameter(torch.rand(N, M))
def forward(self, input):
    if input.sum() > 0:
      output = self.weight.mv(input)
    else:
      output = self.weight + input
    return output

my_module = MyModule(10,20)
sm = torch.jit.script(my_module)
print(sm.code)
print("saving torchscript file")
sm.save(torchscript_file)
print("Completed.  Exiting.")

@rtrahms Maybe this tutorial can help: LOADING A TORCHSCRIPT MODEL IN C++, make sure you use same version of the PyTorch and LibTorch

@yasenh - update.   Using the cpu variant of the exported torchscript file and code for you Detector (and some adjustment of the hard-coded internal 640x640 image input) I was able to load the network in a cpp application and successfully run inference.  CUDA version of torchscript still doesn't load, looking into why (I did change the tensor types like you suggest)...

@yasenh - another update.  Found the issue with CUDA version.  The MS Visual Studio project was manually generated, and did not recognize any CUDA devices.  This was due to a missing linker flag:
/INCLUDE:"?warp_size@cuda@at@@YAHXZ"
https://github.com/pytorch/pytorch/issues/35604
After building with this flag, the CUDA device is detected, and the CUDA DNN loads successfully.
The forward() call now crashes with an exception, but progress.

@yasenh - So I am wondering if the adjustments you suggested to export.py on the ultralytics yolov5 repo are still valid.  The code has changed, so there might be some updates needed to your instructions.  Can you take a look?
Thanks!

@yasenh - update.  So it looks like my CUDA torchscript DNN loads and can run inference... once.  A second time through causes an exception.   I noticed my warm-up forward() call was passing, but passing the real image through afterwards did not work.  Also, skipping the warm-up and passing the image in the first pass also works.  Is there CUDA cleanup that needs to happen after a torch::forward() call?

@rtrahms thanks for the update, I only test it on my 1070 GPU, maybe you are using a GPU with less memory? I will try to figure it out but may take some time due to my current schedule.

@yasenh - Here's a clue... if I insert a call module_.eval(); after processing the detections in the Detector::Run() method, it does work on repeated calls to Run().  Another clue, this eval() call is not needed for the CPU variant, with that Detector::Run() can be called repeatedly with no issue.

module_.eval() is called in the constructor @rtrahms
https://github.com/yasenh/libtorch-yolov5/blob/23c3e5c57addd533d96fcf1c39f0d8fdbf078803/src/detector.cpp#L21

@yasenhu - yes, that gave me the idea to call it after the inference call.  The Detector object inference call works the first time but not after the first time.

Thanks, I found the issue.  I rebuilt with libtorch 1.7, CUDA 11.0, and included ALL DLLs from libtorch distribution and CUDA distribution.  Worked as expected.

Good to hear that!


[RFR] Adds Redfish API client

With this PR we replace the hard-coded value for the stat check with that obtained by the Redfish provider's implementation now uses the Redfish API client.
We initiated a new entity Server and contributed ServerStats enum, which RedfishServer implements with its API specifics. The RedfishSystem is implemented such that it can be used both for testing stats at the provider level and for a selected physical server instance.
Depends on https://github.com/ManageIQ/integration_tests/pull/7735 and https://github.com/ManageIQ/wrapanapi/pull/331.
/cc @tadeboro, @miha-plesko

Lint report for commit 05e606d080916bbc8ff64a0a474811026bf09645:
wrapanapi/systems/redfish.py:

:red_circle: Line 8:1: F401 'json' imported but unused

Please, rectify these issues :smirk: .
No commit flaws detected.
CFME QE Bot

I now added the stat checks for the physical server.

Thank you for the comments, I applied all the pertinent ones.

@mshriver thank you for your comments. I have addressed most of the issues. I will create a server.py entity as you suggested. But maybe that can go into the next PR so that this will unblock the PR that depends on this PR?

I now flipped the dependencies so that the first PRs to look at are the ones that refactor integration tests for physical servers (and Lenovo): https://github.com/ManageIQ/wrapanapi/pull/331 and https://github.com/ManageIQ/integration_tests/pull/7735.

I believe that I have addressed all the comments.

I believe your Travis failures will be resolved by #336

I believe your Travis failures will be resolved by #336

Thanks! I was looking at the errors and they seem unrelated to my changes.

@matejart I rebased after merging in order to trigger travis. Sorry to 'step on toes', just trying to push for release in time for integration_tests merge-freeze.


Find Middle Element in Linked List

fixes #87

Excellent work @hemabhagnani Thanks a lot for making this PR.


new: dev: #154: Make cluster manager pluggable

Pull Request Test Coverage Report for Build 122

0 of 53 (0.0%)  changed or added relevant lines in 8 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.1%) to 23.647%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

rubix-hadoop2/src/main/java/com/qubole/rubix/hadoop2/CachingNativeS3FileSystem.java
0
4
0.0%

rubix-hadoop2/src/main/java/com/qubole/rubix/hadoop2/CachingS3AFileSystem.java
0
4
0.0%

rubix-hadoop2/src/main/java/com/qubole/rubix/hadoop2/CachingDistributedFileSystem.java
0
4
0.0%

rubix-presto/src/main/java/com/qubole/rubix/presto/CachingPrestoS3FileSystem.java
0
4
0.0%

rubix-hadoop2/src/main/java/com/qubole/rubix/hadoop2/CachingNativeAzureFileSystem.java
0
4
0.0%

rubix-core/src/main/java/com/qubole/rubix/core/ClusterManagerInitilizationException.java
0
8
0.0%

rubix-spi/src/main/java/com/qubole/rubix/spi/CacheConfig.java
0
10
0.0%

rubix-core/src/main/java/com/qubole/rubix/core/CachingFileSystem.java
0
15
0.0%

Totals

Change from base Build 121:
-0.1%

Covered Lines:
533

Relevant Lines:
2254

💛  - Coveralls

Added unit tests for existing and non-existing ClusterManagers


Could not resolve "crypto"

I am unable to import crypto in a component while using contentlayer. I never had any issues doing the same thing with mdx-js/react. This error happens on contentlayer build time (when it creates ./contentlayer/generated).
I've spent so many hours on already I've decided to write my own parsing and just run it through MDXProvider instead of using contentlayer. It's a shame because otherwise I loved using contentlayer. If someone knows a fix or if I'm doing something wrong, let me know.
Error: Found 1 problems in 2 documents.

 └── Encountered unexpected errors while processing of 1 documents. This is possibly a bug in Contentlayer. Please open an issue.

     • "blog/some-blog-post.mdx": UnexpectedMDXError: Error: Build failed with 1 error:
     ../../components/crypto-widget.tsx:11:19: ERROR: Could not resolve "crypto"

SourceFetchDataError: {
  "_tag": "HandledFetchDataError"
}

Update on this: I ended up getting rid of contentlayer and rewriting all my logic, instead going with gray-matter and @mdx-js/react, my manual solution is working perfect. If someone knows a solution for this let me know, I still like contentlayer and the features it brings 😄.


Unhandled Exception: System.Collections.Generic.KeyNotFoundException: The given key was not present in the dictionary.

puppeteer sharp throwing following exception
Unhandled Exception: System.Collections.Generic.KeyNotFoundException: The given key was not present in the dictionary.

And the main problem is that all puppeteer sharp code running inside but after that program is breaking.

Could you post this issue following the issue template?
Thanks!

Not understand. Can you please more clarify?

When you create a New Issue you'll find a template to follow.
In order to help you, we need you to help us by following that template, telling us how to reproduce it, which version are you running, etc.

I have had the same issue with the network manager, after some investigation looks like the issue is due to contention over two dictionaries in the NetworkManager class, “_requestIdToRequest” and “_interceptionIdToRequest”. Both these dictionaries were accessed by multiple threads causing the issue of KeyNotFound. After making some code changes to these to make them ConcurrentDictionary and making them thread safe seems to have solved the issue.
This also required similar changes to the “_responses” dictionary in Connection class and the “_callbacks” dictionary in CDPSession class.

@Goregakalack Could you share a piece of code to reproduce that?

@Goregakalack  @kblok  Have you fixed this issue.

@virenderkverma Could you share a piece of code to reproduce the issue?

Sorry it took me so long to get back to you, had quite a few deadlines to meet. Anyway there is code to reporduce, it may take a few attempts to work.
Exception thrown:
System.IndexOutOfRangeException
HResult=0x80131508
Message=Index was outside the bounds of the array.
Source=mscorlib
StackTrace:
at System.Collections.Generic.Dictionary`2.Insert(TKey key, TValue value, Boolean add)
at PuppeteerSharp.CDPSession.d__30.MoveNext()
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd(Task task)
at PuppeteerSharp.Request.d__58.MoveNext()
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
at Puperteer_Sharp_Issues.Program.<>c__DisplayClass0_1.<b__1>d.MoveNext() in

`
var fetcher = new BrowserFetcher();
        fetcher.DownloadAsync(BrowserFetcher.DefaultRevision).Wait();

        var launcher = new Launcher();

        var browser = launcher.LaunchAsync(new LaunchOptions
        {
            Headless = true
        }).Result;

        var urls = new List<string>
        {
            "http://www.google.co.uk",
            "https://uk.yahoo.com",
            "https://www.youtube.com/",
            "https://www.bbc.co.uk",
            "https://www.bbc.co.uk/news",
            "https://www.bbc.co.uk/iplayer",
            //"https://www.microsoft.com/en-gb",
            "https://visualstudio.microsoft.com/"
        };

        var mre = new ManualResetEvent(false);

        var page = await browser.NewPageAsync();

        await page.SetRequestInterceptionAsync(true);

        page.Request += (s, o) =>
        {
            Task.Run(async () =>
            {

                var httpRequest = WebRequest.CreateHttp(o.Request.Url);

                var response = (HttpWebResponse)httpRequest.GetResponse();

                var bytes = new byte[0];

                using (var ms = new MemoryStream())
                {
                    using (var stream = response.GetResponseStream())
                    {
                        stream.CopyTo(ms);
                    }

                    bytes = ms.ToArray();
                }

                var headers = new Dictionary<string, object>();

                foreach (var key in response.Headers.Keys)
                {
                    var strKey = key.ToString();

                    headers.Add(strKey, response.Headers[strKey]);
                }

                // Continue
                await o.Request.RespondAsync(new ResponseData()
                {
                    BodyData = bytes,
                    ContentType = "UTF-8",
                    Headers = headers
                });
            });
        };

        foreach (var url in urls)
        {
            mre = new ManualResetEvent(false);

            page.Load += (o, e) => mre.Set();

            await page.GoToAsync(url, new NavigationOptions());

            mre.WaitOne();

            var screenShotData = await page.ScreenshotDataAsync(new ScreenshotOptions()
            {
                Type = ScreenshotType.Jpeg,
                FullPage = true,
                OmitBackground = false
            });

            var guid = Guid.NewGuid();

            File.WriteAllBytes(@"D:\Projects\SandBox\SandBox\Screenshots\" + guid.ToString() + ".jpeg", screenShotData);
        }

        await page.CloseAsync();
        page.Dispose();
        await browser.CloseAsync();

`

Do you have a piece of code to test @DominicBoettger ?

using System;
using System.IO;
using System.Collections.Generic;
using System.Linq;
using System.Net.Http;
using System.Threading;
using System.Threading.Tasks;
using Newtonsoft.Json.Linq;
using NDesk.Options;
using PuppeteerSharp;
using System.Reflection;
using System.Collections;

namespace com.inspirationlabs.prerenderer
{
    class Prerenderer
    {
        static string Host = "http://localhost:2015";
        static int Threads = Environment.ProcessorCount * 20;
        static string Jsonurl = "https://my.urls";
        static string OutputPath = Path.GetDirectoryName(Assembly.GetEntryAssembly().Location) + Path.DirectorySeparatorChar + "output";
        static DirectoryInfo Cwd;
        static void Main(string[] args)
        {
            var p = new OptionSet() {
               { "host=", "Set the hostname", v => Host = v },
               { "threads=", "Set the amount of paralell threads", (int v) => Threads = v },
               { "jsonurl=", "Set the endpoint url to get the url list", v => Jsonurl = v},
               { "outputpath=", "Set the path to output the contents", v => OutputPath = v }
            };
            List<string> extra = p.Parse(args);

            try
            {
                // delete outputpath if it exists
                if(OutputPath.Length > 0 && Directory.Exists(OutputPath))
                {
                    Directory.Delete(OutputPath, true);
                }
                if(OutputPath.Length > 0)
                {
                    Console.WriteLine("Creating outputpath " + OutputPath);
                    Cwd = Directory.CreateDirectory(OutputPath);
                }
            } catch(Exception e)
            {
                Console.WriteLine(e.Message);
            }
            // wait for MainTask (async)
            Maintask().Wait();
        }

        static async Task Maintask()
        {
            try
            {
                HttpClient client = new HttpClient();
                HttpResponseMessage response = await client.GetAsync(Jsonurl);
                response.EnsureSuccessStatusCode();

                string responseBody = await response.Content.ReadAsStringAsync();
                JObject jObject = JObject.Parse(responseBody);
                JArray urldata = (JArray)jObject["data"];

                var fetcher = await new BrowserFetcher().DownloadAsync(BrowserFetcher.DefaultRevision);
                await DownloadAsync(urldata);
            } catch(Exception e)
            {
                Console.WriteLine(e.Message);
            }
        }

        // download data
        static async Task DownloadAsync(JArray urls)
        {
            Processing processing = new Processing();
            Queue<Page> qt = new Queue<Page>();

            using (SemaphoreSlim semaphore = new SemaphoreSlim(Threads))
            using (Browser browser = await Puppeteer.LaunchAsync(new LaunchOptions
            {
                Headless = false,
                Args =  new[] { "--no-sandbox", "--disable-setuid-sandbox" }
            }))
            {
                for (int i = 0; i <= Threads+50; i++)
                {
                    qt.Enqueue(await browser.NewPageAsync());
                }

                var tasks = urls.Select(async (urldata) =>
                {
                    if ((bool)urldata.SelectToken("published") && (bool)urldata.SelectToken("indexed"))
                    {
                        await semaphore.WaitAsync();
                        Page page = qt.Dequeue();
                        try
                        {
                            page.DefaultNavigationTimeout = 120000;
                            var setIsServer = @"
                            Object.defineProperty(window, 'isServer', {
                                get() {
                                    return true
                                }
                            });
                        ";
                            await page.EvaluateOnNewDocumentAsync(setIsServer);
                            await page.SetRequestInterceptionAsync(true);
                            page.Request += (sender, e) =>
                            {
                                string resType = e.Request.ResourceType.ToString();
                                if (resType == "Image" || resType == "Font")
                                {
                                    e.Request.AbortAsync();
                                }
                                else
                                {
                                    e.Request.ContinueAsync();
                                }
                            };
                            string path = (string)urldata.SelectToken("url");
                            string url = Host + path;
                            await page.GoToAsync(url, new NavigationOptions { WaitUntil = new[] { WaitUntilNavigation.Networkidle0 } });
                            string content = await page.GetContentAsync();

                            // put the result on the processing pipeline
                            processing.QueueItemAsync(content, path, OutputPath);
                        }
                        finally
                        {
                            qt.Enqueue(page);
                            semaphore.Release();
                        }
                    }

                });

                await Task.WhenAll(tasks.ToArray());
                // await processing.WaitForCompleteAsync();
            }
        }
    }
}

@DominicBoettger I'm not getting that error. I created an array of 50 calls to Amazon.
Could you try awaiting e.Request.AbortAsync(); and e.Request.ContinueAsync();

I tried this but this did not work too. And I also only get the error after more than 2000 requests.
I removed:
await page.SetRequestInterceptionAsync(true);
                            page.Request += (sender, e) =>
                            {
                                string resType = e.Request.ResourceType.ToString();
                                if (resType == "Image" || resType == "Font")
                                {
                                    e.Request.AbortAsync();
                                }
                                else
                                {
                                    e.Request.ContinueAsync();
                                }
                            };

After that my code works.

@DominicBoettger I just published v1.9. Could you give it a try?

Fixed on #720


libetpan: Fix CVE-2022-4121 and more patches

Description of changes
There are multiple security and reliability fixes on libetpan's master branch which weren't included into a new release yet. Next to the mentioned CVE-2022-4121, some other small security-related commits were selected.
As a bit of housekeeping, the sha256 field was replaced with hash for the fetchers and an https URL was chosen for the homepage.
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[x] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[x] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
23.11 Release Notes (or backporting 23.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

For the record, I just created an issue to ask for a new release: https://github.com/dinhvh/libetpan/issues/439

BTW this builds fine on macos 10.15, not sure what the linux platform restriction is for.

@risicle: Thanks for testing! It seems like the platform restriction is there since the package emerged in de0a388ccf186119b5fc90fca2549a7a8f328dcb back in 2014. Thus, I have removed it and force pushed an updated version.


Let's call it 1.0? Any features missing?

Hi,
I added a couple of missing features in 0.18.0 (mainly merge.smartStrategy and support for nested fields at merge.strategy). It's starting to feel like 1.0 to me.
Do you think there's anything particular missing? If I don't hear anything in a week or two, I'll publish the current as 1.0.
The interesting thing is that I've started to find the basic merging mechanism useful beyond webpack (Karma, table configuration). Maybe 1.0 would be a good chance to rename the package to reflect this fact?

And published. Thanks for the issues!


added a hint for 'hotkey'

Associated Issue: #
Summary of Changes

change 1
change 2

Pull Request Test Coverage Report for Build 906

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 87.143%

Totals

Change from base Build 514:
0.0%

Covered Lines:
158

Relevant Lines:
180

💛  - Coveralls


Allow installing cli without dependencies

yarn and npm are fine for devs but if you are not a dev or even not a node js dev its annoying and frustrating to need them
see if we can package up the cli for installation

Hello
I would love to engage in your team


Pull request/ef94d0ee

The first commit is a small fix for the previous pull request.
The second commit is more important.
For the SDDP package, I allow to either supply the data using a low level interface using matrices or to use StructJuMP.
The issue is that usually, the number of stages is like 48 with 3 scenarios at each stages which makes 3^48 possible realization of the uncertainty. However the uncertainty at each stages are usually independant which is called "serial independance" so instead of having to do
for s1 in 1:3
  for s2 in 1:3
    for s3 in 1:3
      ...
        for s48 in 1:3

we could do something like
for stage in 1:48
  for s in 1:3

where basically, each of the 3 scenarios has the same 3 children which are the 3 scenarios at the next stage.
To add this functionality to StructJuMP, I see 2 options.

either tell the 2nd and 3rd scenarios that they have the same children than the first scenario and tell each of the 3 scenarios of the next stage that their parent is the 1st scenario (this is the solution I have implemented in the second commit). The variables of the 1st scenarios are used by the children.
or tell each of the 3 scenarios of the next stage that they have 3 parents and give a vector containing the parents. The problem with this solution is as follows : "The variables of which parent are going to be used to model the children ?", we can say that it is the variables of the first one but this seems a bit hacky.

What do you think ?

I think the first solution that you implemented seems more natural. A few minor points:

How about same_children_as rather than same_children_than?
Can you add some error checking after the if same_children_than !== nothing line? To make sure that isa(same_children_than, JuMP.Model) and that haskey(same_children_than, :Stochastic).

Is it better to check with isa or to specify the type Nullable{JuMP.Model} in the signature ?

AFAICT that would mean that a caller to StructuredModel would have to wrap same_children_than=Nullable(model) in a Nullable, which is a bit noisier:
julia> function foo(x; y::Nullable{Int}=Nullable(1))
       y
       end
foo (generic function with 1 method)

julia> foo(1, y=2)
ERROR: TypeError: foo: in typeassert, expected Nullable{Int64}, got Int64

julia> foo(1)
Nullable(1)

julia> foo(1, y=Nullable(2))
Nullable(2)

Done :)

Thanks!


Test harness calls tpm4720-keylime init_tpm_server & tpm_serverd when running TPM 2.0

Environment

OS / version: N/A
Processor architecture: N/A
TPM Manufacturer: mssim
Keylime version: master

Description
When test_restful.py is called during tests, it tries to run scripts init_tpm_server & tpm_serverd here
This will work for  tpm4720-keylime  / tpm 1.2 , but not with the new TPM 2.0 ibmtpm974 simulator. As the files init_tpm_server & tpm_serverd are only available here
Expected behavior vs. actual behavior
Steps to reproduce problem

run_tests.sh using the ibmtpm974 simulator:

test.test_ca_impl_openssl
  OpenSSL_Test
.   test_openssl
test.test_crypto
  Crypto_Test
.   test_aes
.   test_errors
.   test_hmac
.   test_rsa
    test_rsa_signWARNING: Restarting TPM emulator failed!

Fixed here https://github.com/keylime/python-keylime/commit/a32c9568ba40cabd75b5671825f8aa1ca4fba58a


Bump schemas version (0.1.10) and other gems

Description of change
Bump Rails to latest minor release 7.0.5 along with some other gems.
No breaking changes.
Link to relevant ticket
https://dsdmoj.atlassian.net/browse/CRIMAP-382
Notes for reviewer / how to test

Will be superseded by another PR soon.


Traefik: Drop not used Variables

The following variables aren't used in the repositories:

*_traefik_path_prefix
*_traefik_host

This effects all repositories where Traefik vars are used.

That's done.


Cloud chatops docker secrets

Created a docker deployment for Cloud Chatops. Uses GitHub Actions as CD to build and push dev/prod images. Uses watchtower to keep the Cloud Chatops container up to date.

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 81.08%. Comparing base (7394c9a) to head (fa89dca).

:exclamation:  There is a different number of reports uploaded between BASE (7394c9a) and HEAD (fa89dca). Click for more details.
HEAD has 5 uploads less than BASE

Flag
BASE (7394c9a)
HEAD (fa89dca)

6
2

rabbit_consumer
1
0

Additional details and impacted files
@@             Coverage Diff             @@
##           master     #172       +/-   ##
===========================================
- Coverage   96.82%   81.08%   -15.75%
===========================================
  Files          69        3       -66
  Lines        3120       74     -3046
===========================================
- Hits         3021       60     -2961
+ Misses         99       14       -85

Flag
Coverage Δ

rabbit_consumer
?

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Update Text on Experience Page

divs are not properly aligned on newer phones. Confirmed on:

Samsung Galaxy S10

Looks fine on:

Samsung Galaxy S5


Set up pages

I'd like for there to be three pages to this app:

[ ] Accounts: A list of all accounts that belong to some wallet.

Token accounts (with mints displayed).
Stake, Vote accounts.
Custom program accounts (see program).

[ ] Transactions: A list of all transactions done by the wallet.
[ ] Explorer: A very small and lightweight block explorer. Given an
address, signature, or slot, allow users to query on-chain data.

Hey, I set up query to fetch a transaction. How do I get a list of all signatures in the wallet to display list of transactions?

Hey, I set up query to fetch a transaction. How do I get a list of all signatures in the wallet to display list of transactions present in wallet?

You can use the RPC's getSignaturesForAddress method, but we'll need to populate a few mock transactions for now using that script I've initialized. Maybe some simple transfers or token creations via CLI.

I'll create a rest api and pass address, the server queries and returns the signatures to react application. Should I use the addresses in Account.tsx file to create simple transfers?

I'll create a rest api and pass address, the server queries and returns the signatures to react application.

I've got a server going already in server.ts so you can add the endpoint there.

Should I use the addresses in Account.tsx file to create simple transfers?

We can probably just generate one or two keypairs with the CLI and send transfers to them. For now, I think we can use the local keypair (fee payer) as the wallet address, and later we can figure out how to add a user's actual address.
Creating tokens and token accounts will create the accounts on-chain automatically.
This can all be scripted into scripts/generate-transactions.sh.

0] SolanaError: Solana error #-32602; Decode this error by running `npx @solana/errors decode -- -32602 'X19zZXJ2ZXJNZXNzYWdlPUludmFsaWQlMjBwYXJhbSUzQSUyMEludmFsaWQ='`
[0]     at Object.send (/Users/pc/my/code/openSource/graphql-wallet/node_modules/@solana/rpc-spec/src/rpc.ts:79:13)
[0]     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
[0]     at loadTransaction (/Users/pc/my/code/openSource/graphql-wallet/node_modules/@solana/rpc-graphql/src/loaders/program-accounts.ts:98:1)
[0]     at <anonymous> (/Users/pc/my/code/openSource/graphql-wallet/node_modules/@solana/rpc-graphql/src/loaders/transaction.ts:56:14) {
[0]   context: { __code: -32602, __serverMessage: 'Invalid param: Invalid' }

I fetched signatures for corresponding account. However while querying the corresponding transaction, I get the above error.


Assorted fontmake problems when building Schticks (2-axis varfont)

I’ve run into a number of problems with glyphsLib and fontmake when trying to build Schticks Text, a 2-axis variable font based on the STIX Two Text fonts (OFL). The fonts I’m talking about are in the https://github.com/twardoch/schticks-fonts-ofl, Schticks/Sources/25-gly-mm-width1 subfolder.
Configuration

Mac OS X 10.12.3
Glyphs 2.4.1 (965)
Homebrew-installed Python 2.7.13

I've just installed the devel fontmake by running in the repo:
pip install --user -r dev-requirements.txt .
pip install --user .

so I had:

booleanOperations-0.6.4
cu2qu-1.1.2.dev0
defcon-0.2.1:
fonttools-3.6.0
fontmake-1.1.1.dev0
glyphsLib-1.3.1.dev0
MutatorMath-2.0.2.dev0
ufo2ft-0.3.2.dev0

I also did it via:
pip install --user -r requirements.txt .
pip install --user .

So I had:

booleanOperations-0.6.4
cu2qu-1.1.1
defcon-0.2.1
fonttools-3.6.0
glyphsLib-1.3.0
MutatorMath-2.0.1
ufo2ft-0.3.1

In both configurations, when running fontmake like this:
fontmake -g SchticksText-MM.glyphs --verbose 'DEBUG' --keep-overlaps --production-names -o variable

on the SchticksText-MM.glyphs font in 25-gly-mm-width1/bak font from my https://github.com/twardoch/schticks-fonts-ofl repo, I get a number of problems.
Spurious illegal names
fontmake reports one illegal glyph name but such a glyph name is not in the font.
INFO:fontmake.font_project:Checking Glyphs source for illegal glyph names
WARNING:fontmake.font_project:Found 1 glyph names containing hyphens: B: -219
WARNING:fontmake.font_project:Replacing all hyphens with periods.

com.adobe.type.autohint data
The glyphs contain a custom com.adobe.type.autohint entry which makes glyphsLib fail. Stuff in userData that cannot be parsed should be ignored.
userData = {
com.adobe.type.autohint = <ICA8aGludFNldExpc3Q+CiAgICA8aGludHNldCBwb2ludFRhZz0iaHIwMCI+CiAgICAgIDxoc3RlbSBwb3M9Ii0xNjkiIHdpZHRoPSI1MyIgLz4KICAgICAgPGhzdGVtIHBvcz0iLTgiIHdpZHRoPSIxMTUiIC8+CiAgICAgIDxoc3RlbSBwb3M9IjQ2MSIgd2lkdGg9IjIwOCIgLz4KICAgICAgPGhzdGVtIHBvcz0iNjMxIiB3aWR0aD0iMzgiIC8+CiAgICAgIDxoc3RlbSBwb3M9IjczNCIgd2lkdGg9IjUxIiAvPgogICAgICA8dnN0ZW0gcG9zPSIxMCIgd2lkdGg9IjUxIiAvPgogICAgICA8dnN0ZW0gcG9zPSIxMjYiIHdpZHRoPSI3NSIgLz4KICAgICAgPHZzdGVtIHBvcz0iMTI2IiB3aWR0aD0iOTYiIC8+CiAgICAgIDx2c3RlbSBwb3M9IjI0MyIgd2lkdGg9IjExNSIgLz4KICAgICAgPHZzdGVtIHBvcz0iMjgwIiB3aWR0aD0iNDAiIC8+CiAgICAgIDx2c3RlbSBwb3M9IjM5MSIgd2lkdGg9IjkwIiAvPgogICAgICA8dnN0ZW0gcG9zPSI1NDMiIHdpZHRoPSI1MiIgLz4KICAgIDwvaGludHNldD4KICA8L2hpbnRTZXRMaXN0Pgo=>;
};

Traceback
INFO:glyphsLib:Parsing .glyphs file
Traceback (most recent call last):
  File "/Users/adam/Library/Python/2.7/bin/fontmake", line 11, in <module>
    load_entry_point('fontmake', 'console_scripts', 'fontmake')()
  File "/Users/adam/Developer/vcs/github/googlei18n/fontmake/Lib/fontmake/__main__.py", line 171, in main
    project.run_from_glyphs(glyphs_path, **args)
  File "/Users/adam/Developer/vcs/github/googlei18n/fontmake/Lib/fontmake/font_project.py", line 368, in run_from_glyphs
    glyphs_path, family_name)
  File "/Users/adam/Developer/vcs/github/googlei18n/fontmake/src/fonttools/Lib/fontTools/misc/loggingTools.py", line 372, in wrapper
    return func(*args, **kwds)
  File "/Users/adam/Developer/vcs/github/googlei18n/fontmake/Lib/fontmake/font_project.py", line 93, in build_master_ufos
    family_name=family_name)
  File "/Users/adam/Developer/vcs/github/googlei18n/fontmake/src/glyphslib/Lib/glyphsLib/__init__.py", line 91, in build_masters
    filename, include_instances=True, family_name=family_name)
  File "/Users/adam/Developer/vcs/github/googlei18n/fontmake/src/glyphslib/Lib/glyphsLib/__init__.py", line 64, in load_to_ufos
    data = load(file_or_path)
  File "/Users/adam/Developer/vcs/github/googlei18n/fontmake/src/glyphslib/Lib/glyphsLib/__init__.py", line 44, in load
    return loads(fp.read())
  File "/Users/adam/Developer/vcs/github/googlei18n/fontmake/src/glyphslib/Lib/glyphsLib/__init__.py", line 53, in loads
    data = p.parse(value)
  File "/Users/adam/Developer/vcs/github/googlei18n/fontmake/src/glyphslib/Lib/glyphsLib/parser.py", line 43, in parse
    result, i = self._parse(text, 0)
  File "/Users/adam/Developer/vcs/github/googlei18n/fontmake/src/glyphslib/Lib/glyphsLib/parser.py", line 55, in _parse
    return self._parse_dict(text, i)
  File "/Users/adam/Developer/vcs/github/googlei18n/fontmake/src/glyphslib/Lib/glyphsLib/parser.py", line 83, in _parse_dict
    res[name], i = self._parse(text, i)
  File "/Users/adam/Developer/vcs/github/googlei18n/fontmake/src/glyphslib/Lib/glyphsLib/parser.py", line 61, in _parse
    return self._parse_list(text, i)
  File "/Users/adam/Developer/vcs/github/googlei18n/fontmake/src/glyphslib/Lib/glyphsLib/parser.py", line 104, in _parse_list
    list_item, i = self._parse(text, i)
  File "/Users/adam/Developer/vcs/github/googlei18n/fontmake/src/glyphslib/Lib/glyphsLib/parser.py", line 55, in _parse
    return self._parse_dict(text, i)
  File "/Users/adam/Developer/vcs/github/googlei18n/fontmake/src/glyphslib/Lib/glyphsLib/parser.py", line 83, in _parse_dict
    res[name], i = self._parse(text, i)
  File "/Users/adam/Developer/vcs/github/googlei18n/fontmake/src/glyphslib/Lib/glyphsLib/parser.py", line 55, in _parse
    return self._parse_dict(text, i)
  File "/Users/adam/Developer/vcs/github/googlei18n/fontmake/src/glyphslib/Lib/glyphsLib/parser.py", line 83, in _parse_dict
    res[name], i = self._parse(text, i)
  File "/Users/adam/Developer/vcs/github/googlei18n/fontmake/src/glyphslib/Lib/glyphsLib/parser.py", line 70, in _parse
    self._fail('Unexpected content', text, i)
  File "/Users/adam/Developer/vcs/github/googlei18n/fontmake/src/glyphslib/Lib/glyphsLib/parser.py", line 145, in _fail
    raise ValueError('%s:\n%s' % (message, text[i:i + 79]))
ValueError: Unexpected content:
 <ICA8aGludFNldExpc3Q+CiAgICA8aGludHNldCBwb2ludFRhZz0iaHIwMCI+CiAgICAgIDxoc3Rlb

Non-integer underline position
The font contains a weird entry:
name = underlinePosition;
value = -77.5;

which makes glyphsLib fail. I don't know why this entry is there but glyphsLib should round, not fail.
Traceback
INFO:glyphsLib:Parsing .glyphs file
INFO:glyphsLib:Casting parsed values
Traceback (most recent call last):
  File "/Users/adam/Library/Python/2.7/bin/fontmake", line 11, in <module>
    load_entry_point('fontmake', 'console_scripts', 'fontmake')()
  File "/Users/adam/Developer/vcs/github/googlei18n/fontmake/Lib/fontmake/__main__.py", line 171, in main
    project.run_from_glyphs(glyphs_path, **args)
  File "/Users/adam/Developer/vcs/github/googlei18n/fontmake/Lib/fontmake/font_project.py", line 368, in run_from_glyphs
    glyphs_path, family_name)
  File "/Users/adam/Developer/vcs/github/googlei18n/fontmake/src/fonttools/Lib/fontTools/misc/loggingTools.py", line 372, in wrapper
    return func(*args, **kwds)
  File "/Users/adam/Developer/vcs/github/googlei18n/fontmake/Lib/fontmake/font_project.py", line 93, in build_master_ufos
    family_name=family_name)
  File "/Users/adam/Developer/vcs/github/googlei18n/fontmake/src/glyphslib/Lib/glyphsLib/__init__.py", line 91, in build_masters
    filename, include_instances=True, family_name=family_name)
  File "/Users/adam/Developer/vcs/github/googlei18n/fontmake/src/glyphslib/Lib/glyphsLib/__init__.py", line 64, in load_to_ufos
    data = load(file_or_path)
  File "/Users/adam/Developer/vcs/github/googlei18n/fontmake/src/glyphslib/Lib/glyphsLib/__init__.py", line 44, in load
    return loads(fp.read())
  File "/Users/adam/Developer/vcs/github/googlei18n/fontmake/src/glyphslib/Lib/glyphsLib/__init__.py", line 55, in loads
    cast_data(data)
  File "/Users/adam/Developer/vcs/github/googlei18n/fontmake/src/glyphslib/Lib/glyphsLib/casting.py", line 510, in cast_data
    _convert_data(data, True, _TYPE_STRUCTURE)
  File "/Users/adam/Developer/vcs/github/googlei18n/fontmake/src/glyphslib/Lib/glyphsLib/casting.py", line 526, in _convert_data
    _convert_data(cur_data, to_typed, cur_type)
  File "/Users/adam/Developer/vcs/github/googlei18n/fontmake/src/glyphslib/Lib/glyphsLib/casting.py", line 528, in _convert_data
    data[key] = cur_type.convert(data[key], to_typed)
  File "/Users/adam/Developer/vcs/github/googlei18n/fontmake/src/glyphslib/Lib/glyphsLib/casting.py", line 78, in convert
    return self.read(data) if to_typed else self.write(data)
  File "/Users/adam/Developer/vcs/github/googlei18n/fontmake/src/glyphslib/Lib/glyphsLib/casting.py", line 309, in read
    value = int(value)
ValueError: invalid literal for int() with base 10: '-77.5'

TypeError: int() argument must be a string or a number, not 'list'
After I manually cleaned the offending portions in the .glyphs files using a text editor, and stored the font in 25-gly-mm-width1, then run fontmake on it, I get:
TypeError: int() argument must be a string or a number, not 'list'

I have no idea what to do with this, so I cannot proceed any further.
Traceback
INFO:glyphsLib:Parsing .glyphs file
INFO:glyphsLib:Casting parsed values
Traceback (most recent call last):
  File "/Users/adam/Library/Python/2.7/bin/fontmake", line 11, in <module>
    load_entry_point('fontmake', 'console_scripts', 'fontmake')()
  File "/Users/adam/Developer/vcs/github/googlei18n/fontmake/Lib/fontmake/__main__.py", line 171, in main
    project.run_from_glyphs(glyphs_path, **args)
  File "/Users/adam/Developer/vcs/github/googlei18n/fontmake/Lib/fontmake/font_project.py", line 368, in run_from_glyphs
    glyphs_path, family_name)
  File "/Users/adam/Developer/vcs/github/googlei18n/fontmake/src/fonttools/Lib/fontTools/misc/loggingTools.py", line 372, in wrapper
    return func(*args, **kwds)
  File "/Users/adam/Developer/vcs/github/googlei18n/fontmake/Lib/fontmake/font_project.py", line 93, in build_master_ufos
    family_name=family_name)
  File "/Users/adam/Developer/vcs/github/googlei18n/fontmake/src/glyphslib/Lib/glyphsLib/__init__.py", line 91, in build_masters
    filename, include_instances=True, family_name=family_name)
  File "/Users/adam/Developer/vcs/github/googlei18n/fontmake/src/glyphslib/Lib/glyphsLib/__init__.py", line 64, in load_to_ufos
    data = load(file_or_path)
  File "/Users/adam/Developer/vcs/github/googlei18n/fontmake/src/glyphslib/Lib/glyphsLib/__init__.py", line 44, in load
    return loads(fp.read())
  File "/Users/adam/Developer/vcs/github/googlei18n/fontmake/src/glyphslib/Lib/glyphsLib/__init__.py", line 55, in loads
    cast_data(data)
  File "/Users/adam/Developer/vcs/github/googlei18n/fontmake/src/glyphslib/Lib/glyphsLib/casting.py", line 510, in cast_data
    _convert_data(data, True, _TYPE_STRUCTURE)
  File "/Users/adam/Developer/vcs/github/googlei18n/fontmake/src/glyphslib/Lib/glyphsLib/casting.py", line 526, in _convert_data
    _convert_data(cur_data, to_typed, cur_type)
  File "/Users/adam/Developer/vcs/github/googlei18n/fontmake/src/glyphslib/Lib/glyphsLib/casting.py", line 528, in _convert_data
    data[key] = cur_type.convert(data[key], to_typed)
  File "/Users/adam/Developer/vcs/github/googlei18n/fontmake/src/glyphslib/Lib/glyphsLib/casting.py", line 78, in convert
    return self.read(data) if to_typed else self.write(data)
  File "/Users/adam/Developer/vcs/github/googlei18n/fontmake/src/glyphslib/Lib/glyphsLib/casting.py", line 136, in read
    return int(src)
TypeError: int() argument must be a string or a number, not 'list'

@twardoch thanks for the detailed bug report.
About the parser error issue with the custom userData, the issue should go away if you wrap the string value with quotes com.adobe.type.autohint = "<ICA8aGludFNldExp..>".
The glyphsLib parser accepts any character if it is between double quotes (the quote character iteself has to be escaped with backslash); whereas if it is outside of quotes, it only accepts characters [-_./$A-Za-z0-9]+.
Here is the regex used:
https://github.com/googlei18n/glyphsLib/blob/700d96cd9c546c20208606123f34806bb6156a79/Lib/glyphsLib/parser.py#L29
I wonder how that piece of data was written into the glyphs source file? Did Glyphs write it there, or some external tool?

the last traceback, TypeError: int() argument must be a string or a number, not 'list', has to to with the fact that your glyphs file contains color definitions such as:
color = (255,153,153,1);

But glyphsLib is expecting color definitions to be integers:
https://github.com/googlei18n/glyphsLib/blob/4b40d7f0ee34e8637069afb1b6776e4a9d43c6b9/Lib/glyphsLib/casting.py#L473
The inline comment says that this attribute is undocumented, and indeed I couldn't find any references to it in https://github.com/schriftgestalt/GlyphsSDK/blob/master/GlyphsFileFormat.md
maybe @schriftgestalt could tell us how glyphs' colors should be encoded in the source file?
I just tried to set the color of a glyph, saved the file and opened it up in a text editor, and the color is encoded as a decimal integer.
Could it be that the tuple variant was produced by earlier version of Glyphs?

@twardoch can you tell us how that unquoted string com.adobe.type.autohint = <ICA8aGludFNldExp..> was written in the glyphs source file? If it was written by Glyphs.app itself, then we will need to adapt the glyphsLib parser to somehow accept such notation.
@schriftgestalt may I ask again how should glyph colors be encoded? As integers (like glyphsLib expects) or as (RGBA?) tuples like in Adam's .glyphs file?

Was this glyphs file generated with fl6? 😂
On Feb 11, 2017 1:14 PM, "Cosimo Lupo"<EMAIL_ADDRESS>wrote:

@twardoch https://github.com/twardoch can you tell us how that unquoted
string com.adobe.type.autohint = <ICA8aGludFNldExp..> was written in the
glyphs source file? If it was written by Glyphs.app itself, then we will
need to adapt the glyphsLib parser to somehow accept such notation.
@schriftgestalt https://github.com/schriftgestalt may I ask again how
should glyph colors be encoded? As integers (like glyphsLib expects) or as
(RGBA?) tuples like in Adam's .glyphs file?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/googlei18n/fontmake/issues/247#issuecomment-279164802,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAP9y0m-wq2MdrHtwj0EwhHyI-UH7_MUks5rbfp4gaJpZM4L3cMy
.

Color can be encoded as integer or as RGBA tupel.
The color and autohint info most likely comes from a .ufo file that was opened in Glyphs. The RGBA storage is mostly to be able to round trip .ufo.

I see, thanks Georg.
But shouldn't Glyphs.app wrap those userData strings with quotes "?
glyphsLib parser is choking because it only recognizes as a valid "value" token a string comprised of one or more [-_./$A-Za-z0-9] characters, unless that string is included in quotes, in which case .* anything matches.
https://github.com/googlei18n/glyphsLib/blob/700d96cd9c546c20208606123f34806bb6156a79/Lib/glyphsLib/parser.py#L29
value_re = r'(".*?(?<!\\)"|[-_./$A-Za-z0-9]+)'

If this is not correct, then we should modify the parser.

the value in com.adobe.type.autohint is stored as data and that is enclosed in <>. So it is not a string that needs to be enclosed in quotes.

Ok thanks. So it looks like the glyphsLib parser does not properly support binary data values in ASCII plist:
https://developer.apple.com/library/content/documentation/Cocoa/Conceptual/PropertyLists/OldStylePlists/OldStylePLists.html

NSData
Binary data is enclosed in angle brackets and encoded in hexadecimal ASCII. Spaces are ignored. > For example:

<0fbd777 1c2735ae>

I'll file a separate issue over there.

@davelab6 asked:

Was this glyphs file generated with fl6? 😂

I wrote in my report:

Configuration:

Mac OS X 10.12.3
Glyphs 2.4.1 (965)
Homebrew-installed Python 2.7.13

Ps. I shall add that the .glyphs file did indeed originate from opening UFO files which were generated with vfb2ufo from a VFB saved in FontLab Studio 5. I used FontLab VI at an earlier stage. The folder list should give you an idea about the sequence of steps involved.


chore(folio): Update types

The type of this PR is: Chore
Adds some missing interface types.
Description

PR Checklist (tick all before merging)

[x] If I've added a new file to the tree I've exported it from the common index.ts
[x] I've added comments with examples for any new interfaces and ensured that they're in the docs
[x] No platform-specific terminology has been used outside of click and tap (platform is inferred by the DB storing events)

:rocket: PR was released in v4.167.0 :rocket:


Support for adding "selector link" and "selector container" when adding a service.

Support for adding "selector link" and "selector container" when adding a service.

Is this the issue that would resolve this?:

Or is there a way to specify this selectorContainer in some other way.. ?

@kaos There needs to be a way in the UI to set the selectorLink and selectorContainer that is in the API that is something outside of just using labels.

Oh? Ok. It seems to work just fine using labels alone when using a compose
file, though.
mån 14 mars 2016 kl. 21:35 skrev Denise<EMAIL_ADDRESS>
@kaos https://github.com/kaos There needs to be a way in the UI to set
the selectorLink and selectorContainer that is in the API that is something
outside of just using labels.
—
Reply to this email directly or view it on GitHub
https://github.com/rancher/rancher/issues/2270#issuecomment-196510776.

Selector services are in the 2.0 UI.


Upgrade to the latest LLVM

cclyzer++ has only been tested with LLVM 10. Upgrading would involve:

[ ] Reading through the LLVM changelogs for any relevant changes in semantics
[ ] Upgrading the FactGenerator and C++ interfaces to the new LLVM API

We should at least look into supporting multiple versions of LLVM. This makes cclyzer++ applicable in a wider variety of situations (e.g., a project where an older version of LLVM is required). More prosaically, it means we can keep the cclyzer++ submodule in the MATE project up-to-date without spending time upgrading MATE's LLVM support. This is valuable because MATE has a number of tests that could help find bugs in cclyzer++.
That being said, there might be major downsides to supporting multiple LLVM versions, the most likely candidate being an inordinate expense in terms of developer, build, or CI time; or in terms of (Docker) build complexity.

Reopening because LLVM 15 isn't fully supported yet due to opaque pointers.


Email History Role Maintenance with ChatGPT

Because of the reply quotation system of emails, ChatGPT appears to have difficulty maintaining its role (as defined in the system prompt).
A potential solution is to separate each email message and correctly associate them with the appropriate role (assistant or user) in the JSON messages for the ChatGPT API.
To do so, we could use the thread id (work with gmail only).

#2 Fixed