how to add sequence provisioning in the task AzureVmssDeploymentTask

Note
Issues in this repo are for tracking bugs, feature requests and questions for the tasks in this repo
For a list:
https://github.com/Microsoft/azure-pipelines-tasks/tree/master/Tasks
If you have an issue or request for the Azure Pipelines service, use developer community instead:
https://developercommunity.visualstudio.com/spaces/21/index.html )
Required Information
Entering this information will route you directly to the right team and expedite traction.
Question, Bug, or Feature?
Type: Feature/Bug
Enter Task Name: AzureVmssDeploymentTask
list here (V# not needed):
https://github.com/Microsoft/azure-pipelines-tasks/tree/master/Tasks
Environment

Server - Azure Pipelines

Issue Description
How do we add the extension sequence for the vm scale set?
In my scenario we have antimalware extension which supposed to run the before the custom script extension, when i trigger the deployment through the pipeline it works i add the antimalware extension and then will add the custom script extension task.
Im facing issue when i reimage the vmss from the portal, it doesnt guarantee that the custom script runs after the antimalware.
Is there any provision to send the extension sequencing through the pipeline?

Extension sequencing for VM Scale set can be achieved via multiple method. Read HERE
You can try the Template one while creating the VM scale set for the first time. This might retain the sequence even when you try to reimage.
I would also advice you to ask the Azure VM/Compute team for how sequencing can be achieved. I am thus closing this issue, as it seem to be much of a knowledge ask from Azure VM/Compute team.


packed EPUB loading: jQuery deferred promise not resolved when missing resources => infinite wait

To reproduce, try loading any of the PragProg EPUBs:
https://pragprog.com
(drag-n-drop downloaded EPUB on cloud reader window)

Fixed here: https://github.com/readium/readium-js/commit/f485026a514e0ac511e7ac02ab57d4bbc8ae7227


Create update function for the installer script

The Spec-Up-T tool is updated according to NPM's standard operating procedure, namely via npm update.
However, the installation itself (installed via npx create-spec-up-t my-spec) can also use an update function. This will include overwriting the package.json.

I am not so sure if this is a good idea. It's error-prone, and a manual process seems safer, also since it will not be needed that often, especially in the future. It seems a waste of time.

Update procedures like this are usually a manual task. This is for a reason. Let's not do it.


React hook useSwiperSlide() returns null

Check that this is really a bug

[X] I confirm

Reproduction link
https://codesandbox.io/s/swiper-react-state-swiper-8-op0ij8
Bug description
I am trying to use React hook  useSwiperSlide(). To be able to use for example:
{swiperSlide?.isActive ? "ACTIVE" : "not active"}
But it returns null? Do I miss something?
Expected Behavior
No response
Actual Behavior
No response
Swiper version
8.0.7
Platform/Target and Browser Versions
Google Chrome
Validations

[X] Follow our Code of Conduct
[X] Read the docs.
[X] Check that there isn't already an issue that request the same feature to avoid creating a duplicate.
[X] Make sure this is a Swiper issue and not a framework-specific issue

Would you like to open a PR for this bug?

[ ] I'm willing to open a PR

useSwiper must be used by components inside a Swiper element. See this example.

useSwiper must be used by components inside a Swiper element. See this example.


CRDs supports for helm 2, 3 altogether

Toward https://github.com/giantswarm/giantswarm/issues/11075
Changes are to use the same CRD for both Helm2, Helm3.

Testing on both helm2, 3, and both install/upgrade.

@giantswarm/team-batman-engineers I can't assign team-batman :(


Requirements and Feasibility Analysis "Bazelizing Open Image Denoise"

(this is in context of the bounty https://app.bountysource.com/issues/117458567-black-stripes-in-denoised-image)
Article
https://medium.com/@Vertexwahn/bazelizing-open-image-denoise-part-1-the-journey-begins-a16c78ea1b88
Requirement "Bazel"

Open Image Denoise: From CMake to Bazel
The build system of Open Image Denoise is CMake. Instead of CMake I want to use Open Image Denoise  using Bazel. This requires to bazelize Open Image Denoise first.

Is this a hard requirement (e.g. "absolutely non-disputable requirement) ?
what are the main 3 reasons to use bazel / to not use Cmake?

Feasibility

The effort to use a non-supported (by the core team) build-system  for a C++ application can be excessive

Achieving the initial port can become a full-time problem
Maintenance (update to new versions) can become non-feasible

https://github.com/Vertexwahn/rules_oidn/issues/1#issuecomment-1464913690: Bazel is a MUST Requirement, closing.

There can be different reasons why someone Is using Bazel instead of CMake to Bazel. It depends on different things like project size, project type, etc.

Caching: Bazel knows exactly which tests need to be re-executed when something has changed (e.g. modified source code or data file). This leads to significantly slower test times compared to CMake. This is important when doing test-driven development.
Bazel is polyglot - it supports many other languages (such as Python, Rust, Go, Java, etc.) - that gives me the opportunity to combine my python scripts that generate my own weights in one build without having to switch build systems (there is same python support in CMake but that is far beyond what Bazel can do here -> for instance, there is Bazel Python Plugin for PyCharm)
CMake Legacy: The CMake configuration language has a lot of legacy stuff - Policies and Modern CMake guides help you to work around it - but the whole CMake "language" look somehow old and odd. Bazel has Starlark (python subset) and a clear concept and supports windows for deprecating old features.

As a sidenote: I do not like CMake personally. Inconsistencies start with naming:

CMake => cmake
CMakeLists.txt => cmake.txt


Providing examples of simple types

I have some API endpoints that return string arrays; these obviously won't work with the NSwag.Examples provider model, which is looking for complex types. Do you have any thoughts/suggestions on how to provide (possibly inline) example values?
I was hoping that the <example> tag would be picked up and parsed as a JSON object, but that doesn't seem to happen when NSwag.Examples is present e.g.:
/// <example>[ \"value1\", \"value2\", \"value3\" ]</example>
[ProducesResponseType(typeof(string[]), StatusCodes.Status200OK)]
[HttpGet("/string-values")]
public IActionResult GetStringValues()
{
  return Ok(new[] { "value1", "value2", "value3" });
}

I could write my own operation processor easily enough, but I was hoping there might be something more obvious that I might have overlooked?

Closing due to inactivity. Please reopen when needed.


Improve performance saving helper function results

https://github.com/lmu-dbs/feeed/pull/7/files#r1453533368

Maybe with @cache.

maybe also helpful:
https://stackoverflow.com/questions/815110/is-there-a-decorator-to-simply-cache-function-return-values

Specially epa_based features are taking a very long time.
SUCCESS: 23th BPI_Challenge_2017.xes features saved in output/features/real_event_logs/BPI_Challenge_2017.json
     INFO: BPI_Challenge_2019.xes starting at 95, normalized_variant_entropy from complexity took 0:21:56.256513 sec, next sequence_entropy...
     INFO: BPI_Challenge_2019.xes starting at 96, sequence_entropy from complexity took 0:20:16.132700 sec, next normalized_sequence_entropy...
     INFO: BPI_Challenge_2019.xes starting at 97, normalized_sequence_entropy from complexity took 0:20:40.538643 sec, next sequence_entropy_linear_forgetting...                                                                                INFO: BPI_Challenge_2019.xes starting at 98, sequence_entropy_linear_forgetting from complexity took 0:21:22.343736 sec, next normalized_sequence_entropy_linear_forgetting...
     INFO: BPI_Challenge_2019.xes starting at 99, normalized_sequence_entropy_linear_forgetting from complexity took 0:18:59.822657 sec, next sequence_entropy_exponential_forgetting...
     INFO: BPI_Challenge_2019.xes starting at 100, sequence_entropy_exponential_forgetting from complexity took 0:22:27.241479 sec, next normalized_sequence_entropy_exponential_forgetting...
     INFO: BPI_Challenge_2019.xes starting at 101, normalized_sequence_entropy_exponential_forgetting from complexity took 0:20:18.781432 sec, last feature.
SUCCESSFULLY: 100 features for BPI_Challenge_2019.xes took 3:23:33.029262 sec.

Suggestion: Can we cache the epa or create a self.epa for these features to improve computation performance?

I see it is specially epa_based features, which are taking the longest. Other feature types are not as critical.


Why Pair-wise Circle loss explosion?

I add the pair-wise circle loss, but the loss value is very huge, up to 300-400.
I used your defaut  set:  _m=0.25, _s =128, _scale =1.0.
Should set the _scale very small like 0.001? But it is diffierent from circle loss paper.

We are looking forward to the official open-source code of circle loss.

Pairwise circle loss is updated, you can try it.


VoiceOver prepends "1" to every checked menu item

Bug report
Current Behavior
When using dropdown menu primitive with macOS VoiceOver, it prepends "1" when announcing ticked items like so:

This happens with ticked radio and checkbox items only, regardless of whether they are in a group.
Expected behavior
Expected to announce just the item text contents and its functional description.
Reproducible example
https://codesandbox.io/s/0tr78?module=App.js
Your environment
macOS Big Sur 11.4, any browser

This is a VoiceOver bug I believe https://github.com/microsoft/fluentui/issues/14064
Not really sure what we can do here. The same thing happens in the WAI ARIA demo here :( https://www.w3.org/TR/wai-aria-practices/examples/menubar/menubar-2/menubar-2.html

Oh OK, nothing we can do about it I guess. Thanks for checking it out, Jenna.


FancyZones does not respect "space around zones" on/off toggle.

Microsoft PowerToys version
0.64.0
Installation method
PowerToys auto-update
Running as admin
Yes
Area(s) with issue?
FancyZones
Steps to reproduce
Using a custom layout, if the "space around zones" toggle is off, the space/padding in px from the slider is still used and the state of the toggle seems to be ignored. Create a custom zone and set the "space around zones" slider to something obviously/noticeably large like 50 px, then set the toggle to off, then save. Use the custom zone you just made and you'll see the 50 px value is used instead of 0 px value when snapping new windows.
✔ Expected Behavior
With "space around zones" toggle set to off, expect the zones to have a 0 px space/padding between them.
❌ Actual Behavior
With "space around zones" toggle set to off, ends up using the space/padding px value from whatever the slider is set to when it should instead be set to 0 px.
Other Software
No response

Apologies, appears to be a /dup of https://github.com/microsoft/PowerToys/issues/21630.
Seems quite a few duplicates were just opened...


Improve Footer-white-04

Need to improve this component: Footer-white-04
Improvements required
The current design is not as per the Figma file.
Current design:

Can I work on it? @Rajdip019

Is this issue solved?

@Rajdip019 . No, I am working on it.


Add git version prerequisite to DEVELOPMENT.md.

sync/sync.py fails if the installed git version is older than 1.8.5.
That is because sync/sync.py calls git command with -C option internally, and the option was
introduced at git 1.8.5. 1.8.5 is quite old but the default git version is older than it in certain
Linux distribution. Adding git version prerequisite could be more developer-friendly.

Changes

Submitter Checklist
These are the criteria that every PR should meet, please check them off as you
review them:

[ ] Includes docs (if user facing)
[ ] Commit messages follow commit message best practices

_See the contribution guide
for more details.

小林さんレビュー済のためマージ。


Need to be subscription Owner/Contributor?

I'm not sure this statement is true.
"To create Data Factory instances, the user account that you use to sign in to Azure must be a member of the contributor role, the owner role, or an administrator of the Azure subscription."
We just had a developer advise us that they can't create a datafactory instance. They received the error "The client XXX does not have authorization to perform action 'microsoft.datafactory/register/action' over scope xxxx'
We have a subscription called "Dev".
We have a resource group called "Dev-DataFactory"
Developer A has Contributor to "Dev-DataFactory", and no permissions on the subscription.
As the subscription Owner, I registered the datafactories Resource Provider, now Developer A can build Data Factories.

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: cf5455de-d419-3e45-fcd1-e2701b32ff95
Version Independent ID: d25237b1-99b8-1c00-f560-450abba8f821
Content: Roles and permissions for Azure Data Factory - Azure Data Factory
Content Source: articles/data-factory/concepts-roles-permissions.md
Service: data-factory
Sub-service: security
GitHub Login: @nabhishek
Microsoft Alias: abnarain

@mundayn
Thanks for your feedback! We will investigate and update as appropriate.

@mundayn
Please see the link below for further information.
https://learn.microsoft.com/en-us/azure/data-factory/concepts-roles-permissions#roles-and-requirements:~:text=by those roles.-,Roles and requirements,more info about this role%2C see Data Factory Contributor role.,-Resource Manager template

@mundayn
We are going to close this thread as resolved but if there are any further questions regarding the documentation, please tag me in your reply and we will be happy to continue the conversation.


[stable/redis-ha] fix redis-ha NOTE.txt, stable/redis-ha don't create master-0 pod

What this PR does / why we need it:
Fix NOTE.txt.
stable/redis-ha don't create {{ template "redis-ha.fullname" . }}-master-0 pod.
So, user can not execute kubectl exec -it {{ template "redis-ha.fullname" . }}-master-0 bash command.
All pod can use redis-cli command.
This line combined with other line.
Which issue this PR fixes
Special notes for your reviewer:

Hello, approvers! please check me :)
/assign @seanknox

please check me someone 🙏
/assign @lachie83
/assign @linki
/assign @mgoodness
/assign @prydonius
/assign @sameersbn
/assign @viglesiasce
/assign @foxish
/assign @unguiculus
/assign @scottrigby
/assign @mattfarina

/lgtm

Oh, test failed?
This PR change only NOTE.txt, I didn't change application logic... :thinking:

/retest

/retest

Wow, three tests have output different result ... X(

/retest

I think, We need check test infrastructure... X(

Wow, Thank you!


Better support for completion when in the middle of a word

Consider the following file:
int main()
{
    int refactoring_twice;
    refact<cursor>wice;
}

and the cursor is at the indicated place. Now I trigger autocomplete, I get the suggestion for refactoring_twice but after insertion the text becomes:
int main()
{
    int refactoring_twice;
    refactoring_twicewice;
}

The autocompletion is inserted just as clangd says (I verified with debug logs and clangd says to replace the refact part only).
Before I have used ycmd for completion and it would replace the whole refactwice word.
I am using clangd 10.0.0, sorry if this is available in git.

I think this is covered by #387.

I think this is covered by #387.

Indeed, closing this one.


Fix async/await toolchain issues

Successor bug to #7552.
Parts of #7563 had to be reverted because of bugs reported at #7777.
This issue is to investigate a solution that will work for both toolchains for all necessary use cases.
cc: @peterfriese

This is progressing and will probably land early next week. In the meantime, a workaround should be the small updates from two mentioned above after spm installation.


SinusoidalPositionalEmbedding

System Info
Hello, I'm now in TRT - LLM adapter m2m100, but m2m100 USES is SinusoidalPositionalEmbedding, what should I do to make it work。
https://github.com/huggingface/transformers/blob/main/src/transformers/models/m2m_100/modeling_m2m_100.py#L86
Who can help?
@ncomly-nvidia
Information

[ ] The official example scripts
[X] My own modified scripts

Tasks

[ ] An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
[X] My own task or dataset (give details below)

Reproduction
In tensorrt_llm/models/enc_dec/model. The direct reference in py M2M100SinusoidalPositionalEmbedding, and in the EncDecEmbedding initialization, and then directly used in the forward, as shown below:
def forward(
      self,
      input_ids,
      position_ids=None,
      token_type_ids=None,
      prompt_embedding_table=None,
      prompt_tasks=None,
      prompt_vocab_size=None,
  ):
      # position_ids and token_type_ids are provided inputs
      # and should not be formulated determinisitically
      ptuning_args = []
      if self.use_prompt_tuning:
          ptuning_args = [prompt_embedding_table, prompt_tasks, prompt_vocab_size]
      x = self.vocab_embedding(input_ids, *ptuning_args) * self.embedding_scale
      self.register_network_output("word_embeddings", x)
      embed_pos = self.embed_positions(input_ids, x)
      embed_pos = embed_pos.to(x.device)
      hidden_states = x + embed_pos

      # if self.position_embedding:
      #     pos_emb = self.position_embedding(position_ids)
      #     self.register_network_output("position_embeddings", pos_emb)
      #     x = x + pos_emb
      # if self.token_type_embedding:
      #     x = x + self.token_type_embedding(token_type_ids)

      # if self.embedding_layernorm:
      #     x = self.embedding_layernorm(x)

      return hidden_states

Expected behavior
build successfully
actual behavior
File "/root/miniconda3/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 162, in forward
bsz, seq_len = input_ids.size()
ValueError: not enough values to unpack (expected 2, got 1)
additional notes
I think I've made it clear that if you want to replicate it completely, I can upload the code I'm currently using to github

@dingjingzhen Do you still have the problem? If not, we will close it soon.


Update '/networks' api to return PoolId

Consider a case when remote libnetwork and ipam drivers are being used. Two networks are created with subnets having same cidr. The calls to IPAM driver for pool allocation have no info on the network and the calls to the remote driver have no info on poolID. Now when address request reaches to ipam driver, there is no way to determine correct subnet using poolID(which is received from libnetwork).
To address this /network api has been enhanced to give info on PoolIDs. Remote driver can make this call at network creation and maintain network-poolID mapping.
Closes #23025
Signed-off-by: vikaschoudhary16<EMAIL_ADDRESS>

Hi Folks, Would really appreciate any feedback on these changes.
Thanks.

Thanks for the contribution. I think the change makes sense, but discussion should happen over the docker/engine-api repository.

Cc @mavenugo.

@icecrime ,
Thanks a lot for your response.
I am new to docker contribution/development workflow. I can see some checks are failing and some are pending. Not sure if I should be worried about these or how to fix them. Any help or support will be much appreciated.


Ensembl HGVS vs Refseq HGVS in germline variant calling pipeline

Hello!
Thank you for the excellent pipeline!
VEP in variant calling pipeline reports HGVS annotations along the ensembl transcript, i.e.:
ENST00000357730.2:c.1392+1G>A for
ch1 44466835 C T
It is correct, but some people used to RefSeq annotations, i.e.:
c.1347+1G>A NM_001261380.1
It is easy to force VEP to produce RefSeq annotations:

unpack Refseq cache:
ftp://ftp.ensembl.org/pub/current_variation/VEP/homo_sapiens_refseq_vep_87_GRCh37.tar.gz
to
bcbio/genomes/Hsapiens/GRCh37/vep
run VEP with --species homo_sapiens_refseq parameter.

Is it possible to add this functionality to the variant calling pipeline, just to have an option hgvs:ensembl/refseq?
Many thanks!
Sergey

Sergey;
bcbio has been moving to consistently use the Ensembl/GENCODE representations rather than RefSeq. GENCODE is generally considered more comprehensive so handles a wider variety of cases:
https://bmcgenomics.biomedcentral.com/articles/10.1186/1471-2164-16-S8-S2
and we offer the effects_transcripts: canonical flag to limit these to a smaller RefSeq like set:
http://bcbio-nextgen.readthedocs.io/en/latest/contents/configuration.html#variant-annotation
Would this work for your use case?

Hi Brad,
thanks for the article, it is a good reason!
Personally I'm also prefer Ensembl annotations, and I agree that it is the better way.
And I'm using ensembl transcript ID's to get many additional annotations, absent in VEP/GEMINI - Exac scores, OMIM, Orphanet, etc. I need ENSEMBL anyway.
Sometimes clinicians are more conservative, they used to RefSeq coordinates and IDs :).
Ok, I in the meanwhile I'll just use a script to calculate RefSeq coordinates for the report, and in the future I hope we will completely switch to Ensembl/GENCODE in the project!
Thank you for you time!
Sergey


Button component enhancements to support <a> elements

The Button component example should be updated to include the following:

Examples that show <a> elements being used instead of <button> elements.
Updated Accessibility documentation to include attributes that would be used for <a> instead of <button>. For reference, the following attributes are documented for the Dropdown component:

Attribute
Applied
Outcome

disabled="disabled"
button.pf-c-dropdown__menu-item
When the menu item uses a button element, indicates that it is unavailable and removes it from keyboard focus

aria-disabled="true"
a.pf-c-dropdown__menu-item
When the menu item uses a link element, indicates that it is unavailable

tabindex="-1"
a.pf-c-dropdown__menu-item
When the menu item uses a link element, removes it from keyboard focus

:tada: This issue has been resolved in version 1.0.10 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


@Transactional compatibility when using @Actor on Spring Bean

Does not work...

I think you'll need to use the TransactionTemplate within your @Actor annotated Spring bean...

That's exactly what I'm trying to avoid... But might intercept @Transactional and attempt to weave in ActorInterceptor...

Th


Where is this stuff documented?

Hi, I'm loving this tutorial! But I find myself wondering how I could have figured any of this out without a tutorial. Where did you get all this information? I mean things about the fundamental protocols you have to keep to to run code on the Pi 4. Like that the file has to be called kernel8.img, how the SD card has to be formatted, or like the actual format that the .img file should have (i.e. how to put the linker script together, the correct objcopy commands, etc).
Is this stuff even documented officially anywhere? All I can find are forum posts linking to more forum posts with shreds of information.

Hi,
I'm afraid much of this tutorial is pieced together from the official Raspberry Pi/ARM documentation (linked from the tutorial text) or indeed from many forum/blog posts I found along the way. The tutorial actually started because I needed to record all the information I was finding out so I didn't lose track of why I made certain decisions. It was a product of a lot of Google searches!
Sorry that the answer isn't a more pleasant one!
Adam


Endcall Function is not working

If i execute for example
cordova.plugins.CordovaCall.receiveCall('Me');
It launch OK the secreen to decline or response
If I response and call later in my app the function
cordova.plugins.CordovaCall.endCall();
Nothing happened. The native screen call is active all time and when I close my app it show "Error call"
to reproduce the error
cordova.plugins.CordovaCall.receiveCall('Me');

cordova.plugins.CordovaCall.on('answer', () => {

      setTimeout(() => { // 5 sec to end call
         cordova.plugins.CordovaCall.endCall();
      }, 5000);

  });

@videmort
Thanks for sharing your code. Here's what I get when I run it on my iPhone.

As you can see, the call ends after 5 seconds as expected. Is there a call in progress when you call receiveCall? Can you run your project in Xcode and let me know the error message you get? I understand that you get the "Error call" message on your iPhone, but that can happen for several reasons. Xcode might have a more detailed error message.

@videmort Let me know if you got this to work. As you can see in my example above, endCall works fine. My guess is that you have other code that's causing this to fail. I would recommend doing what I did, which is to create a new project and run your code.

Hi dmarcs ! This is the error -->
No se ha podido completar la operación. (Error de com.apple.CallKit.error.requesttransaction 4.)

https://stackoverflow.com/questions/46090002/callkit-call-end-issueerror-domain-com-apple-callkit-error-requesttransaction-c

@dmarcs  property UUID doesn't exist


The codebase parameter incorrectly implies that it will not work if a file does not have a strong-name.

The documentation specifically states that "The assemblyFile argument that you specify with the /codebase option must be a strong-named assembly" which implies that it will not work for assemblies that do not have a strong-name.
We actually use this parameter on dlls that do not have a strong-name, and it allows our application to load them.  Without the codebase argument, our application is unable to find the dlls.
Also, the documentation does not state how this option affects the loading of assemblies into the AppDomain.  In our case, we only use the codebase option on Assemblies that are marked as ComVisible as that is how we access those specific assemblies.  Dependencies of the ComVisible assemblies are accessed the standard .NET way.
In addition, it also does not specify any consequences of loading files this way.  We have recently found an issue where an already loaded assembly, which is a dependency of a ComVisible assembly, was unable to find itself when deserializing an object to one of the types in the same assembly.  This seems more like a bug than a consequence of using the codebase parameter, but we found that the issue does not occur if the exe is in the same folder as said assembly (the dependency), which actually is in the same folder as the ComVisible assembly.
The affects and consequences need not be discussed directly in this article, but could instead be a link to a different article.  Perhaps the one that discusses how the .NET Framework loads assemblies would be appropriate, however the one I found does not address that either (How the runtime locates assemblies).

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 63700a34-9f68-4f17-bfb6-21b1f5ee340c
Version Independent ID: ee5f7263-0985-3624-d51f-fe23dd36a280
Content: Regasm.exe (Assembly Registration Tool) - .NET Framework
Content Source: docs/framework/tools/regasm-exe-assembly-registration-tool.md
Product: dotnet-framework
GitHub Login: @tdykstra
Microsoft Alias: tdykstra

@jeffschwMSFT @AaronRobinsonMSFT who can help with this?

Since this is .NET Framework adding @agocke

ping @agocke

This one delves beyond my knowledge of the Framework APIs. @vitek-karas could you take a look and see if there's anything we oculd clarify in the docs for this? My recollection is that codebase is respected regardless of whether or not the assembly is strong-name signed, so we could at least clarify that.

@mdell-seradex Could you please confirm if you are activating these managed types via COM activation? That is the primary function of the Regasm tool - COM server registration of .NET types.

We actually use this parameter on dlls that do not have a strong-name, and it allows our application to load them.

Yes. This is technically possible but strongly advised against. It would appear we never made it a failure case.

Without the codebase argument, our application is unable to find the dlls.

Yes, this is likely because the application is unable to find the assembly using the default probing paths the application is defining. Adding the assembly to the GAC will ensure the assembly can be found.
The /codebase option is rather poorly named for what it actually does. During COM server registration is stores the current path of the assembly passed to Regasm in the COM registration section of the Win32-Registry. An example is warranted.
Imagine a .NET assembly named Foo.dll is in directory C:\MyServer\. The following command is executed:
C:\MyServer> Regasm.exe Foo.dll /codebase
Along with the appropriate types in the Foo.dll assembly, the absolute path of Foo.dll will be recorded - that is, C:\MyServer\Foo.dll. During activation, this path will be used by the runtime to load the assembly containing the activated .NET type. Instead of using /codebase, one can insert the Foo.dll into the GAC. If I recall, the GAC does require strong-name signing and that is enforced.

We have recently found an issue where an already loaded assembly, which is a dependency of a ComVisible assembly, was unable to find itself when deserializing an object to one of the types in the same assembly. This seems more like a bug than a consequence of using the codebase parameter, but we found that the issue does not occur if the exe is in the same folder as said assembly (the dependency), which actually is in the same folder as the ComVisible assembly.

This is quite possible an issue with the COM apartment the types were activated in and in reality the two assembly instances are distinct - two instances have been loaded in different contexts. This is a very complicated scenario to root out, but would require additional details to confirm the hypothesis.

Could you please confirm if you are activating these managed types via COM activation?

We are accessing these types via COM.  We generally try to keep our files out of the GAC, and in a sub folder of our application instead.  The application's exe file, however is not a .NET exe.  We would like to change to a .NET exe, but we still require the use of many COM dlls, and some of the 3rd party COM files fail to work when run from a .NET exe assembly.  We do not know why.  We believe it has something to do with how the process is initialized.  We also have a few other exe files which are sometimes used, and are often stored outside of this folder structure.
We have never deployed a config file with our exe, but have not experienced any issues with this until now.

The /codebase option is rather poorly named for what it actually does. During COM server registration is stores the current path...

Yes we determined that quite a while ago in our testing.

This is quite possibly an issue with the COM apartment the types were activated in...

To be clear, the assembly that had this issue is not a COM assembly, but rather is loaded by the COM Assembly.  We managed to address the issue manually using the AppDomain.AssemblyResolve Event, however it seems very odd that assembly X essentially cannot find itself.  In reality, it is the .NET framework that is unable to find the assembly that called it to deserialize a byte array to a type in that assembly, which still seems odd to me.
I hope this helps you.

To be clear, the assembly that had this issue is not a COM assembly, but rather is loaded by the COM Assembly.

Yes, that could do it. The reason is not obvious. A potential way to confirm that you have two loaded instances of the same assembly is to get a type that is not equal to itself and check that type's Assembly.HostContext against the other. The serialization mechanisms rely upon the default context if I recall so it can be the case that the COM activated assembly has loaded its dependencies in a new context, but the deserializer is expecting the dependency in the default context. Note that these mechanics are much clearer in .NET Core and exposed via AssemblyLoadContext.
I'm not saying the issue you are seeing is what I am describing but your description is commonly caused by multiple instances of the same assembly loaded in the same process. The behavior can manifest as typeof(Foo) is not equal to typeof(Foo).

That is interesting, however I do not recall that there were multiple instances of the same assembly loaded in the same process, until the issue occurred.  I cannot be completely sure of that, however, as our application is quite large with many dlls in use.
Now that I think of it, I believe I was able to reproduce the issue using a simpler setup that explicitly only instantiated the type in our COM-activated .NET assembly which then created an instance of each needed type in the 3rd party assembly.  So unless the 3rd party assembly (from Telerik) loaded another instance of itself in a different context, I don't think that was the case, if I understand you correctly.
I believe we are resolving it in the AssemblyResolve Event by directing it to the already loaded assembly.

Are you saying that because the one assembly was COM activated, the assemblies it referenced were loaded in a new context instead of the default one?

Perhaps, yes. The way .NET Framework does assembly resolution is very complicated and we've observed situations where application configuration coupled with COM or atypical assembly load patterns can cause unintended code paths to be execute.

I believe we are resolving it in the AssemblyResolve Event by directing it to the already loaded assembly.

Ah, I missed the fact that you were returning the already loaded assembly. I assumed you were loading another copy via one of the Assembly.Load* methods. In this case, I see your confusion.

The odd thing I found though, is that does not occur if the exe is in the same folder as the other assemblies.

This might be an issue that could be fixed with adding a probing path to the CLR configuration file. I recall you mentioning the exe doesn't have one but if it did the sub-directory can be added to where the runtime will look for assemblies. Visual Studio, also a native exe, uses this technique because its .NET assemblies are spread across multiple sub-directories.

Yes, I believe a probing path can also eliminate the issue.  It is just the fact that the required 3rd party assembly is already loaded, and yet .NET cannot find it, that seemed very strange, as we wrote no code to manually load it in the first place.  .NET did that with ease from the COM activated assembly.
Anyway, I did not mean for this to become a full fledged discussion on here, however, if that helps to improve the documentation, then I am all for it. 🙂

however, if that helps to improve the documentation, then I am all for it. 🙂

Appreciate that. The problem with these sorts of issues is that reproducing the issue so we can confirm precisely the issue and then document that is typically costly.
Let me try to describe your situation so I can construct something locally.

Directory
File
Details

.\
native.exe
Loads the CLR via COM activation

.\server
comserver.dll
Managed COM server registered via RegAsm.exe /codebase

dependency.dll
Managed library used by comserver.dll

.\other
lib.dll
Managed library that uses dependency.dll

The lib.dll attempts to serialize something in dependency.dll. Attempting to serialize anything in dependency.dll from lib.dll causes a failure to find dependency.dll.

In my case, if I recall correctly, there is no lib.dll.
Instead, Telerik Reporting is loading a byte stream from a file and asking .NET to deserialize it to an object that is a type in the Telerik Reporting assembly.
I may be wrong, as it was back when I first logged this that I was looking at it.

I reviewed the ticket I posted to Telerik, and the issue was regarding deserializing an image stored in a byte array in the report file to a specific type in the Telerik Reporting assembly.  It seems that the byte array was actually an object, which contains the image, that is one of their types.

Thanks for all your help @mdell-seradex.
I have a local repro for the issue. I've been able to collect a fusion log for the scenario.
*** Assembly Binder Log Entry  (19-Mar-22 @ 16:27:40) ***

The operation was successful.
Bind result: hr = 0x0. The operation completed successfully.

Assembly manager loaded from:  C:\Windows\Microsoft.NET\Framework64\v4.0.30319\clr.dll
Running under executable  ...\DotnetDocs26649\x64\Debug\native.exe
--- A detailed error log follows.

=== Pre-bind state information ===
LOG: Where-ref bind. Location = .../DotnetDocs26649/x64/Debug/server/comserver.DLL
LOG: Appbase = file:///.../DotnetDocs26649/x64/Debug/
LOG: Initial PrivatePath = NULL
LOG: Dynamic Base = NULL
LOG: Cache Base = NULL
LOG: AppName = native.exe
Calling assembly : (Unknown).
===
LOG: This bind starts in LoadFrom load context.
WRN: Native image will not be probed in LoadFrom context. Native image will only be probed in default load context, like with Assembly.Load().
LOG: No application configuration file found.
LOG: Using host configuration file:
LOG: Using machine configuration file from C:\Windows\Microsoft.NET\Framework64\v4.0.30319\config\machine.config.
LOG: Attempting download of new URL file:///.../DotnetDocs26649/x64/Debug/server/comserver.DLL.
LOG: Assembly download was successful. Attempting setup of file: ...\DotnetDocs26649\x64\Debug\server\comserver.dll
LOG: Entering run-from-source setup phase.
LOG: Assembly Name is: comserver, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=null
LOG: Re-apply policy for where-ref bind.
LOG: Binding succeeds. Returns assembly from ...\DotnetDocs26649\x64\Debug\server\comserver.dll.
LOG: Assembly is loaded in LoadFrom load context.

I am not sure I understand how the fusion log identifies what I was experiencing, but as long as it helps you, I guess that is sufficient.

@mdell-seradex After consulting with some colleagues, we believe this could be related to a 4.0 to 4.5.1 compatibility change documented here. I think in your case Telerik would need to respond but that does address the failure in my repro. I'll work on clarifying some of the doc for Regasm soon.

Interesting.  I guess then that this also applies to COM activated objects.
It is unfortunate that it is not mentioned there.  I don't know anywhere in the documentation that explicitly states that COM activated objects are loaded that way, although they are so extensive, it would be easy to miss.  I didn't notice anything in the article this issue is about that mentions it.
I can try mentioning that to Telerik.

@mdell-seradex I've created a PR that has some additional details on how COM servers are loaded. See https://github.com/dotnet/docs/pull/28751.

Awesome.  Thank you.
Do those changes also somehow include a pointer to that issue noted above regarding Binary Formatter?
It would be nice if the documentation included backreferences.  For example LoadFrom is mentioned in ..., BinaryFormatter, ..., but would probably be better as a list, and is well beyond the scope of this. 🙂


Add shellenv --force and shellenv -h

Provide a detailed description of the proposed feature
Add two commands.
shellenv --force               - force output regardless of state of environment
shellenv -h                    - show this help

What is the motivation for the feature?
As explained in https://github.com/Homebrew/brew/issues/11851 shellenv doesn't return any output, and it is hard to debug what's going on without investing some hours to browse this tracker.
I am installing Homebrew in CI and script can not find brew, so I need to see what shellenv produces.
How will the feature be relevant to at least 90% of Homebrew users?
Hopefully they won't have to debug their shell setup when (re)installing brew, but if they will, this will be handy.
What alternatives to the feature have been considered?
Finding the list of variables that shellenv sets by asking here, and reading the source.

You can view the help for shellenv with brew help shellenv. The same is true for any brew subcommand, and I don't think we should make the interface inconsistent (i.e. brew shellenv -h shows the help but, say, brew install -h might not).
#11851 should be fixed by now. You're likely using an outdated version of brew. What's the output of brew config and brew doctor on the machine you're having issues with?

➜  ~ brew shellenv                                                                                /0,0s
➜  ~ brew config
HOMEBREW_VERSION: 3.2.9-64-g6da8630
ORIGIN: https://github.com/Homebrew/brew
HEAD: 6da8630f4e077b38883f6ea57d96e4002f7b8599
Last commit: 4 months ago
Core tap ORIGIN: https://github.com/Homebrew/linuxbrew-core
Core tap HEAD: 51b47956b0713413d134a533759a1873e53a3578
Core tap last commit: 2 months ago
Core tap branch: master
HOMEBREW_PREFIX: /home/linuxbrew/.linuxbrew
HOMEBREW_CASK_OPTS: []
HOMEBREW_DISPLAY: :0
HOMEBREW_EDITOR: /usr/bin/nano
HOMEBREW_MAKE_JOBS: 4
Homebrew Ruby: 2.6.3 => /home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/vendor/portable-ruby/2.6.3_2/bin/ruby
CPU: quad-core 64-bit haswell
Clang: N/A
Git: 2.33.1 => /bin/git
Curl: 7.79.1 => /usr/bin/curl
Kernel: Linux 5.15.11-200.fc35.x86_64 x86_64 GNU/Linux
OS: Fedora release 35 (Thirty Five)
Host glibc: 2.34
/usr/bin/gcc: 11.2.1
/usr/bin/ruby: 3.0.2
glibc: N/A
gcc@5: N/A
xorg: N/A                                                                                /1,9s
➜  ~ brew update
Already up-to-date.

➜  ~ brew doctor
Please note that these warnings are just used to help the Homebrew maintainers
with debugging if you file an issue. If everything you use Homebrew for is
working fine: please don't worry or file an issue; just ignore this. Thanks!

Warning: You have unlinked kegs in your Cellar.
Leaving kegs unlinked can lead to build-trouble and cause formulae that depend on
those kegs to fail to run properly once built. Run `brew link` on these:
  libnghttp2

Warning: Some installed formulae are missing dependencies.
You should `brew install` the missing dependencies:
  brew install curl git

Run `brew missing` for more details.                                                                                /3,3s
➜  ~ git --version
git version 2.33.1                                                                                /0,0s
➜  ~ curl --version
curl 7.79.1 (x86_64-redhat-linux-gnu) libcurl/7.79.1 ...

Is the problem that I am using system git and curl binaries?

No, the problem is that you're using a very old version of brew:
Last commit: 4 months ago

Do brew update before calling brew shellenv.

It appeared that I am on a different branch.
➜  ~ cd /home/linuxbrew/.linuxbrew/Homebrew                                                                                /0,0s
➜  Homebrew git:(desc-v-search) git status
On branch desc-v-search
nothing to commit, working tree clean

I've created the branch when filling https://github.com/Homebrew/brew/issues/11932 and didn't revert back to master. Not sure why brew config and brew update didn't detect that.
After switching back to master I could update to latest 3.3.9.
➜  Homebrew git:(desc-v-search) git checkout master
Switched to branch 'master'
Your branch is up to date with 'origin/master'.                                                                                /0,2s
➜  Homebrew git:(master) git branch --merged | egrep -v "(^\*|master|dev|main)" | xargs git branch -d
Deleted branch desc-v-search (was 6da8630f4).
Deleted branch stable (was 92cf90471).                                                                                /0,2s
➜  Homebrew git:(master) brew update
HOMEBREW_CORE_GIT_REMOTE set: using https://github.com/Homebrew/homebrew-core for Homebrew/core Git remote.
remote: Enumerating objects: 62778, done.
remote: Counting objects: 100% (14984/14984), done.
remote: Compressing objects: 100% (16/16), done.
remote: Total 62778 (delta 14975), reused 14975 (delta 14968), pack-reused 47794
Receiving objects: 100% (62778/62778), 36.39 MiB | 2.43 MiB/s, done.
Resolving deltas: 100% (45288/45288), completed with 4719 local objects.
From https://github.com/Homebrew/homebrew-core
 + 51b47956b07...648dc81da0a master     -> origin/master  (forced update)
==> Downloading https://ghcr.io/v2/homebrew/portable-ruby/portable-ruby/blobs/sha256:cd7fffb18ef9338baa670fc5e8fce99b0e3cc0f0fd7627bcbb56f3c8d54161d4
######################################################################## 100.0%
==> Pouring portable-ruby-2.6.8.x86_64_linux.bottle.tar.gz
==> Migrating formulae from linuxbrew-core to homebrew-core
...

And the shellenv is fixed indeed.
➜  Homebrew git:(master) brew shellenv
export HOMEBREW_PREFIX="/home/linuxbrew/.linuxbrew";
export HOMEBREW_CELLAR="/home/linuxbrew/.linuxbrew/Cellar";
export HOMEBREW_REPOSITORY="/home/linuxbrew/.linuxbrew/Homebrew";
export PATH="/home/linuxbrew/.linuxbrew/bin:/home/linuxbrew/.linuxbrew/sbin${PATH+:$PATH}";
export MANPATH="/home/linuxbrew/.linuxbrew/share/man${MANPATH+:$MANPATH}:";
export INFOPATH="/home/linuxbrew/.linuxbrew/share/info:${INFOPATH:-}";                                                                                /0,1s


[RFC][Quantization] Designing and lowering of quantized ops

[Relay] [Quantization] WIP - This is the continuation of pull request #3367
In this PR I want to discuss the design and implementation of the

Quantize op -> FP32 to i8/u8
Dequantize Op -> i8/u8 -> fp32

I have added test cases to verify the correctness of the ops.

@tqchen @FrozenGene @ZihengJiang @zhiics @wweic @eqy

Is this ready for review? Have we been converged on the design in the quantization RFC?

We have made good progress on the Quantization RFC, achieving clarity and convergence on many points.
For this PR specifically, @tqchen and @FrozenGene, can you please comment if this looks in line with our quantization RFC.

@FrozenGene and @tqchen, any other major comments for the PR?

Mainly organizational issues, please make things consistent with what was discussed in https://github.com/dmlc/tvm/pull/3531

Mainly organizational issues, please make things consistent with what was discussed in #3531

I have addressed the namespace issues and have followed the same convetion as #3531 in the new commit.

@liangfu made the changes you suggested.

Regarding 16/32 bits quantization, I have a discussion in RFC #3591 .

There are quite a lot of changes here that are depndent on #3531 . I am closing the PR for now. I will reopen this once #3531 is pushed.


nix-index: wrap in a separate derivation

Avoids rebuilding nix-index when the only change is the path to nix in the wrapper. Also allows setting nix to null to bypass wrapping.
Also added myself as a maintainer.
cc @Infinisil #51165
Things done

Built on platform(s)

[X] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[X] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[X] Tested execution of all binary files (usually in ./result/bin/)
21.11 Release Notes (or backporting 21.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[X] Fits CONTRIBUTING.md.

@ofborg build nix-index

ping @bennofs

Thanks!

Cleaned up in #139155


fix: hrpc-rs 0.26.0 breaking build

Probably happens for other crates too (which ones?).
Error output:
error: opening file '/nix/store/gzf8jc8jyhf65kzppljkgfxbvc2akhm9-source/crates/*/Cargo.toml': No such file or directory

TODO: also add a test for this in fix PR

This was related to #43, which is now fixed.


Jurst imports dont resolve native visibility

If jurst scope Units contains a native declaration native UnitAlive takes unit u returns boolean, and jurst scope Codezors has import Units, followed by a usage of UnitAlive, its visibility will fail to resolve and be inaccessible. Together with #353 this is a serious problem.

Does it work, when you make the native public? That might be a temporary fix.

I never considered that, yes it works, thanks.


fix: Fix Insight Editor file upload button on mobile

This PR address the fix of the Upload File button when on mobile view from the Create/Edit Insights page.

:tada: This PR is included in version 3.9.4 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Metadata cache load invalid keys when configured without / suffix

When metadata cache path is configured without '/' suffix.
While loading the current files removing the cache dir prefix from the cache key
the file path may include '/' as the prefix of the key.
When using the cache getting the key uses relative prefix without the '/' prefix.
The cache is filled with invalid keys that be evicted.

I think we've had more than enough issues with namespace prefixes that don't end in "/".  Opening a new bug to ensure that all prefixes end in "/"!  (It will come in addition to this, but it might reduce the probability of future occurrences!)


Can't finish build

Description
I am integrating jupyterlite-sphinx into mesa-geo readthedocs but my build keeps failing with the following error on in read the docs (no issues if doing jupyter lite build on my local machine)
Extension error (jupyterlite_sphinx.jupyterlite_sphinx):
--
94 | Handler <function jupyterlite_build at 0x7f477f0ddab0> for event 'build-finished' threw an exception (exception: Command '['jupyter', 'lite', 'build', '--debug', '--contents', 'tutorials/intro_tutorial.ipynb', '--contents', '/home/docs/checkouts/readthedocs.org/user_builds/mesa-geo-test/checkouts/latest/docs/source/_contents', '--output-dir', '/home/docs/checkouts/readthedocs.org/user_builds/mesa-geo-test/checkouts/latest/_readthedocs/html/lite', '--apps', 'notebooks', '--apps', 'edit', '--apps', 'lab', '--apps', 'repl', '--apps', 'tree', '--apps', 'consoles', '--lite-dir', '/home/docs/checkouts/readthedocs.org/user_builds/mesa-geo-test/checkouts/latest/docs/source', '--XeusAddon.environment_file', 'environment.yml']' returned non-zero exit status 3.)
95 | [jupyterlite-sphinx] Running JupyterLite build

Reproduce
You can see the build at ----
https://app.readthedocs.org/projects/mesa-geo-test/builds/24708569/
Expected behavior
Mesa-Geo docs render with xeus-python kernel
Context

JupyterLite version: jupyterlite-xeus>0.1.5
Operating System and version:  "ubuntu-22.04"

Thank you for your time. Attached are my conf.txt
readthedocs.txt and
environment.txt
I feel like this is going to be something very simple but just cant figure out the right combination

Thanks @tpike3 for reporting :+1:
It may be related to jupyterlite-xeus. There were a few issues recently, maybe @martinRenou or @DerThorsten may no more about this.
In the meantime, you could try to use a different kernel (for example https://github.com/jupyterlite/pyodide-kernel), to see if the build passes.

Thanks @jtpio will do.

Your logs don't contain much, it looks like the actual error message is being hidden (we don't know why the jupyterlite build command failed)

Your logs don't contain much, it looks like the actual error message is being hidden (we don't know why the jupyterlite build command failed)

Here is the traceback from the readthedocs build ....
writing additional pages... search done
dumping search index in English (code: en)... done
dumping object inventory... done
Updating searchtools for Read the Docs search...
Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/mesa-geo-test/conda/latest/lib/python3.10/site-packages/sphinx/events.py", line 97, in emit
    results.append(listener.handler(self.app, *args))
  File "/home/docs/checkouts/readthedocs.org/user_builds/mesa-geo-test/conda/latest/lib/python3.10/site-packages/jupyterlite_sphinx/jupyterlite_sphinx.py", line 707, in jupyterlite_build
    completed_process: CompletedProcess[bytes] = subprocess.run(
  File "/home/docs/checkouts/readthedocs.org/user_builds/mesa-geo-test/conda/latest/lib/python3.10/subprocess.py", line 524, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['jupyter', 'lite', 'build', '--debug', '--contents', 'tutorials/intro_tutorial.ipynb', '--contents', '/home/docs/checkouts/readthedocs.org/user_builds/mesa-geo-test/checkouts/latest/docs/source/_contents', '--output-dir', '/home/docs/checkouts/readthedocs.org/user_builds/mesa-geo-test/checkouts/latest/_readthedocs/html/lite', '--apps', 'notebooks', '--apps', 'edit', '--apps', 'lab', '--apps', 'repl', '--apps', 'tree', '--apps', 'consoles', '--lite-dir', '/home/docs/checkouts/readthedocs.org/user_builds/mesa-geo-test/checkouts/latest/docs/source']' returned non-zero exit status 3.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/mesa-geo-test/conda/latest/lib/python3.10/site-packages/sphinx/cmd/build.py", line 337, in build_main
    app.build(args.force_all, args.filenames)
  File "/home/docs/checkouts/readthedocs.org/user_builds/mesa-geo-test/conda/latest/lib/python3.10/site-packages/sphinx/application.py", line 353, in build
    self.events.emit('build-finished', None)
  File "/home/docs/checkouts/readthedocs.org/user_builds/mesa-geo-test/conda/latest/lib/python3.10/site-packages/sphinx/events.py", line 108, in emit
    raise ExtensionError(
sphinx.errors.ExtensionError: Handler <function jupyterlite_build at 0x7effb8c75360> for event 'build-finished' threw an exception (exception: Command '['jupyter', 'lite', 'build', '--debug', '--contents', 'tutorials/intro_tutorial.ipynb', '--contents', '/home/docs/checkouts/readthedocs.org/user_builds/mesa-geo-test/checkouts/latest/docs/source/_contents', '--output-dir', '/home/docs/checkouts/readthedocs.org/user_builds/mesa-geo-test/checkouts/latest/_readthedocs/html/lite', '--apps', 'notebooks', '--apps', 'edit', '--apps', 'lab', '--apps', 'repl', '--apps', 'tree', '--apps', 'consoles', '--lite-dir', '/home/docs/checkouts/readthedocs.org/user_builds/mesa-geo-test/checkouts/latest/docs/source']' returned non-zero exit status 3.)

Extension error (jupyterlite_sphinx.jupyterlite_sphinx):
Handler <function jupyterlite_build at 0x7effb8c75360> for event 'build-finished' threw an exception (exception: Command '['jupyter', 'lite', 'build', '--debug', '--contents', 'tutorials/intro_tutorial.ipynb', '--contents', '/home/docs/checkouts/readthedocs.org/user_builds/mesa-geo-test/checkouts/latest/docs/source/_contents', '--output-dir', '/home/docs/checkouts/readthedocs.org/user_builds/mesa-geo-test/checkouts/latest/_readthedocs/html/lite', '--apps', 'notebooks', '--apps', 'edit', '--apps', 'lab', '--apps', 'repl', '--apps', 'tree', '--apps', 'consoles', '--lite-dir', '/home/docs/checkouts/readthedocs.org/user_builds/mesa-geo-test/checkouts/latest/docs/source']' returned non-zero exit status 3.)
[jupyterlite-sphinx] Running JupyterLite build

Right now I am thinking it is one of 2 things (but could be way off)

That it the way my directories are set up as I am using a source directory and a build directory, but even when I change structure to match jupyterlite sphinx its doesnt work  and  my .jupyterlite.doit.db never ends up where it should be and so never gets cleaned as it should here https://github.com/jupyterlite/jupyterlite-sphinx/blob/d50e2465d5bf67cb03897f68abc9a827658fae69/jupyterlite_sphinx/jupyterlite_sphinx.py#L731

Or that need to build an environment in my pyproject.toml like jupyterlite-sphinx does here https://github.com/jupyterlite/jupyterlite-sphinx/blob/d50e2465d5bf67cb03897f68abc9a827658fae69/pyproject.toml#L43C1-L47C60

Well I should pay closer attention to the documentation I  set jupyterlite_silence = False and I am getting good traceback information.
However, it is not immediately obvious to me what the issue is.
RTD build details.txt

Thanks @tpike3.
These two lines at the top of the logs seem to indicate an issue with jupyterlite-xeus:
ValidationError(ValueError("'emscripten' is not a valid Platform"))
ValidationError(ValueError("'emscripten' is not a valid Platform"))

Which should likely be fixed by https://github.com/conda/conda/pull/13962?
In the meantime, I think the workaround might be to use the latest pre-release of jupyterlite-xeus, and add micromamba to the build environment, similar to https://github.com/voila-dashboards/voici/pull/120 which had the same issue.

@tpike3 looks like this is indeed an issue with jupyterlite-xeus: https://github.com/jupyterlite/jupyterlite-sphinx/pull/179
@martinRenou @DerThorsten maybe we should document the fix somewhere (for example in the jupyterlite-xeus repo) until it is fixed in conda?

Thanks @jtpio!!! I am getting closer but now I am getting this error--- error    libmamba ZSTD decompression error: Unknown frame descriptor

Thanks @tpike3 for reporting 👍
It may be related to jupyterlite-xeus. There were a few issues recently, maybe @martinRenou or @DerThorsten may no more about this.
In the meantime, you could try to use a different kernel (for example https://github.com/jupyterlite/pyodide-kernel), to see if the build passes.

@jtpio Apologies should have done this earlier, I did switch to a pyodide-kernel and the build works. However, since I cannot install packages (e.g Mesa) it does not help in this case. Thank you though.

@tpike3 you could also continue using jupyterlite-xeus but update to the latest pre-release (as in https://github.com/jupyterlite/jupyterlite-sphinx/pull/179), which should fix the issue.

@tpike3 you could also continue using jupyterlite-xeus but update to the latest pre-release (as in #179), which should fix the issue.

@jtpio I still cannot get the xeus-python to build into RTD still.
[LiteBuildApp] creating folder /home/docs/checkouts/readthedocs.org/user_builds/mesa-geo-test/checkouts/latest/_readthedocs/html/lite/static

139 | error    libmamba ZSTD decompression error: Unknown frame descriptor

Of course I realized pyodide won't work with piplite as Mesa doesn't have a pure python wheel

Normally this issue should not be related to Pyodide since you are using xeus-python, and also because this is happening a build time (when the JupyterLite site is built) and not at runtime (when opening JupyterLite in a web browser).
Looks like this could be due to a malformed package on emscripten forge or conda forge?

error    libmamba ZSTD decompression error: Unknown frame descriptor

I believe this should not cause the build to fail. You probably have another error somewhere.

Hi @reox, I think you might need to pip-install jupyterlite-pyodide-kernel if you haven't already, and the Pyodide kernel should then show up.

@agriyakhetarpal yes, I did that, removed all build files and caches and built again - but no kernel shows up...
I also tried with the --pre version, but that just leads to crashes during the build

I see, could you please paste an error trace after setting jupyterlite_silence = False in conf.py and building, as noted above? That might show some extra information

Sure:
[jupyterlite-sphinx] Running JupyterLite build
[LiteBuildApp] Searching ['C:\\Users\\User Name\\git\\gdp2024', 'C:\\Users\\User Name\\.jupyter', 'C:\\Users\\User Name\\AppData\\Roaming\\Python\\etc\\jupyter', 'C:\\Program Files\\Python312\\etc\\jupyter', 'C:\\ProgramData\\jupyter'] for config files
[LiteBuildApp] Looking for jupyter_config in C:\ProgramData\jupyter
[LiteBuildApp] Looking for jupyter_config in C:\Program Files\Python312\etc\jupyter
[LiteBuildApp] Looking for jupyter_config in C:\Users\User Name\AppData\Roaming\Python\etc\jupyter
[LiteBuildApp] Looking for jupyter_config in C:\Users\User Name\.jupyter
[LiteBuildApp] Looking for jupyter_config in C:\Users\User Name\git\gdp2024
[LiteBuildApp] Looking for jupyter_lite_config in C:\ProgramData\jupyter
[LiteBuildApp] Looking for jupyter_lite_config in C:\Program Files\Python312\etc\jupyter
[LiteBuildApp] Looking for jupyter_lite_config in C:\Users\User Name\AppData\Roaming\Python\etc\jupyter
[LiteBuildApp] Looking for jupyter_lite_config in C:\Users\User Name\.jupyter
[LiteBuildApp] Looking for jupyter_lite_config in C:\Users\User Name\git\gdp2024
[LiteBuildApp] [lite] [addon] loading ...
[LiteBuildApp] [lite] [addon] [archive] ... will archive
[LiteBuildApp] [lite] [addon] [archive] ... will status
[LiteBuildApp] [lite] [addon] [contents] ... will build
[LiteBuildApp] [lite] [addon] [contents] ... will check
[LiteBuildApp] [lite] [addon] [contents] ... will post_build
[LiteBuildApp] [lite] [addon] [contents] ... will status
[LiteBuildApp] [lite] [addon] [federated_extensions] ... will post_build
[LiteBuildApp] [lite] [addon] [federated_extensions] ... will post_init
[LiteBuildApp] [lite] [addon] [federated_extensions] ... will pre_build
[LiteBuildApp] [lite] [addon] [icons] ... will build
[LiteBuildApp] [lite] [addon] [icons] ... will status
[LiteBuildApp] [lite] [addon] [jupyterlite-pyodide-kernel-piplite] ... will build
[LiteBuildApp] [lite] [addon] [jupyterlite-pyodide-kernel-piplite] ... will check
[LiteBuildApp] [lite] [addon] [jupyterlite-pyodide-kernel-piplite] ... will post_build
[LiteBuildApp] [lite] [addon] [jupyterlite-pyodide-kernel-piplite] ... will post_init
[LiteBuildApp] [lite] [addon] [jupyterlite-pyodide-kernel-pyodide] ... will build
[LiteBuildApp] [lite] [addon] [jupyterlite-pyodide-kernel-pyodide] ... will check
[LiteBuildApp] [lite] [addon] [jupyterlite-pyodide-kernel-pyodide] ... will post_build
[LiteBuildApp] [lite] [addon] [jupyterlite-pyodide-kernel-pyodide] ... will post_init
[LiteBuildApp] [lite] [addon] [jupyterlite-pyodide-kernel-pyodide] ... will status
[LiteBuildApp] [lite] [addon] [lite] ... will build
[LiteBuildApp] [lite] [addon] [lite] ... will check
[LiteBuildApp] [lite] [addon] [lite] ... will status
[LiteBuildApp] [lite] [addon] [mimetypes] ... will post_build
[LiteBuildApp] [lite] [addon] [mimetypes] ... will status
[LiteBuildApp] [lite] [addon] [report] ... will pre_archive
[LiteBuildApp] [lite] [addon] [serve] ... will serve
[LiteBuildApp] [lite] [addon] [serve] ... will status
[LiteBuildApp] [lite] [addon] [settings] ... will check
[LiteBuildApp] [lite] [addon] [settings] ... will post_build
[LiteBuildApp] [lite] [addon] [settings] ... will pre_build
[LiteBuildApp] [lite] [addon] [settings] ... will status
[LiteBuildApp] [lite] [addon] [static] ... will init
[LiteBuildApp] [lite] [addon] [static] ... will post_init
[LiteBuildApp] [lite] [addon] [static] ... will pre_init
[LiteBuildApp] [lite] [addon] [static] ... will pre_status
[LiteBuildApp] [lite] [addon] [translation] ... will build
[LiteBuildApp] [lite] [addon] [translation] ... will check
[LiteBuildApp] [lite] [addon] [translation] ... will status
[LiteBuildApp] [lite] [addon] [voici] ... will post_build
[LiteBuildApp] [lite] [addon] ... OK 14 addons
[LiteBuildApp] [lite] [tasks] loading ...
[LiteBuildApp] [lite] [tasks] ... OK 18 tasks
static:jupyter-lite.json
.  pre_status:static:jupyter-lite.json
    tarball:         jupyterlite-app-0.3.0.tgz 13MB
    output:          C:\Users\User Name\git\gdp2024\public\lite
    lite dir:        C:\Users\User Name\git\gdp2024
    apps:            notebooks, edit, lab, repl, tree, consoles
    sourcemaps:      True
    unused packages: True
archive:archive
contents:contents
icons:icons
jupyterlite-pyodide-kernel-pyodide:pyodide
lite:jupyter-lite.json
mimetypes:jupyter-lite.json
serve:contents
settings:overrides
translation:translation
.  status:archive:archive
[LiteBuildApp] No archive (yet): gdp2024-jupyterlite.tgz
.  status:contents:contents
[LiteBuildApp] [lite] [contents] All Contents ['C:\\Users\\User Name\\git\\gdp2024\\_contents\\Jupyter1.ipynb',
 'C:\\Users\\Sebastian '
 'Bachmann\\git\\gdp2024\\_contents\\Jupyter_Beispiel.ipynb',
 'C:\\Users\\User Name\\git\\gdp2024\\src\\Jupyter1_Beispiel1a.png',
 'C:\\Users\\User Name\\git\\gdp2024\\src\\Jupyter1_Beispiel1.png']
    contents: 4 files
.  status:icons:icons
[LiteBuildApp] [lite] [icons] All favicons []
    favicon files: 0 files
.  status:jupyterlite-pyodide-kernel-pyodide:pyodide
     URL: None
 archive: []
   cache: 0 files
   local: 0 files
.  status:lite:jupyter-lite.json
[LiteBuildApp]     jupyter-lite.(json|ipynb): []
[LiteBuildApp]     jupyter-lite.(json|ipynb): 0 files
.  status:mimetypes:jupyter-lite.json
    filetypes:         26
.  status:serve:contents
    url: http://<IP_ADDRESS>:8000/
    server: tornado
    headers:
.  status:settings:overrides
    overrides.json: 0
.  status:translation:translation
[LiteBuildApp] [lite] [translation] All Translations []
    translation files: 0 files
static:output_dir
.  pre_init:static:output_dir
static:unpack
.  init:static:unpack
C:\Users\User Name\AppData\Roaming\Python\Python312\site-packages\jupyterlite_core\addons\base.py:294: UserWarning: install libarchive-c for better performance when working with archives: argument of type 'NoneType' is not iterable
  return has_optional_dependency(
contents:copy:Jupyter1.ipynb
contents:copy:Jupyter_Beispiel.ipynb
contents:copy:Jupyter1_Beispiel1.png
contents:copy:Jupyter1_Beispiel1a.png
icons:copy
translation:copy
.  build:contents:copy:Jupyter1.ipynb
[LiteBuildApp] creating folder C:\Users\User Name\git\gdp2024\public\lite\files
.  build:contents:copy:Jupyter_Beispiel.ipynb
.  build:contents:copy:Jupyter1_Beispiel1.png
.  build:contents:copy:Jupyter1_Beispiel1a.png
.  build:icons:copy
[LiteBuildApp] creating folder C:\Users\User Name\git\gdp2024\public\lite\static
.  build:translation:copy
contents:contents:.
federated_extensions:patch
federated_extensions:settings
mimetypes:patch
.  post_build:contents:contents:.
.  post_build:federated_extensions:patch
.  post_build:federated_extensions:settings
.  post_build:mimetypes:patch
[LiteBuildApp] Exiting application: jupyter
[jupyterlite-sphinx] JupyterLite build done

I cannot spot a problem there... Is it possible the problem is actually serving the website? I tried only locally by running python -m http.server in the public folder.

Yes, I can't find a problem either. I don't think actually serving the website would make a difference, though. If it doesn't work with python -m http.server, it will be the same as where you would deploy it. Maybe something to try would be to use a fresh virtual environment, install jupyterlite-sphinx and jupyterlite-pyodide-kernel in it along with other dependencies like sphinx, etc. and see if the problem resurfaces? You might have already tried that, but that's the only suggestion I have right now. Perhaps clearing your browser cache and the user data stored on the page might help as well. From your logs, it does list jupyterlite-pyodide-kernel-pyodide:pyodide, so the kernel is installed indeed.

seems to be a problem with my local environment indeed. I just build it with gitlab CI and deployed it and there it works... Guess I have to look around which caches I can remove :D

Apologies, I was having to fix some other issues and got sidetracked. However, taking a break and readdressing I realized most of my issues was getting the right paths in my conf.py file. However, I am not getting this error

Exception occurred:
--
93 | File "/home/docs/checkouts/readthedocs.org/user_builds/mesa-geo-test/conda/latest/lib/python3.10/site-packages/jupyter_client/kernelspec.py", line 285, in get_kernel_spec
94 | raise NoSuchKernel(kernel_name)

But I cant find where kernelspec is specified. On my machine I am using the xeus-python kernel. I am printing out the kernelspec on the readthedocs build but it only shows a python3 kernel. I am also searhing through my local build and can't find where the kernelspec is.
Thoughts and thank you!

I have a brute force solution where I replace the kernel with one called mesa-geo (see below). I have also narrowed my problem down to what files are available via the various channels.
As this is a bit larger now, would you like me to close the issue?
def create_new_kernel_directory_and_update_json():
    base_path = Path('/home/docs/checkouts/readthedocs.org/user_builds/mesa-geo-test/conda/latest/share/jupyter/kernels/')
    old_kernel_name = "python3"
    new_kernel_name = "mesa_geo"

    old_kernel_path = base_path / old_kernel_name
    new_kernel_path = base_path / new_kernel_name

    try:
        # Check if the old kernel directory exists
        if old_kernel_path.is_dir():
            # Copy the kernel directory to a new directory
            shutil.copytree(old_kernel_path, new_kernel_path)
            print(f"Copied kernel directory from {old_kernel_name} to {new_kernel_name}")

            # Path to the kernel.json file in the new directory
            kernel_json_path = new_kernel_path / 'kernel.json'

            # Update the kernel.json file
            if kernel_json_path.is_file():
                with open(kernel_json_path) as f:
                    kernel_data = json.load(f)
                    print("Original kernel.json data:", kernel_data)

                # Update the display name
                kernel_data['display_name'] = new_kernel_name
                print("Updated kernel.json data:", kernel_data)

                # Write the changes back to the kernel.json file
                with open(kernel_json_path, 'w') as f:
                    json.dump(kernel_data, f, indent=4)

                print(f"Updated display name in kernel.json to '{new_kernel_name}'")
            else:
                print(f"kernel.json not found at {kernel_json_path}")
        else:
            print(f"Old kernel directory '{old_kernel_name}' not found at {old_kernel_path}")
    except Exception as e:
        print(f"Error during kernel directory copy or update: {e}")

# Call the function to create a new kernel directory and update kernel.json
create_new_kernel_directory_and_update_json()


Update sql.bnf to add create statement overrides for use the schema create

Add CREATE stubs that can be overridden in dialects where needed to order statements
SqlDelight TreeUtil forInitializationStatements is responsible for ordering the sql statements when schema.create is generated - https://github.com/cashapp/sqldelight/blob/d7cc00eabc133b43f38a1e75f70af7e786dd00d7/sqldelight-compiler/src/main/kotlin/app/cash/sqldelight/core/lang/util/TreeUtil.kt#L255
Note:- Migrations don't use this ordering and is preferred way of creating more complex schema
Currently CREATE TABLE statements are ordered first - some database support CREATE statements that execute before e.g CREATE SCHEMA sales needs to be run before CREATE TABLE sales.Orders
What an updated TreeUtil would look like using the overrides of create statements that may be available
  val databases= ArrayList<PsiElement>()
  val views = ArrayList<SqlCreateViewStmt>()
  val tables = ArrayList<SqlCreateTableStmt>()
  val creators = ArrayList<PsiElement>()
  val miscellanious = ArrayList<PsiElement>()

  forEach { file ->
    file.sqlStatements()
      .filter { (label, _) -> label.name == null }
      .forEach { (_, sqlStatement) ->
        when {
          sqlStatement.createSchemaStmt != null -> databases.add(sqlStatement.createSchemaStmt!!)
          sqlStatement.createExtensionStmt != null -> databases.add(sqlStatement.createExtensionStmt!!)
          sqlStatement.createTableStmt != null -> tables.add(sqlStatement.createTableStmt!!)
          sqlStatement.createViewStmt != null -> views.add(sqlStatement.createViewStmt!!)
          sqlStatement.createTriggerStmt != null -> creators.add(sqlStatement.createTriggerStmt!!)
          sqlStatement.createIndexStmt != null -> creators.add(sqlStatement.createIndexStmt!!)
          else -> miscellanious.add(sqlStatement)
        }
      }
  }

  databases.forEach { body(it.rawSqlText()) } // some schema elements must exist before tables

e.g Postgresql.bnf would override create_schema_stmt with the dialect implementation and TreeUtil would be able to use the statements
create_schema_stmt ::= CREATE 'SCHEMA' [ IF NOT EXISTS ] schema_name {
   extends = "com.alecstrong.sql.psi.core.psi.impl.SqlCreateSchemaStmtImpl"
   implements = "com.alecstrong.sql.psi.core.psi.SqlCreateSchemaStmt"
   override = true
}

Note:- Migrations don't use this ordering and is preferred way of creating more complex schema

is there a use case for having schema modifiers outside of migration files? In mobile sqlite it makes sense for fresh installs but I was assuming for every other distribution it would just run migration files (and therefore be run in the order written)

Note:- Migrations don't use this ordering and is preferred way of creating more complex schema

is there a use case for having schema modifiers outside of migration files? In mobile sqlite it makes sense for fresh installs but I was assuming for every other distribution it would just run migration files (and therefore be run in the order written)

Initially, I was going to just add grammar for PostgreSql CREATE SCHEMA https://github.com/cashapp/sqldelight/issues/3761 in Sqldelight
🕵 Then, I was looking into a viable approach with regards to "Wrong execution order of CREATE EXTENSION" https://github.com/cashapp/sqldelight/issues/5166
e.g Where the .sq. file is used to create the whole schema without migration files
It could get a bit messy where someone has a use-case for their.sq as a fresh database init script,  even thinking about when DROP SCHEMA ..., DROP TABLE ... would need to be at the top to clear out a schema...
Maybe - I will stick 🏒  with fixing https://github.com/cashapp/sqldelight/issues/3761 for now

yea - I think even for that original issue I would prefer a fix in SQLDelight. We have a similar issue with views where a view can be defined before a table when it needs the table to exist when CREATE VIEW is called. I'm not sure offhand where that sorting logic is but ideally we just extend that to support other schema types.
Or something more generic where we always preserve .sq file ordering unless there's a reason it would break and we need to sort.


Fix git error

This should resolve the issue originally noted in: https://github.com/reviewdog/reviewdog/issues/1158
I recreated the patch made in: https://github.com/reviewdog/action-yamllint/pull/19/files
I've tested it to work on a repo of my own

Thank you for your p-r!
The problem should be fixed by https://github.com/reviewdog/action-pyflakes/pull/18


DynamoDb: Serialization of AttributeValue doesn't skip a None binary field

The JSON serialization of AttributeValue doesn't skip a None binary field which doesn't pass input validation on AWS side. Example:
"ExpressionAttributeValues": {
  ":id": {
    "B": null,
    "N": "130807254563746998"
  },
},

I understand that this is already fixed in https://github.com/rusoto/rusoto/pull/1211 so this is more of a question than a bug report, will there be a hotfix release?

Due to time constraints, I'm not planning a release with just that fix. However, I'm planning on releasing Rusoto 0.36.0 in the next week or so, which will include #1211 . 👍
To get by in the meantime, one can use the git source in Cargo.toml for Rusoto to pick up the fix.
If the fix in #1211 doesn't fix this bug, please re-open this issue and we can take a closer look.

#1211 does fix the bug, thanks. I check out the repo at the fix to make it work for now. Looking forward to v0.36.0.


Auto-rotate doesn't rotate the model pivot if no model is loaded

Fixes #467
This fix checks to make sure that auto-rotation only starts once the poster is no longer visible and the model is interactable.
One thing is that this will not prevent the small stutter that happens when the poster transitions into 3D. That's because the rotation will start when the poster is dismissed, but the poster has a 0.3 second opacity transition. So during the transition you'll see the model rotation while the poster which is an image of the first frame will be fading away. It's not super noticeable, but it's also not perfectly smooth.
It may be more trouble than it's worth solving that case since we'd have to listen for when the opacity transition ends before triggering the rotation. And since the loading mixin and the staging mixin don't know about each other, that could be tricky.

@cdata This is ready for re-review :)


Secondary langauge

This might be a little more niche feature idea, but my use-case is this:

I have the English localization of the app
I've generated a Czech localization (my mother tongue)
Some translations were a little sketchy, so I've have corrected them
--> Now that I have a good English and Czech localization, I think it would be very for GPTs to see both examples. For example, some button labels were in the Czech imperative forms, which did not feel right. I think further translations of slavic languages like Polish, Slovakian etc. would benefit from seeing more.

Also it would be nice to have the system prompt adjustable.
Are you guys accepting PRs?

It's not that niche. It has been brought up to me by someone before. I totally agree that it would probably be beneficial to provide messages in more than one language, especially in the case you mentioned for languages in the same family.
I will probably get around to implementing this eventually, but if you are willing to contribute, PRs are welcome


Set translucent background for description?

Great framework, thx!! Is there a way to add a translucent background to the description label? I have high-res background images and the description sometimes camouflages against the background image in some places, so I would like to place a translucent background color to the description so the text shows no matter how crazy the background image is. Is this possible without having to create a custom intro hopefully?

How about adding blurring or fading whole background image? I can add options to IntroView instance, so it will fade and/or blur its background. Will it work for you?

Yes, that was actually another option that crossed my mind! Blurring would look real elegant and the fading would ensure words would show up in various scenarios I think.

Great, will add this options on this week. Most likely it will be included in 3.0.0 release.

Awesome, thx again for an amazing framework! BTW, you made my top 10 CocoaPods list :)
http://basememara.com/top-10-swift-friendly-cocoapods/

Awesome, thanks for mentioning :)


the docs should include an example of sdk-initiation with custom kin-issuers

`from stellar_base.asset import Asset
kin_asset = Asset('KIN', config.STELLAR_KIN_ISSUER_ADDRESS)
app.kin_sdk = kin.SDK(secret_key=base_seed,
horizon_endpoint_uri=config.STELLAR_HORIZON_URL,
network=config.STELLAR_NETWORK,
channel_secret_keys=channel_seeds,
kin_asset=kin_asset)`

This is being done in kin-core-v2


[ALLUXIO-2554] Make client hostname configurable

https://alluxio.atlassian.net/browse/ALLUXIO-2554

Merged build triggered.

Merged build started.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/13396/
Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/13403/
Test PASSed.

Merged build finished. Test PASSed.

@aaudiber I just had a few followup questions. Thanks!


 We want these to be publicly available.
We shall host them over at https://quay.io
Both are free for open source repos. Quay.io includes RedHat's Clair that gives industry leading security scanning.
Todo

Register account on quay.io using<EMAIL_ADDRESS>with the username iver-wharf (same as on GitHub)
Setup our build pipeline to deploy images to quay.io
Optionally add a badge in the README.md (ex: https://quay.io/repository/iver-wharf/wharf-api/status)
Update docs on how to get and use docker images
Update docker-compose.yml in iver-wharf/wharf-docker-compose to reference the new image names so you don't need to build them locally, but still can

Conflicts
Certificates! Oh the joy!
We need a better way to import our internal self-signed CAs than our current procedure where we're embedding them into the images. But this deserves a different GitHub issue and nothing we should have to worry about here. It can be looked into in #44

Tried building the images via quay.io as a test, and there's this silly issue here:

https://quay.io/repository/iver-wharf/wharf-provider-github/build/12709729-520c-4627-a5a0-79a6b7e60b4e

The base image in that Dockerfile is docker.io/golang:1.13.4, i.e. this: https://hub.docker.com/_/golang
We ultimatetly want to rely on Wharf to build these things, but I'm still rumped that it didn't work :(
Until we get Wharf up and running for GitHub & quay.io integration, we'll resort to just manually building and pushing it locally

Image has been pushed. They are all available over at:
docker pull quay.io/iver-wharf/wharf-api:v4.0.0
docker pull quay.io/iver-wharf/wharf-web:v1.2.0
docker pull quay.io/iver-wharf/wharf-provider-github:v1.1.1
docker pull quay.io/iver-wharf/wharf-provider-gitlab:v1.1.1
docker pull quay.io/iver-wharf/wharf-provider-azuredevops:v1.1.1


Update to slurm-gcp 5.6.2 (from 5.6.0)

Submission Checklist

[ ] Have you installed and run this change against pre-commit? (pre-commit install)
[ ] Are all tests passing? (make tests)
[ ] Have you written unit tests to cover this change?
[ ] Is unit test coverage still above 80%?
[ ] Have you updated all applicable documentation?
[ ] Have you followed the guidelines in our Contributing document?

/gcbrun


Snippet Widgets Interface

Terminology:

Term
Definition

gutter
a CodeMirror concept -- examples would be the line numbers gutter or code folding gutter

line widget
a CodeMirror concept that is a widget either above or below a line of code in the CodeMirror instance

snippet
HB's integrated snippets, such as images, class tables, monsters, etc...

widget
a small utility that performs a function

Description
While discussing things with the DOMT team, one of the biggest topics that came up was snippet options. The overall concept stemmed from there being a solid number of users who are very unfamiliar with HTML/CSS, and having a UI to edit snippets would be really beneficial to them.
The DOMT team was initially thinking something akin to GMB's snippet modal, but this would allow for after-insertion editing as well as initial editing (the editor could be set up to open itself when a snippet is inserted).
Thoughts
Some ideas for what this could look like:

A gutter or line widget that when clicked will create a line widget that will allow for UI-based editing of the snippet
Examples snippet options that could be made editable

LInk URLs/text
Inserting/removing frame/decoration/wide/other classes for known div types (monster, classTable, artist, note, etc...)
Values for vertical and horizontal spacing
Image height, width, position, URL, and alt text
Watercolor image selector and color picker

In chatting with @calculuschild, we realized it could be implemented in such a way that different document themes (tba) have different widgets available to them, and users could create their own widgets. A configurable approach would be ideal here, despite that being a bit more complicated from a programming perspective.

Working proof of concept: https://github.com/jeddai/homebrewery/pull/1
Videos: monster/table frame, artist credit position
Important Notes
CodeMirror handles line widgets normally by passing in dom elements. We do not want to do this, as the redraw process behind the scenes results in a new instance of the DOM elements and focus will be lost. This can be handled by the ReactDOM render method by rendering initially to a parent element and subsequently back into the widget's DOM node. This will allow for consistent editing of text/number inputs, sliders, or any other input types we want to include.
Any react components or fragments can be rendered into the element with the method described above, which should allow for external-to-HB components such as color pickers and more detailed components like image pickers.

I think that these small widgets are incredibly useful for options like frames and width, but would like to make the argument that looking into the future for more complex snippets it would be nice to have a more robust snippet creator at least for creation, but if possible for editing.
I remember @jeddai mentioning that what is triggered for gutter widgets can be customized, meaning that if that is true, in theory a more fleshed out editor in a modal could potentially be triggered.
A use case for this would be opening up a modal to create a table, or position an image.
For instance having 2 menu items for tables, one that is just "table" and another "class table". Table could open a modal that lets you set number of columns and rows, and potentially enter values for the columns and rows. Class Table could let you select third caster/half caster/full caster/blank caster/martial (and decoration/frame/etc) and remove having to have menu items for all options.

I definitely agree, there are some snippets that could really benefit from a pre-insertion editor in a modal. I think it would be good to flesh out what snippets would benefit from something like that. At the moment it seems that tables and images would be the big two to start with, are there any other existing snippets that could use a modal editor like that?

At risk of jumping ahead on such things, "new page" could offer page layouts (single column left, single column right, top half picture, etc).
Table of contents can be customized to varying levels of specificity (only to h2 headers, or all the way to h6 headers). Include all headers, or ignore headers inside divs with class n (monster, spell, etc).

Advanced Snippet Editor
If I were to put the snippets in order for what I think would benefit the community the most to have an "advanced editor" I would say:

Images
Monsters
Tables

I imagine that doing the image advanced editor would open the possibility to do inline images, absolutely positioned images, background images, and splatters with about 90% of the same code. Absolutely positioned images is the most important of that sub-list though, as with absolutely positioned images you can do most of what is needed for the other types with a little massaging. Inline images need relative positioning (to the column its placed in) and an option to wrap the text around it or not, a background image is just an absolutely positioned image but I think having a separate option for it is good UX, and splatters are just absolutely positioned images but with a pre-determined list that can be viewed and chosen from.
I am of the opinion that any snippet could benefit from an "advanced editor" as a popup showing what the snippet is, how to edit it, and what the code means. Educating people on how to easily use the editor is only a boon for everyone involved. However, I consider the listed 3 "essential" rather than "nice to have".
New Page Editor
Gazook's suggestion would also be a nice feature to have. Its not possible to change individual column's widths but its absolutely possible to create a page with only 1 column of any given width. I dont see it as essential as its all very doable with current markup and a simple image element, but having it would be cool and simplify some aspects of layout.

One thing @jeddai and I discussed was adding an easily-readable widget list either to the existing snippet files or as a separate file. Then each theme can easily be given custom widgets without having to hand-code each item:
{
        name         : 'monster',
        prefix       : '{{monster',     // Regex or something to identify the presence of a snippet in the text
        widgets :  [
                {
                        name : frame,
                        type : checkBox,
                        text : frame
                },
                {
                        name : backgroundColor,
                        type : colorPicker,
                        text : background-color:
                },
                {
                        name : positionX,
                        type : slider,
                        text : left:
                }
        ]
}

I can imagine a few types of widgets, although we can start with just a few:

Widget Type
Description
Example

checkBox
checkbox to add or remove related CSS to the curly-brace line
"frame" can be added or removed from a monster div

colorPicker
simple color wheel that changes the hex value of a color
color:#ffffff. Some snippets may have optional colors, such as the color of a note block, etc.

slider or arrowBox
drag a knob left or right / click up & down arrows to set a number value
Used to set x or y position of an image / change image size

button
One-time, repeatable effect
Auto "clean up" the current table, generate lorem ipsum

table editor
popup of a GUI table editor
Used to easily edit table contents, set column alignments, etc.

Random thought: if we scan the text for {{monster to replace with the output of a monster Snippet function, then this is essentially the same functionality as the proposed Brew Variable system.

Yes, I imagine it would be very similar. Although we may be able to approach the variables as just a Markdown extension instead, depending how much we want to do with it.

Mentioned in variables discussion: https://github.com/naturalcrit/homebrewery/discussions/1400#discussioncomment-3090819 .. TL;DR: declaring locally scoped variables could simplify the scanning/parsing involved.
So, instead of scanning for "Armor Class" in a monster block (and failing because someone translated that to french), scan for a variable name (e.g. scan for[_.AC]:=) and updating the variable value (i.e. the text that comes after :=). Then the insertion of the new AC value gets handled by the variables functionality.


VS Code doesn't show predefined launch.json files

I installed mingw64 successfully.
I go to Run -> Add Configuration then choose C++(GDB/LLDB), dropdown for predefined debugging configurations doesn't show anything and create launch.json empty.
Unfortunatelly launch.json is empty.
I can build and create exe file and see the result. But I cannot debug and create launch.json.
My main problem with launch.json and it will be empty.  tasks.json isn't empty and works fine.

VS Code Version: 1.66.1
OS Version: Windows 10 19044.1586
g++ --version 11.2.0
g++ --version  11.2.0

I saw another person has the same problem:
https://stackoverflow.com/questions/71705670/vs-code-doesnt-show-predefined-launch-json-files

Up...

/extCpp


[Shop] Failed to create service

url: /internal/shop/rest/service/put
Failed to create service
Traceback (most recent call last):
  File "/base/data/home/apps/s~mobicagecloudhr/25.399672659706105388/shop/view.py", line 1738, in save_service
    broadcast_to_users=[gusers.get_current_user()])
  File "/base/data/home/apps/s~mobicagecloudhr/25.399672659706105388/add_1_monkey_patches.py", line 205, in _allow_transaction_propagation
    return func(*args, **kwargs)
  File "/base/data/home/apps/s~mobicagecloudhr/25.399672659706105388/add_1_monkey_patches.py", line 184, in wrapped
    r = run(transaction_guid)
  File "/base/data/home/apps/s~mobicagecloudhr/25.399672659706105388/add_1_monkey_patches.py", line 166, in run
    return _orig_run_in_transaction_options(options, *args, **kwargs)
  File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/api/datastore.py", line 2647, in RunInTransactionOptions
    ok, result = _DoOneTry(function, args, kwargs)
  File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/api/datastore.py", line 2667, in _DoOneTry
    result = function(*args, **kwargs)
  File "/base/data/home/apps/s~mobicagecloudhr/25.399672659706105388/mcfw/rpc.py", line 149, in typechecked_return
    result = f(*args, **kwargs)
  File "/base/data/home/apps/s~mobicagecloudhr/25.399672659706105388/mcfw/rpc.py", line 127, in typechecked_f
    return f(**kwargs)
  File "/base/data/home/apps/s~mobicagecloudhr/25.399672659706105388/shop/bizz.py", line 674, in put_service
    search_enabled, qualified_identifier=service.email, broadcast_to_users=broadcast_to_users)
  File "/base/data/home/apps/s~mobicagecloudhr/25.399672659706105388/mcfw/rpc.py", line 149, in typechecked_return
    result = f(*args, **kwargs)
  File "/base/data/home/apps/s~mobicagecloudhr/25.399672659706105388/mcfw/rpc.py", line 126, in typechecked_f
    kwargs)
  File "/base/data/home/apps/s~mobicagecloudhr/25.399672659706105388/mcfw/rpc.py", line 125, in <lambda>
    map(lambda arg: _check_type(arg, kwarg_types[arg], kwargs[arg], accept_missing=accept_missing, func=f),
  File "/base/data/home/apps/s~mobicagecloudhr/25.399672659706105388/mcfw/rpc.py", line 312, in _check_type
    % (name, str(checktype), i, type(x), x))
ValueError: apps: Not all items were of expected type (<type 'str'>, <type 'unicode'>). Encountered an item at index 1 with type <type 'NoneType'>: None.

Failed to create service
Traceback (most recent call last):
  File "/base/data/home/apps/s~mobicagecloudhr/25.399672659706105388/shop/view.py", line 1738, in save_service
    broadcast_to_users=[gusers.get_current_user()])
  File "/base/data/home/apps/s~mobicagecloudhr/25.399672659706105388/add_1_monkey_patches.py", line 205, in _allow_transaction_propagation
    return func(*args, **kwargs)
  File "/base/data/home/apps/s~mobicagecloudhr/25.399672659706105388/add_1_monkey_patches.py", line 184, in wrapped
    r = run(transaction_guid)
  File "/base/data/home/apps/s~mobicagecloudhr/25.399672659706105388/add_1_monkey_patches.py", line 166, in run
    return _orig_run_in_transaction_options(options, *args, **kwargs)
  File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/api/datastore.py", line 2647, in RunInTransactionOptions
    ok, result = _DoOneTry(function, args, kwargs)
  File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/api/datastore.py", line 2667, in _DoOneTry
    result = function(*args, **kwargs)
  File "/base/data/home/apps/s~mobicagecloudhr/25.399672659706105388/mcfw/rpc.py", line 149, in typechecked_return
    result = f(*args, **kwargs)
  File "/base/data/home/apps/s~mobicagecloudhr/25.399672659706105388/mcfw/rpc.py", line 127, in typechecked_f
    return f(**kwargs)
  File "/base/data/home/apps/s~mobicagecloudhr/25.399672659706105388/shop/bizz.py", line 646, in put_service
    is_demo = get_app(service.apps[0]).demo
  File "/base/data/home/apps/s~mobicagecloudhr/25.399672659706105388/mcfw/rpc.py", line 149, in typechecked_return
    result = f(*args, **kwargs)
  File "/base/data/home/apps/s~mobicagecloudhr/25.399672659706105388/mcfw/rpc.py", line 127, in typechecked_f
    return f(**kwargs)
  File "/base/data/home/apps/s~mobicagecloudhr/25.399672659706105388/rogerthat/bizz/app.py", line 68, in get_app
    return trans() if db.is_in_transaction() else db.run_in_transaction(trans)
  File "/base/data/home/apps/s~mobicagecloudhr/25.399672659706105388/rogerthat/bizz/app.py", line 63, in trans
    app = App.get(App.create_key(app_id))
  File "/base/data/home/apps/s~mobicagecloudhr/25.399672659706105388/rogerthat/models/__init__.py", line 195, in create_key
    return db.Key.from_path(App.kind(), app_id)
  File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/api/datastore_types.py", line 527, in from_path
    'received %r (a %s).' % (i + 2, id_or_name, typename(id_or_name)))
BadArgumentError: Expected an integer id or string name as argument 2; received None (a NoneType).


Large bundle size

Describe the bug
When integrating either @corbado/web-js or @corbado/react into frontend frameworks, bundle size tends to get very large.
To Reproduce

React: Taken from our developer panel preview ui, which is a react application with only @corbado/react as added dependencies. The resulting total javascript bundle is 1.13mb large (310kb compressed).
Next.js: Taken from our external page, including vs. excluding the @corbado/react package from the landing page makes around a 300kb difference in first load compressed javascript size.
Angular: Using @corbado/web-js makes the bundle size increase by around 1mb (300kb compressed), which can be seen when building our angular example.

So it's around 300kb of compressed javascript that's added to the bundle size when integrating Corbado.

There are two major reductions in our package code base that we can make:

Remove auto-generated code from the web-core package and write our custom code.
Remove SVG assets from shared-ui package and serve them from a different source

The estimated improved bundle size is:

~840KB uncompressed (~500KB reduction from auto-generated code and ~50KB from svgs)
~210KB compressed (based on the fact that compressed size is approximately 25% of the complete size)

@Aby-JS Please document all your findings here (including how you calculated the numbers).

Here is the approach for finding the impact of three above mentioned factors:

To find the impact of openapi auto-generated code in the web-core package I used webpack to build only that part of the code and then checked the resultant JS size.
The impact of SVGs can be easily found by checking size of the folder as the files are directly converted to React SVGs.
The impact of bundled CSS was found using esbuild