OneDrive and Version Controlled and/or Source Code Directories

Environment

Item
Value

OS, Version / Build
All Windows 10 Builds

Processor Architecture
AMD64/x86/ARM

Processor Type & Model
AMD Ryzen 9 3950X/Irrelevant

Memory
64GB/Irrelevant

Storage Type, free / capacity (e.g. C: SSD 128GB / 512GB)
C: NVMe 400GB/1TB, E: NVMe 600GB/1TB, D: SSD 800GB/ 1TB, Ext4 WSL: NVMe 900GB/ 1TB

Relevant apps installed
OneDrive (Installed on OneDrive dedicated NVMe E:), VS Code, WSL, Git

Description
EDIT: No one is suggesting OneDrive should be used for source control. Read the post.
Among many other uses, OneDrive offers a chance to provide a synchronized developer environment on Windows (including subsystems like WSL), with a massive advantage that directories and projects can be synced without having to make (or undo) commits. This allows project development to happen on a multitude of devices, developers to "pick-up" their work without having to make temporary or incomplete commits. This makes cross-device development cleaner, more efficient and with less cluttered version control histories.
However, OneDrive does not function correctly with version controlled directories, like those with Git produce significantly reduced performance where directories contain many small files that can change rapidly.  This also includes build directories for VS Projects was well as Linux compiled programs with WSL which exist within OneDrive. This can lead to constant syncing, and high resource usage by OneDrive.
These issues and possible solutions have been identified previously by users

https://onedrive.uservoice.com/forums/913522-onedrive-on-windows/suggestions/18449926-reassess-the-possibility-to-ignore-or-exclude-sele
https://onedrive.uservoice.com/forums/262982-onedrive-archive/suggestions/10932591-exclude-folders-inside-onedrive-folder-from-synchi

This issue is being raised here to provide a mechanism to allow MS project managers involved with Developer issues to raise the issue with the OneDrive team.
Steps to reproduce

Use OneDrive on a large project with version control.
Generate many small files, e.g. version controlled directories
Try to sync an existing directory with many small files, e.g. version controlled directories

Expected behavior
One or all of the following behaviors would be expected:
First: It would be expected (hopefully) that OneDrive would sync these files efficiently. However, due to structural reasons this does not happen. So the expected mitigations would allow for using OneDrive in these directories with caveats.

OneDrive will automatically identify such directories, if not all at least those controlled by popular version control software, like git, by using ".git" directories to flag them. These would provide the user a message NOT to place these in a OneDrive controlled folder.
OneDrive will provide a means to disable syncing of said folders, for example with the presence of a .onedriveignore file in the root directory.
Optimal solution would allow a convenient way to "archive" a build directory if needed, with paused syncing while building and cleaning. For example a command line api to OneDrive to pause and resume syncing.

Actual behavior
OneDrive syncs problematic folders without any accounting for the contents, which in the case of developer workloads can be problematic.

Why not just put these kind of things outside OneDrive? Seems to me like the OneDrive app doing its job and syncing files as they come.

Why not just put these kind of things outside OneDrive? Seems to me like the OneDrive app doing its job and syncing files as they come.

Why not just put these kind of things outside OneDrive? Seems to me like the OneDrive app doing its job and syncing files as they come.

I have edited the text of the issue to make it more clear that the problem is syncing many small files, which is a know problem with OneDrive. I have also included a clear, but not exclusive statement about why this is advantageous to do in OneDrive. Final note, the posting of this issue was requested in order to allow it to be brought "up the chain" to the OneDrive team.

Why not just put these kind of things outside OneDrive? Seems to me like the OneDrive app doing its job and syncing files as they come.

I have edited the text of the issue to make it more clear that the problem is syncing many small files, which is a know problem with OneDrive. I have also included a clear, but not exclusive statement about why this is advantageous to do in OneDrive. Final note, the posting of this issue was requested in order to allow it to be brought "up the chain" to the OneDrive team.

Why not just put these kind of things outside OneDrive? Seems to me like the OneDrive app doing its job and syncing files as they come.

I have edited the text of the issue to make it more clear that the problem is syncing many small files, which is a know problem with OneDrive.

I use OneDrive but usually not for Code.  The files of a coding class maybe, but not a code project.  GitHub is far more useful for code efforts and no more expensive :).
One problem I have with OneDrive is that if I sync Music there, any playing of a song causes download of everything.  GitHub has separate actions on small text files, and the related versioning, just right, the same as VSS in its day.    The change to how syncing is handled seems to be an anti-pattern sometimes.
An effort to have OneDrive be one size that fits all strikes me as a bad idea.  If I was gathering .git archives, I would put them on OneDrive as my backup of choice.  But not the source control.

Why not just put these kind of things outside OneDrive? Seems to me like the OneDrive app doing its job and syncing files as they come.

I have edited the text of the issue to make it more clear that the problem is syncing many small files, which is a know problem with OneDrive.

I use OneDrive but usually not for Code.  The files of a coding class maybe, but not a code project.  GitHub is far more useful for code efforts and no more expensive :).
One problem I have with OneDrive is that if I sync Music there, any playing of a song causes download of everything.  GitHub has separate actions on small text files, and the related versioning, just right, the same as VSS in its day.    The change to how syncing is handled seems to be an anti-pattern sometimes.
An effort to have OneDrive be one size that fits all strikes me as a bad idea.  If I was gathering .git archives, I would put them on OneDrive as my backup of choice.  But not the source control.

An effort to have OneDrive be one size that fits all strikes me as a bad idea. If I was gathering .git archives, I would put them on OneDrive as my backup of choice. But not the source control.

No one is suggesting using OneDrive as source control.
I like using OneDrive to host git repositories so I can switch between my laptop and desktop, or work and home without have to make a temporary and messy commit. This was explained in the text of the post.
The issue is, generically, OneDrive has serious performance issues with any collection of small files.

An effort to have OneDrive be one size that fits all strikes me as a bad idea. If I was gathering .git archives, I would put them on OneDrive as my backup of choice. But not the source control.

No one is suggesting using OneDrive as source control.
I like using OneDrive to host git repositories so I can switch between my laptop and desktop, or work and home without have to make a temporary and messy commit. This was explained in the text of the post.
The issue is, generically, OneDrive has serious performance issues with any collection of small files.

serious performance issues with any collection of small files

seem to be a recurring topic for multiple tools

My government organization doesn't allow us to use GitHub, but does have OneDrive.  It would be very useful if we could use it with code version control.

I have found that executing GIT GC will clean up and compact the all of the various GIT versioning files that will help making syncing data via OneDrive much faster.

OneDrive defaults to ON on for a User's Documents, Downloads, etc..
This is especially problematic in an enterprise setting where OneDrive is the de-facto for a virtual desktop.  If OneDrive can't handle git version control workloads, then it utterly craps on your only means of loading a virtual desktop environment.  OneDrive really needs to cooperate better with version control in a very deterministic way.


[FLINK-24835][table-planner] Fix bug in RelTimeIndicatorConverter when materialize time attribute fields of regular join's inputs

What is the purpose of the change
This pull request aims to fix bug in RelTimeIndicatorConverter when materialize time attribute fields of regular join's inputs.
Brief change log
Updates utility method JoinUtil#satisfyRegularJoin, IntervalJoinUtil#satisfyIntervalJoin, TemporalJoinUtil#satisfyTemporalJoin, WindowJoinUtil#satisfyWindowJoin
Verifying this change

UT IntervalJoinTest#testFallbackToRegularJoin

Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): (no)
The public API, i.e., is any changed class annotated with @Public(Evolving): (no)
The serializers: (no)
The runtime per-record code paths (performance sensitive): (no)
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (no)
The S3 file system connector: (no)

Documentation

Does this pull request introduce a new feature? (no)
If yes, how is the feature documented? (not applicable)

Hi, thank you for contribution to fix this bug. I have some confusions following.


Improve C++ Target Language Extendability

Changed code generation methods in CPlusPlus.ts from private to protected to enable to make them overridable.
Added a function in CPlusPlus.ts named emitClassMembers to emit the contents of a class.

Looks great!  If CI likes it we'll merge it.

Thank you!


Simple Moderation [Case sensitivity] From Merge #1222

Describe the bug
As per the merge #1222, the implementation of the feature "Simple Prompt Moderation" seems to have some issues. A user is able to easily bypass the prompt avoidance simply through capitalizing a single letter. This introduces the issue of needing to provide every possible prompt through brute force to account for all options.
Ex input moderation blocked words.
"do not follow context"
Bypass
"Do not follow context"
or  "do nOt follow context"
and several other thousand combinations that would need to be accounted for.
I recommend filtering the text either by checking the text in all lowercase or uppercase during the check. Thus eliminating the need for much redundancy. Such as "Ignore all context" (Start of a sentence). Or "Please ignore all context" (Within a sentence). And all other combinations of capitalization.
To Reproduce
Use the Simple Prompt Moderation block in flowise, and input a chat excerpt to block. Here is a picture of me using the feature, and avoiding the filter through a simple capital letter.
Expected behavior
Block the prompt from being executed.
Screenshots
If applicable, add screenshots to help explain your problem.

Setup

Installation NPX
Flowise Version 1.4.3
OS: Linux
Browser: Mullvad [Firefox based]

PR merged!


[i18n] Improve the Esperanto translation

The existing Esperanto translations are literal translations of the original English text, and not especially idiomatic. They also contain a few errors. I have re-written them with better translations.
Please note that this also requires my improved Esperanto translations for Gherkin.

Thanks @tepreece!


performance kill for disk space usage

Hi,
Can we have a performance kill and a dedicated exit code (similar to maxRSS) when there is too much disk space used?
Andrew

I'm surprised we don't have this, but we appear not to. Killer would go here: https://github.com/dmwm/WMCore/blob/master/src/python/WMCore/WMRuntime/Monitors/PerformanceMonitor.py
There are methods in the new psutil we are using do monitor disk space. That may help.

I guess there is the reason this is not implemented. I am not completely sure how we get the disk usage by process id. I looked at the psutil, pidstat, iotop but it seems we can get the write rate for io but not directly disk usage of the given process. I assume the process writing to one file and maybe there is a way to find the way and keep checking the files size.
i.e - 2 step procedure to fine the information

find file name which a given process is writing. (not sure how to do this)
get the size of the given file.

@ericvaandering, @AndrewLevin, Eric, Andrew, do you know any command to 1? Or is there easier way to get this information (I couldn't find one in psutil)

doesn't condor report the disk space usage?
it is one of the things I see when I do condor_q -l
does wmcore not have access to the same information condor has?

I don't know that unix itself can do disk space of a process. You can get I/O that a process has read/written, but if it writes something and deletes is....
I don't know off the top of my head if we can get this info from Condor on the worker node, but even if we could I'm not sure we'd want to implement it that way vs. using something like psutil to get it from the working directory. We already have monitoring (I believe) that works that way but not with psutil. What we don't have, as I recall, is a way to terminate the job based on that info.

Hi Eric, What we are missing is not just the terminating job part. But we don't get the disk information from the command we use
we use following command to ge the information
https://github.com/dmwm/WMCore/blob/master/src/python/WMCore/WMRuntime/Monitors/PerformanceMonitor.py#L60

doesn't condor report the disk space usage?
it is one of the things I see when I do condor_q -l

I see following in condor_q -l, is DiskUsage updated periodically does that what you want?
Do you know what are the differences between _RAW value and other values?
DiskUsage_RAW = 86370
DiskUsage = 75000
RequestDisk = 400000
RequestDisk_RAW = 370121

does wmcore not have access to the same information condor has?

It uses ps command to get other informaion RSS, VSIZE, etc, I don't know the  command to get disk usage by pid. There must be one if condor can get that information. I think condor command to get the information is tricky. Could we even use the same condor command in worker node?

Question, apart from the technical aspects, how would this actually be used, ie. what would be the value we kill on ? Keep in mind this value needs to make sense compared to the Condor kill value (if Condor kills before WMAgent we haven't gained anything).

Very true. My bad, I was remembering existing code in https://github.com/dmwm/WMCore/blob/6f3570a741779d209f0f720647642d51b64845da/src/python/WMComponent/AlertGenerator/Pollers/System.py but that's the Agent not the WN.

Another question, do we want to stay with fixed disk space assignment per job slot (with the only adjustment being the number of cores assigned to a job) or do we eventually move towards scheduling disk space like we do memory for at least the multi-core pilots ? Technical discussions are all fine and well, but we should make sure we have a sensible policy and an idea if/how that policy will evolve before we make changes.

For instance, high IO job slots don't really fit into the fixed disk space per job slot assignment, they effectively over-commit on disk for multi-core pilots that use them.

IMO, I see killing on running out of disk being potentially useful for RelVal. I don't see it being all that useful for anything else.

Hi Dirk,
For me (release validation), the only important thing is that we set the wmcore disk space kill to be less than the condor/site/glidein kill value, which is the RequestDisk value I guess, which I see is normally set to something like 20 GB, although I am not sure where/how it is set.
Andrew

Yes, the main reason I want it is so that I can report back properly to the requestor why the job failed.

RelVal has very different requirements than normal production. You care about being able to tell requestors why something failed, even if the changes we make for you to do so slightly lower your chances that it will succeed in the first place.
For production the tradeoff is different. We don't debug individual jobs there to the same degree that we do RelVal, so implementing changes that lower the chances of a job to succeed are maybe not the best idea.
Keep in mind that while 20GB is the nominal disk per job slot we ask site to give us, many sites have much more.

Anyways, we do have Condor kill expressions on disk. All I am saying is that anything we deploy on the agent side isn't more aggressive than the Condor ones. How about
Agent kills on (nCores * 20GB) + 1GB
Condor kills on (nCores * 20GB) + 5GB
You could still get unlucky if all jobs on a node use their max disk at the same time and run out of disk on the node before these kills kick in, but the only way to avoid this would be kill below 20GB.

yes, that seems reasonable
(maybe wmagent can also parse the reason why condor killed the job, I think there is a condor kill logic expression that has several different parts to it)


issue 004

hello

hello

Will this see if world is anywhere in the body?

hello

test hello test

hello

test

hello

test

hello


unix: Extend UnixStream and UnixDatagram to send and receive file descriptors

There is a similar PR in the standard library: https://github.com/rust-lang/rust/pull/69864 .
I hope tokio could have an async version of that. https://github.com/tokio-rs/tokio/issues/2975

Seems reasonable to me.

Depends on https://github.com/rust-lang/rust/issues/76915.


Constructors, Destructors and Assignment Operators

Description 📜
Included documentation and video on Constructors, Destructors, and Assignment Operators.
#3035

Type of change 📝

[ ] Au#dio (Should be in mp3 format Includes speech clarity, Concise ,Low distortion)
[x] Video (Animations, screen-recordings, presentations and regular explanatory films are all possibilties etc)
[x] Documentation (Content Creation in the form of codes or tutorials)
[ ] Other (If you choose other, Please mention changes below)

Domain of Contribution 📊

[ ] Android Dev #(Flutter)
[ ] Android Dev #(Java)
[ ] Android Dev #(Kotlin)
[ ] Backend Dev #(Java)
[ ] Backend Dev #(.NET)
[ ] Backend Dev #(PHP)
[ ] Backend Dev #(Python)
[x] C/CPP
[ ] Competitive #Programming
[ ] Cyber #Security
[ ] DS#A
[ ] Data#base
[ ] Datascience with #Python
[ ] Datascience with #R
[ ] Frontend Dev #HTML/CSS/JS
[ ] Frontend Dev #React/Angular/Vue
[ ] Go#lang
[ ] Interview #Prep
[ ] Java_#Domain
[ ] Java#Script
[ ] ME#RN
[ ] Machine #Learning
[ ] Open #Source
[ ] Python_#Domain
[ ] Ru#st
[ ] Statis#tics
[ ] UI/#UX

Checklist ✅

[x] I follow Contributing Guidelines & Code of conduct of this project.
[x] I have performed a self-review of my own code or work.
[x] I have commented my code, particularly in hard-to-understand areas.
[x] My changes generates no new warnings.
[x] I'm GWOC'21 contributor

Screenshots / Gif (Optional) 📸

@Suvraneel @kashishahuja2002 I have made the requested changes in this PR

@elixir14082002 this topic is already there in repo, you can just make a pr of the video part if you want and that too on a separate issue, if it's not been taken
Ps. Please check the issues and branch C/CPP before working on any topic

@gunjan-g The topic is a little different and I asked the supervisors before starting my work. I have removed it from C++14.

@gunjan-g The topic is a little different and I asked the supervisors before starting my work. I have removed it from C++14.

Constructors and destructors are there, we can't repeat the topic. Let me know how this is different from the content present. You can add assignment operators but you have assigned an issue opened by someone else, so you have to open a different issue for that too

@gunjan-g The topic is a little different and I asked the supervisors before starting my work. I have removed it from C++14.

Constructors and destructors are there, we can't repeat the topic. Let me know how this is different from the content present. You can add assignment operators but you have assigned an issue opened by someone else, so you have to open a different issue for that too

Actually, I was told that I can work on this particular issue, I only need to change the information a little, and also there is the topic of the assignment operators. That's why I created the PR. This was an open issue and no one was working on it. @gunjan-g

Looks Good!...Can be Merged!

This topic already exists in doc mode
so you either need to change it to video mode or work on a  new issue.

Looks Good!...Can be Merged!

 You're approval is neither required nor appreciated here. I would suggest you to leave it upto mentors and supervisors, and kindly focus on contributing. :)

Looks Good!...Can be Merged!

So sweet of you, handling the work of a mentor and contributor at the same time. Honestly, we were waiting for your approval only XD

I am closing this PR due to several issues.


chan.sankakucomplex.com

i've added sankakucomplex as a source and then tried dragging and dropping the images in and it doesn't recognize the source or something.  the md5 source codes are in the file names.  and i tested this with the site manually by inputting md5:source code into the sites search engine and it finds it alright.  but grabber still won't recognize it  site server was not offline and they are valid sources.

have you got a solution? i don't think were ever getting one here

I'm not doing what you're doing specifically, but since this is about Sankaku compatibility, I'll share what I know:
From what I understand, there was some sort of problem where the Sankaku site owner bitched at the Grabber dev to stop letting his program access Sankaku.
Now, Sankaku scrapes most of it's content from other sites, so he has no real leg to stand on in this argument, but he was being such a thorn in the Grabber Dev's side that the Grabber Dev gave in and removed compatibility with Sankaku in the next version.
I forget exactly what I did, but here's the basics: There's some sort of special file that tells Grabber how to access different websites. The file that gave grabber compatibility with Sankaku exists in earlier versions of the program, but was removed from later versions. Installing an earlier version and updating will also cause Grabber to delete this compatibility file, and just using the in-program configuration tool isn't enough to make it work with Sankaku.
What I did was I installed an earlier version of Grabber (version 7.1.1, I think), found the special file(s) that let it be compatible with Sankaku, manually marked those files as Read-Only to stop it from being deleted, and then updated the program. There should be some more specific on this if you keep looking around in the issues (solved and unsolved) with "sankaku" in the search term.

No estoy haciendo lo que estás haciendo específicamente, pero como se trata de la compatibilidad de Sankaku, compartiré lo que sé:
Por lo que entiendo, hubo algún tipo de problema en el que el propietario del sitio Sankaku se quejó al desarrollador de Grabber para dejar de permitir que su programa acceda a Sankaku.
Ahora, Sankaku raspa la mayor parte de su contenido de otros sitios, por lo que no tiene una pierna real para sostenerse en este argumento, pero estaba siendo tan agudo en el lado del Grabber Dev que el Grabber Dev cedió y eliminó la compatibilidad con Sankaku en el próxima versión
Olvidé exactamente lo que hice, pero aquí están los conceptos básicos: hay algún tipo de archivo especial que le dice a Grabber cómo acceder a diferentes sitios web. El archivo que dio compatibilidad grabber con Sankaku existe en versiones anteriores del programa, pero fue eliminado de versiones posteriores. La instalación de una versión anterior y la actualización también harán que Grabber elimine este archivo de compatibilidad, y solo usar la herramienta de configuración en el programa no es suficiente para que funcione con Sankaku.
Lo que hice fue instalar una versión anterior de Grabber (creo que la versión 7.1.1), encontré los archivos especiales que permiten que sea compatible con Sankaku, marqué manualmente esos archivos como de solo lectura para evitar que se eliminen y luego actualizó el programa. Debería haber algo más específico sobre esto si sigue buscando en los problemas (resueltos y no resueltos) con "sankaku" en el término de búsqueda.

I'm not doing what you're doing specifically, but since this is about Sankaku compatibility, I'll share what I know:
From what I understand, there was some sort of problem where the Sankaku site owner bitched at the Grabber dev to stop letting his program access Sankaku.
Now, Sankaku scrapes most of it's content from other sites, so he has no real leg to stand on in this argument, but he was being such a thorn in the Grabber Dev's side that the Grabber Dev gave in and removed compatibility with Sankaku in the next version.
I forget exactly what I did, but here's the basics: There's some sort of special file that tells Grabber how to access different websites. The file that gave grabber compatibility with Sankaku exists in earlier versions of the program, but was removed from later versions. Installing an earlier version and updating will also cause Grabber to delete this compatibility file, and just using the in-program configuration tool isn't enough to make it work with Sankaku.
What I did was I installed an earlier version of Grabber (version 7.1.1, I think), found the special file(s) that let it be compatible with Sankaku, manually marked those files as Read-Only to stop it from being deleted, and then updated the program. There should be some more specific on this if you keep looking around in the issues (solved and unsolved) with "sankaku" in the search term.

Hello, I am doing the same procedure as you, only when I put a label, for example, Nekomimi returns 206,253 results, but when I arrive after 1000 pages, all the remaining files mark them as existing, I already verified the files marked as existing in the downloaded folder and there they are, the remaining pages show them as analyzed and all the URLs were analyzed, the question is ?, are there really only 21,706 files of the 206,253 displayed? Or is there a problem?

I'm not doing what you're doing specifically, but since this is about Sankaku compatibility, I'll share what I know:
From what I understand, there was some sort of problem where the Sankaku site owner bitched at the Grabber dev to stop letting his program access Sankaku.
Now, Sankaku scrapes most of it's content from other sites, so he has no real leg to stand on in this argument, but he was being such a thorn in the Grabber Dev's side that the Grabber Dev gave in and removed compatibility with Sankaku in the next version.
I forget exactly what I did, but here's the basics: There's some sort of special file that tells Grabber how to access different websites. The file that gave grabber compatibility with Sankaku exists in earlier versions of the program, but was removed from later versions. Installing an earlier version and updating will also cause Grabber to delete this compatibility file, and just using the in-program configuration tool isn't enough to make it work with Sankaku.
What I did was I installed an earlier version of Grabber (version 7.1.1, I think), found the special file(s) that let it be compatible with Sankaku, manually marked those files as Read-Only to stop it from being deleted, and then updated the program. There should be some more specific on this if you keep looking around in the issues (solved and unsolved) with "sankaku" in the search term.

Hello, I am doing the same procedure as you, only when I put a label, for example, Nekomimi returns 206,253 results, but when I arrive after 1086 pages, all the remaining files mark them as existing, I already checked the files marked as existing in the downloaded folder and there they are, the remaining pages show them as analyzed and all URLs were analyzed, the question is? Are there really only 21,706 files out of the 206,253? Or is there a problem?

First, this is not really sankaku-compliant, so your speeds are abyssmal I guess?
Are your files all named %md5sum%.%ext%? If not, that could be a problem (non-unique names)


[AutoPR policyinsights/resource-manager] typo: policyinsights/resource-manager/Microsoft.PolicyInsights

Created to sync https://github.com/Azure/azure-rest-api-specs/pull/4726

This PR has been merged into https://github.com/Azure/azure-sdk-for-python/pull/3309


GSK-895 Dataset sample & predict caching

Description

[x] Create a sample dataset whenever uploading the dataset (max 1000 rows)
[x] Allow to choose between sample and whole data when creating debugging session
[x] Run all catalog functions on sample dataset for quick preview
[x] Allow to try a test suite on sample dataset (no persistance in database)
[x] Cache model predictions
[x] Various UI improvement

Type of Change

[x] 🥂 Improvement (non-breaking change which improves an existing feature)

TODO: test on this dataset: https://github.com/Giskard-AI/giskard-examples/blob/task/test-fraud-detection-notebook/ieee_fraud_detection_adversarial_validation.ipynb

@andreybavt if for you the backend part is ok we can merge this.

@kevinmessiaen , let me know if you don't have time to resolve the conflicts


project compilation failure

When I run the compile command from step no 3 of the clone project from https://github.com/Azure/azure-sdk-for-java/blob/master/sdk/spring/azure-spring-boot-samples/README.md#ready-to-run-checklist I get the below error

My goal is to run the azure active directory web app samples. What am I missing here?

@stliu can you please take a look?
/cc @conniey

@jeevanbalanmanoj you could try to build the developing version of aad starter by following this instead.

We'll combine these two guidelines together to make it consistent and minimize the number of packages to build when to run samples. @moarychan could you help with this?

@jeevanbalanmanoj , We will only focus on the packages under module spring, please use the following commands first:

Build all modules in spring directory:
mvn -f sdk/spring/pom.xml -Dmaven.javadoc.skip=true -Dcheckstyle.skip=true -Dspotbugs.skip=true -Drevapi.skip=true -Djacoco.skip=true -DskipTests -Dparallel-test-playback clean install

Build the modules associated with the current module, such as azure-spring-boot-sample-active-directory-webapp:
mvn -Dmaven.javadoc.skip=true -Dcheckstyle.skip=true -Dspotbugs.skip=true -Drevapi.skip=true -Djacoco.skip=true -DskipTests -Dparallel-test-playback clean install -pl .\sdk\spring\azure-spring-boot-samples\azure-spring-boot-sample-active-directory-webapp\ -am

Closing this issure because the PR has been merged.


KOPs Integration

Description
Looking at https://karpenter.sh/docs/getting-started/, I don't see any introduction into how Karpenter can be used with a Kops-provisioned cluster. I've been trying to get Karpenter to work with Kops clusters, but with no success yet. The docs only mention EKS.
Can someone add a section on how a generic K8s cluster on AWS can get Karpenter installed? Or at least drop some notes in here.

Hey @RobertLucian. Karpenter is EKS agnostic and should work just fine w/ KOPs. I know @bwagner5 runs it like this. You will need a way to get the KarpenterControllerRole into your pod identity (perhaps https://github.com/jtblin/kube2iam) and then simply helm install Karpenter: https://github.com/aws/karpenter/tree/main/charts/karpenter#installing-the-chart

I see. Thanks @ellistarn! That sounds straightforward then. Any @bwagner5 gotchas I should be aware of?

I think we should have a specific page on kops. It sounds like a prevalent use case.

Kops will most likely get added to Kops as a managed add-on.
In my kops config, I've put Karpenter on the control-plane nodes. You can put them on any node group though.
You'll need to add the IAM policy for Karpenter in the kops cluster spec
spec:
...
 additionalPolicies:
    master: |
      [
        {
          "Effect": "Allow",
          "Action": [
            "ec2:CreateLaunchTemplate",
            "ec2:CreateFleet",
            "ec2:RunInstances",
            "ec2:CreateTags",
            "iam:PassRole",
            "ec2:TerminateInstances",
            "ec2:DescribeLaunchTemplates",
            "ec2:DescribeInstances",
            "ec2:DescribeSecurityGroups",
            "ec2:DescribeSubnets",
            "ec2:DescribeInstanceTypes",
            "ec2:DescribeInstanceTypeOfferings",
            "ec2:DescribeAvailabilityZones",
            "ssm:GetParameter",
          ],
            "Resource": "*"
        }
      ]

and then your helm values.yaml for Karpenter should look something like this (filling in Cluster Name DNS and Cluster Endpoint):
serviceAccount:
  create: true
  name: karpenter
  # Annotations to add to the service account (like the ARN of the IRSA role)
  annotations: {}
controller:
  nodeSelector:
    kubernetes.io/role: master
  tolerations:
    - operator: Exists
      key: node-role.kubernetes.io/master
  affinity: {}
  clusterName: ${CLUSTER_NAME_DNS}
  clusterEndpoint: "https://${KOPS_ENDPOINT}"
webhook:
  env: []
  nodeSelector:
    kubernetes.io/role: master
  tolerations:
    - operator: Exists
      key: node-role.kubernetes.io/master
  affinity: {}

Karpenter will definitely become a kOps addon.
But karpenter overlaps a lot with things kOps relies on ASGs for. So it becomes a bit more than just a plain addon.
E.g should kops get instances also list the karpenter-managed nodes? I think that makes sense. Should kops delete instance work? Probably.
The default provisioner should perhaps also be owned by kOps, and if so, that needs some configuration options as well. Maybe each Provisioner should get its own launch template as well, so one can define things like AMIs per Provisioner.
I have a PoC in a branch https://github.com/olemarkus/kops/tree/karpenter
This branch will unconditionally install karpenter and it depends on irsa being enabled.

Karpenter will definitely become a kOps addon.

Amazing! Do you have an issue we can cross reference?

But karpenter overlaps a lot with things kOps relies on ASGs for. So it becomes a bit more than just a plain addon.
E.g should kops get instances also list the karpenter-managed nodes? I think that makes sense. Should kops delete instance work? Probably.

Keep in mind, kubectl delete nodes on a node created by karpenter will trigger node deletion using a finalizer and our cordon/drain workflow. Not that kops delete instance shouldn't work, but we need to figure out which interface to recommend to customers.

The default provisioner should perhaps also be owned by kOps, and if so, that needs some configuration options as well. Maybe each Provisioner should get its own launch template as well, so one can define things like AMIs per Provisioner.

We've discussed ideas like wiring up spec.provider.additionalUserData, spec.provider.ami. I'd lean towards allowing Karpenter to own the launch template, to allow our controller to manage node versions, security groups, tags, etc.
I'd love to discuss all of this at https://github.com/awslabs/karpenter/blob/main/WORKING_GROUP.md next week.

Karpenter will definitely become a kOps addon.

Amazing! Do you have an issue we can cross reference?

For now, I am just using this one. I'll cross reference anything a user may create on kOps side.

But karpenter overlaps a lot with things kOps relies on ASGs for. So it becomes a bit more than just a plain addon.
E.g should kops get instances also list the karpenter-managed nodes? I think that makes sense. Should kops delete instance work? Probably.

Keep in mind, kubectl delete nodes on a node created by karpenter will trigger node deletion using a finalizer and our cordon/drain workflow. Not that kops delete instance shouldn't work, but we need to figure out which interface to recommend to customers.

We have to consider that kOps clusters always will have non-Karpenter nodes and that it should be possible to delete instances without the K8s API being available.

The default provisioner should perhaps also be owned by kOps, and if so, that needs some configuration options as well. Maybe each Provisioner should get its own launch template as well, so one can define things like AMIs per Provisioner.

We've discussed ideas like wiring up spec.provider.additionalUserData, spec.provider.ami. I'd lean towards allowing Karpenter to own the launch template, to allow our controller to manage node versions, security groups, tags, etc.

Karpenter can own the launch template itself if it takes input from kOps. Especially user data, the node SG, IP addressing flags and things like Resource Based Naming.
A lot of users use kops upgrade cluster to set the correct AMI version (or the corresponding template functions).
Node version in kOps is not determined by the launch template. It is set in the Cluster spec and then provisioned by the kOps server-side component.
Allowing power users to specify their own launch template may be useful, but they would really know what they are doing and ensure that they apply changes needed between kOps versions. That is already possible today though by users creating Provider resource manually.

I'd love to discuss all of this at https://github.com/awslabs/karpenter/blob/main/WORKING_GROUP.md next week.

I think a good start would be to consider how Cluster updates should work. Both upgrades where the API is available and upgrades where the API is not used (kops rolling-upgrade cluster --cloudonly)

We had a brief discussion in the kops-dev channel on how to go about karpenter provisioner. What we found is that the Instance Group resource and Provisioner has a lot in common. So because InstanceGroup is designed to be somewhat cloudprovisioner agnostic, it was trivial to create provisioners based on Instance Groups. This allows us to reuse all logic for launch templates, node configuration etc etc.
The changes we have to do is introduce a new InstanceManager field that can be either CloudGroup or Karpenter. If the latter, we'll create Provisioners based on a template.
This also makes it trivial for kOps users to create Provisioners with the same flexibility we allow for ASGs, and kOps can manage the cluster upgrade cycles without much change to the logic.
Power users who really know what they are doing can still create their own Provisioner.
Appreciate any feedback on this approach.

 We've discussed removing support for a full LT passthrough and instead only allowing a select set of fields like userDataExtensions and ami.
I think of Karpenter provisioners as heterogenous, self-scaling node groups. However, Karpenter has quite a bit more behavior than standard nodes groups (e.g. cordon/drain, expiration, LT Generation, upgrade, defrag (soon)). I worry that while they look similar today, it may evolve in a way that may seem a bit unnatural. The other challenge is that you may end up wrapping/replacing behavior that Karpenter tries to own, which would turn into a tricky multi-master problem.
Can you send me a slack invite? I'd love to chat in real time.

See https://github.com/kubernetes/kops/pull/12906

Started on docs in #1055
Also see docs on the kOps side here: https://kops.sigs.k8s.io/operations/karpenter/


Update Tini to fix photon build-rpm compilation errors

Should fix #28548 — building locally to make sure.
It updates tini to not require xxd for the license, see https://github.com/docker/docker/issues/28548#issuecomment-261343734. Thanks a lot @krallin
diff --git a/CMakeLists.txt b/CMakeLists.txt
index e16d8c2..b5b93ba 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -58,16 +58,6 @@ set (CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -Wl,-Bsymbolic-functions

 # Build

-execute_process(
-  COMMAND xxd -i "LICENSE" "${PROJECT_BINARY_DIR}/tiniLicense.h"
-  WORKING_DIRECTORY "${PROJECT_SOURCE_DIR}"
-  RESULT_VARIABLE xxd_ret
-)
-
-if(NOT "${xxd_ret}" EQUAL 0)
-	message(SEND_ERROR "xxd on LICENSE failed: ${xxd_ret}")
-endif()
-
 configure_file (
 	"${PROJECT_SOURCE_DIR}/src/tiniConfig.h.in"
 	"${PROJECT_BINARY_DIR}/tiniConfig.h"
diff --git a/ci/install_deps.sh b/ci/install_deps.sh
index da7412d..ce5d104 100755
--- a/ci/install_deps.sh
+++ b/ci/install_deps.sh
@@ -7,7 +7,7 @@ apt-get update
 apt-get install --no-install-recommends --yes \
   build-essential git gdb valgrind cmake rpm \
   python-dev libcap-dev python-pip python-virtualenv \
-  hardening-includes gnupg vim-common \
+  hardening-includes gnupg \
   gcc-aarch64-linux-gnu binutils-aarch64-linux-gnu libc6-dev-arm64-cross \
   gcc-arm-linux-gnueabihf binutils-arm-linux-gnueabi libc6-dev-armhf-cross

diff --git a/ci/run_build.sh b/ci/run_build.sh
index 728da5c..7658ac7 100755
--- a/ci/run_build.sh
+++ b/ci/run_build.sh
@@ -118,7 +118,7 @@ if [[ -n "${ARCH_NATIVE:=}" ]]; then
       "${tini}" -h

       echo "Testing $tini for license"
-      "${tini}" -l | grep -q -i "mit license"
+      "$tini" -l | diff - "${SOURCE_DIR}/LICENSE"

       echo "Testing $tini with: true"
       "${tini}" -vvv true
diff --git a/src/tiniLicense.h b/src/tiniLicense.h
new file mode 100644
index 0000000..db230ad
--- /dev/null
+++ b/src/tiniLicense.h
@@ -0,0 +1,95 @@
+unsigned char LICENSE[] = {
+  0x54, 0x68, 0x65, 0x20, 0x4d, 0x49, 0x54, 0x20, 0x4c, 0x69, 0x63, 0x65,
+  0x6e, 0x73, 0x65, 0x20, 0x28, 0x4d, 0x49, 0x54, 0x29, 0x0a, 0x0a, 0x43,
+  0x6f, 0x70, 0x79, 0x72, 0x69, 0x67, 0x68, 0x74, 0x20, 0x28, 0x63, 0x29,
+  0x20, 0x32, 0x30, 0x31, 0x35, 0x20, 0x54, 0x68, 0x6f, 0x6d, 0x61, 0x73,
+  0x20, 0x4f, 0x72, 0x6f, 0x7a, 0x63, 0x6f, 0x20, 0x3c, 0x74, 0x68, 0x6f,
+  0x6d, 0x61, 0x73, 0x40, 0x6f, 0x72, 0x6f, 0x7a, 0x63, 0x6f, 0x2e, 0x66,
+  0x72, 0x3e, 0x0a, 0x0a, 0x50, 0x65, 0x72, 0x6d, 0x69, 0x73, 0x73, 0x69,
+  0x6f, 0x6e, 0x20, 0x69, 0x73, 0x20, 0x68, 0x65, 0x72, 0x65, 0x62, 0x79,
+  0x20, 0x67, 0x72, 0x61, 0x6e, 0x74, 0x65, 0x64, 0x2c, 0x20, 0x66, 0x72,
+  0x65, 0x65, 0x20, 0x6f, 0x66, 0x20, 0x63, 0x68, 0x61, 0x72, 0x67, 0x65,
+  0x2c, 0x20, 0x74, 0x6f, 0x20, 0x61, 0x6e, 0x79, 0x20, 0x70, 0x65, 0x72,
+  0x73, 0x6f, 0x6e, 0x20, 0x6f, 0x62, 0x74, 0x61, 0x69, 0x6e, 0x69, 0x6e,
+  0x67, 0x20, 0x61, 0x20, 0x63, 0x6f, 0x70, 0x79, 0x0a, 0x6f, 0x66, 0x20,
+  0x74, 0x68, 0x69, 0x73, 0x20, 0x73, 0x6f, 0x66, 0x74, 0x77, 0x61, 0x72,
+  0x65, 0x20, 0x61, 0x6e, 0x64, 0x20, 0x61, 0x73, 0x73, 0x6f, 0x63, 0x69,
+  0x61, 0x74, 0x65, 0x64, 0x20, 0x64, 0x6f, 0x63, 0x75, 0x6d, 0x65, 0x6e,
+  0x74, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x20, 0x66, 0x69, 0x6c, 0x65, 0x73,
+  0x20, 0x28, 0x74, 0x68, 0x65, 0x20, 0x22, 0x53, 0x6f, 0x66, 0x74, 0x77,
+  0x61, 0x72, 0x65, 0x22, 0x29, 0x2c, 0x20, 0x74, 0x6f, 0x20, 0x64, 0x65,
+  0x61, 0x6c, 0x0a, 0x69, 0x6e, 0x20, 0x74, 0x68, 0x65, 0x20, 0x53, 0x6f,
+  0x66, 0x74, 0x77, 0x61, 0x72, 0x65, 0x20, 0x77, 0x69, 0x74, 0x68, 0x6f,
+  0x75, 0x74, 0x20, 0x72, 0x65, 0x73, 0x74, 0x72, 0x69, 0x63, 0x74, 0x69,
+  0x6f, 0x6e, 0x2c, 0x20, 0x69, 0x6e, 0x63, 0x6c, 0x75, 0x64, 0x69, 0x6e,
+  0x67, 0x20, 0x77, 0x69, 0x74, 0x68, 0x6f, 0x75, 0x74, 0x20, 0x6c, 0x69,
+  0x6d, 0x69, 0x74, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x20, 0x74, 0x68, 0x65,
+  0x20, 0x72, 0x69, 0x67, 0x68, 0x74, 0x73, 0x0a, 0x74, 0x6f, 0x20, 0x75,
+  0x73, 0x65, 0x2c, 0x20, 0x63, 0x6f, 0x70, 0x79, 0x2c, 0x20, 0x6d, 0x6f,
+  0x64, 0x69, 0x66, 0x79, 0x2c, 0x20, 0x6d, 0x65, 0x72, 0x67, 0x65, 0x2c,
+  0x20, 0x70, 0x75, 0x62, 0x6c, 0x69, 0x73, 0x68, 0x2c, 0x20, 0x64, 0x69,
+  0x73, 0x74, 0x72, 0x69, 0x62, 0x75, 0x74, 0x65, 0x2c, 0x20, 0x73, 0x75,
+  0x62, 0x6c, 0x69, 0x63, 0x65, 0x6e, 0x73, 0x65, 0x2c, 0x20, 0x61, 0x6e,
+  0x64, 0x2f, 0x6f, 0x72, 0x20, 0x73, 0x65, 0x6c, 0x6c, 0x0a, 0x63, 0x6f,
+  0x70, 0x69, 0x65, 0x73, 0x20, 0x6f, 0x66, 0x20, 0x74, 0x68, 0x65, 0x20,
+  0x53, 0x6f, 0x66, 0x74, 0x77, 0x61, 0x72, 0x65, 0x2c, 0x20, 0x61, 0x6e,
+  0x64, 0x20, 0x74, 0x6f, 0x20, 0x70, 0x65, 0x72, 0x6d, 0x69, 0x74, 0x20,
+  0x70, 0x65, 0x72, 0x73, 0x6f, 0x6e, 0x73, 0x20, 0x74, 0x6f, 0x20, 0x77,
+  0x68, 0x6f, 0x6d, 0x20, 0x74, 0x68, 0x65, 0x20, 0x53, 0x6f, 0x66, 0x74,
+  0x77, 0x61, 0x72, 0x65, 0x20, 0x69, 0x73, 0x0a, 0x66, 0x75, 0x72, 0x6e,
+  0x69, 0x73, 0x68, 0x65, 0x64, 0x20, 0x74, 0x6f, 0x20, 0x64, 0x6f, 0x20,
+  0x73, 0x6f, 0x2c, 0x20, 0x73, 0x75, 0x62, 0x6a, 0x65, 0x63, 0x74, 0x20,
+  0x74, 0x6f, 0x20, 0x74, 0x68, 0x65, 0x20, 0x66, 0x6f, 0x6c, 0x6c, 0x6f,
+  0x77, 0x69, 0x6e, 0x67, 0x20, 0x63, 0x6f, 0x6e, 0x64, 0x69, 0x74, 0x69,
+  0x6f, 0x6e, 0x73, 0x3a, 0x0a, 0x0a, 0x54, 0x68, 0x65, 0x20, 0x61, 0x62,
+  0x6f, 0x76, 0x65, 0x20, 0x63, 0x6f, 0x70, 0x79, 0x72, 0x69, 0x67, 0x68,
+  0x74, 0x20, 0x6e, 0x6f, 0x74, 0x69, 0x63, 0x65, 0x20, 0x61, 0x6e, 0x64,
+  0x20, 0x74, 0x68, 0x69, 0x73, 0x20, 0x70, 0x65, 0x72, 0x6d, 0x69, 0x73,
+  0x73, 0x69, 0x6f, 0x6e, 0x20, 0x6e, 0x6f, 0x74, 0x69, 0x63, 0x65, 0x20,
+  0x73, 0x68, 0x61, 0x6c, 0x6c, 0x20, 0x62, 0x65, 0x20, 0x69, 0x6e, 0x63,
+  0x6c, 0x75, 0x64, 0x65, 0x64, 0x20, 0x69, 0x6e, 0x0a, 0x61, 0x6c, 0x6c,
+  0x20, 0x63, 0x6f, 0x70, 0x69, 0x65, 0x73, 0x20, 0x6f, 0x72, 0x20, 0x73,
+  0x75, 0x62, 0x73, 0x74, 0x61, 0x6e, 0x74, 0x69, 0x61, 0x6c, 0x20, 0x70,
+  0x6f, 0x72, 0x74, 0x69, 0x6f, 0x6e, 0x73, 0x20, 0x6f, 0x66, 0x20, 0x74,
+  0x68, 0x65, 0x20, 0x53, 0x6f, 0x66, 0x74, 0x77, 0x61, 0x72, 0x65, 0x2e,
+  0x0a, 0x0a, 0x54, 0x48, 0x45, 0x20, 0x53, 0x4f, 0x46, 0x54, 0x57, 0x41,
+  0x52, 0x45, 0x20, 0x49, 0x53, 0x20, 0x50, 0x52, 0x4f, 0x56, 0x49, 0x44,
+  0x45, 0x44, 0x20, 0x22, 0x41, 0x53, 0x20, 0x49, 0x53, 0x22, 0x2c, 0x20,
+  0x57, 0x49, 0x54, 0x48, 0x4f, 0x55, 0x54, 0x20, 0x57, 0x41, 0x52, 0x52,
+  0x41, 0x4e, 0x54, 0x59, 0x20, 0x4f, 0x46, 0x20, 0x41, 0x4e, 0x59, 0x20,
+  0x4b, 0x49, 0x4e, 0x44, 0x2c, 0x20, 0x45, 0x58, 0x50, 0x52, 0x45, 0x53,
+  0x53, 0x20, 0x4f, 0x52, 0x0a, 0x49, 0x4d, 0x50, 0x4c, 0x49, 0x45, 0x44,
+  0x2c, 0x20, 0x49, 0x4e, 0x43, 0x4c, 0x55, 0x44, 0x49, 0x4e, 0x47, 0x20,
+  0x42, 0x55, 0x54, 0x20, 0x4e, 0x4f, 0x54, 0x20, 0x4c, 0x49, 0x4d, 0x49,
+  0x54, 0x45, 0x44, 0x20, 0x54, 0x4f, 0x20, 0x54, 0x48, 0x45, 0x20, 0x57,
+  0x41, 0x52, 0x52, 0x41, 0x4e, 0x54, 0x49, 0x45, 0x53, 0x20, 0x4f, 0x46,
+  0x20, 0x4d, 0x45, 0x52, 0x43, 0x48, 0x41, 0x4e, 0x54, 0x41, 0x42, 0x49,
+  0x4c, 0x49, 0x54, 0x59, 0x2c, 0x0a, 0x46, 0x49, 0x54, 0x4e, 0x45, 0x53,
+  0x53, 0x20, 0x46, 0x4f, 0x52, 0x20, 0x41, 0x20, 0x50, 0x41, 0x52, 0x54,
+  0x49, 0x43, 0x55, 0x4c, 0x41, 0x52, 0x20, 0x50, 0x55, 0x52, 0x50, 0x4f,
+  0x53, 0x45, 0x20, 0x41, 0x4e, 0x44, 0x20, 0x4e, 0x4f, 0x4e, 0x49, 0x4e,
+  0x46, 0x52, 0x49, 0x4e, 0x47, 0x45, 0x4d, 0x45, 0x4e, 0x54, 0x2e, 0x20,
+  0x49, 0x4e, 0x20, 0x4e, 0x4f, 0x20, 0x45, 0x56, 0x45, 0x4e, 0x54, 0x20,
+  0x53, 0x48, 0x41, 0x4c, 0x4c, 0x20, 0x54, 0x48, 0x45, 0x0a, 0x41, 0x55,
+  0x54, 0x48, 0x4f, 0x52, 0x53, 0x20, 0x4f, 0x52, 0x20, 0x43, 0x4f, 0x50,
+  0x59, 0x52, 0x49, 0x47, 0x48, 0x54, 0x20, 0x48, 0x4f, 0x4c, 0x44, 0x45,
+  0x52, 0x53, 0x20, 0x42, 0x45, 0x20, 0x4c, 0x49, 0x41, 0x42, 0x4c, 0x45,
+  0x20, 0x46, 0x4f, 0x52, 0x20, 0x41, 0x4e, 0x59, 0x20, 0x43, 0x4c, 0x41,
+  0x49, 0x4d, 0x2c, 0x20, 0x44, 0x41, 0x4d, 0x41, 0x47, 0x45, 0x53, 0x20,
+  0x4f, 0x52, 0x20, 0x4f, 0x54, 0x48, 0x45, 0x52, 0x0a, 0x4c, 0x49, 0x41,
+  0x42, 0x49, 0x4c, 0x49, 0x54, 0x59, 0x2c, 0x20, 0x57, 0x48, 0x45, 0x54,
+  0x48, 0x45, 0x52, 0x20, 0x49, 0x4e, 0x20, 0x41, 0x4e, 0x20, 0x41, 0x43,
+  0x54, 0x49, 0x4f, 0x4e, 0x20, 0x4f, 0x46, 0x20, 0x43, 0x4f, 0x4e, 0x54,
+  0x52, 0x41, 0x43, 0x54, 0x2c, 0x20, 0x54, 0x4f, 0x52, 0x54, 0x20, 0x4f,
+  0x52, 0x20, 0x4f, 0x54, 0x48, 0x45, 0x52, 0x57, 0x49, 0x53, 0x45, 0x2c,
+  0x20, 0x41, 0x52, 0x49, 0x53, 0x49, 0x4e, 0x47, 0x20, 0x46, 0x52, 0x4f,
+  0x4d, 0x2c, 0x0a, 0x4f, 0x55, 0x54, 0x20, 0x4f, 0x46, 0x20, 0x4f, 0x52,
+  0x20, 0x49, 0x4e, 0x20, 0x43, 0x4f, 0x4e, 0x4e, 0x45, 0x43, 0x54, 0x49,
+  0x4f, 0x4e, 0x20, 0x57, 0x49, 0x54, 0x48, 0x20, 0x54, 0x48, 0x45, 0x20,
+  0x53, 0x4f, 0x46, 0x54, 0x57, 0x41, 0x52, 0x45, 0x20, 0x4f, 0x52, 0x20,
+  0x54, 0x48, 0x45, 0x20, 0x55, 0x53, 0x45, 0x20, 0x4f, 0x52, 0x20, 0x4f,
+  0x54, 0x48, 0x45, 0x52, 0x20, 0x44, 0x45, 0x41, 0x4c, 0x49, 0x4e, 0x47,
+  0x53, 0x20, 0x49, 0x4e, 0x0a, 0x54, 0x48, 0x45, 0x20, 0x53, 0x4f, 0x46,
+  0x54, 0x57, 0x41, 0x52, 0x45, 0x2e, 0x0a
+};
+unsigned int LICENSE_len = 1099;

/cc @crosbymichael @andrewhsu @vieux
Signed-off-by: Vincent Demeester<EMAIL_ADDRESS>

Can you post the diff in the description ? ;)

I can :angel:

LGTM

thanks to https://github.com/krallin/tini/issues/62


Use browser locale to set default language of new user.

This PR finishes the work of #16938.
Major differences from the previous PR -

Used Django's inbuilt function to parse the accept-language header. The code in previous PR was incorect
as it just returned the first language irrespective of the quality value which can be helpful if some of the locales
are not supported by us.
Refactored some part of the code added in previous PR keeping the logic same.

Would add commits to use the correct language when sending initial messages while registration.

Have added the commit to set realm's setting during realm creation as per locale of user creating the realm, as it is done in the previous PR.
Also, I am not sure do we want realm-level setting, for cases where the user's browser locale is not supported by us or we can just use the default en.
If we want this do we want this to be controlled by RealmUserDefault. It is not done now as we assumed that we would remove this setting when setting setting for new users as per browser's locale.

And regarding the initial messages sent during registration, for which commits for added in the previous PR, I can not reproduce that bug and it works fine without adding override_language, and I think it should work without that.

And regarding the initial messages sent during registration, for which commits for added in the previous PR, I can not reproduce that bug and it works fine without adding override_language. Would be good if someone can confirm this by manual testing though.

I tried this again and realised that I was not able to reproduce it due to cookies, so added the commits.
When sending these messages, the translation language is not set explicitly using translation.activate (it is done after login), so django instead first checks the cookies and then the browser locale if required. So, we need to add override_language for cases when the browser locale with highest priority is not supported by us.

I talked through how this works with @alya, and I think the individual user part of it is good to merge aside from some /help/ documentation updates. I did a first pass at the documentation updates and pushed that here.
The global realm one seems problematic -- we think that we should revise the model for it to display a dropdown or similar on the new organization registration page to pick one's preferred default language for the organization, because this choice generates a set of permanent messages to the default streams, and it's annoying if, e.g., those are Russian because you personally prefer that language even though most of your organization speaks English.
I think it makes sense to do the realm part in a new PR; so I'm going to try using this PR to merge just the user part of it.


Whitespace and scrollbar in Flimstrip view in develop branch

When in Flimstrip view in develop branch, there is whitespace at the bottom of the screen when scrolled. It appears to be size of cookie notification bar (... This website uses coolies.... I understand)

Thank you for the report! We will fix it!

This happens because elements inside the videos in the filmstrip are too big for their respective videos and hence overflows out below.

This is now a duplicate of #115


Can I use  Snorkel for  multi-label problems ?

I want to do a multi-label classification(14 labels) using snorkel , but when i am using MajorityLabelVoter it is giving me this error "IndexError: index 5 is out of bounds for axis 0 with size 2" .As per this issue  https://github.com/snorkel-team/snorkel/issues/1041 i have used the naive approach i.e treating each label as a separate binary classification problem but i am not satisfied with the end result. Can you please recommend best course of action here .
Thanks.

Hi @bpandey23, thank you for the question. Yes, multi-label problems are very common in the wild. This repo primarily contains the code for reproducing some of the early papers for the Snorkel research project, many of which were multi-class, but none of which were multi-label. Since forming Snorkel AI, we've done additional research and development to create multi-label compatible LF templates, label models, analysis tools, etc., and all of those are in Snorkel Flow, but none are backwards compatible with this library. So I don't have any open source implementations to point you to, unfortunately.

@bhancock8 , no problem . Thank for the update.


Fix: ReferenceFilter doesn't display the correct number of elements

Hello,
There is currently a bug related to ReferenceFilter. On a list page, when counter is shown, the number is incorrect when resource elements are more than 25.
Example :

On this page: https://archipel.assemblee-virtuelle.org/Organization?filter={"pair%3AhasType"%3A"https%3A%2F%2Fdata.virtual-assembly.org%2Ftypes%2Fscop"}&order=ASC&page=1&perPage=25&sort=pair%3Alabel it lists 3 organizations whereas the filter on the right side panel displays SCOP (2).

The bug was due to an incorrect pagination on the useGetList hook. By default, react-admin limits to 25 results if no pagination parameters is provided (see https://github.com/marmelab/react-admin/blob/master/packages/ra-core/src/dataProvider/useGetList.ts#L60).
I fixed this issue by explicitly adding a pagination parameter with no limit. It currently works because the semantic-data-provider doesn't paginate results in Sparql queries (see https://github.com/assemblee-virtuelle/semapps/blob/master/src/frontend/packages/semantic-data-provider/src/dataProvider/utils/fetchSparqlEndpoints.js#L100) and so has no performance impact. I think in the future it can be perfected by introducing a sparql count query to retrieve this information instead of fetching all the data in the browser...
I also converted the component to Typescript. I split in 2 commits :

7868dc2c5429a9c5c394834dbc07c6b830873e01 focused on Typescript conversion of the component
5798bedcf8366a5b7eb6b5b0f0b3a094116370c0 focused on the bug fix

I think in the future it can be perfected by introducing a sparql count query to retrieve this information instead of fetching all the data in the browser...

Yes this is a much needed improvement. Unfortunately paging is not so easy when using complex SPARQL queries.


getVAOId parameter cleanup

Looks good.
I am working on a different cleanup of vaoid -- avoid deleting it if the mesh becomes dirty.
Do we have any sample where we set the GVRMesh  vertices directly in the app? A grep couldn't find any and I am wondering if we have any old samples that do that.

I am not aware of any sample that uses setVertices. Since it is a public API some people may have used it..
Since Vittal gave the thumbs up I am planning to merge this lunch time.


Convert wav2vec2 fine tuned model to torch script model

I am unable to convert wav2vec2 fine tuned model to torchscript. I want to prune/quantize or shorten the size of the model so that i can load as many models as i can.
Can anyone tell me how to do that?
I have followed vakyansh/convert_hf scripts but couldn't do so because of the errors.

Hi @harveenchadha,
I'm trying to convert wav2vec2 model to onnx for inference use.
I face below error, can you please help me
`
print('Exporting onnx model to', onnx_model)
torch.onnx.export(
model,                                                  # model being run
# tuple(encoder_input.values()),
(
encoder_input['source'],                            # model input (or a tuple for multiple inputs)
encoder_input['padding_mask']
),
onnx_model,                                             # where to save the model (can be a file or file-like object)
opset_version=12,                                       # the ONNX version to export the model to
verbose=True,
do_constant_folding=True,                               # whether to execute constant folding for optimization
input_names=[                                           # the model's input names
'feats',
'input_mask'
],
output_names=[                                          # the model's output names
'encoder_out',
'encoder_padding_mask',
'padding_mask'
],
dynamic_axes={
'feats': {0: 'batch_size', 1: 'max_seq_len'},        # variable length axes
'input_mask': {0: 'batch_size', 1: 'max_seq_len'}
}
)
error:
Exporting onnx model to model.onnx
Traceback (most recent call last):
File "generate_onnx_model.py", line 191, in
result = main(args_local)
File "generate_onnx_model.py", line 173, in main
with torch.no_grad(): generate_onnx_model(args_local, model, sample)
File "generate_onnx_model.py", line 125, in generate_onnx_model
torch.onnx.export(
File "/opt/miniconda/envs/fairseq_fb_18-11-21/lib/python3.8/site-packages/torch/onnx/init.py", line 316, in export
return utils.export(model, args, f, export_params, verbose, training,
File "/opt/miniconda/envs/fairseq_fb_18-11-21/lib/python3.8/site-packages/torch/onnx/utils.py", line 107, in export
_export(model, args, f, export_params, verbose, training, input_names, output_names,
File "/opt/miniconda/envs/fairseq_fb_18-11-21/lib/python3.8/site-packages/torch/onnx/utils.py", line 724, in _export
_model_to_graph(model, args, verbose, input_names,
File "/opt/miniconda/envs/fairseq_fb_18-11-21/lib/python3.8/site-packages/torch/onnx/utils.py", line 493, in _model_to_graph
graph, params, torch_out, module = _create_jit_graph(model, args)
File "/opt/miniconda/envs/fairseq_fb_18-11-21/lib/python3.8/site-packages/torch/onnx/utils.py", line 437, in _create_jit_graph
graph, torch_out = _trace_and_get_graph_from_model(model, args)
File "/opt/miniconda/envs/fairseq_fb_18-11-21/lib/python3.8/site-packages/torch/onnx/utils.py", line 388, in _trace_and_get_graph_from_model
torch.jit._get_trace_graph(model, args, strict=False, _force_outplace=False, _return_inputs_states=True)
File "/opt/miniconda/envs/fairseq_fb_18-11-21/lib/python3.8/site-packages/torch/jit/_trace.py", line 1166, in _get_trace_graph
outs = ONNXTracedModule(f, strict, _force_outplace, return_inputs, _return_inputs_states)(*args, **kwargs)
File "/opt/miniconda/envs/fairseq_fb_18-11-21/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
return forward_call(*input, **kwargs)
File "/opt/miniconda/envs/fairseq_fb_18-11-21/lib/python3.8/site-packages/torch/jit/_trace.py", line 127, in forward
graph, out = torch._C._create_graph_by_tracing(
File "/opt/miniconda/envs/fairseq_fb_18-11-21/lib/python3.8/site-packages/torch/jit/_trace.py", line 118, in wrapper
outs.append(self.inner(*trace_inputs))
File "/opt/miniconda/envs/fairseq_fb_18-11-21/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
return forward_call(*input, **kwargs)
File "/opt/miniconda/envs/fairseq_fb_18-11-21/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1090, in _slow_forward
result = self.forward(*input, **kwargs)
TypeError: forward() takes 1 positional argument but 3 were given
`
Same error wile converting to torchscript
    jit_module = torch.jit.trace(model, tuple(encoder_input.values()), strict=False, check_trace=False)   File "/opt/miniconda/envs/fairseq_fb_18-11-21/lib/python3.8/site-packages/torch/jit/_trace.py", line 741, in trace     return trace_module(   File "/opt/miniconda/envs/fairseq_fb_18-11-21/lib/python3.8/site-packages/torch/jit/_trace.py", line 958, in trace_module     module._c._create_method_from_trace(   File "/opt/miniconda/envs/fairseq_fb_18-11-21/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl     return forward_call(*input, **kwargs)   File "/opt/miniconda/envs/fairseq_fb_18-11-21/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1090, in _slow_forward     result = self.forward(*input, **kwargs) TypeError: forward() takes 1 positional argument but 3 were given

The errors for hf_scripts have been fixed now!


Update electron-builder to version 13.11.1 and electron to 1.6.5

electron-builder is still pretty old but I did not want to deal with conflicts that happened after upgrading to anything above the 13.x series.
To document them, this is what happened when trying the last version of 14.x:
Error: Config is invalid:
{
  "homepage": "Unknown option",
  "category": "Unknown option",
  "linux": [
    {
      "arch": "Unknown option"
    },
    "Invalid option object"
  ],
  "win": [
    {
      "target": [
        "Should be array",
        "Invalid option object"
      ]
    },
    "Invalid option object"
  ]
}

Raw validation errors: [
  {
    "keyword": "additionalProperties",
    "dataPath": "",
    "schemaPath": "#/additionalProperties",
    "params": {
      "additionalProperty": "homepage"
    },
    "message": "should NOT have additional properties"
  },
  {
    "keyword": "additionalProperties",
    "dataPath": "",
    "schemaPath": "#/additionalProperties",
    "params": {
      "additionalProperty": "category"
    },
    "message": "should NOT have additional properties"
  },
  {
    "keyword": "additionalProperties",
    "dataPath": ".linux",
    "schemaPath": "#/additionalProperties",
    "params": {
      "additionalProperty": "arch"
    },
    "message": "should NOT have additional properties"
  },
  {
    "keyword": "type",
    "dataPath": ".linux",
    "schemaPath": "#/properties/linux/anyOf/1/type",
    "params": {
      "type": "null"
    },
    "message": "should be null"
  },
  {
    "keyword": "anyOf",
    "dataPath": ".linux",
    "schemaPath": "#/properties/linux/anyOf",
    "params": {},
    "message": "should match some schema in anyOf"
  },
  {
    "keyword": "type",
    "dataPath": ".win.target",
    "schemaPath": "#/properties/target/anyOf/0/type",
    "params": {
      "type": "array"
    },
    "message": "should be array"
  },
  {
    "keyword": "type",
    "dataPath": ".win.target",
    "schemaPath": "#/properties/target/anyOf/1/type",
    "params": {
      "type": "null"
    },
    "message": "should be null"
  },
  {
    "keyword": "anyOf",
    "dataPath": ".win.target",
    "schemaPath": "#/properties/target/anyOf",
    "params": {},
    "message": "should match some schema in anyOf"
  },
  {
    "keyword": "type",
    "dataPath": ".win",
    "schemaPath": "#/properties/win/anyOf/1/type",
    "params": {
      "type": "null"
    },
    "message": "should be null"
  },
  {
    "keyword": "anyOf",
    "dataPath": ".win",
    "schemaPath": "#/properties/win/anyOf",
    "params": {},
    "message": "should match some schema in anyOf"
  }
]
    at C:\Users\Benjamin\Documents\VCS\Git\cerebro\node_modules\electron-builder\src\util\readPackageJson.ts:151:11
From previous event:
    at validateConfig (C:\Users\Benjamin\Documents\VCS\Git\cerebro\node_modules\electron-builder\out\util\readPackageJson.js:1
55:22)
    at C:\Users\Benjamin\Documents\VCS\Git\cerebro\node_modules\electron-builder\src\packager.ts:128:11
    at Generator.next (<anonymous>)
    at runCallback (timers.js:666:20)
    at tryOnImmediate (timers.js:639:5)
    at processImmediate [as _immediateCallback] (timers.js:611:5)
From previous event:
    at Packager.build (C:\Users\Benjamin\Documents\VCS\Git\cerebro\node_modules\electron-builder\out\packager.js:257:11)
    at C:\Users\Benjamin\Documents\VCS\Git\cerebro\node_modules\electron-builder\src\builder.ts:217:33
    at Generator.next (<anonymous>)
From previous event:
    at build (C:\Users\Benjamin\Documents\VCS\Git\cerebro\node_modules\electron-builder\out\builder.js:70:21)
    at Object.<anonymous> (C:\Users\Benjamin\Documents\VCS\Git\cerebro\node_modules\electron-builder\out\cli\build-cli.js:71:4
1)
    at Module._compile (module.js:571:32)
    at Object.Module._extensions..js (module.js:580:10)
    at Module.load (module.js:488:32)
    at tryModuleLoad (module.js:447:12)
    at Function.Module._load (module.js:439:3)
    at Module.runMain (module.js:605:10)
    at run (bootstrap_node.js:427:7)
    at startup (bootstrap_node.js:148:9)
    at bootstrap_node.js:542:3
error Command failed with exit code<PHONE_NUMBER>.

So, it is not clear for me from you comment. With this update everything is fine and everything works, but if we want to use latest electron-builder – we should resolve some conflicts. Correct?

@KELiON correct. I'm just trying to learn a bit of JS/yarn/npm atm so those PRs are me experimenting but I hope it is at least of some use for you. :)

@Stanzilla it is useful, thanks! Only one thing to change: when we update version of electron we should also update it in build script in app/package.json

Indeed! Done.

Thanks, @Stanzilla!


Update npm packages to their latest to fix build errors

Update npm packages to the latest to fix build errors
Remove the yarn.lock file which is obviously outdated
Created npm package lock file to ensure one can restore a working version

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.:x: Radon222 sign nowYou have signed the CLA already but the status is still pending? Let us recheck it.

@bowdenk7 Could you please have a look while you are free?


Highlight code blocks in treesitter

https://github.com/neovim/neovim/pull/21154
PSA: Code Blocks can now be highlighted in vimdoc
Preferably add an option called treesitter = true
>lua
-- This code will be highlighted as lua
<

Preview

You can add the code below to your github workflow to replace any code block with a lua code block:
- name: Code Blocks
  run: sed -i 's/^>$/>lua/' doc/your_plugin.nvim.txt

Thanks for opening the issue and bringing this feature to my attention. I should add this as a feature directly but haven’t gotten around to it.
I’ve been dealing with some stuff in my personal life and my time on open source projects has taken a hit. But I can review and merge a PR if required!

No hurry. The work-around works perfectly fine for me for now :)

Thanks for bringing this to my attention! Adding treesitter: true will result in codeblocks with the language in vimdoc.
See this commit for an example of how to use it: https://github.com/kdheepak/panvimdoc/commit/0c1c4e1edd0b54ac2f459fbe7af8e57db7118045
And see this commit for an example of what the vimdoc output is with that option enabled: https://github.com/kdheepak/panvimdoc/commit/460adc9085639d110ef7b9dbb783e404596fe956

Thanks!

https://github.com/catppuccin/nvim/commit/ee756bf71c444303d9ea378dc4996e59edb1aa2b Works 🥳


2023 catchup

Existing PRs and Issues

fix(hubot): Can we please get rid of the SIGTERM catch? #1403
fix(robot): Set max json request body size with EXPRESS_LIMIT #1405
fix(robot): express basicAuth #1470
fix(robot): Robot uses deprecated require.extensions prevents using Jest for testing #1483
fix(http): Remove node-scoped-http-client #1485
docs(windows): Fix setx typo to set and put double quotes in the correct place #1509
docs(roadmap): Roadmap for Hubot #1530
fix(robot): return results from send and messageRoom #1546
fix(http): HUBOT_HTTPD can now disable httpd #1548
docs(patterns): use scoped npm package as adapter #1564

New from hubot-new/hubot fork

fix(package): Add package-lock.json generated with Node v14 / NPM v6 #32
fix(shell): Every other character is skipped when typing at the prompt with the default Shell adapter in Node.js versions > 16.19.0 #39
fix(actions): GitHub Actions for tests #45
chore(deps): bump lodash from 4.17.11 to 4.17.21 #47
chore(deps): bump minimatch and mocha #48
chore(deps): bump semver-regex and semantic-release #49
chore(deps): bump shelljs and standard #51
chore(deps): bump path-parse from 1.0.6 to 1.0.7 #54
chore(deps): bump hosted-git-info from 2.7.1 to 2.8.9 #55
chore(deps): bump handlebars from 4.1.0 to 4.7.7 #56
chore(deps): bump y18n from 4.0.0 to 4.0.3 #57
chore(deps): bump yargs-parser and nyc #58
chore(deps): bump express-basic-auth from 1.1.5 to 1.1.7 #59
fix(tests): Include CI for Windows, OSX #62

@technicalpickles @joeyguerra I think we should see Github Actions kick in on this MR - does that need to be enabled on the repo?
Is there a task also to remove the Travis CI checks as well, if they're sufficiently replaced by Github Actions here? They didn't show up on hubot-new/hubot but we wouldn't have had the expected configuration in place.

https://github.com/hubotio/hubot/pull/1581/commits/3ceeb0a4881270ecc9d134e25aaf1806f0236526 may want to be revisited - that change can be updated to retain the existing org.

Whooo, this PR adds 10K LoC. Is that all from .coffee > .js?!

It's the package-lock.json file. 9,494 LOC.

does that need to be enabled on the repo?

It is already enabled 🤔  They do require an initial approval for first time contributors, but I am not seeing a way to enable them. Will see if I can figure it out.

I have no idea why I can't approve or run GitHub Actions for this branch (followed the docs and there's not an approve workflow as suggested).
I was able to npm install and npm test locally on macOS w/ Node 18.x. Two caveats though.

npm install had some warnings about an old lockfile. May want to npm install and commit the updated package.json at some point (doesn't need to be now)

npm WARN old lockfile
npm WARN old lockfile The package-lock.json file was created with an old version of npm,
npm WARN old lockfile so supplemental metadata must be fetched from the registry.
npm WARN old lockfile
npm WARN old lockfile This is a one-time fix-up, please be patient...
npm WARN old lockfile

npm test is successful, but seems to hang after printing out the summary:

load: 7.47  cmd: node 44076 waiting 1.04u 0.12s
load: 7.47  cmd: node 44076 waiting 1.04u 0.12s
load: 7.47  cmd: node 44076 waiting 1.04u 0.12s
^C----------|---------|----------|---------|---------|-------------------
File      | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s
----------|---------|----------|---------|---------|-------------------
All files |       0 |        0 |       0 |       0 |
----------|---------|----------|---------|---------|-------------------

Looks like it may be nyc which is doing... coverage reporting? I'm not very worried about this, but should be fixed up as a followup.

That "pending" thing was raised in https://github.com/hubot-new/hubot/issues/33 which is closed but IDK if the issue was resolved.
Is it that Travis failing prevents Github Actions from proceeding perhaps?

Is it that Travis failing prevents Github Actions from proceeding perhaps?

Doubtful. Looking more closely, it's a branch protection for the default branch, and 'waiting for status to be reported' suggests to me it's not getting kicked off at all. I can remove that requirement for now.

Something I'd like to double check before approving & merging is trying to upgrade an existing hubot. https://github.com/hubotio/hubot-for-hubot is one we can use to test. I'm more worried about loading and running the tests than anything else, so testing with the shell adapter is fine instead of getting it hooked back up to the hubotio slack.

@joeyguerra don't forget the commit message syntax in https://github.com/hubotio/hubot/blob/master/CONTRIBUTING.md for semantic-release. (I'll eventually add a workflow to ensure that, but not here.)

@xurizaemon any suggestions on what to put for the title since this PR includes a mixture of multiple fixes, features, doc updates?

Uhmm ... C-C-C-C-C-C-COMBO? Not sure about the issue title sorry.
AFAIK it's commit messages that semantic-release would care about, and it's likely to use the individual branch commits rather than the merge commits. Apologies if I didn't make my thinking explicit enough when suggesting this a couple days ago!

Ah. Now I see what you're saying. I was thinking, "surely we don't have to go back and update every commit message". But, in fact, you're saying we do. But those individual commits were not "atomic" in that they encapsulated a complete thought or change. Any suggestions for that situation?

The upside of travis not working is it won't be enforced. I can also bypass it as a maintainer even if it did.
As a followup, using PR titles instead of commits would be more practical and easier to manage instead of each commit.

When testing with hubot-for-hubot, http client bugs were exposed. buildOptions (extend) isn’t working as expected. I pushed a fix to this PRs branch. I’ll test it again with hubot-for-hubot later this evening.

I'm done with my testing as well. I pushed some bug fixes to the httpclient module after testing with hubot-for-hubot. @technicalpickles good call! Thank you.
@technicalpickles do you know how to do a dry run semantic-release so we can see how the commit messages are going to look? I was wondering if we should create a different branch, put all the changes that are in this PR into that new one and start from there instead to shore up the commit messages, but I don't know how semantic-release builds the change log from the commit messages. The documentation says it parses the commit messages between the previous release (git tag) and the new one.

I'm not super familiar, as I wasn't the main proponent of using this. I pulled it down locally, and looks like there is a --dry-run command at least. It looks like it needs some GitHub and/or NPM tokens to run.

The documentation says it parses the commit messages between the previous release (git tag) and the new one. But I'm still unclear what commit messages it's going to get.

My hunch is that it wouldn't even run, because it was part of Travis CI which isn't running.

Ok. I'm ready whenever you are :) What are the next steps?

FWIW - I proposed (https://github.com/hubot-new/hubot/issues/44) we adopt semantic-release pattern based on seeing it documented in CONTRIBUTING.md here.
I supported that approach because it's easier to have a clear process and adopting an existing established process would make more sense for community adoption than change. It seems low-maintenance which I think is great. But it's a little opaque - I tried a few combos just now and couldn't work out how to get it to simulate generating release notes.
OTOH if there's a better method, we're free to do that too! So this comment is mostly to say we can be flexible on it :)

when you say, "doing the release manually", do you mean execute the semantic-release cli command from your local machine to publish an updated package to NPM? Or adding the semantic-release command to one of (or create a new one) the GitHub Actions workflow yaml file?

I mean publishing with the npm command, and tagging git. I can create new "Release" really easily, because it can figure most of that out automatically nowadays (that didn't exist when semantic-release was being used)

Ok. I'm ready whenever you are :) What are the next steps?

I'm going to:

merge
add y'all as maintainers
see if I can release a one-off w/o using semantic release

I'm trying a dry run with semantic-release and running into errors. First one is "can't login with password" :(


Arier-Nachweis importieren?

@SebastianWolf-SAP, something for you?
I consider this as "Unacceptable behavior" according to the Code of Conduct, see https://github.com/corona-warn-app/cwa-wishlist/blob/master/CODE_OF_CONDUCT.md#our-standards:
"Examples of unacceptable behavior include:
[…]

Trolling, insulting or derogatory comments, and personal or political attacks"

I'd consider this as "Trolling", and very bad and disrespectful trolling (comparing digital vaccination certificates to the "Arier-Nachweis is just unacceptable IHMO).

@RobinLinus This is a clear violation of our Code of Conduct as outlined by @Ein-Tim. As a consequence, you receive an official warning. Other violations in the future will lead to a temporary or permanent ban.


Upgrade optimum-habana diffusers dependency from 0.26.3 to 0.29.2

Upgrade optimum-habana diffusers from 0.26.3 to 0.29.2

Updated optimum-habana diffusers to 0.29.2 in setup.py
Fixed SVD (GaudiStableVideoDiffusionPipeline) to handle updated API in 0.29.2 for tensor2vid and instead use video_processor.postprocess_video
Fixed GaudiStableDiffusionControlNetPipeline to handle updated check_inputs API in 0.29.2
Fixed tests/test_diffusers.py to use to(dtype=...) instead of to(torch_dtype=...) which is deprecated starting diffusers 0.27.0 (See here)
Clean-up SVD to remove all inheritable functions
Updated warmup logic in SVD
Updated documentation example with command line for SVD that yields improved performance

Tests
Ran tests on G2:
export RUN_SLOW=true
export GAUDI2_CI=1
make test_installs
python -m pytest tests/test_diffusers.py

Diffusers test results before diffusers upgrade:
17 failed, 106 passed, 13 skipped, 65 warnings in 2304.15s (0:38:24)

Diffusers test results after diffusers upgrade:
16 failed, 107 passed, 13 skipped, 316 warnings in 2157.94s (0:35:57)

Note:  Update is needed for supporting enabling newest pipelines such as Stable Diffusion 3, etc.

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.


Phpstan level 6

Pull Request Test Coverage Report for Build 955992922

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 100.0%

Totals

Change from base Build 955897493:
0.0%

Covered Lines:
378

Relevant Lines:
378

💛  - Coveralls


Array assert_eq doesn't check for equality on scalars

Not sure if this is intentional, but I thought I would check:
In [41]: import dask.array as da; import dask.array.utils

In [42]: import numpy as np

In [43]: a = da.from_array(np.array([0]), chunks=1)

In [44]: x = np.array([1])

In [45]: da.utils.assert_eq(a[0], x[0])
# None

This check seems fragile: https://github.com/dask/dask/blob/a1a75a766243f1f84ac1df502de3faf8ce4ebc81/dask/array/utils.py#L99-L103
I think that will short-circuit when a is empty.
(Pdb++) _not_empty(a) and _not_empty(True)  # the second condition isn't checked
()
(Pdb++) _not_empty(True)  # it is hit now, raises an error
*** AttributeError: 'bool' object has no attribute 'shape'

Huh
In [40]: np.isscalar(np.array(1))
Out[40]: False

I'm not quite sure how to distinguish between empty arrays and a scalar.

I think that will short-circuit when a is empty.

I imagine it should probably have been or rather than and.

I'm not quite sure how to distinguish between empty arrays and a scalar.

getattr(x, 'size', 1) will be 1 for scalars and 0 for empty arrays.
np.isscalar() has a pretty strange design. It checks for instances of np.generic types or their built-in Python equivalents (int, float, str, etc).


doc: update visuals in code doc & user guide

all changes done in the user guide are in the Definitions section
we need absolute link in the rust doc if we want the figures to appear on docs.rs; its probably possible to prevent that with a custom doc build script, but I won't look into it now

Scope

[x] Documentation

Type of change

[x] Refactor

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 83.97%.

Additional details and impacted files
@@           Coverage Diff           @@
##           master      #95   +/-   ##
=======================================
  Coverage   83.97%   83.97%
=======================================
  Files          27       27
  Lines        2571     2571
=======================================
  Hits         2159     2159
  Misses        412      412

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


[tiles] add initial support for day/night cycle

Add initial support for swapping tiles depending on the time of the day

done via new TileSwapScheduler behavior:


I can’t invite the bot

Hi I meet all the requirements, and when I try to add it to the right server it says something like ?this bot cannot be invited to more servers, contact owner to verify the bot”

The bot has been verified and has been registered for all necessary intents, multiple users have already been able to add it and use it, including myself. It might need a moment for Discord to register it but once it is, it should be able to join as long as you meet the requirements.

Make sure you're using the correct invite link from the Getting Started page.


Export / import full bot dump (incl. stories, rules, answers, etc.)

Export / import feature for a full single-file Bot dump including nlp, stories, rules, answers, etc.:

Required in Tock Studio (aka Admin) in Applications view
(Possibly in Bot Integrated mode too, eg. importXxxDump in BotInstaller)

It is possible to create a new type of "bot dump", or add missing content to the existing application dump (ie. #779).

Hi,
I don't know if this is part of the 'etc' but if we can also include connector configurations into the dump this would be great and avoid having to create them manually after importing a dump

Hi @jygdev thanks for noticing that. Currently, connector configurations are not included in application dumps. It would be nice to export/import them through dumps, indeed.
Yet their specific params may embed secrets and passwords. Should we put them in dumps too? Or the configurations, without the params/secrets?
I propose we rather work on this (Configurations dumps) through another/new issue. Could you please create it @jygdev ? I'd be interested in your opinion about how to deal with secrets, by the way - you might add it to the issue as a comment?


It does not vibration on complete.

Could you double check you are on the head of development-fp branch (0282ce499e0cdff7d26660be259e4a1cd1a46833)
Also check "Vibrate on Ring" and "Vibrate on Silent" are on in "Sounds & Haptics" setting.


hack addressing tauri issue #9127 (context menus overwrite WNDPROC dwrefdata on windows)

See https://github.com/tauri-apps/muda/issues/162 for more details.
This is not a good fix, but it demonstrates the nature of the problem.

Thank you, I have opened https://github.com/tauri-apps/muda/pull/166 to fix this behavior.


Wrong apiVersion in getting started examples

When I apply the example resource as is I get the following error:
error: unable to recognize "resource.yaml": no matches for kind "Kubegres" in version "io.reactive-tech/v1"
When I modify the apiVersion to kubegres.reactive-tech.io/v1, it works.
I also think the namespace requires an indent, so it will be inside metadata.

@deorder thank you for reporting this issue and @heschlie thank you for fixing it so promptly.
We made some API changes after upgrading the SDK used by Kubegres from Kubebuilder version 2 to version 3. And we forgot to update the documentations which were written before the upgrade.


JEP-290 documentation

Related to #4583
This issue adds an include document that describes the JEP-290 approach in Helidon.
I could not find a Security document that I could use to include it (seems to be gone in 3.x). Can @Verdent or @dalexandrov point me to the right place?
Signed-off-by: Tomas Langer<EMAIL_ADDRESS>

Do you want to include this file to SE and MP security articles?

I am not sure if this belongs to security or somewhere else. It is valid for both SE and MP, yes.


Remove show-example default

Hi, I am working on the package Catppuccin and I am currently writing up the documentation, where I am using the theme that the library provides.
Currently, Tidy makes it very difficult to render examples that adhere to theming. For example, this line makes it impossible to set the preview theme, even if using the preamble argument in parse-module.
Could you please revise this function and remove any such defaults? I do not feel that explicitly setting the fill to white is necessary.
Thanks :)

Perhaps you could allow a dict argument as a solution. I also noticed that here and here are some other rather rigid configurations.
If you require that I make a pull request, please let me know and I would be happy to, based on your preference as to how to solve this.

Hi @TimeTravelPenguin
It is true that the provided themes are not very customizable, that is why you can make your own themes too. Especially the example rendering has proven to be a little tricky. There have been plans to redesign how themes and customization work in tidy but they require the long-awaited custom types in Typst and I don't want to break everything twice.
This is just for the background, however - for show-example we can totally find a good solution for now. Let me look at the issue.
PS: Nice color themes by the way :)

Okay, so the thing is that we could of course add parameters for all these hard-coded values to show-example() but then you'd sortof have to provide these arguments every time you call example in the documentation. This can be overcome by adding some #example = example.with(...) to the preamble argument in parse-module() but that's all not very clean and satisfying.
Again, ideally example would be a type so one can do set and show rules like
#set tidy.example(fill: black, spacing: 1em)

or similar. Unfortunately, they're not there yet.
One thing you can do now is copying the default theme and insert your own modified implementation of show-example. As you pointed out, there are so many hard-coded details of the implementation of show-example that in the end you might want to change even more.
Modifying a theme does not actually require copying everything, you can write
#import "/src/styles/default.typ"

#let show-example(..args) = {} // your own implementation of show-example

#let my-theme = dictionary(default)
#{my-theme.show-example = show-example}

#show-module(docs, style: styles.default, colors: styles.default.colors-dark)

because styles also accepts a dictionary and since recently it is possible to turn a module into a dictionary.
Still, if you can think of another (elegant) solution to solve this issue, let me know!

PS: Nice color themes by the way :)

Haha thanks, but I can't take credit for the Catppuccin themes. I am just a random community member implementing it in Typst to save my poor eyes from a white screen!

Okay, so the thing is that we could of course add parameters for all these hard-coded values to show-example() but then you'd sortof have to provide these arguments every time you call example in the documentation.

There are a few ways to get around this. Or at least, to make it a bit easier to work with, as I will shortly discuss.

There have been plans to redesign how themes and customization work in tidy but they require the long-awaited custom types in Typst and I don't want to break everything twice.

Yes, I am very excited about custom types. For a temporary workaround, I personally like using the Valkyrie package for schema validation.
What I am doing in my project is having some
#import "@preview/valkyrie:0.2.1" as z

#let config-schema = z.dictionary(( ... ))

#let my-func(config: (:)) = {
    let config-defaults = (...)
    config = config-defaults + config
    let valid-config = z.parse(config, config-schema)
}

You don't need Valkyrie for the main point here, but it shows how it can be used. So, the approach I take is that the default user-provided config is empty. I then use the + behaviour on dictionaries to update the functions local default (e.g., setting the fill to white). Then, if the user provides a fill entry, it overwrites your function's defaults.
Then, Valkyrie is helpful because you can assert that the provided dictionary configuration contains only the expected values for the thing you're plugging it into. But like I said, it isn't the necessary part of this. I just wanted to mention it as I am a personal fan.
Ultimately, a redesign is certainly looking like the way to go, but depending on how you want to go about it, the complexity can greatly differ.
Just off the top of my head, I would probably take a kind of functional approach, seeing as there are no object/types. The tidy show-module function would need to be dependent on the remaining other functions. Either these are given as arguments, found through introspection/parsing of a module, or be the result of another function who, itself, if provided the required components to build show-module.
One other approach, which would be much harder to implement and would require some big planning would be to work with functions that give and take contexts (not the typst context), similar to how CeTZ does.
This is just some spitballing of ideas I had while studying your library :)

Haha thanks, but I can't take credit for the Catppuccin themes. I am just a random community member implementing it in Typst to save my poor eyes from a white screen!

Ah okay haha. Still good work porting it to Typst 👍

You don't need Valkyrie for the main point here, but it shows how it can be used. So, the approach I take is that the default user-provided config is empty. I then use the + behaviour on dictionaries to update the functions local default (e.g., setting the fill to white). Then, if the user provides a fill entry, it overwrites your function's defaults.

Yes, I use the same approach in other packages as well.

Just off the top of my head, I would probably take a kind of functional approach, seeing as there are no object/types. The tidy show-module function would need to be dependent on the remaining other functions. Either these are given as arguments, found through introspection/parsing of a module, or be the result of another function who, itself, if provided the required components to build show-module.

Indeed, the dependance is one of the main issues. For example implementations of show-function call show-type etc. Another issue is passing lots of data and settings down to the different styling functions.
I really want to go for user-defined types in the long run because they allow for composable, flexible rules. Additionally, methods on types will allow outsourcing common transformations/preprocessing of the content.
Alas, the different ways that other packages currently deal with this problem are all a bit clunky and have disadvantages of their own. For me it does not make too much sense rebuilding the framework now and then changing it up yet again so I will wait for custom types since everything is possible (I think/I hope) now by building your own themes.
I would still keep this issue open so we can check that this is treated with the needed care once the time has come :)
If you need help defining or modifying a style/theme come back to me.

Ah okay haha. Still good work porting it to Typst 👍

I have so much to do, still, but the package is slowly coming together.

For me it does not make too much sense rebuilding the framework now and then changing it up yet again so I will wait for custom types since everything is possible (I think/I hope) now by building your own themes.

That's perfectly valid. There is a lot of potential opportunity depending on what kinds of features get added in the future. Any changes you do now should probably only complement the future, in the event you don't start from the ground up.

If you need help defining or modifying a style/theme come back to me.

Thanks! I have been having some issues with a slightly hacked version of the default show-example, but it is partly due to my own package's configuration and styling. I need to chip away at yours some more when I have the time. I will certainly reach out if I need help.

One thing that I do want to mention is that one place to perhaps introduce where the user may be able to configure their docs would be by making this code an external dependency.
Just wanted to leave this here as a future consideration.

Yes, this will definitely be customizable at some point!

I am just commenting to update the original issue to be in the context of the recent v0.4.0 release.
In my original issue:

Currently, Tidy makes it very difficult to render examples that adhere to theming. For example, this line makes it impossible to set the preview theme, even if using the preamble argument in parse-module.

The problematic line is now located here.

Thanks for your attention @TimeTravelPenguin !
I'd like to add that customizing the example output is now a little easier than before! The show-example function has a layout argument that expects a function that it will pass the code raw element and the rendered preview to.
If you don't want to write a custom style file, you can use it like this:
#import "@preview/tidy:0.4.0"

#let my-show-example = tidy.show-example.show-example.with(
  layout: (code, preview) => grid(
    columns: 2,
    align: horizon,
    code,
    preview
  )
)

#let docs = tidy.parse-module(src)
#tidy.show-module(
  docs,
  style: dictionary(tidy.styles.default) + (show-example: my-show-example)
)

Still not perfect but better than before!


Download Link 404s

Hi xtreme-up team! Thanks for the great work. I'm excited to evaluate on some of the newly released datasets in the benchmark!
However, the dataset download link in the repo (https://storage.googleapis.com/xtreme-up/v1.0/xtreme-up.jsonl.tgz) currently leads to a dead end.
<Error>
NoSuchKey
<Message>The specified key does not exist.</Message>
No such object: xtreme-up/v1.0/xtreme-up.jsonl.tgz
</Error>

Let me know if I can provide any other details to help diagnose the issue.
Cheers,
Will Held

Okay, I botched the path when I bumped the version to v1.1. Please find the current version at https://storage.googleapis.com/xtreme-up/xtreme-up-v1.1.jsonl.tgz.
This will be reflected on the README.md shortly.

Thanks!


Duplicate case value in cpr/error.cpp causes failure to build

Error during make process. Specifically, duplicate case of CURLE_SSL_CACERT and CURLE_PEER_FAILED_VERIFICATION.
/home/alex/cpp-opencage-geocoder/cpr/cpr/error.cpp:41:9: error: duplicate case value
         case CURLE_SSL_CACERT:
         ^~~~
/home/alex/cpp-opencage-geocoder/cpr/cpr/error.cpp:25:9: note: previously used here
         case CURLE_PEER_FAILED_VERIFICATION:
         ^~~~

Resolution found on the cpr issues page works for me link.
(Apologies if poor formatting, first ever issue post).

Thanks for raising this, and good job on the first issue post.
If you're interested in getting your first PR, feel free to submit a PR with that fix and I'll help guide it through. Otherwise I'll fix this in the next couple of days.

I've pulled in changes from the upstream cpr repo and updated this project's cpr subtree accordingly.
@A-M-Simmons could you check that master now builds fine? I wasn't able to reproduce the original issue, which I guess is platform-dependent.


Use the new linked-hash-map entry API

:umbrella: The latest upstream changes (presumably #307) made this pull request unmergeable. Please resolve the merge conflicts.


Feature Request: Ability to run command on node using api

Hi all,
We currently have ~130 vms that configured using selenium grid extras and it's working great.
From time to time we need to run some sh/bat commands on those nodes (internal stuff) and it will be great Selenium Grid Extras will support running commands using the api,
for example as we have here: http://selenium-node.xyz:3000/api
just lets say execute post with command line as "xcopy c:\a.txt c:\b.txt" or in linux "touch /tmp/hello.txt"
This kind of feature will give the grid extras another powerful management aspect

You can implement a servlet to do that and add a servlet section to grid extra config.


How is CTR calculated from logged data?

Can you please give more details about how is the CTR calculated from the data that is logged?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 3d15016b-314c-5757-2ea0-76bd04cf87c6
Version Independent ID: bc6c02ee-9770-7af5-e496-34e5ef8d0865
Content: implement search traffic analytics - Azure Search
Content Source: articles/search/search-traffic-analytics.md
Service: search
GitHub Login: @HeidiSteen
Microsoft Alias: heidist

@aasis21 Thank you for your feedback! We will review and provide an update as appropriate.

@aasis21, Currently, we do not do much with CTR since we do not track what a user does after we return the search results.  Generally, this needs to be done on the client side and then leverage something like App Insights to store this data.

@aasis21, We will now proceed to close this thread. If there are any further questions regarding this matter, please tag me in your reply we will gladly continue the discussion.

Sorry for late reply. The documentation states that "Click through Rate (CTR): ratio of users who click on a specific document to the number of total searches " and When to use the power BI desktop to see the CTR by scoring profile. So, It does some averaging and stuff, also the CTR goes above 100. Basically I didn't get the definition given in documentation and how the CTR is calculated in PowerBI report. @AjayKumar-MSFT

aasis21, Thanks for following-up on this with additional details, I'm checking on this further and will get back to you soon.

@HeidiSteen, Requesting your comments.

@AjayKumar-MSFT @aasis21  Apologies for the delayed response.  The report in question is currently lacking an owner.  I don't know if this helps, but you should be able to replace the CTR calculation with one that works better with your data.  This link looks promising:  https://community.powerbi.com/t5/Desktop/Calculating-CTR-and-display-in-the-right-format-in-Power-PI/td-p/254225


[3.x] Issue #4719 - Helidon DBClient does not trigger an Exception when no sane DB connection can be obtained

Signed-off-by: Tomas Kraus<EMAIL_ADDRESS>Fix for 3.x branch.

/trigger


cli tool to find available ports

Low priority, it will take a while before finding a open port will be hard.

I wrote a basic shell script to get all the used ports:
netstat -ln[u/t] --inet[6] | perl -ne 'print "$1\n" if /((?<=:)[0-9]+)/gm'

and puts it into a form that is easier for a cli app like you describe to parse, something like this: (pseudocode)
ports = [array with every number 1024 - 65535]
proto = true if user wants tcp and false if udp
ipv = true if user wants ipv6 and false if ipv4
usedPorts = result of `netstat -ln${proto ? t | u} --inet${ipv ? '6' : ''} | perl -ne 'print "$1\n" if /((?<=:)[0-9]+)/gm'`
for port in usedPorts
  remove that port from ports
return random item from array ports


Alamofire 5.2.0 Error:Server trust evaluation failed due to reason: No certificates were found or provided for evaluation.

Alamofire 5.2.0 Openssl self-signed certificate
A requirement for HTTPS one-way authentication:
I bind the domain name to PinnedCertificatesTrustEvaluator().
bundle下有一个.cer文件.
Code:

跟踪代码发现:

certificateData Like is empty.
po:

You can see that you've got a.cer file under path
Error:
Server trust evaluation failed due to reason: No certificates were found or provided for evaluation.

What was the issue?