[BUG] Delete only site leads to 'Site matching query does not exist.'

I followed the documentation https://docs.django-cms.org/en/latest/introduction/01-install.html to install django-cms manually.
I added a superuser and was able to login.
When going to /admin/sites and deleting example.com I run into this error:
Request Method: | GET
-- | --
http://localhost:8000/admin/sites/site/
3.2
DoesNotExist
Site matching query does not exist.
/usr/local/lib/python3.9/site-packages/django/db/models/query.py, line 435, in get
/usr/local/bin/python
3.9.4
['/app/app',  '/usr/local/lib/python39.zip',  '/usr/local/lib/python3.9',  '/usr/local/lib/python3.9/lib-dynload',  '/usr/local/lib/python3.9/site-packages']
Thu, 15 Apr 2021 18:41:52 +0000

from which it seems impossible to recover without manual database manipulation.
I would expect an empty table with the option to create a site.

Sorry, I actually did not add the label priority high.

Hi, @soerendip thanks for the bug report. I tested it and have same effect. Need some discuss

Hi @soerendip, while this does cause an error, it's not an error from django-cms.
This is an error from django itself, and I would expect an error here.
To confirm, here's the stacktrace;
  File "/Users/mwalker/Sites/consoles/.venv/lib/python3.8/site-packages/django/contrib/sites/shortcuts.py", line 13, in get_current_site
    return Site.objects.get_current(request)
  File "/Users/mwalker/Sites/consoles/.venv/lib/python3.8/site-packages/django/contrib/sites/models.py", line 58, in get_current
    return self._get_site_by_id(site_id)
  File "/Users/mwalker/Sites/consoles/.venv/lib/python3.8/site-packages/django/contrib/sites/models.py", line 30, in _get_site_by_id
    site = self.get(pk=site_id)
  File "/Users/mwalker/Sites/consoles/.venv/lib/python3.8/site-packages/django/db/models/manager.py", line 85, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/Users/mwalker/Sites/consoles/.venv/lib/python3.8/site-packages/django/db/models/query.py", line 435, in get
    raise self.model.DoesNotExist(
django.contrib.sites.models.Site.DoesNotExist: Site matching query does not exist.

Django generally operates using Site objects, you'll notice django.contrib.sites in your INSTALLED_APPS if you're using the framework. Django creates the first Site object using the domain example.com after migrations are ran.
Generally, if you check your settings file, you'll see SITE_ID = 1 (unless you're running a multisite project). So what you've done is delete the site matching the ID in your settings file, so django can't load it's own Site object anymore.

I thought that is what I did. Though, if the interface becomes unusable if a user by accident or by curiosity deleted the object, maybe it would be wise to hide that functionality or at least add a comment. Or better would be to catch it.

From: Mark Walker @.>
Sent: Thursday, April 15, 2021 6:26:34 PM
To: django-cms/django-cms @.>
Cc: Sören @.>; Mention @.>
Subject: Re: [django-cms/django-cms] [BUG] Delete only site leads to 'Site matching query does not exist.' (#7023)
Hi @soerendiphttps://github.com/soerendip, while this does cause an error, it's not an error from django-cms.
This is an error from django itself, and I would expect an error here.
To confirm, here's the stacktrace;
File "/Users/mwalker/Sites/consoles/.venv/lib/python3.8/site-packages/django/contrib/sites/shortcuts.py", line 13, in get_current_site
return Site.objects.get_current(request)
File "/Users/mwalker/Sites/consoles/.venv/lib/python3.8/site-packages/django/contrib/sites/models.py", line 58, in get_current
return self._get_site_by_id(site_id)
File "/Users/mwalker/Sites/consoles/.venv/lib/python3.8/site-packages/django/contrib/sites/models.py", line 30, in _get_site_by_id
site = self.get(pk=site_id)
File "/Users/mwalker/Sites/consoles/.venv/lib/python3.8/site-packages/django/db/models/manager.py", line 85, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
File "/Users/mwalker/Sites/consoles/.venv/lib/python3.8/site-packages/django/db/models/query.py", line 435, in get
raise self.model.DoesNotExist(
django.contrib.sites.models.Site.DoesNotExist: Site matching query does not exist.
Django generally operates using Site objects, you'll notice django.contrib.sites in your INSTALLED_APPS if you're using the framework. Django creates the first Site object using the domain example.com after migrations are ran.
Generally, if you check your settings file, you'll see SITE_ID = 1 (unless you're running a multisite project). So what you've done is delete the site matching the ID in your settings file, so django can't load it's own Site object anymore.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/django-cms/django-cms/issues/7023#issuecomment-820822898, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAZ4A7WV6MW4YW44ACJX7X3TI573VANCNFSM43ABTGTA.

Yeah, I don't disagree @soerendip, but any change would be for django itself, not django-cms.
There's nothing we can do about that in this app. I'm going to close this as it's a third party issue.

You cannot remove it from the admin pannel, for example, and set the site name with an environment variable, or a variable in settings?

Yeah, I don't disagree @soerendip, but any change would be for django itself, not django-cms.
There's nothing we can do about that in this app. I'm going to close this as it's a third party issue.

Actually we can restrict deletion of the last site

You cannot remove it from the admin pannel, for example, and set the site name with an environment variable, or a variable in settings?

Remove the sites section?
Yeah you could do that. You'd just have to unregister the model from admin. Basically the opposite of how you get your models in there.

Yeah, I don't disagree @soerendip, but any change would be for django itself, not django-cms.
There's nothing we can do about that in this app. I'm going to close this as it's a third party issue.

Actually we can restrict deletion of the last site

While that may be true, it would be something better done at a project level if the developers wanted to do that.


Using version test in Ansible to check if version is below 1.0.0 in…

…stead of replace and int comparison

Hi. Good work on this role, not sure if you are enhancing to merge back or to use yourself but I've been using it very recently and thought this would be a good improvement for it
Reference:
https://docs.ansible.com/ansible/latest/user_guide/playbooks_tests.html#version-comparison


Feature: Preserve file extension when duplicating files

VSCode Version: 1.20.1
OS Version: macOS High Sierra 10.13.2
Settings: https://gist.github.com/Epigene/a65bbf5e88d847b5e0622d42019fb678
Wrongly opened issue in VSC repo: https://github.com/Microsoft/vscode/issues/44254
Needed behavior:
When duplicating a file it would be nice to have it append the "-copy" just before the file extension.
Current behavior:
Duplication overrides file extension with "-copy".
Discussion:
As can be seen in the example image below, I choose to duplicate a file named "calendar.apib" and the default duplicate file-name is "calendar-copy" where "calendar-copy.apib" would be much more useful.

Hello, @Epigene,
Sorry for delay. By default, the extension remains unchanged.

Original: index.d.ts
Popup with input: index.d (.d, because we don't know how many points is an extension)
Saved file: index.d.ts

But if the extension does not match the original one, it will be changed:

Original: index.d.ts
Popup with input: index.d → index.d.ts2
Saved file: index.d.ts2

Hi, thanks for the extension! It looks like duplication doesn't work as expected if the filename contains more than one period (which is not an uncommon idiom in the *nix world).
How to reproduce:

Create foo.dev.json.
Hit Duplicate file or directory – the new filename is displayed minus the .json suffix.
Enter foo.test
The duplicated filename is now foo.test.

Perhaps this is the intended behaviour but it seems counterintuitive.

IMO a better option (if possible with Code APIs) would be to preserve extension(s) while selecting the filename part in the prompt, i. e. for file foo.d.ts a prompt [foo].d.ts ([] denoting a selection) is presented, from where user can:

rename file while keeping extension by just typing new name,
add a suffix before the extension (→ and type the suffix),
change the extension (End Ctrl+Backspace and type new extension),
finally, rename file completely (Ctrl+A and type new filename).

I can work out a PR if this kind of functionality is okay with you.

Try this one.
https://marketplace.visualstudio.com/items?itemName=axelrindle.duplicate-file


Update the CMakelists.txt - [merged]

In GitLab by @MaximilienNaveau on Aug 1, 2019, 13:17
Merges mpi/crocoddyl-topic/fix_numpy_search -> cpp_devel
Add the search of Numpy in the cmakelists.txt upon building of the python bindings.
Otherwize numpy.h is not found during the "make" step.

In GitLab by @cmastalli on Aug 1, 2019, 14:12
I am not sure if this is needed because I think eigenpy is including this. Did you install eigenpy?

In GitLab by @MaximilienNaveau on Aug 1, 2019, 15:25
If you installed eigenpy from source maybe, but from binaries it does not (yet).
I think it is safer to add it now and remove it later than to force people to install eigenpy from source.

In GitLab by @cmastalli on Aug 1, 2019, 15:27
I think I tried both cases. In any case, I think this doesn't hurt.

In GitLab by @cmastalli on Aug 1, 2019, 15:27
merged

In GitLab by @cmastalli on Aug 1, 2019, 15:27
mentioned in commit e3e5d68d584bf92fcf98d2e22a864cf04768c0c1

In GitLab by @MaximilienNaveau on Aug 1, 2019, 15:33
thanks


fix wikidata 403 bug

Added

HTTP connector backoff for 10 retries on 403 from server
rki/mex user agent is sent with query requests via wikidata connector

Changes

update wikidata search organization request query, with optional language parameter
wikidata query search can be enhanced by specifying the language.
EN is the default language.

@mr-kamran-ali can retrying really fix a 403 error? 🤔

@mr-kamran-ali can retrying really fix a 403 error? 🤔

It probably doesn't, issue was fixed by the User-Agent headers, I took this step to give 10 backoff attempts because it was still working on some requests and failing on others. Also with this extractor will fail bit more elegantly.


enforce lowercase class name in .stylelintrc

"selector-class-pattern": "^[a-z]+([a-z0-9]?|[a-z0-9\\-]*[a-z0-9])$"
I think this could be a useful addition, to prevent the use of a class name against the projects guidelines - mixed case or using underscores etc.
REF: http://codeguide.co/#css-classes

I think ^[a-z][a-z0-9\\-]*[a-z0-9]$ should do the job?

@coliff, can you add an exception to _syntax.scss?

I believe this regex needs more work, so don't merge this yet.

OK, it seems the regex was fine after all: https://regex101.com/r/m7ynnv/3


Make NPM package publishing sane with Lerna

Our NPM publishing is hairy and easy to get wrong. That, and it'd be nice to get CHANGELOG updates for free via lerna-changelog.

Merging into https://github.com/facebookincubator/fbt/issues/76


Images not working on Microsoft Edge

None of the images served by Refile are working on Microsoft Edge. They are coming across as application/octet.

This can be fixed if you add metadata informations into you database as described on github page.
I've checked this on a simple refile example app with refile 0.6.2 and MS Edge 25.10586.0.0 / EdgeHTML 13.10586.
I don't know if it must considered as a refile issue or edge issue.

Thanks! I must have missed this. Let me try that out and see if it works. As to whether it's a refile issue or edge....not sure. Edge is the only one where it's happening, but on the other hand it's not like all images are broken on Edge, so I guess a little bit of both?
cc/ @seque1990

@maxcodes Running into the same issue but may be due to edge's inability to distinguish image/jpg which isn't a valid mime type from the valid image/jpeg

Interesting. Where is the relevant code? @ChanChar Is it something we can fix in Refile?

@maxcodes May just be an edge issue, still investigating it on my end but the only broken images on our end are from uploading 'image/jpg'. A suggested, but untested in production, workaround was to check for image/jpg before uploading/downloading and labeling it as image/jpeg since they're the same format.

@maxcodes I suspect Refile ( or Sinatra / Rack ) to only use the extension of a filename to determine the file content-type.
I've done a test with extensions .jpeg and .jpg  both have been served as image/jpeg.

@maxcodes we had the same issue in edge until we added the meta columns, and then it's worked just fine.

Closing due to the lack of activity.
I believe this is an Edge issue but I don't have access to it.
Feel free to reopen or create a new issue if you find it again.


Show only palettte?

Is there an option to show only palette-based colour picker?

Do you mean you only want them to be able to select from a preselected group of colors and not be able to slide around the hue and saturation/value?  There is no option for that right now, but I see where that could be useful in some situations.  If so, this could be added in as an option showColorpicker: false.

Thanks. Yes, that's what I needed. I managed to achieve that by specifying theme: option and the following css:
.myTheme .sp-fill {
    display: none;
}
.myTheme .sp-val {
    display: none;
}


How to load your pre-trained model and train on my dataset?

Hi,
thank you so much for your great work. I'm doing a project with multi-label classification so I wonder how I can apply your pretrained model for image feature extraction? what I need is to extract feature of an image. Could you please give me some hints?
Best regards,
Hui

We use famous backbones like ResNet and Swin-Transformer for feature extracting as well in our methods. What we proposed is a framework for multilabel classification other than a new backbone. Our framework is backbone agnostic, that is, our method could be used for any backbone and introduce improvement consistently.
If you want to extract features only, some traditional backbones like TResNet, ResNet, and Swin-Transformer are suitable enough.   While if you'd like to train a multilabel classifier for your own dataset, you can use our model in your datasets and modify some parameters, like the number of labels, correspondingly.


Intent open account and check mail

Please review it.

Thx for quick answer ...
Did you mean https://github.com/k9mail/k-9/blob/master/src/com/fsck/k9/remotecontrol/K9RemoteControl.java ? I found a comment: "not yet implemented".
Or is there just another way to trigger a sync?

Oh. That's disappointing :/
On Mon, Nov 07, 2011 at 07:10:55PM -0800, mad654 wrote:

Thx for quick answer ...
Did you mean https://github.com/k9mail/k-9/blob/master/src/com/fsck/k9/remotecontrol/K9RemoteControl.java ? I found a comment: "not yet implemented".
Or is there just another way to trigger a sync?

Reply to this email directly or view it on GitHub:
https://github.com/k9mail/k-9/pull/33#issuecomment-2663334

--

Oh realy it is. Is the next release already planned? Until when will I have to commit a fix if it should be included?


[MODPATBLK-63] Update RMB to v31.1.5, update Vert.x to v3.9.4

Resolves MODPATBLK-63

INFO: lint-raml: Using lint-raml version: 1.4.4
INFO: lint-raml: Investigating and determining configuration: mod-patron-blocks/ramls
INFO: lint-raml-schema: Assessing schema files (https://dev.folio.org/guides/describe-schema/):
INFO: lint-raml-schema: Found 21 JSON schema files under directory './ramls'
ERROR: lint-raml-schema: loan-policy.json: Missing top-level "description".
ERROR: lint-raml-schema: loan-policy.json: Missing "description" for: description, holds, loanable, loansPolicy, name, pages, recalls, renewalsPolicy, requestManagement
ERROR: lint-raml-schema: period.json: Missing top-level "description".
INFO: lint-raml: Not assessing RAML/Schema or examples against schema, as per option '--json-only'.
ERROR: lint-raml: There were processing errors. See list above.

Note: When those errors are fixed, then the INFO messages will be gone too. See Jenkins "Artifacts" tab.


Do not call check_last_version if no .git directory found

Describe the bug
When running a script in a non-git folder, you see:
fatal: not a git repository (or any parent up to mount point /)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).

To Reproduce
Steps to reproduce the behavior:

Run bashew script in your root folder
Execute the new script with -h

Expected behavior
No git error should occur
Desktop (please complete the following information):

Shell type : bash - version 5.0.17(1)-release
OS Version : Distributor ID:	Ubuntu (Linux) Release:	20.10 on x86_64
Program: bashew 1.15.0
Updated: 2021-02-28 05:39

added a check if the script is in a git repo, before checking for updates


WIP CI Zulu J8

Depends on #2142 #2144

Using this image would let cattle move up to Java 8 and stay on trusty for a bit longer (delaying the db headaches). The tradeoff though is picking up an extra 130MB on disk in the rancher/server image.
I think the only way to avoid bloat in rancher/server is to migrate to Alpine. Which means db migration and other testing. I kinda like this until all the other work gets sorted out.

@dx9 Any chance you can do a corresponding PR to github.com/rancher/rancher/server?  Once we release Rancher 1.2 I can merge this.  As we are heavy into the test cycle of Rancher 1.2 we don't really want to upset anything at the moment.

@ibuildthecloud yep https://github.com/rancher/rancher/pull/6627

@dx9 I finally merged https://github.com/rancher/cattle/pull/2142 so I'd like to merge this one next.  For maven installation though I was wondering if we could just use https://github.com/takari/maven-wrapper.  If we add the mvnw artifacts then the mvnw will handle installing maven.  Which is cool for local dev too.

@ibuildthecloud Sure. I'll tidy this up and put the maven changes into a separate PR.


Deploy Azure AppConfiguration Settings through ARM template

I have the ARM template generated from the Azure portal to deploy the Azure AppConfiguration resource but is there a way we can deploy the settings as well through the ARM template?

This work is in the plan. I am closing this issue as it is a dup of #268.


When PLC is not reachable an inappropriate exception is thrown

Disconnecting the PLC from power supply after plc.Open() is invoked produces an exception specified below:
The variable'DB111.DBX1.2' could not be read. Please check the syntax and try again.

I think this is really misleading, because a syntax correction definitely won't help. An exception with message similar to "PLC can not be reached" should be thrown in my opinion.

An exception with message similar to "PLC can not be reached" should be thrown in my opinion.
Yes, you're right, provided that your application polls continuosly such lib properties like 'IsConnected'.
And based on this event your app disable a var polling.

Mallowan

Right, similar to issue #112 where WrongVarFormat is returned.
Methods Read(string) and Write(string, object) should be affected. Needs better exception handling (Exception forwarding).

Addressed in #156.


[BUG]: Court Staff Not able to see Summons Document

Describe the bug
Court Staff unable to see summons document
To Reproduce
https://github.com/user-attachments/assets/4615b79e-f22c-4c1b-8ab5-fc9586551c30

Hi @devarajd94  This issue is now fixed.
You'll be able see the documents for new summons.
Jam video for reference.
https://jam.dev/c/b29eed89-e25c-4096-9b97-139341ef6ccf
cc :- @Ramu-kandimalla

Hi @Ramu-kandimalla,verified in UAT env.
Court Staff able to see summons document.
https://jam.dev/c/0e71c1b0-0db8-431e-b57f-a2b64fa14825

Hi @devarajd94  This issue is fixed

Verified


Validate uniqueness of gs.DETS

Inspired by @ericdill in NSLS-II/ophyd#138

To state the obvious, gs.DETS is a traitlet so validation is very easy.


Allow customizing the focus element per fragment rule

Describe the problem

Most fragment visits don't require setting focus to the main element on navigation
Especially filter UIs should either not move focus or move it to the updated list which won't be a main element

Describe the proposed solution

I'd suggest adding a focus option to fragment rules that is passed into the new visit.a11y.focus option, if present
Should fragment visits be false here by default? Or better to make people be explicit?

How important is this feature to you?

[ ] Nice to have
[x] Would make the life of screen-reader users a lot easier
[ ] I cannot use swup without it

@hirasso If I'm correct, the current behavior of a11y + fragment plugins is to move the focus to main even for fragment visits. A default value of false would mean not setting focus, so a breaking change.

Ah! Right. What would you prefer?

An optional focus key with a default of null sounds good. The current behaviour seems to me like a good default solution, i.e. focus the main element as configured by the a11y plugin. That's already a better experience for keyboard users than traditional MPAs that return focus to the top of the document on each navigation — here we're already skipping the header and jumping right to the content.
Depending on the markup of your fragments, this could well be enough, assuming the new fragment is a main tag and comes first in the document. For other cases like filter UIs, specifically setting or disabling a new focus option on the fragment rules would be a great way for fine-tuning this.

Great, let's go with a default of undefined then, just like with the rule option for name!


add TypeScript typing definition file.

Allow TypeScript users to import tiny-emitter easily through:
import EventEmitter = require("tiny-emitter")

Cheers!

So I'm unfamiliar with Typescript definitions. Any screenshots are tests you can provide just I feel safer merging?
This is great by the way! Been hoping this would happen soon!

Sure @scottcorgan,
I've made some changes and copied this index.d.ts file my project's node_modules/tiny-emitter/index.d.ts and it worked. Don't know how to programatically test this thing, but here's a screenshot of VisualStudio Code evaluating it successfully:

TypeScript will resolve index.d.ts in the module's root directory (see module resolution), so after merging and publishing it should hopefully be ready to use!

Great stuff! Thanks!
Published as 1.1.0


Thesaurus expansion not working or is not active

The PEP thesaurus is implemented in the alpha preview.  However, currently, as I've been search tuning, it is not active in the current release.

the alpha 2 and 3 versions do have thesaurus expansion, and a checkbox was added to the supporting version of PEP-Easy to allow it to be tested in vivo.


Laravel 11.28.0 Shift

This pull request includes updates for the recent minor version release of Laravel as well as bumps your package dependencies. You may review the full list of changes in the Laravel Release Notes, or highlighted changes and tips in the weekly Shifty Bits newsletter.
Before merging, you need to:

Checkout the shift-ci-v11.28.0 branch
Review all pull request comments for additional changes
Run composer update
Thoroughly test your application (no tests?, no CI?)

:warning: The Shift build servers run PHP 8.3. If you are running a different version of PHP, you should run composer update to ensure you have the proper versions of your dependencies for your environment.


Hide Menu?

Hey, I like how easy Parchment is to setup, but don't want to use the menu. Is there any way to hide the menu entirely?

hi @aaronn. what is your solution to hide the menu entirely? i need this function too

@Sopheak0 I decided to just use a pageviewcontroller.


Fix(uif-button): Issue 268 disabled not watched

Added watch for disabled and new tests and demos for the directive.

attribute disabled was missing
link default action was not prevented
added watch for ng-disabled
added tests for disabled and ng-disabled
added demo for ng-disabled

References #268.
Sorry about the change history. I had troubles with the interactive rebase. Files changed:

buttonDirective.spec.ts
buttonDirective.ts
demoDisabled\index.html
demoDisabled\index.js

Coverage decreased (-0.2%) to 98.453% when pulling f442da1c0548f19a2297ff7ce82edb88865d6cec on andikrueger:issue-268-button-disabled into 11b4fb0e7ca8238d15d97e10531673b9e1019ed5 on ngOfficeUIFabric:dev.


update anno_img: add a parameter merge and merge_width. assert if all the images have the same shape.

Hi Wubin,
Thanks for merging the pull request and adding the use of anno_img to the documentation. I appreciate your feedback and suggestions.
Regarding the image size issue, my approach assumes that all images are exactly the same size. If this is not the case, the current code will indeed encounter errors during the np.hstack or np.vstack operations. To address situations where images may vary in size and aspect ratio, implementing a comprehensive solution becomes complex. Therefore, I decided to use assertions to enforce that images within the same cluster must have identical sizes.
https://github.com/dakomura/PyComplexHeatmap/blob/951ecb9f15d765bdd3f811dd228d8c1e7cb744ac/PyComplexHeatmap/annotations.py#L1149-L1155
As for the merge parameter, while the need to merge images is rare, there are scenarios where it could be beneficial to display a single image for a particular category or cluster. Since the implementation was straightforward, I included this feature.
https://github.com/dakomura/PyComplexHeatmap/blob/951ecb9f15d765bdd3f811dd228d8c1e7cb744ac/PyComplexHeatmap/annotations.py#L1163-L1169
df = pd.DataFrame(['AAAA1'] * 5 + ['BBBBB2'] * 5, columns=['AB'])
df['CD'] = ['C'] * 3 + ['D'] * 3 + ['G'] * 4
df['F'] = np.random.normal(0, 1, 10)
df.index = ['sample' + str(i) for i in range(1, df.shape[0] + 1)]
df_box = pd.DataFrame(np.random.randn(10, 4), columns=['Gene' + str(i) for i in range(1, 5)])
df_box.index = ['sample' + str(i) for i in range(1, df_box.shape[0] + 1)]
df_bar = pd.DataFrame(np.random.uniform(0, 10, (10, 2)), columns=['TMB1', 'TMB2'])
df_bar.index = ['sample' + str(i) for i in range(1, df_box.shape[0] + 1)]
df_scatter = pd.DataFrame(np.random.uniform(0, 10, 10), columns=['Scatter'])
df_scatter.index = ['sample' + str(i) for i in range(1, df_box.shape[0] + 1)]
df_bar1 = pd.DataFrame(np.random.uniform(0, 10, (10, 2)), columns=['T1-A', 'T1-B'])
df_bar1.index = ['sample' + str(i) for i in range(1, df_box.shape[0] + 1)]
df_bar2 = pd.DataFrame(np.random.uniform(0, 10, (10, 2)), columns=['T2-A', 'T2-B'])
df_bar2.index = ['sample' + str(i) for i in range(1, df_box.shape[0] + 1)]
df_bar3 = pd.DataFrame(np.random.uniform(0, 10, (10, 2)), columns=['T3-A', 'T3-B'])
df_bar3.index = ['sample' + str(i) for i in range(1, df_box.shape[0] + 1)]
df_bar3.iloc[5,0]=np.nan
df_bar4 = pd.DataFrame(np.random.uniform(0, 10, (10, 1)), columns=['T4'])
df_bar4.index = ['sample' + str(i) for i in range(1, df_box.shape[0] + 1)]
df_bar4.iloc[7,0]=np.nan
df_img = pd.DataFrame([f"{i:02}.jpg" for i in range(1,11)], columns=['path'])
df_img.index = ['sample' + str(i) for i in range(1, df_box.shape[0] + 1)]
df_heatmap = pd.DataFrame(np.random.randn(30, 10), columns=['sample' + str(i) for i in range(1, 11)])
df_heatmap.index = ["Fea" + str(i) for i in range(1, df_heatmap.shape[0] + 1)]
df_heatmap.iloc[1, 2] = np.nan

plt.figure(figsize=(12, 16))

#df_img_col = pd.DataFrame([f"{i:02}.jpg" for i in range(10)], columns=['Image'])
df_img_col = pd.DataFrame([f"01.jpg" for i in range(10)], columns=['Image'])
df_img_col.index = ['sample' + str(i) for i in range(1, 11)]
#df_img_row = pd.DataFrame([f"{i:02}.jpg" for i in range(30)], columns=['Image'])
df_img_row = pd.DataFrame([f"02.jpg" for i in range(30)], columns=['Image'])
df_img_row.index = ["Fea" + str(i) for i in range(1, df_heatmap.shape[0] + 1)]

col_ha = HeatmapAnnotation(Image=anno_img(df_img_col,  height=16, border_width=10, merge=True, merge_width=1))#, axis=1)
row_ha = HeatmapAnnotation(Image=anno_img(df_img_row, border_width=13, merge=True, merge_width=2, height=20), axis=0)

cm = ClusterMapPlotter(data=df_heatmap, top_annotation=col_ha, left_annotation=row_ha, col_split=2, row_split=3, col_split_gap=10,
                     row_split_gap=10,label='values',row_dendrogram=True,show_rownames=True,show_colnames=True,
                     tree_kws={'row_cmap': 'Dark2'},cmap='Spectral_r',
                       legend_gap=5,legend_hpad=2,legend_vpad=5)
plt.show()

I will be sure to send you the Jupyter notebook when it is finished and contains a better example of anno_img used with real biological data. This example will be suitable for inclusion in the documentation website.
Best regards,
Daisuke

Awesome, thank you very much. @dakomura
I really appreciated it.

Hi @dakomura ,
I made some big changes to the anno_img:
(1). It supports remote URL as input.
(2). It supports missing values now.
(3). Fix bugs about the order and rotation of rows and columns image annotations.
Please see here for some example
Could you please install the latest version from github and help me test those new features?

Hi @DingWB ,
I apologize for not making any progress on the examples. I've been extremely busy lately and haven't had the opportunity to work on them.
Thank you so much for implementing those new features in anno_img! I sincerely appreciate your effort.
I'll make sure to install the latest version from GitHub and thoroughly test it next week.
If I have any feedback or suggestions, I'll be sure to let you know.


Adding Custom Url Endpoints and Headers

Related Issues/PRs

Adding custom base url and headers to access additional OpenAI endpoint deployments
What changes are proposed in this pull request?

Adding trait HasCustomHeader that checks if a customHeader was assigned.
Adding customHeader to the addHeaders function to iterate through the Map of headers provided and assign them to the request.
Added customRootUrl - if provided will be used instead of the prepareUrlRoot() function

How is this patch tested?

[X] I have written tests (not required for typo or doc fix) and confirmed the proposed feature/bug-fix/change works.

Does this PR change any dependencies?

[ ] No. You can skip this section.
[X] Yes. Make sure the dependencies are resolved correctly, and list changes here.

deploymentName parameter was changed to not required

Does this PR add a new feature? If so, have you added samples on website?

[X] No. You can skip this section.
[ ] Yes. Make sure you have added samples following below steps.

Find the corresponding markdown file for your new feature in website/docs/documentation folder.
Make sure you choose the correct class estimators/transformers and namespace.
Follow the pattern in markdown file and add another section for your new API, including pyspark, scala (and .NET potentially) samples.
Make sure the DocTable points to correct API link.
Navigate to website folder, and run yarn run start to make sure the website renders correctly.
Don't forget to add <!--pytest-codeblocks:cont--> before each python code blocks to enable auto-tests for python samples.
Make sure the WebsiteSamplesTests job pass in the pipeline.

/azp run

/azp run

Codecov Report
Attention: Patch coverage is 0% with 22 lines in your changes missing coverage. Please review.

Project coverage is 13.54%. Comparing base (a8820fe) to head (e455006).

Files
Patch %
Lines

...ure/synapse/ml/services/CognitiveServiceBase.scala
0.00%
21 Missing :warning:

...soft/azure/synapse/ml/services/openai/OpenAI.scala
0.00%
1 Missing :warning:

Additional details and impacted files
@@             Coverage Diff             @@
##           master    #2232       +/-   ##
===========================================
- Coverage   86.23%   13.54%   -72.69%
===========================================
  Files         327      327
  Lines       16715    16731       +16
  Branches     1495     1508       +13
===========================================
- Hits        14414     2267    -12147
- Misses       2301    14464    +12163

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

/azp run

/azp run

/azp run


Move setup step to init in Abiotic and AbioticSimple models

Description
It was done already for the Hydrology model but I forgot to do the same for the other two. The main changes have more to do with fixing the tests than the actual change in the code.
Fixes # N/A
Type of change

[x] New feature (non-breaking change which adds functionality)
[ ] Optimization (back-end change that speeds up the code)
[ ] Bug fix (non-breaking change which fixes an issue)

Key checklist

[x] Make sure you've run the pre-commit checks: $ pre-commit run -a
[x] All tests pass: $ poetry run pytest

Further checks

[x] Code is commented, particularly in hard-to-understand areas
[x] Tests added that prove fix is effective or that feature works
[ ] Relevant documentation reviewed and updated

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 95.06%. Comparing base (864a5c4) to head (05a6952).

Additional details and impacted files
@@             Coverage Diff             @@
##           develop     #524      +/-   ##
===========================================
+ Coverage    94.93%   95.06%   +0.12%
===========================================
  Files           73       73
  Lines         4048     4052       +4
===========================================
+ Hits          3843     3852       +9
+ Misses         205      200       -5

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

@TaranRallings - The animal model also uses  a setup method, which is currently also called from within __init__, so runs twice.


Try Cachix Deploy

https://blog.cachix.org/posts/2022-07-29-cachix-deploy-public-beta/

services.cachix-agent.enable = true;


allowing for external containers to be referenced in the link configuration

can I review this before #1099 ?

Dropping this idea of implementing external container links for a seperate external kind.


[CI] Add a script to generate releases and precompiled archives

@cvanaret
I reused the CI script that I developed for GALAHAD, CUTEst, and SIFDecode.
The script automatically generates an archive here when you add a tag v* to a commit on the branch main of Uno.
For example, you just need these commands:
git tag v1.2.0
git push --tags

to trigger the CI script (--> example <--), and it automatically generates the release with precompiled artifacts and dependencies.
Here are some examples:

https://github.com/ralna/GALAHAD/releases
https://github.com/ralna/CUTEst/releases
https://github.com/ralna/SIFDecode/releases

Thank you @amontoison!


Allow users to add additional options to the select widget directly

As per the react-select module, it would be great if Netlify CMS could allow users to add additional options to the select widget directly from itself.

The difficulty here is whether this added option can be plugged into the options list in the config.yml file.

@erezrokah @erquhart  Interested in tackling this issue. Does this ticket cover adding new options to the relation widget as well, or should a new ticket be created for that?
My first thoughts for an implementation would be:
Display a modal to add a new option to the select. The modal would contain the relations fields for the user to fill out. The modal would have Publish and Cancel buttons (you won't be able to save a draft of the relation you want to add). Once the relation you want to add is Published, it's added to the relation widget. Side note: If this option is taken, might need to resolve #1571 first before working on this, as I assume only collections that do not have editorial workflow enabled will be supported here.
Motivation/Use Case
The simplest use case where I see this being extremely useful is with tags, where you may have a tags folder collection with just a single title field.
For example, imagine you have tags and posts collections, and tags is implemented as a folder collection in config.yml. I want to be able to create new tags from a relation widget inside a post entry without having to leave the post entry. From a UX perspective, it's quite tiresome to have to click back to the dashboard, click on the tags collection, create new tag, add the tag, save, set ready status, publish, go back to dashboard, find original post entry, then finally be able to add the tag to your entry. Would be very nice to be able to just create it inline on the post entry.
Thoughts?

Thanks @kwyoung11, I think this would only cover the select widget for now.
Instead of changing the config.yml (which will probably trigger an unwanted CI build) the widget could support an additional allow_add flag and then save the additional options under the entry data, e.g.
options:
 selected: string | string[]
 additional: string[]

That would mean the widget would need to support persisting an object instead of the current list or string.
WDYT?

Note: this ties into #192, which calls for making related in changes in one entry from another entry - in this case adding a tag to a centralized tags data file from an entry that will reference the tag. That part of the request is duplicate.
The core ability to add arbitrary values to a select, without persisting that value to any centralized, reusable set of values, should be really simple. I would even suggest that the selected/additional dichotomy may not be necessary - allow_add could just allow any value to be added.
Also wondering if we can serve the tags use case with the relation widget as it currently works, with added logic for reducing the related values list to unique values only. That might serve the majority of tags requirements quite well.

Any plans for adding this? or a workaround to implement this to my project?

Hi @AbdallahAbis, the best way to move this forward is through a contribution.

@erezrokah any guidelines for someone who wants isn't a contributor and wants to contribute? Any help.

Hi @AbdallahAbis you can see our guide here.
The select widget code is here.
Please let me know if you have any other questions.

@erezrokah Does cloning the widget's sub repo will allow me to test locally?

@erezrokah Could you tell me where should those newly created options be stored?

@erezrokah Could you tell me where should those newly created options be stored?

Hi @AbdallahAbis, well I'm not sure. See https://github.com/netlify/netlify-cms/issues/3821#issuecomment-647410593 and https://github.com/netlify/netlify-cms/issues/3821#issuecomment-647547887.
The additional options can be saved as a part of the entry. That however means they will only be available per entry and not globally.

@erezrokah Why can't we extend the widget options and pass it the additional options by updating config.yaml?
What do you mean by unwanted CI build

@erezrokah Why can't we extend the widget options and pass it the additional options by updating config.yaml?

Hi @AbdallahAbis, updating the config.yml means committing it back to GitHub (either directly to the default branch or via a PR). That means a build will be triggered due to the code change and also the CMS would need to wait for the build to complete to reload the config.yml file.

@erezrokah So I've thought about it and I have an idea but I don't know if it will work or not as I haven't yet tried it.

Let's say we have a collection of articles
We have a field of tags on that article which is a list widget and can be either single or multiple.

We can change the list widget to be a Creatable component when the allow_add flag is true.
The new values added will be stored in the data entry (e.g. article.tags), the issue here is that they will not be available globally to other articles.
We can mimic the way Relation widget queries for fields, and we can query for the  tags field and filter those tags to be unique then show them in the tags options in different articles.
My questions is:

can we extend the list options when the user clicks on the field to choose an option so they can see all the queried and filtered tags? Just the way Relation widget works.

WDYT?

Let's say we have a collection of articles
We have a field of tags on that article which is a list widget and can be either single or multiple.

We can change the list widget to be a Creatable component when the allow_add flag is true.

Hi @AbdallahAbis, do you mean a select widget here?
I'm not sure we need the relation widget. Having an allow_add field will allow users to add any value which we will save on the entry. Next time the entry is loaded any custom values saved with the entry will be available for selection.
Please let me know if that works

@erezrokah

do you mean a select widget here?

Yea, sorry about that, I wrote it at about 3AM 🥴

Next time the entry is loaded any custom values saved with the entry will be available for selection.

Will those added values be available to all entries?
Adding tag1 to article one, will I be able to see that tag1 on article 2?

Will those added values be available to all entries?

No they won't.
I think I'm start to understand you suggestion. You would like to query all entries (in the same collection/all collections) and extract all values and merge them?
Please note that this can have a performance hit when dealing with many entries.

@erezrokah

You would like to query all entries (in the same collection/all collections) and extract all values and merge them?

Yes, exactly.
But I don't think adding an option to an entry and having it only show at that entry is that helpful. For example a tag should be available on all entries ao the content editor can easily select it on a different entry without having to remember how did they exactly write it the first time.

Hi @AbdallahAbis, please consider you can have multiple select widgets in multiples collections and also nested ones in lists/objects.
This means will have to read all entries from all collections each time we load a single entry. I think the performance hit will be too big, but I don't see any harm in testing it out.

I implemented my own custom widget and I am going to add it here in case it is helpful to anybody (the widget is still in early development). My approach is different from the one proposed by @AbdallahAbis. The widget uses the page building tool to populate the select entries. An example can be seen in this demo in the Jekyll Tags (HTML) entry where the built page itself is used to populate the select.

Hey,
Don't know is anybody already did it or commented it, but you could use relations widget type for it.
you need 2 collections

main collection with your posts
second collection with tags

Then use the relation widget to query the tags collection inside the posts collection.

The full sample:
collections:
  - name: "art"
    label: "Art"
    folder: "public/docs/portfolio"
    create: true
    slug: "{{year}}-{{month}}-{{day}}_{{slug}}"
    fields:
      - label: "Tags3"
        name: "tag3"
        widget: "relation"
        collection: "tags"
        search_fields: [ "name" ]
        value_field: "name"
        display_fields: [ "name" ]
        multiple: true
  - name: "tags"
    label: "Tag"
    folder: "public/docs/tags"
    create: true
    slug: "{{slug}}"
    fields:
      - { label: "Name", name: "name", widget: "string" }

The downside to this approach is that you have to do 2/3/4 commits depending on how many new tags you want to add to a new post. On the positive side - you don't need to integrate any plugins and get to maintain all of your tags in the CMS itself.
HAVING SAID THAT, I would still prefer if the CMS allowed us to:

Add new tags to select as the author of the post suggested
Load a list of tags from external collection/file i.e. source:  'selectValues.json' as an example

This would give me, personally, a better control over the select. For instance it would then be possible to generate a fresh selectValues.json file in CI pipeline . This would be a nice addition to other widgets as well if people wanted to move some of the dependencies out into separate files for whatever reason.


Cannot read categories labels on charts when using a large period

When using a really large period ("Tout le temps" in french), the dates label are missing some vertical space and we cannot read the categories labels anymore :

Wow, even I don't have that much data in my instance! Did you just use Kresus, or did you hack to import external data from another source?
Anyways, thanks for the report. That calls either for a period selector + sliding window combo, or just a quick dive into d3.js to configure the height between the labels and the legend.

I just used kresus :). I have got to say, when I launched it for the first time I thought it crashed. I tried the next day and then I saw everything worked :).


fix(jest): Update deprecated ts-jest config option.

What does this PR do?
Resolves a warning running tests — this was consistently reproducible both locally and in CI. See e.g.,
the latest Travis build:
ts-jest[backports] (WARN) "[jest-config].globals.ts-jest.tsConfigFile" is deprecated, use "[jest-config].globals.ts-jest.tsConfig" instead.

Simple one-line fix. 👌

Cool, yep - this works for me. Thanks @jessestuart


Tag types

Problem

There is no straight way to encode a predicate (some checking) about a value in its type.

Details
There are situations when a value has to pass some sort of check/validation prior to being used. For example: a min/max functions can only operate on a non-empty array so there must be a check if a given array has any elements. If we pass a plain array that might as well be empty, then we need to account for such case inside the min/max functions, by doing one of the following:

crashing
returning undefined
returning a given default value

This way the calling side has to deal with the consequences of min/max being called yet not being able to deliver.
function min<a>(values: a[], isGreater: (one: a, another: a) => boolean) : a {
   if (values.length < 1) { throw Error('Array is empty.'); }
   // ...
}
var found;
try {
   found = min([]);
} catch (e) {
   found = undefined;
}

Solution
A better idea is to leverage the type system to rule out a possibility of the min function being called with an empty array. In order to do so we might consider so called tag types.
A tag type is a qualifier type that indicates that some predicate about a value it is associated with holds true.

const enum AsNonEmpty {} // a tag type that encodes that a check for being non-empty was passed

function min<a>(values: a[] & AsNonEmpty) : a {
   // only values that are tagged with AsNonEmpty can be passed to this function
   // the type system is responsible for enforcing this constraint
   // a guarantee by the type system that only non-empty array will be passed makes
   // implementation of this function simpler because only one main case needs be considered
   // leaving empty-case situations outside of this function
}

min([]); // <-- compile error, tag is missing, the argument is not known to be non-empty
min([1, 2]); // <-- compile error, tag is missing again, the argument is not know to be non-empty

it's up to the developer in what circumstances an array gets its AsNonEmpty tag, which can be something like:
// guarntee is given by the server, so we always trust the data that comes from it
interface GuaranteedByServer {
    values: number[] & AsNonEmpty;
}

Also tags can be assigned at runtime:

function asNonEmpty(values: a[]) : (a[] & AsNonEmpty) | void {
    return values.length > 0 ? <a[] & AsNonEmpty> : undefined;
}

function isVoid<a>(value: a | void) : value is void {
  return value == null;
}

var values = asNonEmpty(Math.random() > 0.5 ? [1, 2, 3] : []);
if (isVoid(values)) {
   // there are no  elements in the array, so we can't call min
   var niceTry = min(values); // <-- compile error;
} else {
    var found = min(values); // <-- ok, tag is there, safe to call
}

As was shown in the current version (1.6) an empty const enum type can be used as a marker type (AsNonEmpty in the above example), because

enums might not have any members and yet be different from the empty type
enums are branded (not assignable to one another)

However enums have their limitations:

enum is assignable by numbers
enum cannot hold a type parameter
enum cannot have members

A few more examples of what tag type can encode:

string & AsTrimmed & AsLowerCased & AsAtLeast3CharLong
number & AsNonNegative & AsEven
date & AsInWinter & AsFirstDayOfMonth

Custom types can also be augmented with tags. This is especially useful when the types are defined outside of the project and developers can't alter them.

User & AsHavingClearance

ALSO NOTE: In a way tag types are similar to boolean properties (flags), BUT they get type-erased and carry no rutime overhead whatsoever being a good example of a zero-cost abstraction.
UPDATED:
Also tag types can be used as units of measure in a way:

string & AsEmail, string & AsFirstName:

var email = <string & AsEmail><EMAIL_ADDRESS>var firstName = <string & AsFirstName> 'Aleksey';
firstName = email; // <-- compile error

number & In<Mhz>, number & In<Px>:

var freq = <number & In<Mhz>> 12000;
var width =   <number & In<Px>> 768;
freq = width; // <-- compile error

function divide<a, b>(left: number & In<a>, right: number & In<b> & AsNonZero) : number & In<Ratio<a, b>> {
     return <number & In<Ratio<a, b>>> left  / right;
}

var ratio = divide(freq, width); // <-- number & In<Ratio<Mhz, Px>>

A few thoughts, in the typescript compiler we have used brands to achieve a similar behavior, see: https://github.com/Microsoft/TypeScript/blob/master/src/compiler/types.ts#L485; https://github.com/Microsoft/TypeScript/issues/1003 could make creating tags a little bit cleaner. A nominal type would be far cleaner solution (https://github.com/Microsoft/TypeScript/issues/202).

one more use case just came across:
var scheduled = setTimeout(function() { }, 1000);
clearInterval(scheduled); // <-- BUG

with type tag this situation could have been avoided

Unfortunately as long as there is bi-variance assignability checks for function parameters (as of today) is there the whole idea of tag types in TypeScript is worthless: #5741

With Typescript 2 you can now simulate the behavior you want:
declare class MinValue<T extends number> {
    private __minValue: T;
}

// MinValue type guard
function hasMinValue<T extends number>(value: number, minValue: T): value is number & MinValue<T> {
    return value >= minValue;
}

// Use it
function delay(fn: Function, milliSeconds: number & MinValue<0>) { }

const delayInMs = 200;
delay(() => { }, delayInMs); // error: number not assignable to MinValue<0>

if (hasMinValue(delayInMs, 0)) {
    delay(() => { }, delayInMs); // OK
}

if (hasMinValue(delayInMs, 100)) {
    delay(() => { }, delayInMs); // error: MinValue<100> not assignable to MinValue<0>
}

with this concept, you can also create Ranges:
// MaxValue
declare class MaxValue<T extends number> {
    private __maxValue: T;
}

// MaxValue type guard
function hasMaxValue<T extends number>(value: number, maxValue: T): value is number & MaxValue<T> {
    return value <= maxValue;
}

// RangeValue
type RangeValue<TMin extends number, TMax extends number> = MinValue<TMin> & MaxValue<TMax>;

// RangeValue type guard
function inRange<TMin extends number, TMax extends number>(value: number, minValue: TMin, maxValue: TMax): value is number & RangeValue<TMin, TMax> {
    return value >= minValue && value >= maxValue;
}

// Range example
//-----------------
type Opacity = RangeValue<0, 1>;
function setTransparency(opacity: number & Opacity) {
   // ...
}

const opacity = 0.3;
setTransparency(opacity); // error: 'number' not assignable to MinValue<0>

if (inRange(opacity, 0, 1)) {
    setTransparency(opacity); // OK
}

if (inRange(opacity, 0, 3)) {
    setTransparency(opacity); // error: MinValue<0> & MaxValue<3> not assignable to MinValue<0> & MaxValue<1>
}

IMO this feature is long over due - mainly because there are so many types of strings: UUID, e-mail address, hex color-code and not least entity-specific IDs, which would be incredibly useful when coupling entities to repositories, etc.
I currently use this work-around:
export type UUID = string & { __UUID: undefined }

This works "okay" in terms of type-checking, but leads to confusing error-messages:
Type 'string' is not assignable to type 'UUID'.
  Type 'string' is not assignable to type '{ __UUID: void; }'.

The much bigger problem is, these types aren't permitted in maps, because an index signature parameter type cannot be a union type - so this isn't valid:
interface ContentCache {
    [content_uuid: UUID]: string;
}

Is there a better work-around I don't know about? I've tried something like declare type UUID extends string, which isn't permitted, and declare class UUID extends String {}, which isn't permitted as keys in a map either.
Is there a proposal or another feature in the works that will improve on this situation?

I am struggling hard with this in a model I've been building this past week.
The model is a graph, and nodes have input and output connectors - these connectors are identical in shape, and therefore inevitably can be assigned to each other, which is hugely problematic, since the distinction between inputs and outputs (for example when connecting them) is crucial, despite their identical shapes.
I have tried work-arounds, including a discriminator type: "input" | "output" or distinct type "tags" like __Input: void and __Output: void respectively.
Both approaches leave an unnecessary run-time footprint in the form of an extra property, which will never be used at run-time - it exists solely to satisfy the compiler.
I've also attempt to simply "lie" about the existence of a discriminator or "tag" property to satisfy the compiler, since I'll never look for these at run-time anyway - that works, but it's pretty misleading to someone who doesn't know this codebase, who might think that they can use these properties to implement a run-time type-check, since that's literally what the declaration seem to say.
In addition, I have a ton of different UUID key types in this model, which, presently, I only model as strings, for the same reason - which means there's absolutely no guarantee that I won't accidentally use the wrong kind of key with the wrong collection/map etc.
I really hope there's a plan to address this in the future.

Great issue. Added two thoughts here:
The original usecase from @aleksey-bykov sounds a lot like dependent-types, implemented most famously in Idris, where you can define a type "array of n positive integers", and a function that appends two arrays and returns a third array of (n+m) positive integers. Non-empty is a simple case of that. If you like this power, take a look at https://www.idris-lang.org/example/
The simpler usecase from @mindplay-dk is actually what I am struggling with right now. In my example, I have Uint8Array, which may be a PrivateKey or a PublicKey for a cryptographic hash function and I don't want to mix them up. Just like the input and output nodes. This was my solution...
interface IPrivateKey extends Uint8Array {
  readonly assertPrivateKey: undefined;
}

function asPrivate(bin: Uint8Array): IPrivateKey {
  // tslint:disable-next-line:prefer-object-spread
  return Object.assign(bin, { assertPrivateKey: undefined });
}

function signMessage(msg: string, secret: IPrivateKey): Uint8Array {
 // ...
}

Same for public key. When I just did type IPublicKey = Uint8Array, tsc would treat them interchangeably.

@ethanfrey How about:
function asPrivate(bin: Uint8Array): IPrivateKey {
  return bin as IPrivateKey
}

@ethanfrey @dgreensp see comments by @SimonMeskens here - the unique symbol solution is very close to what I was looking for.

@dgreensp I tried what you said, but tsc complained that Uint8Array didn't have the property assertPrivateKey. The problem with using ghost properties to define types.
@mindplay-dk That solution looks perfect, that I can cleanly cast with as IPrivateKey or as IPublicKey, and the information is carried around throughout tsc (cannot convert one to the other), but has no runtime footprint. Thank you for that link.
I'm still learning typescript, and want to thank you all for being a welcoming community.

@SimonMeskens great link. not just the repo, but its references (rambda and lens) also. I have played with dependent types with LiquidHaskell, and studied a bit of Idris, but it seems to need a symbolic algebra solved many times... Two big examples are tracking the length of an array and tracking the set of items in a container.
I was looking at typical to see how they tracked that and found: https://github.com/tycho01/typical/blob/master/src/array/IncIndexNumbObj.ts commented out.... Am I missing something?
This example seems quite nice. https://github.com/gcanti/typelevel-ts#naturals But it seems they had to predefine all possible numbers to do math: https://github.com/gcanti/typelevel-ts/blob/master/src/index.ts#L66-L77
This also looks interesting https://ranjitjhala.github.io/static/refinement_types_for_typescript.pdf but seems to be a demo project and currently inactive: https://github.com/UCSD-PL/refscript

@ethanfrey Interesting, I wonder why my IDE didn't seem to complain.  Well, the bottom line is you just need to case through any:
function asPrivate(bin: Uint8Array): IPrivateKey {
  return bin as any
}

The other example you mention also uses an any-cast, on an entire function signature no less:
const createInputConnector:
    (node: GraphNode) => InputConnector =
    <any>createConnector;

The general principle at work, in both cases, is that the compile-time type need not bear any particular relationship to the runtime type.  Given this fact, there is very little constraining what you can do.  The fact that you need a "dirty" any-cast to mark something as public/private or input/output is a feature, not a bug, because it means that only your special marker functions can do it.

@dgreensp Ahh... the any cast did the trick.
I was doing
const key : IPrivateKey = Buffer.from("top-secret") as IPrivateKey;

which was complaining. but using any fixed that.
const key : IPrivateKey = Buffer.from("top-secret") as any as IPrivateKey;

You can just do:
const key : IPrivateKey = Buffer.from("top-secret") as any;

I've been trying to find an elegant solution to the "primitive obsession" problem. I want my interfaces and method arguments to use data types like UniqueID, EmailAddress, and Priority rather than string, string, and number. This would lead to more type safety and better documentation. Tagging seems overly complicated to me. Type aliases ALMOST do what I want. I think I could get 99% of the way there with syntax like the following.
type EmailAddress < string; // rather than '='

const emailAddressRegExp = new RegExp("[a-z]+@.com");

function newEmailAddress(e: string): EmailAddress {
    if (!emailAddressRegExp.test(e)) {
        throw new TypeError("That isn't a valid email address");
    }
    return new EmailAddress(e.trim());
}

let email1: EmailAddress = newEmailAddress("bob@yahoo.com"); // ok
let email2: EmailAddress = "fail"; // compile error, unsafe cast
let email3 = "banana" as EmailAddress; // compiles ok, forcing the cast
let s: string = email1; // ok implicit cast to wider type always works

I've been trying to find an elegant solution to the "primitive obsession" problem. I want my interfaces and method arguments to use data types like UniqueID, EmailAddress, and Priority rather than string, string, and number. This would lead to more type safety and better documentation. Tagging seems overly complicated to me

Makes me wonder why the unique keyword is tied to the symbol keyword?
Why can't you just say:
type Email = unique string;
type UserID = unique number;
type AuthorizedUser = unique User;

In other words, unique simply meaning "treat the resulting type as unique", which is essentially what it means in conjunction with symbol - this could work for any type, couldn't it?

For that matter, this seems really counter-intuitive:
declare const Foo: unique symbol;

Compared with:
type Foo = unique symbol;

Why do you have to declare a non-existent "fake" constant get the effect of a unique type in the first place?
Hmm.

@mindplay-dk This isn't really what unique means in the context of a unique symbol.  It just so happens that unique symbols can be "abused" to simulate tag types, but if it makes sense to somebody to call tag types "unique types" that is just a coincidence, as far as I'm concerned.
You could tag a numeric type with a string constant type instead of a unique symbol type:
type Foo = number & 'Foo'

The unique symbol is just taking the place of 'Foo' here.

@mindplay-dk does have a point actually. In theory, it would be a nice addition to the language to have unique as a keyword for nominality. Anything you slap it on becomes nominal. Once you start thinking about how that would work though, it quickly becomes clear this is a hard problem to solve and type Foo = unique symbol; probably wouldn't be the way to do it.

Just to add in my two cents:
I've found constructs as mentioned by @mindplay-dk such as
export type UUID = string & { __UUID: undefined }

immensely useful in my codebase. I haven't found any way to resolve the main issue with this though, that the type isn't a valid type as the key for an index type. Has anyone found a way to workaround this?
And I'm obviously in favor of some cleaner way to create nominal types and/or tag types! But until then, I'd settle for any decent workarounds 😄

Does the work done for indexed types in 2.9 make it more feasible to revisit this feature now @mhegazy?

We have talked about this one a few times recently. @weswigham volunteered to write up a proposal.

Just curious, @weswigham / @mhegazy if there is a proposal for this now? Or what the general idea you're considering is even if not in proposal form?
I'm always a little hesitant in asking for features because I know how tough maintaining OSS can be, so though I'd love to see this feature, my appreciation for all the work you've done on TS is great no matter what happens!

For what it is worth, I got quite good mileage from the unique Symbol approach mentioned above:
declare const UsernameSymbol: unique symbol;
type Username = typeof UsernameSymbol;
export type UsernameString = Username & string;

declare const PasswordSymbol: unique symbol;
type Password = typeof PasswordSymbol;
export type PasswordString = Password & string;

I know if the string is a Username or a Password, and never can switch the order of them with a bad api call. The second line is not really needed, you can collapse it into the third (export type UsernameString = typeof UsernameSymbol & string;) but my fellow devs thought this was the most readable.
It would be nice to have some more language support and easier syntax, but this works quite well, albeit a few lines in the type definitions, the usage is easy and intuitive.

unique symbol as a tag turned out to be problematic for us, few cases:

string & typeof uniqueSymbol is not compatible to string
other strange incompatibilities

how about string & { [sym]: typeof uniqueSymbol }

i like the following better:
declare class As<Tag extends string> { ' _ _ _ ': Tag; }
type UserId = string & As<'user-id'>

mhegazy commented 18 hours ago
how about string & { [sym]: typeof uniqueSymbol }

Whoa... I got lost. Why is this { [a: string]: ... }-wrapping of typeof uniqueSymbol necessary for string & but not for something like MyClass &?

@SlurpTheo you're mistaking computed properties with indexers. And none of this is "necessary", just multiple options

Since Tyepscript 2.9, these above constructs no longer work, at least the issue with
string & typeof uniqueSymbol as that is converted to a never type....
This is documented in #25268 and considered desired behavior by the typescript team. Any ideas on how to build tag types that still work with the stricter type rules?

@ethanfrey I use this, found on a related issue thread (sorry but I can't find the original)
declare class OpaqueTag<S extends string> {
  private tag: S;
}

type Opaque<T, S extends string> = T & OpaqueTag<S>;

type UserUUID = Opaque<string, 'UserUUID'>;
type TransactionId = Opaque<string, 'TransactionId'>;

const a: UserUUID = '...' as UserUUID; // assigning a string literal requires a cast
const b: TransactionId = UserUUID // errors

It's not perfect, but in my experience gets the job done and it's really readable

@agos the problem with that solution is that it's not type safe. Symbols are the only nominal type, so you need a solution that uses them. That much is clear.
Here's the sample adapted to be type-safe:
// Using namespace here simply to show that external files
// should NOT have access to OpaqueTagSymbol or OpaqueTag.
// Put this in its own module, without the namespace
namespace Tag {
    declare const OpaqueTagSymbol: unique symbol;
    declare class OpaqueTag<S extends symbol> {
        private [OpaqueTagSymbol]: S;
    }

    export type Opaque<T, S extends symbol> = T & OpaqueTag<S>;
}

// Simple alias, if in a module, same as:
// import { Opaque } from "tagmodule";
type Opaque<T, S extends symbol> = Tag.Opaque<T, S>;

// Since these are ghost symbols, you probably don't want to export them
// Create actual symbols if you do
declare const UserUUIDSymbol: unique symbol;
declare const TransactionId: unique symbol;

type UserUUID = Tag.Opaque<string, typeof UserUUIDSymbol>;
type TransactionId = Opaque<string, typeof TransactionId>;

const a: UserUUID = '...' as UserUUID; // assigning a string literal requires a cast
const b: TransactionId = a // errors

Nice solution, thank you @SimonMeskens

@SimonMeskens it seems to me that the only case where the type safety of the solution I posted earlier would be that two Tag types with the same string passed as second type parameter? In that case it might be still a worthy tradeoff

@sledorze I didn't use symbol at all, it's all just in declarative type land, so this solution will work even in environments without symbol :)
Magic!
@agos That's correct, it's mostly a collision problem, but there's also the issue that with strings, you can't encapsulate creation, as any other part of the code can construct a valid one by providing the same string. With symbols, if the symbol isn't exported, only that part of the code is able to create that specific tagged type.

@SimonMeskens Oh yes! wonderful ;)

I liked @SimonMeskens example, but it leaves the string prototype methods available on the resulting type--which I did not like.
I made the following change which seems to preserve some some checks for the original type:
export type Opaque<T, S extends symbol> = T & OpaqueTag<S> | OpaqueTag<S>;

By adding the intersection, Typescript will not let such a variable be used as a string without a type assertion.
For example:

type UserId = Opaque<string, UserIdSymbol>;

const userId: UserId = "123" as UserId;

userId.replace('3', '2'); // This works on the Union type, but not the union+intersected

function takesAString(str: string) {
  // ...
}

takesAString(userId); // This works on the Union type, but not the union+intersected
takesAString(userId as string); // This works on both versions

const num: number = userId as number; // this doesn't work on either version

I'm sure there's probably a better way to implement this, and certainly room for set of semantics.

I built https://github.com/ForbesLindesay/opaque-types which uses a transpiler to support opaque and nominal types, with optional runtime validation for the cast operation. It may be of interest to people here.

@ForbesLindesay way to go!
What do you think about https://github.com/gcanti/newtype-ts?

@qm3ster that looks pretty cool. The iso and prism functions are a neat approach to avoiding runtime cost for multiple opaque types in a single project. The main issue I see with it is that you've used functional terminology, instead of plain english words. You've also tightly coupled your solution to functional programming concepts like Options, which are not commonly used in JavaScript.  The other small points are that you have _URI and _A show up as properties on your opaque values, when in fact those properties do not exist at runtime.
The main advantage I see to using my transpiled approach, is that it's very easy to completely change, in the event that new versions of typescript break the current approach.  The secondary advantage is that I get to use clean syntax, and ensure that the type name is the same as the name of the object that provides utility functions for cast/extract.
I mainly wanted to prototype how I thought they might behave if added to the language.

@ForbesLindesay It's not my project :v I just used it a few times.
I see how the notation might be an impedance to wider adoption.
It's positioning itself as part of a larger gcanti/fp-ts ecosystem, hence the Option type.

@ProdigySim @SimonMeskens Nice solution but it seems to has some cons as below.

The notation declare const symbolName:unique symbol is long. And therefore, the user should write multi line when define opaque type alias.
The symbolName in declare const symbolName:unique symbol is necessary not.  However the user should define symbolName in user's namespace.

Then I tried improvement. And it seems working.
Please let me know if there are problems.
The definition of Opaque:

interface SourceTag{
    readonly tag:symbol;
}

declare const OpaqueTagSymbol: unique symbol;

declare class OpaqueTag<S extends SourceTag>{
    private [OpaqueTagSymbol]:S;
}

export type Opaque<T,S extends SourceTag> = T & OpaqueTag<S> | OpaqueTag<S>;

usage:
type UserId = Opaque<string,{ readonly tag:unique symbol}>;

type UserId2 = Opaque<string,{ readonly tag:unique symbol}>;

const userId:UserId = 'test' as UserId ;

const userId2:UserId2 = userId; // compile error

The notation  Opaque<string,{ readonly tag:unique symbol}>  can be written in one line.

@ProdigySim
True. I think it is trade off between "easy to understand error" and "the syntax is shorter".

I checked out how Flow handles opaque types in comparison to our solutions. They have some interesting behavior.
Notably:

Opaque types are treated differently in the file they're created in. Implicit conversions from underlying-type to Opaque Type are allowed in the same file the type is created in.
The Opaque Types behave similarly to @SimonMeskens 's original solution (string & { [TagSym]: typeof UserIdSymbol }). That is, implicit conversions TO the underlying type are allowed.
Opaque Types can be used as index types. e.g. { [idx: UserId]: any } is a valid type. This is not currently possible in typescript afaict.

I put together a typescript playground link demonstrating different constructions of Opaque<T> and their capabilities.

"Super Opaque": OpaqueTag<S>-- no reference to underlying type, no valid casts to an underlying type.
"Weak Opaque":  T & OpaqueTag<S> -- matches flow behavior closely, automatic downcasts to T
"Strong Opaque": T & OpqaueTag<S> | OpaqueTag<S> -- keeps some reference to underlying type for explicit conversions, but doesn't allow implicit casting.

I think each could have uses; but a first-party Typescript solution could definitely allow the best of all worlds here.

For others coming across this, a fairly succinct solution that results in short but readable error messages can be found over here.  Comes with caveats, since it is setup to ignore a compiler warning, but so far I like the UX of it the most out of all of the options I have seen so far.  Need to test it cross-module still though.

Wrote a small tag type library based on some ideas here. Will change when TS gets nominal types.
https://github.com/StephanSchmidt/taghiro
Happy for feedback.

I guess, it is necessary for typescript to  support  official opaque type alias.
Because both(@ProdigySim and me) suggestions has the number of writing characters bit too much.

FYI: An issue with the solutions in https://github.com/Microsoft/TypeScript/issues/4895#issuecomment-401058832 and https://github.com/Microsoft/TypeScript/issues/4895#issuecomment-401067935 is that if your build process involves interfacing between different modules using .d.ts files from tsc -d, they'll need to be adjusted to make the private members public. Otherwise, the type of the private member gets stripped, making the helpers lose their intended effect.

Another minor difference I thought of today while debugging an issue with a manual Opaque<T,K> solution: These "official" tag types do not extend object.
You can try this code against the test build the bot created:
// Experimental Tag Types from this PR
type Builtin = string & Tag<'builtin'>;

// true
type yy = Builtin extends string ? true : false;
// false
type zz = Builtin extends object ? true : false;

// DIY tag type solution
type RetroTag<T, K> = T & { __tag: K }
type Retro = string & RetroTag<string, 'retro'>;

// true
type aa = Retro extends string ? true : false;
// true
type bb = Retro extends object ? true : false;

In my case, today I had to special case a type conditional to avoid having my tag types be treated as objects. With the new behavior exhibited by the builtin implementation here, I wouldn't have had to. So, I'd consider this an improvement over the DIY solution. But probably worth noting for anyone trying to convert from DIY to builtin.

We have talked about this one a few times recently. @weswigham volunteered to write up a proposal.

@mhegazy @weswigham did this ever happen?

There are two proposals in the form of experimental implementations in PRs.

@weswigham sorry to raise this issue again. Is there is roadmap for this?

I had a recent need for a tag type that none of the existing workarounds can solve.  I rebased @weswigham's #33290 (at shicks/TypeScript:structural-tag) and found that it works more or less perfectly.
To summarize this solution, it introduces a new type operator, tag, that creates a new type tag T from any existing type T, such that tag T <: tag U only if T <: U, but otherwise tag T behaves essentially like unknown in terms of inference and other capabilities.  I believe this is the ideal solution to this issue for a handful of reasons:
Branding without the lies
Type branding is a very common practice, seen in libraries, numerous blogs, and even in the TypeScript codebase itself.  But to this day, the standard approach (workaround, rather) is to lie to the type checker by adding fake properties:
type Brand = {brand: void};
type BrandedString = string&Brand;

This may be "free" at runtime, but you can get into some trouble while type checking, since this brand property doesn't actually exist.  Additionally, if a primitive type (such as string) isn't intersected with the brand, the checker assumes it's an object, which can lead to wildly inaccurate narrowing.  The tag T type makes no such promises.
Reimagining some code from the TS codebase:
type Branded<T, B> = T & tag {[K in B]: void};
export interface ClassElement extends Branded<NamedDeclaration, 'ClassElement'> {
  readonly name?: PropertyName;
}

Inherently structural, but nominal-friendly
These tag types are structural, and behave consistently to all the existing structural types (unlike the unique type proposal in #33038): you can even reflect into them via conditional types (T extends tag infer U ? ...).  This makes them a very natural addition to the language.
That said, tag presents a relatively clean emergent solution to nominal typing (#202) as well, by simply combining with unique symbol types: tag {readonly _: unique symbol} generates a guaranteed-unique type everywhere it shows up in source.  Nominal typing falls out more or less automatically when you either intersect with such a type, or add a brand-typed property to an interface.
Introducing a type operator/primitive solely for the purpose of nominal typing would be going against the grain (hence the repeated complaint on #202 and elsewhere that TypeScript is structural, not nominal), but this would provide the necessary tools for those who want nominal typing to achieve it, while still remaining essentially structural.  For those who want to ensure that multiple different versions of their API are compatible, they can use structural tags with fixed strings.  For those who want to guarantee that nothing outside their module is assignable to their nominal type, they can use unique symbols to get that guarantee.
True opacity
In addition to allowing authors to avoid lies and enabling brands and nominal types, this solution introduces a new type with important capabilities that's currently impossible to express.  As mentioned above, the current status quo for brands is to use an object literal type (e.g. {brand: void}).  If this type is intersected with a primitive, the type checker will assume its typeof is that primitive, but if not, the checker will (potentially erroneously) assume it's object.  This makes it impossible to accurately use these faux brands to refer to types whose actual runtime representation (e.g. primitive vs object) is truly opaque.  In fact, the only type that currently prevents narrowing by typeof is unknown, but unknown will clobber any other type in a union (since unknown|T === unknown), and provides no actual safety.
The beauty of tag T is that it prevents typeof narrowing (like unknown), but (unlike unknown) it's still preserved in union types.  This gives it a unique capability that no type currently expressible in TypeScript can do.  To wit, adapting the "impossible" example from above:
type SafeBigint = tag {readonly __safeBigint: unique symbol};
declare const x: string|SafeBigint;
if (typeof x === 'string') {
  use(x);
  //  ^? const x: string | (string & SafeBigint)
} else {
  use(x);
  //  ^? const x: SafeBigint
}

Because the tag type is completely opaque, the narrowing recognizes that it could show up in either branch, and thus avoids potentially-incorrect usage of the SafeBigint wrapper type.  And because it's not a supertype of all other types, it doesn't clobber unions.  It's also impossible to cast directly to/from (without casting through unknown or any), which can provide good type safety for API design.

Final words
I'm asking that the TypeScript team reconsider the tag T approach as a solution to both tagged/branded types (this issue) and nominal types (#202).  The closed PR #33290 or my rebase of it demonstrates clear utility.  There are no backwards-compatibility risks.  And I'd argue that it's the best solution to nominal typing that stays true to TypeScript's structural roots.  It allows using it in flexible, emergent, ways that lead to better, sounder, types.
Thank you for your consideration.

Hello, I just came across this issue and it reminds me a problem I encountered, that led to a quick PoC I made several months ago. The idea to solve this problem seems to be kinda the same than the solutions mentionned, therefore I thought it could be interesting to provide a link to my repository : https://github.com/Schrubitteflau/type-all-the-things
Thank you, and happy coding

Another library example is zod .brand


GDK Tools dialog auto-saves

Description
Made the GDK Tools dialog auto-save. Also moved the reset button to the bottom instead of the top to make it a bit cleaner.
Documentation

[x] Changelog
[ ] Documentation

LGTM, though I'm not entirely sure about that timer. Not sure what could replace it though.

I'm not a huge fan of the timer, either. I wish Unity had a concept of "finished entering in text, go ahead and update whatever you need to". I've copied the timer from other sections of code in the GDK, so it's at least consistent.


Duplicates multi-word tags (etc.) when using string values and SELECT element

TL;DR: Any tag string that is modified by JS escape() is prone to unwanted duplication!
First, thanks for this plugin! It's been hassle-free for us until just today. As described in this GitHub issue, I've found a bug that results in creating duplicate tags when using bootstrap-tagsinput with a select widget. If the select is pre-populated with tags in its options (which means we're using simple strings as our values), any multi-word tag — actually, any tag that is modified by escape() or encodeURIComponent() — will be duplicated each time the TagsInput behavior is applied.
In this setup, the tags widget reads option values from a chosen select widget, and scans its own TagsInput object for matching values. But for simple string values, it messes things up:
When it first scans their value attributes, the values are read "directly" without transformation. So for this option, the value is read as "delete me":
    <option value="delete me">delete me</option>

But when it looks for matching option elements (to decide whether a new one needs to be created), it checks for an URL-encoded version of this value. This would only match on an option like:
    <option value="delete%20me">delete me</option>

If we try to pre-escape the value attributes, we'll start with options like the one above. But then these are scanned as initial tag values, and the test ends up looking for this:
    <option value="delete%2520me">delete me</option>

As far as I can see, this wil always fail with a multi-word tag (any string modified by encodeURIComponent).
For now, I've worked around the problem by switching from simple string tag values to simple objects like
    {"displayText": "delete me", "attrValue": "delete%20me" }

... then specifying these properties in my tags options:
    itemValue: 'attrValue',
    itemText: 'displayString',

I checked the latest code, and it seems both conflicting operations are still unchanged. If the Right Thing to do is to URI-encode attribute values, then this should be done here and also specified for any intial option elements in the page. And the initial reading of option elements into tag values should un-escape these strings before adding them as tags:
  var tagValue = decodeURIComponent( $(this).attr('value') );
  self.add(tagValue, true);

If needed, I can put together a proper pull request for this, but I won't have time for a week or so.

I just noticed the same behaviour described above, and discovered this issue when I was browsing to see if anyone else has reported it. Thanks for posting such a detailed description. Did you find a way to continue using simple string values with multi-word tags? Surprised nobody else has been bitten by this.

I have also been bitten by this bug, Only noticed it yesterday.
Thanks for the detailed post, if you haven't written a pull request already I will.

+1 this is still an issue.

The pull request has been merged and the problem is fixed, but please note that the current release (0.5.0) does not contain this fix.

@Dr4K4n, can you link to the merged pull request? I'm curious to see whether it handles quotes, angle brackets, etc. safely.

@jimallman Well, sry, I must have looked in the wrong places... It doesn't seem to be merged yet.

this is still a problem, and it also applies to any tag with special characters that would be escaped by encodeURI. The problem is that the check for existing options checks with the value escaped with encodeURI, so of course it will not match (not sure exactly how this results in a double entry, but it does); my fix was to gather the current options first, then check if the value is among them, since removing the encodeURI from the selector causes errors in the case of values containing double quotes. at line 140 change this:
if (self.isSelect && !$('option[value="' + encodeURIComponent(itemValue) + '"]',self.$element)[0]) {
to this:
      var opts = self.$element.find('option').map(function() {return $(this).val();}).get();       if (self.isSelect && !opts.includes(itemValue)) {


Dataset Viewer issue for Matthijs/cmu-arctic-xvectors

Link
https://huggingface.co/datasets/Matthijs/cmu-arctic-xvectors
Description
The dataset viewer is not working for dataset Matthijs/cmu-arctic-xvectors.
Error details:
Error code:   ClientConnectionError

Thanks for reporting. It sometimes appears, and it's due to a timeout between the Hub backend and the datasets server. We'll be looking at this. Closing in favor of #877


useStorage().getKeys() returns different result for dev and build

Environment

Operating System: Linux
Node Version:     v18.18.0
Nuxt Version:     3.8.2
CLI Version:      3.10.0
Nitro Version:    2.8.0
Package Manager<EMAIL_ADDRESS>Builder:          -
User Config:      -
Runtime Modules:  -
Build Modules:    -

Reproduction
https://stackblitz.com/edit/nuxt-starter-zcakjn?file=server%2Froutes%2Ftest.ts
Describe the bug
Go to the /test endpoint in dev and build:

dev -  there are keys in the list
build - the list is empty

Additional context
Similar issue https://github.com/unjs/nitro/issues/374
Logs
No response

Hi dear @enkot. This is an expected behavior indeed. build:, root: and src: mounts are only in dev (auto configured by nitro using devStorage configuration).
If you need a persistent data storage, you can manually mount a new one or use useStorage('data').


Kubectl delete should handle stopping an rc while it's starting replicas

Since kubectl stop currently polls status.Replicas, if one creates an rc and stops it soon after there's a chance the polling hits status.Replicas=0 when in fact the rc has not updated status.replicas yet.
An elegant solution to this problem probably involves kubectl not deleting the rc, but somehow signaling to the rc manager that it wants the rc gone. A halfway solution involves kubectl not deleting the rc while it's working. One solution is as follows (hacks like defaulting status.replicas to -1 aside):
The current stop flow is:

resize
poll on status.Replicas
delete rc

At the time of the resize, the rc could be in 3 states:

stable (status.Replicas == spec.Replicas)
dormant (waiting on watch events from previously created/deleted replicas)
working (currently creating/deleting replicas)

It is not safe to poll status.Replicas while the rc is in working. If we create it in working and move it to dormant/stable after updating status.replicas, we can add a stage between 1 and 2 that blocks till this happens.
@lavalamp @davidopp

We've encountered this race previously in other contexts. See also #7328.

So kubectl will compare the rv it got via its resize update with the rv in the rc's status? that would work but i think we've stayed away from comparing rvs for various reasons in many other clients. Fyi I think we should fix this for 1.0, either hacky or clean, or people will be confused. We started rate limiting the controller-manager to 20qps, so it takes 10s to create an rc with 200 pods. Stopping during those 10s has undefined consequences.

The proposal in #7328 would add a new sequence number, not use resourceVersion.
If we don't want to bite the bullet and properly implement graceful termination (#1535) in the server, then another alternative would be to just delete the replication controller, and then the pods matching its selector. Rolling update already has such a cleanup loop:
https://github.com/GoogleCloudPlatform/kubernetes/blob/master/pkg/kubectl/rolling_updater.go#L244

Just cleaning up orphaned pods won't cut it because deleting the rc != the rc manager noticing the delete. So if we cleanup orphaned pods and the rc manager is processing the rc, it will still create pods. Looping around till there are none left might, but feels dirty. I'm going to take a stab at a few experimental implementations and see what works best given our 1.0 constraints.

I think I'm going to stick with watching pods instead of status.Replicas in kubectl, that'll fix a majority of the confusion and confine the changes to kubectl, though some races will still be possible. In theory graceful termination is a feature and we have feature freeze on friday, so that's probably not getting.

This is too error prone to do without some changes to the rc manager. The best change discussed so far (and one that will be helpful long term) is to add a sequence number to the spec and status of the rc. Everytime the rc manager takes any action it mirrors the sequence number from the spec into the rc, and kubectl does:

resize to 0
poll on status.sequenceNumber from previous step so it knows the rc has seen the reisze
poll on status.Replicas
delte rc

Fixed via https://github.com/GoogleCloudPlatform/kubernetes/pull/9739


Graal 20.3.0

Release notes
https://www.graalvm.org/release-notes/20_3/

Most interesting update is probably
Added support for method handles that represent a call to a single method accessible by the Reflection API, with no transformations applied through the Method Handles API, such as calls to MethodHandle.bindTo() or MethodHandle.asType(). To enable this feature use --report-unsupported-elements-at-runtime.

Failing tests appear unchanged, largely cache file on Windows, and lack of support for kotlin suppressed exceptions.
Test run finished after 345552 ms
[       256 containers found      ]
[         2 containers skipped    ]
[       254 containers started    ]
[         0 containers aborted    ]
[       254 containers successful ]
[         0 containers failed     ]
[      2081 tests found           ]
[       109 tests skipped         ]
[      1972 tests started         ]
[        38 tests aborted         ]
[      1906 tests successful      ]
[        28 tests failed          ]


QUIC hot restart 6 [WIP]

Commit Message: QUIC hot restart 6 [WIP]
Additional Description: Make the child instance not read from the UDP socket until the parent instance stops reading. From the prior changes, the parent instance forwards UDP packets it doesn't recognize to the child instance via a side-channel, so not reading doesn't mean not responding - it just means not intercepting packets which may be intended for the parent instance.
In this change there is no drain optimization - if the parent has no more live connections we will still wait and forward packets until the drain time completes. I think it is most likely not worth optimizing for that, since the eventual goal is to migrate the QUIC-packet-sorting behavior onto eBPF at which point, for systems that care about performance, all of this manual forwarding support will be unused anyway.
I've tried to keep entirely of the performance hot path, so the change only touches constructors and enable/disable operations and stays out of the read path.
It's awkward because the point at which we detect that the socket is being forwarded (the creation of the UdpListenSocket) the UdpListener and ActiveQuicListener don't exist, but those are where we need to disable or enable reading. Also awkward is the potential race - the unpause callback can potentially arrive after the listener was deleted, and must go to the right thread, so there's a bit of a tangle ensuring that that's the case. Finally, there's a potential race of disable/enable and this paused state, so I've added enough state to the UdpListener that the sequence start_paused->disable->unpause does not accidentally end with the listener enabled.
Keeping the additional context on only NetworkListenSocket and UdpListener (and not exposing it through Socket) minimizes the impact of the change, but does require a dynamic_cast to UdpListenSocket (the socket of a UdpListener must, of course, always be a UdpListenSocket).
Risk Level:
Testing:
Docs Changes:
Release Notes:
Platform Specific Features:

I think this is a reasonable approach overall. But I think you'll need a way to un-register for these callbacks, in the case that the quic listener is removed via LDS after the child has started, but before the parent has finished draining.

That's handled as-is by the callback self-aborting, rather than by unregistering. I don't think unregistering can work reliably because the destructor may not be called on the same thread.

I think this is a reasonable approach overall. But I think you'll need a way to un-register for these callbacks, in the case that the quic listener is removed via LDS after the child has started, but before the parent has finished draining.
I'd also like to get at least one more opinion on this, maybe oneof @wbpcode @soulxu @alyssawilk @yanavlasov.

Or we only register the worker as the callbacks, then we can use the existing disableListeners/enableListeners (maybe add a new flag to disable all the udp listener) of ConnectionHandler to pause/unpause the listener.
https://github.com/envoyproxy/envoy/blob/d797293ee8f31328ae664304ad6d6ef85d2e4f2c/source/common/listener_manager/connection_handler_impl.h#L56-L57

I think this is a reasonable approach overall. But I think you'll need a way to un-register for these callbacks, in the case that the quic listener is removed via LDS after the child has started, but before the parent has finished draining.
I'd also like to get at least one more opinion on this, maybe oneof @wbpcode @soulxu @alyssawilk @yanavlasov.

Or we only register the thread worker as the callbacks, then we can use the existing disableListeners/enableListeners method (maybe add a new flag to disable all the udp listener) of ConnectionHandler to pause/unpause the listener.

The problem with using the existing functions is there are already calls to those functions which could potentially make something racily do the wrong thing, e.g. if a listener starts up and is disabled by a call to disableListeners for hot restart purposes, then is disabled for some other reason, then hot restart draining completes and calls enableListeners, now the socket would be listening despite the other caller of disableListeners expecting it not to be. Tracking it separately ensures that the socket is only reading if both things want it to be.
So my rationale for separating out this paused state is that it can play well with other enable/disable operations with no risk of conflict. Another way to achieve this would be to add a "disabledness" counter so that two disables must be matched to two enables, but knowing whether that would be safe or not would require digging a lot more into existing code to ensure it doesn't currently expect one enable to be capable of overriding two prior disables.

/coverage

/retest

/retest


Simulator module takes time to download

From the simulators main page, clicking on simulators creates a delay as the module is loaded. From the user's point of view, it seems as if the application is frozen.
Using preloading may fix this.

The simulators module is 4.6mb uncompressed. There might be an erroneous import that is causing the large size. This should be fixed independently of the preloading. Tracked in #1290

fixed in #1291


Sliders flickering on Safari

Issue description:  I Observed that slider items are flickering on Safari. The problem has been on observed on safari.
Demo link/slider setting: https://codesandbox.io/s/d0yoz
(It's a random example I found on the codesandbox)
Tiny-slider version:   tested with 2.8.8 - 2.9.3
Browser name && version:  Safar 14.0.3
OS name && version:   OSX 10.15.7

Yes, it's very noticeable on the Customize demo slider. It kinda looks like a loazyload thing but it flickers after it's loaded and shown. Half of the image or 2 even images flicker after sliding.
ios 14.4 / ipad 2017 / safari + Chrome

I've seen these kind of problems before on Safari and it seems to be related with Translate3D, which is being used for the sliding effect.

@dijkermans any known "hacks" to prevent this? unfortunately, I'm more a JS than CSS person 😬

Hi, try this CSS on the slides and/or img:
-webkit-backface-visibility: hidden;
-webkit-perspective: 1000;
Please let me know if it works.
I guess the best fix is to use TranslateX instead of Translate3D, but 3d renders faster.

@dijkermans 🙇
.tns-item, tns-item img {
  perspective: 1000;
  backface-visibility: hidden;
  transform: translate3d(0, 0, 0);
  -webkit-perspective: 1000;
  -webkit-backface-visibility: hidden;
  -webkit-transform: translate3d(0, 0, 0);
}

... id did fix the problem for me 😎

btw I think only the webkit namespaces props are relevant

I think it should be like this:
-webkit-backface-visibility: hidden;
-moz-backface-visibility: hidden;
backface-visibility: hidden;
btw, is this one really required?:  -webkit-transform: translate3d(0, 0, 0);
Please try and share which lines were required to fix it.

-webkit-transform: translate3d(0, 0, 0); finally fixed it for me

Ok, I double-checked, the minimum required CSS (in my case) is:
.tns-item {
  -webkit-transform: translate3d(0, 0, 0);
}

Ok, I double-checked, the minimum required CSS (in my case) is:
.tns-item {
  -webkit-transform: translate3d(0, 0, 0);
}

Works for me on iPhone 8.


google chrome has CORS issues on mac

open -a Google\ Chrome --args --disable-web-security --user-data-dir

is necessary to avoid exceptions with
zone.js:2707          GET http://localhost:8080/ net::ERR_CONNECTION_REFUSED

was solved with
	config := cors.DefaultConfig()
	config.AllowOrigins = []string{"https://localhost:443"}                   // Allow requests from localhost:8080
	config.AllowMethods = []string{"GET", "POST", "PUT", "DELETE"}            // Allow specific HTTP methods
	config.AllowHeaders = []string{"Origin", "Content-Type", "Authorization"} // Allow specific headers
	r.Use(cors.New(config))

and client on "https://localhost:443/"