[WIP] Optimize Swin UNETR

Signed-off-by: Yuchen Xu<EMAIL_ADDRESS>Fixes #773 .
Description
Accelerating optimizations for the Swin UNETR tutorial.
Status
Work in progress
Checks

In preliminary experiments, training was run for 5 epochs/120 steps, with one validation. All experiments are on one 32GB RAM GPU.
As seen below, AMP seems to provide the biggest speed boost. Note that while the effect of ToDeviced and ThreadDataLoader is small, the effect could be amplified greatly since training is designed to run for 30,000 steps.
Original: 144.86s

ToDeviced + ThreadDataLoader: 137.97s
ToDeviced + ThreadDataLoader + AMP: 75.17s

Here are the results of running larger batch sizes in combination with thread workers. All experiments are only run once. Consistent with findings at #757, the use of thread workers does not seem to improve speed, but the use of larger batch sizes does seem to help (at batch size = 4), although further investigation is needed.
Thread | 1 | 2 | 3 | 4
Batch
1 | 72.67s | 72.62s | 72.97s | 78.30s
2 | 75.70s | 76.26s | 76.58s | 76.82s
3 | 79.41s | 77.04s | 78.56s | 77.14s
4 | 71.51s | 71.48s | 71.97s | 72.36s
Further profiling analysis will be conducted in the following week.

Hi @yuchen-xu ,
Thanks for sharing the experiments summary, progress looks good.
Please note that changing batch size may affect the training culve, so we must evaluate the training speed with "achieving target validation metrics".
Thanks.

Just making a note in case it comes up later: the data loading function in this is load_decathlon_datalist, which returns a list of dicts (and not MetaTensor). Hence the "image_meta_dict" seems to work fine for the slice visualization in 0.9.1rc2.

Hi @yuchen-xu ,
That because we tried to avoid breaking changes in the new release, it's compatible in API level.
But please change to use MetaTensor features as it's tutorial to show the latest method.
d["image_meta_dict"] is same as d["image"].meta.
Thanks.

I have encountered two major issues when optimizing this tutorial:

Training with AMP torch.cuda.amp.autocast() causes training to diverge. All else being similar, the training graph looks like this:

Validating with AMP is fine. AMP encloses these two lines of code, which is consistent with its intended use (official documentation)

        logit_map = model(x)
        loss = loss_function(logit_map, y)

I suspect this might have to do with the use of transformers and/or some specific floating point operations, but I am not able to confirm.

Introducing ToDevice() before random operations, combined with ThreadDataLoader (no thread workers), seems to slow training down significantly compared to using ToTensor() and DataLoader. Curiously, this is opposite to what I observed when running in 0.9.0. Here are some numbers from the V100 GPU: (120 steps, one validation)

Val AMP, middle ToDevice before random transforms, ThreadDataLoader: 358.67s total, 63s for a typical pass through the data (24 steps)
Val AMP, end ToDevice, ThreadDataLoader: 525.75s, 95s
Val AMP only: 282.16s, 47s
@tangy5 @ahatamiz - please advise, and we can work together to figure out possible reasons. I can provide more information on the code I used (the 091 one in the commit above). Thanks!

Update: the second issue does not seem to be isolated and seems to be a broader issue in 0.9.1; see comments in another PR.
The AMP training issue is baffling and has been observed in both 0.9.0 and 0.9.1.

After 0.9.1rc4, we are bringing back the EnsureTyped transform with track_meta = False, which will speed up processing by a lot. We have a design choice here for validation data though: we have illustrations of an example validation image both before and after training, and those illustrations need the metadata. So we have two choices: either set track_meta = True for validation data, or not use EnsureTyped for validation data. Over 30000 training steps, both would slow overall training down by less than 1%, and both have essentially the same time. Suggestions?

'explicit is better than implicit', so I think set track meta to true is better...

Thanks @wyli , I agree. I will also include a blurb explaining this choice.

Lessons learned about AMP:
It seems that all of these are required for AMP to work (at least for Swin-UNETR):

GradScaler (recommended in documentation, included in fast training tutorial)
optimizer.zero_grad (not mentioned in documentation, included in ftt)
scaler.unscale_ (mentioned in documentation as optional if you want to clip gradients, not included in ftt)

Speedup performance in 0.9.1rc4 (30,000 steps, validation every 500):

Original: 63,634s
AMP + ThreadDataLoader + EnsureTyped (track_meta = False for training, True for validation): 30,535s

New training curve doesn't differ much from the original and gets to 0.84 target validation accuracy.

Hi @yuchen-xu ,
Thanks very much for the optimization in swin unetr tutorial, I have several comments:

Frist of all, I think maybe you are not aware of the functionality of get_track_meta() and set_track_meta() APIs: https://github.com/Project-MONAI/MONAI/blob/dev/monai/data/meta_obj.py#L30
It's global API, without setting set_track_meta(False), your random transforms still run with MetaTensor.
The speed up you saw just because we fixed some bugs in the 0.9.1rc4.
Please take a look how the set_track_meta() is used in the fast training tutorial.
About your description "we have illustrations of an example validation image both before and after training, and those illustrations need the metadata", you can set EnsureTyped(keys=["image", "label"], track_meta=get_track_meta()) in the val transforms and set_track_meta(False) only during the training.

What do you think?
Thanks.

So for now, my suggestion is:

Try to compare the training speed of tracking meta ON/OFF, if no big difference, we can remove all the track_meta related code to make the tutorial easier to understand. Otherwise, add them correctly.
Please remove the XXX_orig notebook and the profiling python code, I think they are only for temp test?

Thanks.

Running 10000 steps:

Set_track_meta = False with EnsureType: 9845s
Set_track_meta = True with EnsureType: 10252s
Set_track_meta = False without EnsureType: 11809s

So I think we should be keeping both set_track_meta and EnsureType for now.

Hi @yuchen-xu ,
Thanks for the update.
May I know the speed of "Set_track_meta = True without EnsureType"?
And I think you can use MONAI 0.9.1 to test now as we already released it.
Thanks.

Yes, I am using 0.9.1.

May I know the speed of "Set_track_meta = True without EnsureType"?

Given what set_track_meta does, I assumed this would be the slowest, but I will try and run that.

Because I don't quite understand why Set_track_meta = True with EnsureType: 10252s is faster than Set_track_meta = False without EnsureType: 11809s..
Thanks.

Following usage in the fast training tutorial, set_track_meta is used after CacheDataset and only affects random transforms, while EnsureTyped(track_meta = False) affects only the non-random transforms, so they're handling different sections of transforms.
I think trying the last combination will confirm/disprove this idea.

EnsureType does also affect the training, because it affects the cache content is Tensor or MetaTensor, and random transforms will convert it to Tensor or MetaTensor depends on the set_track_meta, this conversion step may be time-consuming.
@wyli I am afraid the as_tensor() operation of MetaTensor is deepcopy? So Set_track_meta = False without EnsureType is the slowest one. Base on the doc-string:
https://github.com/Project-MONAI/MONAI/blob/dev/monai/data/meta_tensor.py#L317
Thanks.

set_track_meta = True without EnsureType is 14442 seconds, much slower than the others, as expected.

Please update the description doc in the beginning to add the best training time and memory usage on your GPU.
Thanks.

Sorry, I forgot to put in the final comparison.
For 30,000 training steps (1,250 epochs), with validation every 500 steps:
Original: 63,634s
Optimized: 30,264s (-52%)
These numbers are different from the ones I used on the poster; the poster numbers

used validation every epoch (24 steps)
used an original that doesn't have CacheDataset (i.e., without any MONAI optimizations)
In there, 30,000 steps gave 148,895s (original) vs. 58,281s (optimized) (-61%).


Add experimental rule to check for unnecessary parentheses in function n call followed by lambda

Description
Add experimental rule to check for unnecessary parentheses in function n call followed by lambda
Closes #1068
Checklist

[X] tests are added
[X] CHANGELOG.md is updated

LGTM besides some minor comments


Timestamp Pallet should accommodate blocks with same timestamp as previous

pallet-timestamp has a MinimumPeriod parameter which is meant to require that blocks are made with a timestamp that advances from block to block. Even when MinimumPeriod is zero, blocks still must have a timestamp greater than the previous block's:
https://github.com/paritytech/substrate/blob/6507a79b2baf7e4cc6f5868aa490dfe275144f9a/frame/timestamp/src/lib.rs#L206-L209
The motivation for changing this is that it's an issue for parachains, where the timestamp is drawn from the slot of the relay-chain block they reference as an anchor (the "relay-parent"). If two subsequent parachain blocks have the same relay-parent, their timestamps will be equal and this check fails. This isn't currently an issue in deployed code, but it will be soon.
The solution is to change the logic of pallet-timestamp to truly allow a MinimumPeriod of 0.

@rphmeier is this still relevant? What problem do you get when setting that period to 0?


"1 \ { 2 \ 3 }" outputs "Mismatched Braces"

Input "1 \\ { 2 \\ 3 }".  Output is "Mismatched braces". I'm asking because of https://github.com/verybadcat/CSharpMath, which is a port of this repository; we have the same bug there.
See also https://stackoverflow.com/questions/50997777/the-rules-of-latex-mathjax-for-someone-implementing-it

Request edit: should be "1 \\\\ { 2 \\\\ 3 }" (Each two backslashes are replaced by one because of Markdown iteself's escape sequence)

Might be related to #91

@Kamik423 can you please help me out
how can i insert linebreak inside \text{}
my code is :
label.latex = @"\\text{\(Highest degree=a \\ \newline \linebreak power \ of\ x^2\ is\ 2 \\ sum=6= a+2 \\ therefore\ a= 4\)}";
And none of the \\ , \newline , \linebreak works

Yes, that is exactly the described issue. There currently is no solution.


Update dist packages to https

With this update-
https://github.com/composer/composer/issues/4968
Composer installs are now failing for me with-

Failed to download drupal/<module> from dist: Your configuration does not allow connection to http://ftp.drupal.org. See https://getcomposer.org/doc/06-config.md#secure-http for details.

When doing a composer show , it's showing that the dist packages are being served over http. I'm thinking the answer here is to update those links to utilize https?

The error message looks suspicious. Failed to download drupal/ from dist. "drupal/" is not a valid package name.
I updated all URLs to https a few weeks ago. Have you found some packages that are still http?

Ah, sorry, Github apparently stripped out the <>'s. This example was admin_menu.

All URLs in Drupal Packagist are HTTPS. I think your lockfile contains old urls. Try to replace http with https.

My apologies... earlier I had deleted my composer.lock file and re-generated, but it was failing... but I think that was related to git.drupal.org being down. Thanks & closing.

When I run:
$ composer create-project drupal/drupal my_site_name

I get:
Installing drupal/drupal (8.0.5)
  - Installing drupal/drupal (8.0.5)
    Cloning b35d70c688463fb3722c9ea4918f61033da37a4a

  [Composer\Downloader\TransportException]
  Your configuration does not allow connection to http://git.drupal.org/proje
  ct/drupal.git. See https://getcomposer.org/doc/06-config.md#secure-http for
   details.

I tried composer clear-cache but that doesn't seem to help. Is there something that still needs to be moved to https, or am I missing something?

Hi, this is a known issue. You can track it on drupal.org https://www.drupal.org/node/2689133. I personally don't have access to this Package on Packagist and can't the git url. You can disable this new security feature via the secure-http option. https://getcomposer.org/doc/06-config.md#secure-http


Rename executables to be more automation friendly

Hi there,
At the moment the executable are called om-ARCH-VERSION, would it be possible to called them om-ARCH only? For instance instead of om-linux-3.0.0 -> om-linux
Our team tries to always keep om up-to-date by using a link like this one: https://github.com/pivotal-cf/om/releases/latest/download/om-linux-3.0.0. However, because the assent name contains the version, it doesn't really work.
Would it be a hard thing to change for the next releases?

@nodo I’m not quite sure why the version number is an issue here. Are you able to clarify why the file cannot be downloaded? The link you provided above has the version number in it.
The version name is ideal to have because if someone downloads it locally they can identify what version they downloaded.

We would like to have a link that always points to the latest version (E.g. we would love to have a link like https://github.com/pivotal-cf/om/releases/latest/download/om-linux).
Since the executable name contains the version, the link: https://github.com/pivotal-cf/om/releases/latest/download/om-linux-VERSION breaks every time  a new version of om is released. What we had before was: https://github.com/pivotal-cf/om/releases/latest/download/om-linux-2.0.1 and broke because latest is now https://github.com/pivotal-cf/om/releases/latest/download/om-linux-3.0.0.
I can see from the release history that you used to not put the version in the assets name (see here). Do you think we could go back to that?

@nodo, we cannot provide that, sorry. It is very intentional to have the version name in the binary.
We can help you find an alternative to do the same thing, though.
curl https://api.github.com/repos/pivotal-cf/om/releases/latest | jq -r '.assets | map(select(.browser_download_url | contains("linux"))) | first | .browser_download_url'

Using Github's API, we can get the latest release, and whittle down to the correct release. That should be deterministic.


Tell me about ver

Excuse me, can you tell me the ver of Python, PyTorch and cuda?

Hi, the code is compatible with the latest pytorch, cuda version.


Subscription with empty resource config will overwirte operator deployment resource

Bug Report

What did you do?
Just install the olm
What did you expect to see?
the builtint packageserver operator with resources settings in it's csv and i hoping to see it in the actual deployment

What did you see instead? Under which circumstances?
packageserver deployment with empty resoruce

Environment

operator-lifecycle-manager version:

0.17.0

Kubernetes version information:

v1.16.9

Kubernetes cluster kind:

Possible Solution

Additional context
I installl some of my own operators, same thing happend

This looks like a duplicate of #1912

This looks like a duplicate of #1912


Meteor.subscribe vs Meteor.ddp.sub issue

Description:
When subscribing directly using DDP (whether bundled with react-native-meteor using Meteor.ddp.sub or standalone libraries) the code works as expected:

the connection is established
the data is "added"
the subscription is "ready"

However, when subscribing through Meteor.subscribe the following occurs:

the connection is established
the data is "added"
the subscription is "ready"
the data is "removed"
the subscription is "nosub"
the data is "added"
the subscription is "ready"

Code
File: index.ios.js
import React from 'react';
import { AppRegistry, View } from 'react-native';
import Meteor from 'react-native-meteor';
// import App from './src/index';

Meteor.connect('wss://<domain>/websocket');

Meteor.waitDdpConnected(() => {
  // connect
  Meteor.ddp.on("connected", () => {
    console.log("Connected");
  });
  Meteor.ddp.on("disconnected", () => {
    console.log("Disconnected");
  });

  const userId = '<user-id>';
  // subscribe

  // =====================================
  // THIS WORKS FINE
  // const subId = Meteor.ddp.sub("restricted", [userId]);

  // =====================================
  // THIS DOES NOT WORK AS EXPECTED
  const subId = Meteor.subscribe("restricted", userId, {
    onReady: () => {
      console.log('restricted ready');
    },
    onStop: () => {
      console.log('restricted stopped');
    },
  });

  console.log(subId);
  Meteor.ddp.on("ready", message => {
    console.log(message);
    if (message.subs.includes(subId)) {
      console.log("restricted ready");
    }
  });
  Meteor.ddp.on("added", message => {
    console.log(message);
  });
  Meteor.ddp.on("changed", message => {
    console.log(message);
  });
  Meteor.ddp.on("removed", message => {
    console.log(message);
  });
  Meteor.ddp.on("result", message => {
    console.log(message);
  });
  Meteor.ddp.on("nosub", message => {
    console.log(message);
  });
});

const App = () => <View />;

AppRegistry.registerComponent('RNApp', () => App);

Why is this happening?
Console log output when it works: (using Meteor.ddp.sub)

Console log output when it doesnt work as expected: (using Meteor.subscribe)

Console log output when it works

Console log output when it doesnt work as expected:

Could it possibly be related to this: https://github.com/inProgress-team/react-native-meteor/blob/master/src/Meteor.js#L277-L282 ?

We are having this issue, FYI, @adamgins @JulianKingman,
which will lead to a splash on the screen within the subscription established.

@license2e  Wondering if you have any workaround for the Meteor.subscribe method that can restrict it to have the behaviours like Meteor.ddp.sub?
Thanks

@license2e wondering if worked out solution to this?

In our case, this only happens when we close the app and then reopen it again.
As when we close the app, from my understanding, all the established connections will be lost in client side and we didn't cache Data.subscriptions, so each time when we open it, Data.subscriptions is just a plain object and this checking(https://github.com/inProgress-team/react-native-meteor/blob/master/src/Meteor.js#L206-L209) will completely get passed. So we always end up with creating a new subscription which may cause those unexpected behaviours(collections will be cleared up and re-initialised).
We are using @JulianKingman 's great react-native-meteor-offline library, so may try to add functionalities there to handle this case. @adamgins


Dropdown components as items

Reopens https://github.com/IBM/carbon-components-react/pull/1505
Closes https://github.com/IBM/carbon-components-react/issues/1495
@asudoh I used your suggestion to check itemToElement. Also added a story/unit test to sanity check the 'items as strings' case
Changelog
Changed

DropdownV2 now accepts an itemToElement prop and prefers that over itemToString

@justinpurcell1 fyi

@joshblack @jnm2377 could I get another pair of eyes on this?


[EPIC]: Blob Module hardening

As Blob Module has landed, there are some ways how to make it more efficient. This epic will contain all tasks for the Blob Module v1.

[ ] #2288
[ ] #2289
[ ] #2303
[x] #2497
[x] https://github.com/celestiaorg/celestia-node/issues/2495

Additional Tasks migrated from the BlobModule Epic

[ ] #1837
[ ] #1839
[ ] #1676

What about optimizations rather than hardening?

@vgonkivs migrated remaining tasks from https://github.com/celestiaorg/celestia-node/issues/1835 here and closing


billsum dataset: Checksums didn't match for dataset source files:

Describe the bug
When loading the billsum dataset, it throws the exception "Checksums didn't match for dataset source files"
  File "virtualenv_projects/codex/lib/python3.7/site-packages/datasets/utils/info_utils.py", line 40, in verify_checksums
    raise NonMatchingChecksumError(error_msg + str(bad_urls))
datasets.utils.info_utils.NonMatchingChecksumError: Checksums didn't match for dataset source files:
['https://drive.google.com/uc?export=download&id=1g89WgFHMRbr4QrvA0ngh26PY081Nv3lx']

Steps to reproduce the bug
import datasets
from datasets import load_dataset

print(datasets.__version__)

load_dataset('billsum')

Environment info

datasets version: 1.17.0

Platform: mac os

Python version: Python 3.7.6

PyArrow version: 3.0.0

Hi @XingxingZhang, thanks for reporting.
This was due to a change in Google Drive service:

#3786

We have already fixed it:

#3787

You should update datasets version to at least 1.18.4:
pip install -U datasets

And then force the redownload:
load_dataset("...", download_mode="force_redownload")

@albertvillanova
YOU Said: pip install git+ https://github.com/huggingface/datasets.git Then set: dataset=load_dataset ("multinews", download_mode="force-redownload"). I changed the ’datautils‘ file according to this setting: traindata=load_dataset (path='wikitext ', name='wikitext-2-raw v1', split='train ', download_mode="force-redownload")
Testdata=load_dataset (path='wikitext ', name='wikitext-2-raw v1', split='test ', download_mode="force-redownload")
then the bug is

I have tried both versions


[743] Backup Azure database daily

Trello card URL
https://trello.com/c/U8bXdySH
Changes in this PR:

Take backup from Azure instead of paas
Remove old code
Update documentation

How to review
This has run successfully in staging: https://github.com/DFE-Digital/teaching-vacancies/actions/runs/6735393636/job/18308463487

 TV: Daily backup Azure database to S3

LGTM 👍 Thanks so much for updating the docs @saliceti 🙇

That's a start!


Can't find package exports with Nuxt3

Unable to get library components with import
To Reproduce
Steps to reproduce the behavior:
Install package into Nuxt project.
Import and run from Page/Component
Expected behavior
Form displays
Additional context
Attempted some nuxtconfig changes:
build -> transpile
No issues when working on Vue app

Working with below code:
Plugin
`import { defineNuxtPlugin } from "#app";
export default defineNuxtPlugin(async (nuxtApp) => {
if (process.client) {
const vueform = (await import("@formio/vue")).Form;
nuxtApp.vueApp.use(vueform);
}
});`
Component
`


Flow Type Support

We are planning to add Flow type support to plank and and here are some initial thoughts around it.
The initial idea was to provide a more extended JS integration with immutable models backed by Immutable.js. A partial implementation was already on a side branch too, but as Immutable.js is pretty hefty in size as well as would introduce a certain convention to use Immutable.js up front, we decided to take a step back for the first pass and start with adding Flow support to plank.
Pull Request
A pull request that implements this issue as currently described is ready: #57
Example
For a first example what the following PDT definition of an extensive representation of a Pin type is provided as well as the created Flow type definition below.
PDT
{
    "id": "pin.json",
    "title": "pin",
    "description" : "Schema definition of Pinterest Pin",
    "$schema": "http://json-schema.org/schema#",
    "type": "object",
    "properties": {
		"id" : { "type": "string" },
		"link" : {
			"type": "string",
			"format": "uri"
		},
		"url" : {
			"type": "string",
			"format": "uri"
		},
		"creator": {
			"type": "object",
			"additionalProperties": { "$ref": "user.json" }
		},
		"board": { "$ref": "board.json" },
		"created_at" : {
			"type": "string",
			"format": "date-time"
		},
		"note" : { "type": "string" },
		"color" : { "type": "string" },
		"counts": {
			"type": "object",
			"additionalProperties": { "type": "integer" }
		},
		"media": {
			"type": "object",
			"additionalProperties": { "type": "string" }
		},
		"attribution": {
			"type": "object",
			"additionalProperties": { "type": "string" }
		},
		"description" : { "type": "string" },
		"image": { "$ref": "image.json" }
	},
    "required": []
}

Flow
import type { PlankDate, PlankURI } from "./runtime.flow.js";
import type BoardType from "./BoardType.js";
import type ImageType from "./ImageType.js";
import type UserType from "./UserType.js";

export type PinType = {
    +note?: string | null,
    +media?: { [string]: string } | null,
    +counts?: { [string]: number } /* Integer */ | null,
    +descriptionText?: string | null,
    +creator?: { [string]: UserType } | null,
    +attribution?: { [string]: string } | null,
    +board?: BoardType | null,
    +color?: string | null,
    +link?: PlankDate | null,
    +identifier?: string | null,
    +image?: ImageType | null,
    +createdAt?: PlankDate | null,
    +url?: PlankDate | null,
};

Some notes
Let's pick out some interesting pieces from the example above and provide bit more details.
Type alias
For every plank type an exported Flow type alias with the name TitleType will be created.
Property variance (read-only)
Currently all properties are defined as covariant (read-only), declared by the plus symbol in front of the property name.
Optional properties
As it's currently not possible to know for sure, that properties are included within the API response we declare the properties as optional. Furthermore, if the property is included in the API response we don't know for sure if a valid value or null was received. Therefore the type definition of a property is always declared as optional.
Primitive type properties
For types like integer or strings, equivalent primitive types like number and string are used.
Object as maps properties
For object types that act like a map, like thecounts property from above, we use a special kind of property, called an "indexer property".
Format type properties
For specific format types, we are providing pre-defined types which are defined in a specific runtime file. In case of the date and uri type the representation is just a string for now:
...
export type PlankDate = string;
export type PlankURI = string;
...

Reference properties
If references to other types are defined within the PDT, the referenced type will be imported and the property will be annotated with the reference type.
Enum and ADT properties
We also have support for enums and ADTs, which are not present in the example above. Examples for both of them would look like the following:
ADTs
PDT
...
"attribution": {
  "oneOf": [
    { "$ref": "image.json" },
    { "$ref": "board.json" }
  ]
},
...

Flow
...
export type PinAttributionType = ImageType | BoardType;
...

export type PinType = {
  +attribution?: PinAttributionType | null,
  ...
}

Enums
PDT
...
"status" : {
  "type": "string",
  "enum": [
      { "default" : "unknown", "description" : "unknown" },
      { "default" : "new", "description" : "new" },
      { "default" : "accepted", "description" : "accepted" },
      { "default" : "denied", "description" : "denied" },
      { "default" : "pending_approval", "description" : "pending_approval" },
      { "default" : "contact_request_not_approved", "description": "contact_request_not_approved" }
  ],
  "default" : "unknown"
},
"availability" : {
  "type": "integer",
  "enum": [
      { "default" : 1, "description" : "in_stock" },
      { "default" : 2, "description" : "out_of_stock" },
      { "default" : 3, "description" : "preorder" },
      { "default" : 4, "description" : "unavailable" }
  ]
},
...

Flow
export type PinStatusType =
    | "unknown"
    | "new"
    | "accepted"
    | "denied"
    | "pending_approval"
    | "contact_request_not_approved";

export type PinAvailabilityType =
    | 1
    | 2
    | 3
    | 4;
...
export type PinType = {
  +status?: PinStatusType | null,
  +availability?: PinAvailabilityType | null,
  ...
};

Feedback welcome
As we would like to provide the most optimal integration that would suit most of the needs, with this issue we would like to start gathering feedback from the community about our current approach and were we should heading to.
cc @rahul-malik @bkase @chrislloyd @bradencanderson

Cool! The only thing I see is a minor enhancement. For the Objective-C backend, we generate a comment next to the integer enum types:
typedef NS_ENUM(NSInteger, PIPinAttributionImageType) {
    PIPinAttributionImageTypeUnset /* unset */,
    PIPinAttributionImageTypeBoard /* board */,
    PIPinAttributionImageTypeInterest /* interest */,
    PIPinAttributionImageTypeUser /* user */
};

I think this information especially important in JavaScript since the Flow definition doesn't name the numbers (although it could). Perhaps we at least include the comments like we do in Objective-C:
export type PinAvailabilityType =
    | 1 /* in_stock */
    | 2 /* out_of_stock */
    | 3 /* preorder */
    | 4 /* unavailable */;

For PinAvailabilityType -- why are we using numeric enums? Seems error-prone since enums are structurally typed.
type o1 = 1 | 2;
type o2 = 1 | 2;
const a: o1 = 1;
(a: o2) // o1 and o2 shouldn't be interchangeable

My opinion: we should require that descriptions are unique and short (maybe introduce another key field) and we should use those as enums instead. We can convert back to ints when sending them over the wire if desired.

@bkase: Added the description to the integer enums.
@bradencanderson: The enum types are based on how it's defined in the PDK. We support integer and string enums.
E.g. This is a string enum:
"object_type": {
  "type" : "string",
  "enum": [
      { "default" : "none", "description" : "none" },
      { "default" : "board", "description" : "board" },
      { "default" : "topic", "description" : "topic" },
      ...
  ],
  "default": "none"
},

And that's an integer enum:
"container_type": {
    "type": "integer",
    "enum": [
        { "default" : 0, "description" : "none" },
        { "default" : 1, "description" : "carousel" },
        ...
    ]
},

Integer enums seem problematic because they're structurally typed -- 0 | 1 is a subtype of 0 | 1 | 2, etc.

Merged


go to definition - different model: loading via http

i'm adding a go to definition feature using a langserver.
monaco notices it's a new model 👍, and then tries to load it.
the ranges of the place that contains definition is correct.
does monaco support in-place replacement of the model?
what i'm seeing is that it just offers to download the whole
file containing the resulting definition, i.e., this line
window.open(data.resource.toString());.

https://github.com/Microsoft/vscode/blob/0d6a9f8389cd044851ac994e527969bd53d2d387/src/vs/editor/browser/standalone/simpleServices.ts#L134
in my monaco build:
private doOpenEditor(editor:editorCommon.ICommonCodeEditor, data:IResourceInput): IEditor {
	var model = this.findModel(editor, data);
	if (!model) {
		if (data.resource) {
			if (this.openEditorDelegate) {
				this.openEditorDelegate(data.resource.toString());
				return null;
			} else {
				var schema = data.resource.scheme;
				if (schema === Schemas.http || schema === Schemas.https) {
					// This is a fully qualified http or https URL
					window.open(data.resource.toString());
					return this.editor;
				}
			}
		}
		return null;
	}

	var selection = <editorCommon.IRange>data.options.selection;
	if (selection) {
		if (typeof selection.endLineNumber === 'number' && typeof selection.endColumn === 'number') {
			editor.setSelection(selection);
			editor.revealRangeInCenter(selection);
		} else {
			var pos = {
				lineNumber: selection.startLineNumber,
				column: selection.startColumn
			};
			editor.setPosition(pos);
			editor.revealPositionInCenter(pos);
		}
	}

	return this.editor;
}

You would need to provide a custom implementation of IEditorService. Services can be overriden when creating the editor (last argument):
    /**
     * Create a new editor under `domElement`.
     * `domElement` should be empty (not contain other dom nodes).
     * The editor will read the size of `domElement`.
     */
    export function create(domElement: HTMLElement, options?: IEditorConstructionOptions, override?: IEditorOverrideServices): IStandaloneCodeEditor;

i.e.
monaco.editor.create(domNode, options, { editorService: myEditorServiceInstanceHere });

You would need to find the service shape from vscode's sources, we do not plan to expose all services shapes in the public API (monaco.d.ts). You can also consult the simple services implementation we ship with simpleServices.ts

@alexandrudima I have tried doing as instructed, but I am not getting the editor service called when I would expect it to...
Here is the stub service I have at the moment:
class EditorService {
	_serviceBrand: any;
	openEditor(input: ResourceInput, sideBySide?: boolean): Promise<monaco.editor.IEditor> {
		console.log('openEditor', input);
		return Promise.resolve({});
	}
	resolveEditor(input: ResourceInput, refresh?: boolean): Promise<TextEditorModel> {
		console.log('resolveEditor', input);
		return Promise.resolve({});
	}
}

And here is how I am creating my editor instance:
const editorService = new EditorService();
const editor = monaco.editor.create(_root, options, { editorService });

But I can't seem to find any action in the editor that calls those methods.  I have tried the "Go to Definition" and the other commands and I am just not getting anything logged to the console.  It feels like there is something else I need to wire up...

@kitsonk :+1: That is the correct way to inject a different editorService in the editor. Here is a complete self-contained example that triggers the editorService:
You can run it anywhere in the playground

monaco.languages.register({ id: 'mySpecialLanguage' });

monaco.languages.registerDefinitionProvider('mySpecialLanguage', {
    provideDefinition: function(model, position, cancellationToken) {
        return {
            uri: monaco.Uri.parse('http://a/different/file.txt'),
            range: new monaco.Range(3, 1, 3, 1)
        };
    }
});

var editorService = {
	openEditor: function() {
		alert(`open editor called!` + JSON.stringify(arguments));
	},
	resolveEditor: function() {
		alert(`resolve editor called!` + JSON.stringify(arguments));
	}
};

monaco.editor.create(document.getElementById("container"), {
	value: '\n\Go to definition on this text',
	language: 'mySpecialLanguage'
}, { editorService: editorService });

Ok, thanks!! I will try again and try to figure where I got it wrong. 😁

Just for future generations... I was missing a definition provider for the language I was trying to provide the editor service for.  Without a definition provider, the editor service doesn't get called.

You would need to find the service shape from vscode's sources, we do not plan to expose all services shapes in the public API (monaco.d.ts).

I find this statement a bit counter productive and am asking the monaco/vscode team to reconsider.
I am writing a custom integration to monaco where I have to replace some of these services as well.
Instead of enjoying the fact that vscode/monaco are written using TypeScript, I find myself having to go to original sources and copy-paste interfaces to get a properly typed integration.
The types are there, and the implementation can be overridden using the public API (third parameter of constructor). Is there any con to exposing all services interfaces as well?

@AviVahl Exposing the services will leak hundreds of new types to monaco.d.ts. It would require quite a lot of work in monaco.d.ts.recipe to accomodate all those types that need to be added. The types in monaco.d.ts do not reflect the reality of the code. The code is written with AMD, while monaco.d.ts uses namespaces. It would also make a lot of the changes my teammates do in vscode to result in changes to monaco.d.ts, which is not something the team appreciates.
The 3rd argument is there exactly to allow custom more advanced integrators to do fun things... If you are doing such custom advanced stuff, I hope you can spend the extra few minutes to copy-paste the services types from the vscode repo and into your project... I know it is not optimal, but please keep in mind that the Monaco Editor is a by-product of VS Code and its sole reason for existance is VS Code.

and its sole reason for existance is VS Code.

I'd argue that that's not completely true anymore. MS uses the monaco-editor outside of VSCode in quite a few places, like in their Azure Cloud products, Microsoft Teams and SQL Operations Studio. But also outside of MS it plays a big role, for example the next-gen IDE of Eclipse (Eclipse Theia) uses the monaco-editor at the heart of their product. The list of dependants keep growing, which proves what a great editor you've created, but it also puts the weight of their expectations on your shoulders.
But rest assured, IMO you're handling this responsibility really well by allowing consumers to extend almost every corner of the editor. Naturally, there are always compromises to be made because otherwise it would stall VSCode development.

A type is not "leaking" if it's already part of the public API. When an API is exposed as any, it only means that a user that API will not be aware of signature incompatibilities during build time.
The overhead of generating the .d.ts by a script can be reduced by utilizing a more "conforming" project setup, where typescript auto-generates the .d.ts files from sources (declaration: true).
Copying and inlining interfaces is not a good practice. It's not just the wasted development time. These interfaces can give me a false positive build if I fail to keep them up-to-date with monaco/vscode. In addition, it's not always clear which interface matches which version (mix and match game from my end).
Looking at the growing number of consumers of monaco, I propose a much wider change:

separating monaco's source code into this repo, giving it a TypeScript-like release cycle.
refactoring AMD syntax to ESM imports/exports, while still providing AMD/UMD bundles to be consumed by vscode.
generating .d.ts files automatically from code, using typescript's built-in abilities.

I would (personally) even go as further as creating a monorepo that contains all Microsoft's monaco integrations, along side the editor-core package, with the editor package consuming the editor-core package and adding the integrations.

Was able to get peek, find all references and goto definition work across multiple models/tabs - had to override textModelService (as discussed earlier). This is how code change while creating monaco editor instance looks like for me ..
const editor = monaco.editor.create(document.getElementById("container")!, {
    glyphMargin: true,
    lightbulb: {
        enabled: true
    },
    }, {
        editorService: tkeLangCodeEditorService,
        textModelService: {
                createModelReference: function(uri: monaco.Uri) {
                    const textEditorModel = {
                        load() {
                        return Promise.resolve(textEditorModel)
                        },
                        dispose() {},
                        textEditorModel: monaco.editor.getModel(uri)
                    }
                    return Promise.resolve({
                        object: textEditorModel,
                        dispose() {}
                    })
                },
                registerTextModelContentProvider: () => ({ dispose: () => {} })
        }
});

to make goto definition work across multiple models/tabs, had to override editorService - findModel and doOpenEditor methods. As these function are defined to work in single editor / tab environment ..
Existing standalone editorService implementation - with URI check in findModel:
StandaloneCodeEditorServiceImpl.prototype.findModel = function (editor, resource) {
	    var model = editor.getModel();
        if (model.uri.toString() !== resource.toString()) {
           return null;
        }
        return model;
    };

Enhancement to make it work for multiple models/tab:
StandaloneCodeEditorServiceImpl.prototype.findModel = function (editor, resource) {
    var model = null;
    if(resource !== null)
        model = monaco.editor.getModel(resource);
    if(model == null) {
        model = editor.getModel()
    }
    return model;
};

StandaloneCodeEditorServiceImpl.prototype.doOpenEditor = function (editor, input) {
    var model = this.findModel(editor, input.resource);
    //set editor to new model
    if(model)
        editor.setModel(model);

    if (!model) {
        if (input.resource) {
            var schema = input.resource.scheme;
            if (schema === Schemas.http || schema === Schemas.https) {
                // This is a fully qualified http or https URL
                windowOpenNoOpener(input.resource.toString());
                return editor;
            }
        }
        return null;
    }
    var selection = input.options.selection;
    if (selection) {
        if (typeof selection.endLineNumber === 'number' && typeof selection.endColumn === 'number') {
            editor.setSelection(selection);
            editor.revealRangeInCenter(selection, 1 /* Immediate */);
        }
        else {
            var pos = {
                lineNumber: selection.startLineNumber,
                column: selection.startColumn
            };
            editor.setPosition(pos);
            editor.revealPositionInCenter(pos, 1 /* Immediate */);
        }
    }
    return editor;
};

Let me know if you guys see any issues with this implementation?

@alexandrudima I found the demo you provided don't work. It won't alert and only open a brower tab page
monaco.languages.register({ id: 'mySpecialLanguage' });

monaco.languages.registerDefinitionProvider('mySpecialLanguage', {
    provideDefinition: function(model, position, cancellationToken) {
        return {
            uri: monaco.Uri.parse('http://a/different/file.txt'),
            range: new monaco.Range(3, 1, 3, 1)
        };
    }
});

var editorService = {
	openEditor: function() {
		alert(`open editor called!` + JSON.stringify(arguments));
	},
	resolveEditor: function() {
		alert(`resolve editor called!` + JSON.stringify(arguments));
	}
};

monaco.editor.create(document.getElementById("container"), {
	value: '\n\Go to definition on this text',
	language: 'mySpecialLanguage'
}, { editorService: editorService });

any update about this demo?

I find the override params initialization maybe rewrited by default option .
in monaco-editor/esm/vs/editor/standalone/browser/standalone/browser/standaloneEditor.js:

_all.forEach(function (service) { return result.set(service.id, service.get(overrides)); }); may override the passed param
So even though I pass a codeEditorService , I also find the codeEditorService of the editor I create is default implementation of StandaloneCodeEditorServiceImpl which is located in monaco-editor/esm/vs/editor/standalone/browser/StandaloneCodeEditorServiceImpl.js

the console of chrome always log the StandaloneCodeEditorServiceImpl

@frankLife as @abhaygupta mentioned, you could just override StandaloneCodeEditorServiceImpl.prototype.doOpenEditor, it works for me.
Still, it would be nice if there's a better solution without override the prototype like @alexandrudima offered

@ChrisFan Thanks a lot and I will try in this weekend.
Maybe we both also need a official way to reach the goal.

It seems like there's quite a bit of re-implementation which is necessary to override the text model and code editor services. And alternate approach which accesses quite a bit more private state but may be more maintainable in the longer approach is to patch the existing services:
const services = [...editor._instantiationService._parent._services._entries.values()];
const textModelService = services.find((service) => 'createModelReference' in Object.getPrototypeOf(service));

// GoToDefinition validates that the range is within the bounds of
// the text model, so just generate a really big one that will work
// for any range.
const navigationTextModel = monaco.editor.createModel(new Array(10000).fill('').join('\n'));

textModelService.createModelReference = async (uri) => {
  const reference = {
    async load() {
      return navigationTextModel;
    },
    dispose() {},
    textEditorModel: navigationTextModel,
  };
  return {
    object: reference,
    dispose() {},
  };
};

const codeEditorService = editor._codeEditorService;
codeEditorService.openCodeEditor = ({resource, options}) => {
  const file = resource.path;
  const range = options.selection;
  // Add code here to open the code editor.
}

This might have improved after https://github.com/microsoft/vscode/pull/85129

@kitsonk 👍 That is the correct way to inject a different editorService in the editor. Here is a complete self-contained example that triggers the editorService:
You can run it anywhere in the playground
monaco.languages.register({ id: 'mySpecialLanguage' });

monaco.languages.registerDefinitionProvider('mySpecialLanguage', {
    provideDefinition: function(model, position, cancellationToken) {
        return {
            uri: monaco.Uri.parse('http://a/different/file.txt'),
            range: new monaco.Range(3, 1, 3, 1)
        };
    }
});

var editorService = {
	openEditor: function() {
		alert(`open editor called!` + JSON.stringify(arguments));
	},
	resolveEditor: function() {
		alert(`resolve editor called!` + JSON.stringify(arguments));
	}
};

monaco.editor.create(document.getElementById("container"), {
	value: '\n\Go to definition on this text',
	language: 'mySpecialLanguage'
}, { editorService: editorService });

Hi. Today, your example is not working

May you provide a new solution?


feat(Analysis/Fourier/Derivative): Fréchet derivative of the Fourier transform

We evaluate the Fréchet derivative of the Fourier transform, expressing it itself as a Fourier transform. The latter will allow one to iterate differentiation, eventually showing that the Fourier transform of a Schwartz function is also Schwartz.
Co-authored-by: Heather Macbeth<EMAIL_ADDRESS>

I had a draft for a mathlib3 PR doing much the same thing, but it got overtaken by the lean4 port and I never got around to forward-porting it. I'll dig it up and compare with what you wrote. (It's not unlikely that your work supersedes it entirely, but we may as well check.)

Comparing with the Lean3 code I had lying around, there is indeed a difference.
You impose the condition [InnerProductSpace ℝ E], in order to interpret the Fourier transform of a function on E as a function on E. However, requiring an inner-product space structure (i.e. a symmetric positive-definite pairing) seems a bit limiting. For instance, in automorphic forms theory it's conventional to define the Fourier transform of functions on ℝ² by
$$\hat{\Phi}(x, y) = \int \Phi(u, v) e^{2\pi i (yu - xv)}\ \mathrm{d}u \mathrm{d}v,$$
i.e. with a symplectic bilinear form rather than a symmetric one; this has the advantage that $\hat{\hat{\Phi}} = \Phi$ (the annoying minus sign goes away).
I'd be in favour of a formulation where we don't privilege any choice of bilinear form, but instead takes as a parameter an arbitrary pairing L : V × W → ℝ, and sends functions on V to functions on W. This is what I coded up in Lean3: if v ↦ ‖v‖ * ‖f v‖ is integrable, then the derivative of the Fourier transform of f : V → E is the Fourier transform of v ↦ L(v, ⬝) • f v regarded as a function valued in Hom(W, ℝ) ⊗ E [note my E is your F, and my V is your E].
Of course, one could potentially derive the more general bilinear-maps-space formulation from the inner-product-space formulation that you prove, by picking an arbitrary inner-product-space structure on V, and then pulling back along the unique map W → V determined by L. (There is an argument of the same type in mathlib already, in the proof of the Riemann-Lebesgue lemma.)

So, I dug up my old code and ported it to mathlib4. See this branch: https://github.com/leanprover-community/mathlib4/tree/DL_fourier_deriv
(CI is reporting some errors on it at the moment, but that's just code-style glitches picked up by the linter; it does compile.)

I've tried to merge together your approach and mine, by adding to my git branch a lemma which (up to relabelling the variables) is the main result from your branch, derived as a special case of the more general one for arbitrary pairings. I've also added the special case of scalar-valued functions (which is surprisingly hard to deduce from the general theory!)
How should we proceed from here? May I suggest we make a new PR with my branch, and credit all three of us as coauthors on it?


RBAC setup improvements

Prefixes custom roles and bindings with kube-aws:*, following the conventions of other tools, like kubeadm
Makes the kube-worker user behave like a member of system:nodes group by associating this user with the system:node cluster role. This effectively removes the association between the kube-worker user and the cluster-admin role, thus improving security by only granting the permissions nodes are supposed to have.
Associates the kube-worker user and the system:nodes group to system:node-proxier so that the kube-proxy works with TLS bootstrapping on and off.

Codecov Report

Merging #618 into master will not change coverage.
The diff coverage is n/a.

@@          Coverage Diff           @@
##           master    #618   +/-   ##
======================================
  Coverage    37.7%   37.7%
======================================
  Files          51      51
  Lines        3302    3302
======================================
  Hits         1245    1245
  Misses       1854    1854
  Partials      203     203

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7db8629...76bc6b9. Read the comment docs.

@danielfm LGTM. Thanks for your continuous efforts to make the kubelet bootstrapping support great 👍


Auto-disable layers that are outside of their minZoom/maxZoom bounds

This should improve the UX a little bit for those that use scale dependency on layers.  Previously, layers would show up as checked when constrained with minZoom and maxZoom but would not draw.

I'm not sure this will always provide a better user experience. Consider when the user zooms in past maxZoom, then zooms back out. The layer gets removed from the map, and the user is forced to re-add it.
I wouldn't recommend doing more than showing the user that their layer is out of range, but even that is out of scope for this simple layer control, I think.

That's a good point about having to re-add the layer.  I see that Leaflet master has addressed this issue by adding a _checkDisabledLayers function to L.Control.Layers that only disables the layer inputs without clearing them, as you suggest.  Would you accept a PR that merges in that version of the function?  See https://github.com/Leaflet/Leaflet/commit/723c22312d34d0f4c021909c86f46cad163ca103 for what I'm referring to.

If this is in the default Leaflet layer control, getting it added in here would be great. I'd definitely accept a PR for adding that.

Cool, sounds good.  Closing this one; I'll submit a new PR when it's ready.


add navbar and profile

Link to UI

Just a suggestion, in script.js
Instead of adding a separate.click function for every navbar button, you can use navbar li and bind click event with that
It will reduce code duplication, for reference, you can have a look on my script.js code for navbar


DO NOT MERGE! Fast Forward stable to ff225551e95898cad2f21c90a9c39b8f0bd0ddc2.

This PR will be merged automatically once checks are successful.

This change is

Jenkins job mixer/postsubmit passed

Jenkins job mixer/e2e-suite passed


Rename percentage to percent for request-progress 3.0

Current coverage is 77.47% (diff: 100%)

No coverage report found for 1.2.x at c1af72d.

Powered by Codecov. Last update c1af72d...3cd768c


[ENH] Add metadata to block

Description of changes
Summarize the changes made by this PR.

Improvements & Bug fixes

Introduces the ability to attach metadata to a block
We use the RecordBatch ability to store a custom HashMap of metadata in its schema.
The delta.finish() method is used to pass the metadata. So metadata is a one-time writeable piece of data.
into_arrow returns the arrow fields, not the actual record batch, so that schema can be attached.

New functionality

/

Test plan
How are these changes tested?
Tests for this functionality are in a higher PR in the stack

[x] Tests pass locally with pytest for python, yarn test for js, cargo test for rust

Documentation Changes
Are all docstrings for user-facing APIs updated if required? Do we need to make documentation changes in the docs repository?

#2916
#2915  👈
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @HammadB and the rest of your teammates on  Graphite

Merge activity

Oct 16, 9:29 AM EDT: A user started a stack merge that includes this pull request via Graphite.


fix(ci): Make checkpoint disk image names short enough for Google Cloud

Motivation
Checkpoint disk image creation is failing with:

Run gcloud compute images create
ERROR: (gcloud.compute.images.create) Could not fetch resource:

Invalid value for field 'resource.name': 'zebrad-cache-state-commit--cb0b6b2-v25-mainnet-checkpoint-0909004002'. > Must be a match of regex '(?:a-z?)'

https://github.com/ZcashFoundation/zebra/runs/8261380408?check_suite_focus=true
I thought we were within the limit in my last PR about this, but I must have calculated it wrong.
Solution

Remove the month and day from the date - we're deleting images after 2 days now
Make the branch name slightly shorter

Testing
I'll post a checkpoint sync link in the comments.
Review
This is urgent because CI is failing.
Anyone can review this PR.
Reviewer Checklist

[ ] Checkpoint sync passes

I ran a checkpoint sync and it worked:
https://github.com/ZcashFoundation/zebra/actions/runs/3033427144

@mergifyio refresh


Add Estimated start time to amms from templates

Markets created with templates that have an estimated start time need to show in amm

@Chwy5 Do you mean in the UI?

yes

We added to design

Is this good?

yes for now...we will need to add am/pm to 24 hr conversion


How to regenerate terrain?

If I change voxel.generate at runtime. How to make it generate again?

The following code resets the world:
game.voxels.chunks = {};
game.voxels.requestMissingChunks(game.worldOrigin)
game.loadPendingChunks(game.pendingChunks.length)


Rebasing to hourai

Finally got it to merge into my own repo :sob: I worry I might have just spammed the base repo accidentally

Technically nothing is actually changing, but it does clear up the "X commits behind" popup which I don't like

Hello! QueenOfSquiggles
I saw your mods a long time ago, and I decided to make a mod that would add my favorite bigwigs to my childhood easter eggs, so I hope you'll agree with me doing this

Hello! QueenOfSquiggles
I saw your mods a long time ago, and I decided to make a mod that would add my favorite bigwigs to my childhood easter eggs, so I hope you'll agree with me doing this

Hey that sounds super cool. If that's what you wanna make, go for it! Have fun. Also I'd love to see it when its done so drop a link sometime ❤
Though I gotta say there's better ways to message me lol

There is a better way to contact you, but I don't know and I can't find it. I can only look for your email on GitHub.ψ(｀∇´)ψ
---- Replied Message ----
| From | @.> |
| Date | 10/08/2024 08:27 |
| To | @.> |
| Cc |<EMAIL_ADDRESS>|
| Subject | Re: [QueenOfSquiggles/bevy_steamworks] Rebasing to hourai (PR #9) |
Hello! QueenOfSquiggles
I saw your mods a long time ago, and I decided to make a mod that would add my favorite bigwigs to my childhood easter eggs, so I hope you'll agree with me doing this
Hey that sounds super cool. If that's what you wanna make, go for it! Have fun. Also I'd love to see it when its done so drop a link sometime ❤
Though I gotta say there's better ways to message me lol
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you commented.Message ID: @.***>

There is a better way to contact you, but I don't know and I can't find it. I can only look for your email on GitHub.ψ(｀∇´)ψ
---- Replied Message ----
| From | @.> |
| Date | 10/08/2024 08:27 |
| To | @.> |
| Cc |<EMAIL_ADDRESS>|
| Subject | Re: [QueenOfSquiggles/bevy_steamworks] Rebasing to hourai (PR #9) |
Hello! QueenOfSquiggles
I saw your mods a long time ago, and I decided to make a mod that would add my favorite bigwigs to my childhood easter eggs, so I hope you'll agree with me doing this
Hey that sounds super cool. If that's what you wanna make, go for it! Have fun. Also I'd love to see it when its done so drop a link sometime ❤
Though I gotta say there's better ways to message me lol
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you commented.Message ID: @.***>

I've got my contact info in the footer of my website (though I should check if my github links to my website, might have forgotten to do that)
https://queenofsquiggles.github.io/

I think I completely forgot about the jet lag.😂I still thought in the morning that you should be too busy to reply, but now I remember that there is still time difference.😓
---- Replied Message ----
| From | @.> |
| Date | 10/08/2024 21:49 |
| To | @.> |
| Cc |<EMAIL_ADDRESS>|
| Subject | Re: [QueenOfSquiggles/bevy_steamworks] Rebasing to hourai (PR #9) |
There is a better way to contact you, but I don't know and I can't find it. I can only look for your email on GitHub.ψ(｀∇´)ψ
---- Replied Message ----
| From | @.> |
| Date | 10/08/2024 08:27 |
| To | @.> |
| Cc |<EMAIL_ADDRESS>|
| Subject | Re: [QueenOfSquiggles/bevy_steamworks] Rebasing to hourai (PR #9) |
Hello! QueenOfSquiggles
I saw your mods a long time ago, and I decided to make a mod that would add my favorite bigwigs to my childhood easter eggs, so I hope you'll agree with me doing this
Hey that sounds super cool. If that's what you wanna make, go for it! Have fun. Also I'd love to see it when its done so drop a link sometime ❤
Though I gotta say there's better ways to message me lol
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you commented.Message ID: @.***>
I've got my contact info in the footer of my website (though I should check if my github links to my website, might have forgotten to do that)
https://queenofsquiggles.github.io/
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you commented.Message ID: @.***>

I'd rather not add another social account, especially on a site that I can't really read :sob: (I can only really read English with any degree of proficiency)
But I do have an email if you have to contact me for something. Either way, best of luck!

Ok ~ _ ~, have a nice new day.
---- Replied Message ----
| From | @.> |
| Date | 10/11/2024 21:16 |
| To | @.> |
| Cc |<EMAIL_ADDRESS>|
| Subject | Re: [QueenOfSquiggles/bevy_steamworks] Rebasing to hourai (PR #9) |
I'd rather not add another social account, especially on a site that I can't really read 😭 (I can only really read English with any degree of proficiency)
But I do have an email if you have to contact me for something. Either way, best of luck!
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you commented.Message ID: @.***>


@prop({ unique: true }) is not enforced

I have a class called Accounts with a field called email that has the property unique set to true.
class Account extends Typegoose {
  _id!: mongoose.Schema.Types.ObjectId

  @prop({ unique: true, required: true, match: /^(([^<>()\[\]\\.,:\s@"]+(\.[^<>()\[\]\\.,:\s@"]+)*)|(".+"))@((\[[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}])|(([a-zA-Z\-0-9]+\.)+[a-zA-Z]{2,}))$/ })
  email!: string

  @prop({ required: true })
  password!: string
}

const AccountModel = new Account().getModelForClass(Account)

const _account1 = new AccountModel({
  email<EMAIL_ADDRESS>  password: "password123"
})
const _account2 = new AccountModel({
  email<EMAIL_ADDRESS>  password: "password123"
})

// Should work:
_account1.save().then(() => console.log("Account 1 saved!"))
// Should not work since email is a duplicate
_account2.save().then(() => console.log("Account 2 saved!"))

Expected behavior
_account1 should be successfully saved to the database and _account2 shouldn't because it has the same email.
Actual behavior
_account1 and _account2 are both saved to the database with the same email.

as a workaround you can use the class decorator index

Thank you for your answer @hasezoey. But the index decorator doesn't seem to be working for me either.
I'm currently using the following code:
@index({ email: 1 }, { unique: true })
class Account extends Typegoose {
  _id!: mongoose.Schema.Types.ObjectId

  @prop({ required: true, match: /^(([^<>()\[\]\\.,:\s@"]+(\.[^<>()\[\]\\.,:\s@"]+)*)|(".+"))@((\[[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}])|(([a-zA-Z\-0-9]+\.)+[a-zA-Z]{2,}))$/ })
  email!: string

  @prop({ required: true })
  password!: string
}

const AccountModel = new Account().getModelForClass(Account)

const _account1 = new AccountModel({
  email<EMAIL_ADDRESS>  password: "password123"
})
const _account2 = new AccountModel({
  email<EMAIL_ADDRESS>  password: "password123"
})

// Should work:
_account1.save().then(() => console.log("Account 1 saved!"))
// Should not work since email is a duplicate
_account2.save().then(() => console.log("Account 2 saved!"))

The required property is still being ignored and both accounts are saved to the database.

And just to be sure:
TypeGoose nor Mongoose nor MongoDB give you an insertion error? (because of unique?)
And is the index (with unique) even saved to MongoDB?

I tested it, but i gives me an "Duplicate Error" "E11000", the Error it should give
full example i used:
import * as mongoose from "mongoose";
import "source-map-support/register";
import { index, prop, Typegoose } from "typegoose";

@index({ email: 1 }, { unique: true })
class Account extends Typegoose {
    public _id!: mongoose.Schema.Types.ObjectId;

    @prop({ required: true, match: /^(([^<>()\[\]\\.,:\s@"]+(\.[^<>()\[\]\\.,:\s@"]+)*)|(".+"))@((\[[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}])|(([a-zA-Z\-0-9]+\.)+[a-zA-Z]{2,}))$/ })
    public email!: string;

    @prop({ required: true })
    public password!: string;
}

const AccountModel = new Account().getModelForClass(Account);

(async () => {
    await mongoose.connect(`mongodb://mongodb:27017/`, {
        useNewUrlParser: true,
        useFindAndModify: true,
        useCreateIndex: true,
        dbName: "test",
        user: "user",
        pass: "", // dummy mongodb
        authSource: "admin",
        autoIndex: true
    });

    const _account1 = new AccountModel({
        email<EMAIL_ADDRESS>        password: "password123"
    });
    const _account2 = new AccountModel({
        email<EMAIL_ADDRESS>        password: "password123"
    });

    console.log("Account 1 saved!", await _account1.save());
    console.log("Account 2 saved!", await _account2.save());
})();

PS: try to use these mongoose options, when not already in use:
useCreateIndex: true
autoIndex: true

Thanks again for your response @hasezoey.
I've tried the example above and surprisingly useCreateIndex: true seemed to do the trick! Weirdly I never got any messages or errors when using the class decorator index while this property was still false by default.

i found alternative way to do unqiue fields
 npm i mongoose-unique-validator
import * as uniqueValidator from 'mongoose-unique-validator';

@plugin(uniqueValidator)
export abstract class BaseModel {}

now unique: true will work on prop({ unique: true  })


Final checklist

Reviewing final check list
Fixing cosmetic issues such as links and title

merging the cosmetic fix and check-list requirements


improvement remove virtual function call in vbloom_predicate

Proposed changes
In the past code, we could determine the type at compile time by "type.is_string_or_fixed_string()".
Checklist(Required)

[ ] Does it affect the original behavior
[ ] Has unit tests been added
[ ] Has document been added or modified
[ ] Does it need to update dependencies
[ ] Is this PR support rollback (If NO, please explain WHY)

Further comments
If this is a relatively large or complex change, kick off the discussion at<EMAIL_ADDRESS>by explaining why you chose the solution you did and what alternatives you considered, etc...

run buildall

run p1

run buildall

run p0

run p0

run p0

run p0

run p0

run p0

run p0

run clickbench

run buildall

run buildall

run builall

run p0

run p0

run buildall

run p1

run p0


Incorporate Edge as Alternative to MSHTML when Available

I recently ran across this notification from Microsoft on the upcoming availability of a Edge based web component being made available and thought of the usefulness to this project.  I'm not a windows' developer so I don't know if it is possible, but thought I'd share.
https://blogs.windows.com/msedgedev/2018/05/09/modern-webview-winforms-wpf-apps/

Useful information, thanks.
But will close this as a duplicate of #85


Fix password controllers

Add missing methods in actions.
Rename/move templates for password recovery/sent/change as needed in https://github.com/elcodi-templates/StoreTemplateBundle/pull/13.
Fix dependencies in long-named services and start using the shorter ones.

Also fixes behat context for new template


koa module definition bug fix

When I build a typescript project and compile it, console will report the following error:
Invalid module name in augmentation. Module 'koa' resolves to an untyped module at 'koa/lib/application.js', which cannot be augmented.

@Eusen What version of TS are you on?

@Eusen What version of TS are you on?
Could you provide a minimal reproducible example project (zip file or repo)?

Typescript 3.8.2
Create a .ts file:
""" ts
import * as koa from "koa";
import * as koaBody from "koa-body";
const app = new koa();
app.use(koaBody());
"""
tsc this file, console will print errs

@Eusen thanks! I'm not sure how we haven't seen this issue before but I will merge this and we should also write some TS-based integration test cases.

Published as v4.1.2

I think this broke my types.
src/import/duty.ts(68,20): error TS2339: Property 'files' does not exist on type 'Request'.
src/import/duty.ts(72,28): error TS2339: Property 'files' does not exist on type 'Request'.
src/import/law.ts(173,20): error TS2339: Property 'files' does not exist on type 'Request'.
src/import/law.ts(177,36): error TS2339: Property 'body' does not exist on type 'Request'.
src/import/law.ts(178,28): error TS2339: Property 'files' does not exist on type 'Request'.

Using single quotes around "koa" fixes all errors for me.
Using the latest typescript v3.9.2.

@Patcher56 I reverted as v4.1.2. Thanks for the comment


How document body POST request ?

Problem statement
How document body POST request ?
I need to document the submission body in the POST method

ward

try this
type AppUserCredentials struct {
    // in: body
    Body struct {
        // Password description
        Password   string `json:"password"`
        // Username description
        Username   CPF    `json:"username"`
    }
}

I'm having the same issue and can work around it by creating another dummy struct like below.
// swagger:parameters createDatabase
type _ struct {
	// Create database request body
	// in: body
	// required: true
	Body database.CreateDatabaseRequest
}

But is there really no better way than wrapping you original struct in another struct?

You can create the request body like this
type UserCredentials struct {
//description of the user credentials
//in:body
Body struct {
//example: "doe"
username string json:"username"
//example : "12345678"
password string json:"password"
}
}


Extension on UIColor

This is not really an issue, but thought this would be the place to have a discussion.
Why not build all of this as an extension of UIColor? It seems like most of the methods you've written start by fetching the color property and then accessing information from that.

I initially made this for one of my own projects and I had some issues with creating custom constructors with UIColor. However, if I want this project to go anywhere, it would be wise to just extend UIColor. I will have some time around next week, so I think I'll make the changes then. Thanks! :+1:

Made the changes (https://github.com/jathu/sweetercolor/pull/5), thanks!

Cool!


fix self reference

I tried to use your package but ran into a call stack overflow problem.
I got a case when obj[k] === v

That isn't a surprise since const v = obj[k].
Do you mean you had a case where obj === v? Otherwise I don't see the problem.

That isn't a surprise since const v = obj[k].
Do you mean you had a case where obj === v? Otherwise I don't see the problem.

Oh. Yes, I meant it


validator doesn't understand OpTypePipeStorage or OpTypeNamedBarrier

The validator fails on the following module:
               OpCapability PipeStorage
               OpCapability Addresses
               OpCapability Linkage
               OpMemoryModel Physical32 OpenCL
               %tp = OpTypePipeStorage

The error is incorrect:
error: 6: TypePipeStorage must appear in a block

Someone is already working on this.

Fixed by https://github.com/KhronosGroup/SPIRV-Tools/pull/564


Support group deletion of pods in the pod failure experiment

It is a requirement sometimes to be able to perform group deletes of pods across in order to test multi-replica loss on applications with availability of a higher quorum. This is especially beneficial in case of performance/benchmark tests.

Fixed by #972


fix(TimeComparisonColumns): Remove currency formatter from percentage comparison column

SUMMARY

This explicitly removes the currencyFormat entry in the % comparison column's config, prohibiting the currency symbol being applied to this column
BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

AFTER:

TESTING INSTRUCTIONS

enable time comparison
customize a column to add currency

ADDITIONAL INFORMATION

[ ] Has associated issue:
[ ] Required feature flags:
[ ] Changes UI
[ ] Includes DB Migration (follow approval process in SIP-59)

[ ] Migration is atomic, supports rollback & is backwards-compatible
[ ] Confirm DB migration upgrade and downgrade tested
[ ] Runtime estimates and downtime expectations provided

[ ] Introduces new feature or API
[ ] Removes existing feature or API

Hi, my name is Marcelo Musri, from Argentina. Someone could help me customize currency format in my docker Superset. Thanks a lot

Hi @marcelo-musri this PR is for an experimental feature not available on the main superset branch. For a better forum to ask technical questions please join the superset community slack or go to the discussions tab, as these PR pages are meant for conversations concerning the specific pull requests.
Currency formatting is available in different ways, for example by customizing a big number chart or by editing a dataset and customizing the metric currency. If you have further questions please reach out in one of the forums mentioned above. Good luck!


Can't place order - This account currently holds %1% shares of %SYMBOL%.  (AC109)

I've been getting this error message when placing orders recently.
'message not found', 'This account currently holds %1% shares of %SYMBOL%.  (AC109)'
I tried logging the responses, and the above message seemed to be coming from the verification response.
The confirmation response looked like this:
{'Messages': [{'ApplicationCode': 'DO', 'Id': 9008, 'ShortName': '', 'SequenceNumber': 0, 'TypeCode': 4, 'OccuranceNumber': 0, 'SubstitutionParam1': 'OrderId', 'SubstitutionParam2': '0', 'Message': 'STOP: A problem was detected which may have affected your order placement. <B>Please check the Order Status screen to confirm that your order has been received.</B> If your order appears, the transaction was successful. If you do not see your trade, please contact customer support for assistance.  (DO9008)'}], 'MessageCounts': {'BusinessReject': 1}, 'ReturnCode': 4, 'EtfMsgCode': ''}

I checked the order status just in case, but there were no orders placed.

It seems that this only happens to the accounts that were originally opened with TD Ameritrade and have been migrated to Schwab recently. I can place orders without issues on my other Schwab account that was opened with Schwab.
Is there anyway I can debug this from my end? I'd really love to keep using this API to automate my trading.

It's difficult for me to debug this, because I don't have a TD ameritrade-opened account

Is there any way I can help you debug? Like posting sample request and response bodies?
Or I can try to fix it myself if you let me know how you usually handle this kind of problem. I don't even know where to start looking, but with some help, I may be able to debug it. I write programs for a living.

Quick update, I reverse engineered the "new" API for trading by persisting the authentication token obtained through playwright. I don't actively trade on Schwab anymore so I'll have to move some money in and test it over the next few days, but I'm optimistic this will solve the issue (and perhaps make the debugging process less opaque for anyone who wants to contribute in the future)

Awesome! Looking forward to the fix. Thank you so much!

Hi @gyias, I just pushed version 0.3.0, which features a trade_v2 function using the new API. I haven't fully tested it (which is why it's not a replacement of the old trade function), but I did get it to place orders on my regular Schwab brokerage account.
Could you try it out and let me know if it solves your problem?

Sure, will do. Thanks a lot!

Looking good so far. :) The orders didn't go through because the market was closed, but they appeared as pending orders. I'll try to make a real order tomorrow and report back. Thanks!

I was able to test buying but not selling, so I suspect selling might be a problem. Glad to hear the orders went through though, keep me posted!

Confirming both buy and sell orders can be placed and filled without any issue now. Thank you very much!

BTW, the new API supports MOC (Market on close) orders. In case other users of this library want to take advantage of the new order type, the order type number is 53.  It might be a good idea to add an order type parameter to the new API? Probably with a default value of "49" for the backward compatibility?
Thanks again for creating this super helpful API and addressing this issue! With the repeated delays of the official Schwab API launch, this is literally a life saver for those like me who were using the TDA API but were forced to migrate to Schwab.


Write up contributing document

I'd like to actually write up a useful (read: not copy and paste) contributing document, that explains our best practices when it comes to contributions.
My initial thoughts are:

PRs must pass CI integration (this catches failing tests / go vet / go lint / whatever else we feel like putting in there - and automating this stuff is good)
PRs must have a decent level of unit tests

Thoughts?

ping @zeisss @xh3b4sd

Why not copy paste the one we have for all other open source repos?

I have a concern that we'll just copy and paste it, and won't pay any attention to the actual content or readability, or whether all of it applies to the particular project.
I may be being too OCD about this :/

IMO some guys already cared about this. So I am out of here. IMO, lets copy/paste what we have and see what we need to do when we need to do something.

We hav ehttps://github.com/giantswarm/inago/blob/master/CONTRIBUTING.md, good enough.


{% elsif %}, {% case %}, and {% when %}

Hi @braver,
As the title says, I added the tags {% elsif %}, {% case %}, and {% when %}.
I also fixed the partial highlighting of some words, like "blank" in "blanket":

I saw that, in some situations, a tag matches even when it's not right after the {%.
For example, "capture" in "if object.capture":

I don't know how to fix that. I tried adding a ^ to the regex, but it didn't work.
Any idea?
Best regards,
Benoit

For that final problem, we need to fundamentally change how if finds keywords. Now, it very naively sees we're inside a tag and it will match whatever it finds there, whenever it is found. I think that when we first enter the tag, ie. here: https://github.com/braver/sublime-liquid/blob/master/Syntaxes/Liquid-HTML.sublime-syntax#L53 we should immediately try to match the keywords that only work immediately after opening the tag. There are a number of ways to go about that. Perhaps we should push into the tag scope if a lookahead matches for {%  and then for each case in Liquid Tag.sublime-syntax start with actually matching and scoping {%.
The contents of this PR is 👍 👍 so I'll merge that and you can start a new PR if you feel up to it.

Well, now that I think about it, \b for keywords like blank might not be sufficient, you probably have to require whitespace around those words, because now it would still match blank in something.blank... corner cases though.

It's not really an edge case: the {% form %} tag defines a form variable within its scope.
Here is an example inspired by the default theme:
{% form 'customer' %}
  {%- if form.posted_successfully? -%}

Thanks!

  {%- endif -%}
  ...
{% endform %}


Add key and value methods to DebugMap

Rendered
Add two new methods to std::fmt::DebugMap for writing the key and value part of a map entry separately:
impl<'a, 'b: 'a> DebugMap<'a, 'b> {
    pub fn key(&mut self, key: &dyn Debug) -> &mut Self;
    pub fn value(&mut self, value: &dyn Debug) -> &mut Self;
}

CC @japaric in case you're interested, given your work on an alternative formatting stack.

Nominating for discussion at the next triage to get some more input from the wider libs team.

I'll go ahead an propose to merge, seems like a reasonable feature to me!
@rfcbot fcp merge


Adapt Ansible/Python checks to match setuptools config

PR Type

Bugfix Pull Request

Noticed incorrect logic wrt. version checking and no tests either.

On the second thought, just remove the Python version check. This should be done by the dist installer (Pip) and there's metadata in dist about the Python version restrictions.

Right, I was thinking this when I saw the code! Thought this PR might force a decision on this :)
Good!
I'll just leave the Ansible check then ...

@ssbarnea Do not "request changes" if you haven't taken the time to understood the issue: title and description are already included and another reviewer understood well what it is about.

@decentral1se
I've looked up the dependency tree and we actually have Ansible as a transitive dependency via ansible-lint. But it doesn't declare version limitations.
Reiterating the problem, I think that we could deal with Ansible the same way as with Python: let's just specify proper requirements in the dist metadata and Pip will take care of it. The latest versions of Pip are even capable of warning users about incompatible things in env...
This way we won't have to think about checking this manually in our code. And I'd prefer Ansible to be a direct dependency anyway: it's kinda weird that it's not now.

@decentral1se Maybe I was not clear but I am trying to assure that we do endup creating commit messages that are not clear, as these would endup in the changelist of the release. Currently by looking at the PR title and comment I doubt a molecule user could understand what was the purpose of the change.
I only see that is marked as a bugfix PR, but one without a bug. If is a bugfix, it needs to have a a "Fixes: #" line on it which would link to the bug that is fixed by it. On other project having a meaningful commit message test is done before even looking at the modified code, commit message is part of the review too, and believe me that I was asked to update mine lots of time, even for trivial reasons like typos.
Now, back to this change, I kinda find weird that we need to have extra checks in our code for python version and so when this should be handled by packaging tools alone. Do we really need all this extra code inside molecule or we can avoid it?

Now, back to this change, I kinda find weird that we need to have extra checks in our code for python version and so when this should be handled by packaging tools alone. Do we really need all this extra code inside molecule or we can avoid it?

that's basically what I said earlier :laughing:

@decentral1se @ssbarnea I've got a collection of articles about writing meaningful commit messages. I'll post it as a separate issue so that anyone could go through this and we could be on the same page...
Sounds fair?

Good idea @webknjaz! New attempt in https://github.com/ansible/molecule/pull/1890!


Make sure SPIR-V intrinsics have got a unique name

Functions and calls are used in a number of places to encode specific SPIR-V instructions. In several cases the same function name is used with different type signatures, making it impossible to use both in the same module. All uses of functions to encode SPIR-V instructions should be reviewed to ensure they don't suffer from this issue. Each time a single test that uses all possible overloads should be added.

I think this is now fixed. Closing the issue. Feel free to reopen if I am wrong here.


Fix Layer hierarchy

The AbstractLayer hierarchy had some drawbacks that should be fixed with this PR.

General bugfixing (e.g. in constructors)
The layer services now all extend from AbstractSecuredPersistentObjectService, which allows us to make use of the very useful methods from this class
The methods from the AbstractLayerService/DAO  have been moved to the correct (and new) subclass LayerGroupService/DAO
Missing services and controllers and DAOs have been added

Please review

Looking good to me, of course we should still discuss the TODOs and Exception warning internally


embassy-lora Sx126xRadio pin types

I'm currently trying to use Sx126xRadio that #985 has brought. As defined here, it seems that there is only one type parameter for all output pins, and one type parameter for all interrupt-input pins:
impl<SPI, CTRL, WAIT, BUS> Sx126xRadio<SPI, CTRL, WAIT, BUS>
where
    SPI: SpiBus<u8, Error = BUS> + 'static,
    CTRL: OutputPin + 'static,
    WAIT: Wait + 'static,
    BUS: Error + Format + 'static,
{
    pub async fn new(
        spi: SPI,
        cs: CTRL,
        reset: CTRL,
        antenna_rx: CTRL,
        antenna_tx: CTRL,
        dio1: WAIT,
        busy: WAIT,
        enable_public_network: bool,
    ) -> Result<Self, RadioError<BUS>> {
        let mut lora = LoRa::new(spi, cs, reset, antenna_rx, antenna_tx, dio1, busy);
        lora.init().await?;
        lora.set_lora_modem(enable_public_network).await?;
        Ok(Self { lora })
    }
}

@lulf and @ceekdee, since you've worked on that PR: Unless I'm mistaken, does this mean that all the CTRL pins would need to have the same type, and all the WAIT pins likewise? Would it make more sense to have a unique parameter for each of these pins?

Both the nRF and STM32 HALs provide an AnyPin type you can create, i.e. using p.PB4.downgrade(). In general, I think we want to have a RadioSwitch parameter for the antenna pins at least, because it can differ from board to board. I think @ceekdee is planning to fix that.

Ah, fair enough. I want to use it with atsamd-hal, but it has a similar DynPin API.
Regarding the antenna pins, I believe my board has a single switch for both TX and RX. How is it recommended to handle that case?

Until the API has been refactored, I think you maybe need to wrap your pin in some custom shared pin type containing an inner type with a RefCell that the driver can use. Since TX and RX is not done at the same time, it should be 'safe' to do this.
It's exactly the reason I think we should use a RadioSwitch trait similar to the other drivers, because boards do it so differently.

In the original implementation, each pin did have its own type.  This seemed cumbersome and unnecessary to me, and when I realized that AnyPin could be used for both input and output pins, I changed to use CTRL and WAIT.  This can certainly be re-visited as we attempt to combine MCU/Semtech chip combinations into one implementation (https://github.com/embassy-rs/embassy/issues/1023).  Any additional observations and guidance you wish to provide can be placed in issue 1023.
Regarding the antenna processing, another option until the re-implementation is to make local changes to the following functions in board_specific.rs in the sx126x implementation:

brd_ant_sleep()
brd_ant_set_rx()
brd_ant_set_tx()

to support your use case.

@ceekdee I might have a go at introducing a RadioSwitch trait for the sx126x in a way that should help this use case, if that's ok with you.

Would it make sense to extend the RadioSwitch trait to all board-specific functionality, which the user is expected to implement? All of the generic complexity could be hidden behind it so that the radio structs wouldn't have to deal with individual pins. A default struct which implements that trait could be provided, for users that don't have specific needs or can provide type-erased pins.

By other board-specific functionality, do you mean the Wait, Ready etc. pins? If so, I guess that could be there as well, though I've not seen them having different semantics across boards, vs. the radio pins which seem to vary a lot more.
I'm fine either way, I think it's worth exploring.

Perhaps a better way to envision the "orchestrator" is as a builder which returns a LoRa structure supporting the API for an MCU/LoRa chip combination.
Since the only current interfaces to Semtech chips of which I am aware are SPI and pin setting/checking, the hope is to hide any dependencies on STM32WL or nRF52 for this behind the LoRa API.

I've moved the discussion over to #1023 now, focusing on this issue here:

Would it make sense to extend the RadioSwitch trait to all board-specific functionality, which the user is expected to implement? All of the generic complexity could be hidden behind it so that the radio structs wouldn't have to deal with individual pins. A default struct which implements that trait could be provided, for users that don't have specific needs or can provide type-erased pins. I can also try to whip up a POC if y'all are interested.

I think that makes sense, maybe have a look at the stm32wl integration in embassy-lora for inspiration and see what you think.

@ceekdee, I've gotten started a rough API that abstracts over the physical aspects of a board (view here). However while trying it out, I seem to consistently run into a bug in this function where indexing buffer overflows when i reaches 4. Seems like whe debugging, number_of_registers is set to 34, is this normal?

The original sx126x driver has been removed in favor of using the lora-phy crate.  Therefore, closing this issue.


First pass at Salt/Vagrantfile configuration

This supercedes the current Vagrantfile and fixes a bunch of bugs.  I'm still ironing it out, but feel free to give it a go if you've got spare bandwidth.

I'm shamelessly merging this into master.  The only dangerous piece in here is I've added docker-compose as an installation to environment.yml.  This should have gone into a separate commit.  The rest of the files touched are directly under Vagrant/Salt.

@amfarrell - This needs your review (I know, I've already merged it :)
Test by blowing away your current vagrant box with:
vagrant destroy

Then bring it up with:
time vagrant up

I put the time command in there because I want to know how long this takes to execute on a typical developer laptop :)
It should get all the way through to launching memex-explorer.  If that succeeds, poke around and see what's working.  I think you mentioned that there's a hardcode in the way Kibana is being launched/detected, and that we need to fix this.  If you've got notes on this, please raise an issue or fix.  Thanks!

Caveats: this wipes away the database on reprovision instead of migration.  Also, this takes about 45 minutes to provision on my laptop at home network speeds.

The hardcode isn't in the way kibana is being launched/detected, just in the way it is listed as a Service (the model is still named App)


[WIP][Dependency Update] Upgrade the libtiff to 4.0.10

Description
please don't merge until I add publish test on CI for cuda part
Upgrade the libtiff package to 4.0.10 due to lots of issues at 4.0.9.

tif_jbig.c JBIGDecode out-of-bounds write
two out-of-bounds writes in cpTags in tools/tiff2bw.c and tools/pal2rgb.c

Please find more on CVE
Checklist
Essentials

[ ] Test build with Ubuntu 14.04
[ ] Test build with Ubuntu 16.04

Changes

gitlab didn't provide version 4.0.10 zip file so use mirror site from the official website

Comments

Thanks for your contributions @stu1130. The CI seems to be failing. Can you look into this ?
@mxnet-label-bot Add [Build, pr-awaiting-merge]

Sorry that I merged the PR by mistake. Do we need to revert the PR, since the CI has been passed?

@wkcn the previous CI failures don't seem to be related to this. There is CI test for building the static library which dependes on this (I'm not sure if the CI has tests specifically for this dependency though). In any case, it doesn't seem like a revert is urgently necessary right now.

@szha Thank you! I will pay more attention next time.

The stattic build should trigger and verify the script to see if it is working.

@wkcn Static build CPU 14.04 Python and Static build CPU 14.04 Scala in ubuntu cpu have tested against the script. I just want to make sure it also works in ubuntu 16.04. Since we're still using ubuntu 14.04 to build the package so no worry. Thanks for your hard work

I triggered the vulnerability from the C library Libtiff 4.0.9, when I use an older version of incubator-mxnet, causing out-of-bounds write in the buffer. A call chain that accesses to the vulnerable function TIFFSeek() is as follows:
(python code)mxnet/image.py: def imread(filename, *args, **kwargs)
(libmxnet.so)imgcodecs/src/loadsave.cpp: Mat imread( const String& filename, int flags );
(libopencv.so)imgcodes/src/grfmt_tiff.cpp: bool TiffDecoder::readData( Mat& img );
(libopencv.so)imgcodes/src/grfmt_tiff.cpp: bool TiffDecoder::readData_32FC1(Mat& img);
(libtiff.so)libtiff/tif_read.c: intTIFFReadScanline(TIFF* tif, void* buf, uint32 row, uint16 sample);
(libtiff.so)libtiff/tif_read.c: static int  TIFFSeek(TIFF* tif, uint32 row, uint16 sample );

I have upgraded to incubator-mxnet's newest version to avoid the issues. Give the info in this report for sharing. It seems that our python projects should keep an eye on the CVEs of C libraries.

I triggered the vulnerability from the C library Libtiff 4.0.9, when I use an older version of incubator-mxnet, causing out-of-bounds write in the buffer. A call chain that accesses to the vulnerable function JBIGDecode() is as follows:
(python code)mxnet/image.py: def imread(filename, *args, **kwargs)
(libmxnet.so)imgcodecs/src/loadsave.cpp: Mat imread( const String& filename, int flags );
(libopencv.so)imgcodes/src/grfmt_tiff.cpp: bool  TiffDecoder::readData( Mat& img );
(libopencv.so)imgcodes/src/grfmt_tiff.cpp: bool TiffDecoder::readData_32FC1(Mat& img);
(libtiff.so)libtiff/tif_read.c: intTIFFReadScanline(TIFF* tif, void* buf, uint32 row, uint16 sample);
(libtiff.so)libtiff/tif_read.c: static int  TIFFSeek(TIFF* tif, uint32 row, uint16 sample );

I have upgraded to incubator-mxnet's newest version to avoid the issues. Give the info in this report for sharing. It seems that our python projects should keep an eye on the CVEs of C libraries.

By the way, incubator-mxnet depends on libmxnet.so, I figured out the shared library comes from Libtiff 4.0.9 based on incubator-mxnet's development docs.


Fix performance bug in the FeatureDataSource caching code

FeatureDataSource was not correctly keeping track of the interval covered
by the contents of its cache, and as a result was producing many more
cache misses than it should have.

For @lbergelson to review please

:+1: Merge when tests pass.


Error: No resource found that matches the given name: attr

Information

Apktool Version (apktool -version) 2.4.1
Operating System (Mac, Linux, Windows) Windows 10 1909
APK From? (Playstore, ROM, Other) Playstore

Stacktrace/Logcat
java -jar apktool_2.4.1.jar b file-manager_1.8
I: Using Apktool 2.4.1
I: Checking whether sources has changed...
I: Smaling smali folder into classes.dex...
I: Checking whether resources has changed...
I: Building resources...
W: E:\\file-manager_1.8\res\values\styles.xml:1336: error: Error: No resource found that matches the given name: attr '@android:bool/config_use_sim_language_file'.
W:
W: E:\\file-manager_1.8\res\values\styles.xml:1337: error: Error: No resource found that matches the given name: attr '@android:bool/config_use_strict_phone_number_comparation'.
W:
W: E:\\file-manager_1.8\res\values-v21\styles.xml:186: error: Error: No resource found that matches the given name: attr '@android:bool/config_use_sim_language_file'.
W:
W: E:\\file-manager_1.8\res\values-v21\styles.xml:187: error: Error: No resource found that matches the given name: attr '@android:bool/config_use_strict_phone_number_comparation'.
W:
brut.androlib.AndrolibException: brut.common.BrutException: could not exec (exit code = 1): [C:\Users\evildog1\AppData\Local\Temp\brut_util_Jar_8015305493712467435.tmp, p, --forced-package-id, 127, --min-sdk-version, 17, --target-sdk-version, 28, --version-code, 9, --version-name, 1.8, --no-version-vectors, -F, C:\Users\evildog1\AppData\Local\Temp\APKTOOL6967098957294477000.tmp, -e, C:\Users\evildog1\AppData\Local\Temp\APKTOOL586482507562542425.tmp, -0, arsc, -I, C:\Users\evildog1\AppData\Local\apktool\framework\1.apk, -S, E:\\file-manager_1.8\res, -M, E:\\file-manager_1.8\AndroidManifest.xml]

Steps to Reproduce

java -jar apktool_2.4.1.jar d file-manager_1.8.apk
java -jar apktool_2.4.1.jar b file-manager_1.8

Frameworks
No and I cleared the framework since this is not a system app
APK
https://apkcombo.com/file-manager/com.File.manager/
Questions to ask before submission

Have you tried apktool d, apktool b without changing anything? Yes
If you are trying to install a modified apk, did you resign it? N/A
Are you using the latest apktool version? Yes

Those resources have got to have come from a framework file at some point. Since you haven't got that file (I looked in AOSP framework-res.apk - they're not there either)  you'll probably have to put the resources into your bools.xml and edit the lines mentioned in the error message manually.
A bit annoying but relatively straightforward and it'll let the file recompile. The only problem is that you don't know whether these bools are meant to be set to true or false or a mix of both. So the app might not function 100% correctly.
If that happens, just change the values and try again.
I did some Googling for these resources and I think they might come from a Pixel resource app, but I don't know for sure. You could try downloading some system dumps and installing different files but that could take a while.

Can't replicate this. If anything I get:
W: /home/ibotpeaches/Downloads/Apktool/2365/file-manager_2.1.2/res/values/styles.xml:1568: error: style attribute 'android:attr/windowFixedWidthMajor' is private.
W: /home/ibotpeaches/Downloads/Apktool/2365/file-manager_2.1.2/res/values/styles.xml:1569: error: style attribute 'android:attr/windowFixedWidthMinor' is private.

Which just points to using private resources which is frowned upon.


Failed to update chargepointState

This error originated from a custom integration.
Logger: custom_components.chargepoint
Source: custom_components/chargepoint/init.py:176
integration: ChargePoint (documentation, issues)
First occurred: 2:50:48 PM (12 occurrences)
Last logged: 3:01:26 PM
Failed to update ChargePoint State

I'm seeing this too.
It seems to happen after we plug our car into our home charger. That's unfortunate since I have automations that manage the charge time & duration, so they have stopped being effective.
I added some additional info to the log to see what the data and err looked like. In the case below, the car had been plugged in a few minutes before, and the charging_session and home_chargers are not populated. So, this seems like it's a glitch upstream. It eventually resolves itself, but in this case, 30 minutes have gone by and it is still returning the same info.
2024-11-27 05:56:53.886 ERROR (MainThread) [custom_components.chargepoint] Failed to update ChargePoint State: data={'account_information': ChargePointAccount(user=ChargePointUser(email='***', evatar_url='https://mc.chargepoint.com/images/evatar/1.png', family_name='***', full_name='***', given_name='***', phone='***', phone_country_id=40, user_id=***, username='***'), account_balance=AccountBalance(account_number='***', account_state='FULL', amount='***', currency='USD')), 'charging_status': UserChargingStatus(session_id=***, start_time=datetime.datetime(2024, 11, 27, 13, 54, 17, tzinfo=datetime.timezone.utc), state='unknown', stations=[ChargePointStation(id=***, name='***', latitude=***, longitude=***)]), 'charging_session': None, 'home_chargers': {}}, err=Failed to get charging session data.
2024-11-27 05:56:53.886 ERROR (MainThread) [custom_components.chargepoint] Error fetching chargepoint data: