[ENH] Convert stock symbol to company name ?

Brief Description

I was thinking of a function that would either convert stock symbols to company names or create a new column of company names based on stock symbols from a different column, The only issue would be to find a stable site or api that would provide the company names in return for stock symbols and handling errors when you weren't connected to the internet.
Example API
def stock_to_name(df, symbol_column_name):

Before
Stock
F
O
GE
Stock2Name(DF.Stock)
Stock
Ford
Realty Income
General Motors.

There's definitely a bunch of APIs!
I've got very little domain knowledge on finance to know which one of these would be most reputable. Do you have an opinion?

I don't have much on the ground experience.  If I had to make a decision right now, I would choose Bloomberg because the company is big and been around for a while. I don;t know anything about the API though.
I sent you an email through shortwhale about a related issue.

Would it be possible to find a solution that generally didn't require an internet connection? Since it's not pulling stock quotes, and the dictionary of stock symbols is mostly stable, it'd be nice not to send a request each time.

In this case, I think caching the dictionary locally would be ideal. Doesn't have to be complicated: JSON/YAML or a sqlite database. (Actually, sqlite would be overkill, IMO!)

After a brief Google search it looks like there’s only about 4000 publicly traded companies on the NYSE/Nasdaq.
Wouldn’t it make sense to just store this in a Pandas DataFrame instead of json or YAML? Especially cause YAML has security vulnerabilities: https://tools.cisco.com/security/center/viewAlert.x?alertId=58783

That could work too! So we'd cache the pandas DataFrame, then? I think using the feather format for caching would be awesome here, especially since it's a binary format.

Are you proposing to build this and distribute with pyjanitor, or it's
built on the user's end, with the ability to rebuild at will?
On Sat, May 25, 2019, 7:51 PM Eric Ma<EMAIL_ADDRESS>wrote:

That could work too! So we'd cache the pandas DataFrame, then? I think
using the feather format for caching would be awesome here, especially
since it's a binary format.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/ericmjl/pyjanitor/issues/366?email_source=notifications&email_token=AFSDFRBAMZMJ2ITHB5M7YZDPXH3LPA5CNFSM4HNY36NKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODWH455I#issuecomment-495963893,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AFSDFRG6X7MFLU2MZY4KF33PXH3LPANCNFSM4HNY36NA
.

Are you proposing to build this and distribute with pyjanitor, or it's
built on the user's end, with the ability to rebuild at will?

I think the latter is the better way to approach this.

Just a thought on how to extend this:
Coming from an R background, R has a number of included datasets that are readable as a data.frame.
Similarly, scikit-learn has a number of included datasets, but those are included as arrays but not Pandas DataFrames.
Maybe for the submodules there could be added “informational” DataFrames? Obviously the stock names for finance, but maybe a periodic table for chemistry (I have no idea if that would be necessary) and similar ones for the others?

Ahhh, I see! Yes, we could also definitely make this part of the build pipeline, i.e. including scripts to build those dataframes as part of the CI or deployment story.
Gives me more impetus to do the auto-deployment thing right!

I like that idea @szuckerman ! What would be some other general purpose DataFrames?

So, besides the periodic table for chemistry (which may or may not be useful for chemists - I have no idea, it’s not my field) I’m not sure what would be general purpose for biology and other sciences.
However, what I’ve used frequently recently are ones like “country abbreviations” I.e Ireland being IE, Mexico MX, etc. Similarly, for US cities having a zip code, city, state table is sometimes useful.
Basically these are just lookup tables to give more verbose information than is in the default. For example, if I had data that only had country abbreviations and I want to know how many of those countries use Euros as their main currency, it would be a cinch to just do a join with the lookup table that has this data stored there already.
Obviously the size of the data should be taken into account; it’s not worth it to force everyone to download a gigantic pyjanitor package for something they won’t use. (There are some libraries that take this into account and have separate modules to download for different data people want, such as the spaCy package.)

For chemistry & biology, which are the two domains I am in right now, the periodic table mappings to atomic weights and proton numbers are definitely useful. RDKit provides the ability to compute more complicated properties on the fly. My intern @bhallaY has PR'd them into pyjanitor.
For biologists, DNA mapping to their molecular weights is generally helpful, as is the amino acid mappings to molecular weights and isoelectric points. I definitely will be PR-ing these in shortly.
Those few examples, in all, very tiny to construct - essentially a small dictionary that can be hard-coded, and are stable over time, given that these are fundamental properties of the chemical and biological universe :smile:.

Obviously the size of the data should be taken into account; it’s not worth it to force everyone to download a gigantic pyjanitor package for something they won’t use.

Definitely agree! Do you know how to enable this, @szuckerman? I'm a bit at a loss here, as I've never distributed a package that has optional downloads.

These seem like good places to start. I guess the biggest contribution will getting the packaging/distribution down, and having that in place to add more in the future as needed. How big do you imagine these files would be (in relation to pyjanitor)?

@ericmjl I think there's a few ways to serve optional data files, but the easiest would most likely be to have another package, something like pyjanitor_datasets. That way it gets hosted on PyPI and people can just download the package.
Another way to do it would be to host the files in a CDN (or even github) and then add a load_data() method to pull each file when the user wants.

Looking back on this issue, yes, definitely a good idea to host this on GitHub. I like this idea. I suggest we use @chrisfs’ original use case as a testing ground for how to host data files.


Infallible factorization

It would be nice to have a factors-like function that always returns the factorization of the input, even if it has to fall back to trial division. And in the same vein as #1, you might want to consider renaming factors (e.g. to factors_fast), since integer factorization is always possible in principle, and having a function with such a generic name "fail" feels strange from a math POV.

integer factorization is always possible

This is true, but the factorization is intractable when the integer gets bigger. I let the factors function be fallible so that the factorization function will not run forever, and the execution time can be estimated. Even if the factorization failed, the found factors will still be returned.

IMO, the API design prevalent in this crate mixes two unrelated concerns:

The function should give the correct result.
The user should have control over how long the function runs.

The first of those is the job of a math crate. The second is the job of a scheduler, a timeout system, or similar. If I want to try to factorize a number, but am unwilling to wait for more than five minutes for the answer, I can run timeout 5m factor N. In Rust, I can use a subprocess, and terminate it programmatically if it is still running after the deadline. But under no circumstances do I want the factors function itself to do that, especially not by default. That's just fundamentally wrong.
Imagine an HTTP client library with a function for making requests such as
fn get(url: &str) -> Option<String>

This function makes an HTTP GET request and returns the response body as a string – but only if the response is less than 1 GB in size. If it is bigger, it just returns None, and I have to instead call
fn get_no_really_i_mean_it(url: &str) -> String

to get what I want. I'm sure you'd agree that this would be a terrible API, yet this is essentially how num-prime currently works.
When I call a function named factors, I expect it to always return the correct result, even if it takes a million years to compute. If I'm worried that my input number may be too large, I can wrap the call in a timeout, or call an appropriately-named probabilistic function instead (which might not be able to completely factor some large semiprimes or similar). But I never, ever, want factors to make such decisions for me.

Let user terminate execution by killing the process is far from a robust and correct API. In the specific application of factorization, we can get the factors from the process even if it's not fully factorized, but you can't get these factors if you killed it by termination. Partial factorization is already very useful for many applications, and you can even turn to other parallelized factorization algorithm if you found some factors, but the full factorization failed with given trials.
For the example you gave, it's not the exact same case. For http downloading, the common way is to split the file into chunks and each chunk is downloaded as a whole. So, you can think of a function fn get_chunk(url, start_pos) -> Result<(Content, Finished), Failure>, and you download the file by calling get_chunk until it's finished. Similarly, to fully factorize a target, you will need to call factors for several times with different configuration until it's finished.
Indeed, there're a lot of file servers that don't provide the protocol to download the file by chunks, but that will results in terrible user experience (unable to pause and resume downloading) and should not be followed.
On the other hand, I agree with you that there're a mixture of two concerns in the API design. I think I will let the factors function to remain the current shape and create a factorize function that won't return until the target is fully factorized. Though I still believe that it's important to provide users with the choice of early termination and the choice of which factorization algorithm is chosen under the hood.

The API factorize() is introduced in v0.3, and factors64 is renamed to factorize64(). Both functions are infallible factorization.


2.8.0 Release Announcement

Photograph by Margaret Krzepkowski
Thank you everyone for your contributions.
While this release still doesnt support OpenAPI 1.0 spec, it does bring support for the new swagger-ui look thanks to @kasecato!! 🙇
Apologies for the delayed release schedule of 2.8.0. Much of the work in this release was to shore up the library for significant work in the near future to support in 2.9.0

OpenAPI spec 3.0
Set the stage for fixing bugs related to Swagger 1.2 Spec. This is the penultimate release with support for swagger1

2.9.0 will be the last supported release for Java 6/Java 7. Much of 2.9.0 will focus on Improvements in the model rendering thanks to @MaksimOrlov! 🙇  (support @JsonView, better support for rendering the same models in request/response, inheritance support). Please note the deprecation warnings to prepare for breaking changes in 3.0.0.
Pull Requests
(#2178) Change regex in Paths.java to handle expressions/constraints correctly @nobe0716
(#2174) fix fmt   maintenanc @silenceshell
(#2169) Swagger ui 3.x suggested improvements   feature @kasecato
(#2160) Fix conversion of byte to integer with max / min   bug @avdv
(#2153) Upgrading Swagger UI to 3.7.0  @kasecato
(#2144) Allow ApiModelProperties on methods to be discovered from superclasses  @RoyJacobs
(#2106) Add support for exclusive ranges handling  @filiphr
(#2103) Fix some tests under windows  @apixandru
(#2101) Fix Remapping issue  @apixandru
(#2081) Fix child expansion context creation   bug @gzsombor
(#2069) Optimise HandlerMethodResolver.getMemberMethods   maintenance @simongajdosech
(#2066) Added error handling around "duplicate" request handlers  @mate1983
(#2048) Support explicit ordering for Tags  @jroweboy
(#2040) Use Guava 20.0 throughout the project  maintenance @Thunderforge
(#2014) Fix markdown  @koppor
(#2013) Fixed merging headers from the already existing request with the supplied request  @pvanassen
(#1988) custom the web page title as swagger.title when had set it  @rainplus
(#1974) Add default property support on model properties   feature @matrosovs
(#1956) @ApiModelProperty example string does not escape char "" from JSON example  @heapifyman
(#1952) Fix to recognize @Param as query parameter in EntitySearchExtractor know  @viruscamp
(#1943) Wrong API resource path in Swagger 1.2  @mathieuales
(#1942) Model classes having names containing integers are not detected as array  @mathieuales
(#1917) Consumes / Produces media-types on the document level aren't copied and merged anymore with the operation level consumes / produces media-types.  @mzeijen
(#1914) Provide proprty pattern annotation support  @simonamc
(#1897) Correct spelling and typos  @naXa777
(#1878) Preserve tags order in documentation builder  @rainoko
(#1868) Create EntitySaveExtractor.java  @jadhavsuhas
(#1838) Fix a mixed up part in the Getting Started guide for Docket  @PeterWippermann
(#1837) Minor update of Docket's JavaDoc  @PeterWippermann
(#1829) JacksonEnumDeterminer to handle JsonFormat.Shape.Object  @yelhouti
Features
(#2177) Paths.sanitizeRequestMappingPattern fix @nobe0716
(#2139) Should support "title" property, set via @ApiModel annotation.  @ngbalk
(#2088) @ApiParam(allowableValues = "range(0, infinity)") does not work @filiphr
(#2063) Added support for Pageable resolved parameter @avillev
(#2057) Swagger-ui don't render additionalProperties duplicate @deblockt
(#2026) Produces/Consumes do not maintain order @jgaribay21
(#2023) Is it possible to disable globalResponseMessage configuration partially. duplicate @dohoon
(#2021) @ApiModelProperty.allowEmptyValue = true/false does not emit "allowEmptyValue" in swagger.json @bill
(#2000) Upgrade to latest version of Swagger UI (3.1.5) duplicate @madheshr
(#1960) Upgrade to swagger-ui 3.0 duplicate @alex
(#1957) springfox doesn't work with spring boot 2.0 and spring data Kay-RC2 duplicate @shashankitmaster
(#1955) Add support for inclusive and exclusive ranges for allowable values @JohnNiang
(#1946) collectionFormat problem @vitek499
(#1936) configuration for adding dynamic api-key(access token value). duplicate @akashgupta08
(#1919) Add support to rename ApiModel property name in Model Attributes @peterjurkovic
(#1901) Pattern Bean Validations API (JSR-303) support for Request Parameters  help wanted @simonamc
(#1900) Tag custom ordering @rainoko
(#1818) JsonFormat for enum and other cases @yelhouti
(#1729) Status of support for v3.0.2 of Swagger UI duplicate @JLLeitschuh
Maintenance
(#2161) How to set a default value to a field of a model? in progress @michele
(#2093) swagger-ui.html appears to be empty documentation  @silentsnooc
(#2090) When using AlternateTypeRuleConvention ApiModelProperty annotation does not work @snimavat
(#2031) How to get object in response body in autogenerated swagger.json file documentation  @rajat
(#2029) ApiImplicitParam with empty datatype fails when we try it out documentation @ljp510016132
(#1995) View APIs from different Spring Cloud Instances registered in Eureka documentation  @s
(#1971) Vavr/Javaslang Jackson module support documentation  @Sir4ur0n
(#1954) Multiple swagger JSON's in swagger-ui.html documentation  @dreambrother
(#1950) Document support customized param using HandlerMethodArgumentResolver documentation  @neil4dong
(#1916) Consumes and produces media-types defined on Docket are incorrectly merged together with consumes/produces media-types that are defined on a resource level @mzeijen
(#1913) Space getting added to oAuth scope while making authorization request documentation  @mojaiq
(#1904) Nondeterministic output for Models used in multiple controllers documentation  @kevinm416
(#1899) Upgrade libraries @dilipkrish
(#1896) Wrong spelling and typos in code @naXa777
(#1882) @RepositoryRestResource -- ApiParam definition for the JPA methods always defines the @Param as "body" type parameter @aniruthmp
(#1875) Tags should be orderable  @rainoko
(#1870) The lasted version supported for Swagger UI 3.x?  @maliqiang
(#1865) Can't test the configuration 404 not found documentation @pinkyjain26
(#1833) Different guava versions in dependencies @Dimok74
(#1704) Document springfox oauth2 documentation duplicate @kidshg
Bugs
(#2165) AlternateTypeRules doesn't work as expected @crmky
(#2148) 2.7.1-SNAPSHOT NullPointerException when attempting to view http://localhost:8080/v2/api-docs @beardy247
(#2138) java.util.Optional<java.time.OffsetDateTime> disappears from request params  duplicate @bohdan
(#2135) No qualifying bean error when launching spring 5.0.x application with springfox  duplicate @gauravphoenix
(#2133) Optional @kitsjory
(#2132) @ApiModelProperty has no effect on some variables (name starting with one lowercase)  not-reproducable @bbrenne
(#2118) Request type mapping doesn't work if using both RequestBody and ModelAttribute on the same parameter @andrea
(#2114) @RequestParam and @PathVariable annotated parameters should not be expanded @loxal
(#2111) Application startup failed org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'documentationPluginsBootstrapper' @shobhit921
(#2107) @ApiParam ignores certain properties  duplicate @milosonator
(#2097) Swagger 2 : Getting "type": "ref" when using @RequestPart  duplicate @jrishabh198
(#2096) @ApiParam is ignored on ValueObjects wrapped with @JsonCreator @dkellenb
(#2080) NullPointerException in handlerPackage  duplicate @ejuniorasas
(#2072) Springfox generates Api-Doc for non-exported Repositories @stoetti
(#2053) @PathVariable work with Parameter Converter strangely  duplicate @cxl086
(#2039) Endpoints with the same path and parameters but different headers some times cause java.lang.IllegalArgumentException: Multiple entries with same key @joaoacmota
(#2025) SpringFox-Data-Rest 2.7.0: Missing camelCase with generated Associations operationIds @stoetti
(#2015) Add support for generics  not-reproducable wontfix @raderio
(#2012) Duplicated swagger.json document  not-reproducable @cesartl
(#2011) OperationBuilder mergeResponseMessages overwrites headers @pvanassen
(#1999) Generated JSON for endpoints with PathVariables converted to non-trivial objects is incorrect  duplicate @joel
(#1998) PathVarible composed of Custom Objects are not parsed correctly  duplicate @frbo42
(#1965) @RequestPart annotation not rendering models correctly @pratapyelugula
(#1963) Spring Data Rest Integration doesn't document repository method parameters correctly @thombergs
(#1961) Springfox - Authorization value didn't update after Authorize confirmed..  @thomasharin
(#1941) swagger-ui does not remove java string escape char "" from @ApiModelProperty example @heapifyman
(#1932) BasePathAwareController docs aren't base path aware @fquinner
(#1926) Can't be filtered by ignoredParameterTypes  not-reproducable @heyuxian
(#1924) Unable to define host in Swagger 1.2 @mathieuales
(#1906) Swagger shows wrong id field using Spring Data Rest  duplicate @drenda
(#1894) @EnableSwagger2 breaking my unit tests  not-reproducable @rawadrifai
(#1890) Can't declare Docket in multiple Configuration classes  not-reproducable @lorenzobenvenuti
(#1887) @ApiResponses tag does not override default responses  not-reproducable @muff1nman
(#1880) Wrong API resource path in Swagger 1.2 @mathieuales
(#1876) NullPointerException with Spring Data Rest integration @drenda
(#1866) 2.7.0 does not list PATCH methods from Spring Data Rest @jadhavsuhas
(#1864) @NotNull not working to mark field as required  @sddakoty
(#1860) [Spring boot] @EnableAspectJAutoProxy cause endpoint scanning failed  duplicate   @jdupont22
(#1841) Overloaded method does not respect the "tag" option @jackmatt2
(#1839) Primary keys are added to path parameters for Spring Data Rest Entities POST request after upgrading to 2.7.0 @jadhavsuhas
(#1830) CORS headers disappeared after upgrading to 2.7.0  not-reproducable @gionn
(#1804) Api key input missing in swagger ui after upgrading version to 2.6.1 @prajapatkiran
(#1781) Swagger JSON generated by SpringFox shows 'Consumes' for GET and DELETE operations  @dcp65
(#1672) Swagger-UI giving 405 (Method not allowed) when called @GarrettMosier  not-reproducable

Hello, I found a bug for this version 2.8.0, I use the following libraries springfox-swagger2 and springfox-swagger-ui (2.8.0) from Maven.
I attach the code of my application
//Swagger Config Class
@Configuration
@EnableSwagger2
public class SwaggerConfig {
@Bean
public Docket api() {
return new Docket(DocumentationType.SWAGGER_2)
.select()
.apis(RequestHandlerSelectors.basePackage("xxxx.yyyy.zzzzz"))
.paths(PathSelectors.any())
.build()
.apiInfo(apiInfo())
.securitySchemes(Collections.singletonList(apiKey()))
.securityContexts(Collections.singletonList(securityContext()));
}
private SecurityContext securityContext() {
    return SecurityContext.builder()
            .securityReferences(defaultAuth())
            .forPaths(PathSelectors.regex("/anyPath.*"))
            .build();
}

List<SecurityReference> defaultAuth() {
    AuthorizationScope authorizationScope
            = new AuthorizationScope("global", "accessEverything");
    AuthorizationScope[] authorizationScopes = new AuthorizationScope[1];
    authorizationScopes[0] = authorizationScope;
    return Collections.singletonList(new SecurityReference("Bearer", authorizationScopes));
}

@Bean
public ApiListingScannerPlugin listingScanner() {
return new SwaggerManualApiPlugin();
}
}
//SwaggerManualApiPlugin
public class SwaggerManualApiPlugin implements ApiListingScannerPlugin {
@Override
public List apply(DocumentationContext context) {
return new ArrayList(
Arrays.asList(
new ApiDescription(
"/fondos/generartoken",
"Generar Token Basic OAUTH",
Arrays.asList(
new OperationBuilder(
new CachingOperationNameGenerator())
.authorizations(new ArrayList())
.codegenMethodNameStem("basicAuth0001")
.method(HttpMethod.POST)
.notes("Endpoint responsable de la Autenticacion Básica de XXXXXX")
.parameters(
Arrays.asList(
new ParameterBuilder()
.description("Basic Auth")
.type(new TypeResolver().resolve(String.class))
.name("Authorization")
.defaultValue("Basic XXXXXXXXX")
.parameterType("header")
.parameterAccess("access")
.hidden(true)
.required(true)
.modelRef(new ModelRef("string"))
.build(),
new ParameterBuilder()
.description("Tipo de Grant")
.type(new TypeResolver().resolve(String.class))
.name("grant_type")
.defaultValue("password")
.required(true)
.hidden(true)
.parameterType("query")
.parameterAccess("access")
.modelRef(new ModelRef("string"))
.build(),
new ParameterBuilder()
.description("Tipo de Documento")
.type(new TypeResolver().resolve(String.class))
.name("tipoDocumento")
.parameterType("query")
.parameterAccess("access")
.required(true)
.modelRef(new ModelRef("string"))
.build(),
new ParameterBuilder()
.description("Numero de Documento")
.type(new TypeResolver().resolve(String.class))
.name("numeroDocumento")
.parameterType("query")
.parameterAccess("access")
.required(true)
.modelRef(new ModelRef("string"))
.build(),
new ParameterBuilder()
.description("Origen del canal")
.type(new TypeResolver().resolve(String.class))
.name("origen")
.defaultValue("web")
.parameterType("query")
.parameterAccess("access")
.required(true)
.modelRef(new ModelRef("string"))
.build(),
new ParameterBuilder()
.description("Codigo del canal")
.type(new TypeResolver().resolve(String.class))
.name("canal")
.defaultValue("PLD000501")
.parameterType("query")
.parameterAccess("access")
.required(true)
.modelRef(new ModelRef("string"))
.build()))
.build()),
false)));
}
@Override
public boolean supports(DocumentationType delimiter) {
    return DocumentationType.SWAGGER_2.equals(delimiter);
}

}
The problem here is that this new operation is creating is not showing response on version 2.8.0.
I tried with version 2.7.0 and if it's showing response, however when I try with other operation the new operation's response dissapears, but this is not problem for me.
Maybe I forgot any configuration, if so, throw me a cable.
Thanks in advance.

@mhersc1 Try just returning true from the supports method and see if that fixes the issue. It might just be a problem with the DocumentationType equals check, possibly due to different media types.

@mhersc1 please try 2.9.0, if it still doesnt work please create a new issue


Create 2022 Cloud SIEM release notes archive

Purpose of this pull request
This pull request (PR) creates an archive for Cloud SIEM 2022 release notes:
https://help.sumologic.com/release-notes-cse/
Issue number: Inspired by issue #2861

Select the type of change:

[ ] Minor Changes - Typos, formatting, slight revisions
[x] Update Content - Revisions and updating sections
[ ] New Content - New features, sections, pages, tutorials
[ ] Site and Tools - Updates, maintenance, and new packages for the site, Gatsby, React, etc

You're welcome, @kimsauce! Thanks for reviewing the PR.


Default environment not recognized

Description
I guess this is related to #10 as my PyCharm doesn't seem to want to add an environment called default. The Use existing environment-dropdown is just empty and I can't proceed:

Yes, the Load Environments-button has been clicked 😉
Versions
❯ pixi --version
pixi 0.19.1
❯ pixi list | grep pixi-pycharm
pixi-pycharm                   0.0.3           h707e725_0               7.8 KiB    conda  pixi-pycharm-0.0.3-h707e725_0.conda

PyCharm version: PyCharm 2024.1 (Professional Edition)
Platform: macOS Sonoma

Pixi-pycharm writes logs to pixi-pycharm.log. Do you see anything of interest there?

Here is the entire output of the logs:
Attempting to run: ['info', '--envs', '--json']
Attempting to run: ['info', '--envs', '--json']
Attempting to run: ['info', '--envs', '--json']
Attempting to run: ['info', '--envs', '--json']
Attempting to run: ['info', '--envs', '--json']
Attempting to run: ['info', '--envs', '--json']
Attempting to run: ['info', '--envs', '--json']
Attempting to run: ['info', '--envs', '--json']
Attempting to run: ['info', '--envs', '--json']
Attempting to run: ['info', '--envs', '--json']
Attempting to run: ['info', '--envs', '--json']
Attempting to run: ['info', '--envs', '--json']
Attempting to run: ['info', '--envs', '--json']
Attempting to run: ['info', '--envs', '--json']
Attempting to run: ['info', '--envs', '--json']
Attempting to run: ['info', '--envs', '--json']
Attempting to run: ['info', '--envs', '--json']
Attempting to run: ['info', '--envs', '--json']
Attempting to run: ['info', '--envs', '--json']
Attempting to run: ['info', '--envs', '--json']

./.pixi/envs/default/libexec/conda info --envs --json
{
    "envs_dirs": [
        "/Users/marcowo/Documents/pypypy/woody/.pixi/envs"
    ],
    "conda_prefix": "/Users/marcowo/Documents/pypypy/woody/.pixi/envs",
    "envs": []
}

Hmm interesting, could you share your pixi.toml?

Do you by any chance use pyproject.toml instead of pixi.toml? This is currently not supported as pixi.toml is implicitly assumed: https://github.com/pavelzw/pixi-pycharm/blob/75c3a9731d7a2e96315c9a19f6a396263a1cbab6/conda#L18

I guess that will be it. I have used a pyproject.toml. Any chance this will be supported in the near future given that pixi supports both formats now?

Yes, it's now on my radar 😄
I'll try to implement it soon, should be a fairly simple fix :)


Refactor KILToKORE to use new FlatModule data-structures in outer.scala

Also check if Configuration code in outer-ext.scala is dead-code, and remove it if so.


Volumetric Ray Casting from Viv

This brings the work from Viv https://github.com/hms-dbmi/viv/pull/415 here.
This also completes the next step in https://github.com/hubmapconsortium/portal-ui/issues/1673
Also fixes #969 and fixes #968.
The main new additions are the new coordination types spatialRotationX spatialRotationY spatialRotationZ spatialRotationOrbit spatialOrbitAxis and updates to spatialRasterLayers with slicing props, useFixedAxis and use3d.
This PR also adds spatial view state coordination types to LayerController which are needed for re-centering operations and uses memoization to prevent updates from re-rendering the component, which is expensive, especially with an even more complicated UI.

The main outstanding question now is what to do about the initialization of coordination scopes.  Right now the layer controller is not hooked up to the view state types (SPATIAL_TARGET_Z SPATIAL_ROTATION_Y etc.) by default to preserve old Vitessce behavior (judging by the comment in the coordination.js file) and also because just from a more reasonable standpoint, independence of spatial views by default makes sense.  But should we be asking people to coordinate these things even though the functionality is needed by default?  We could just document it and then also make it the default in all of the wrapper packages.  Let me know what you think @keller-mark - I would be open to auto-coordinating which would be breaking behavior but we could also hard-code something somewhere to do it without the "auto" feature.  Also, I can try to be clearer if this doesn't make sense.

This is a bit nit-picky but I do not really like the prefix use in the coordination types. How about spatialAxisFixed: boolean or if we anticipate other axis "modes" spatialAxisMode: string (e.g. "free", "fixed")

As far as the auto-coordinating, I think we should implement the auto-coordination so that the result is whatever seems most reasonable, even if it means a "breaking" change. If someone wants to avoid future breaking changes they should be using initStrategy: "none" (rather than initStrategy: "auto") anyway

Done and done!

@keller-mark Ok, I will figure that out and try out all the other demos again.  I had before, but I guess something changed.

@keller-mark Should be resolved.


Fix versioning so AppVeyor doesn't complain.

@MitchBodmer,
Thanks for your contribution as a Microsoft full-time employee or intern. You do not need to sign a CLA.
Thanks,
Microsoft Pull Request Bot


fix Selective Copying data generation bug

Fixes #7.

looks good to me. thanks for spotting this!


[🐛]: where our beloved Otterkit?

Can I ask what happens and why Otterkit as it was no more?

Found https://github.com/orgs/otterkit/discussions/42


Update OVS version from 2.14.0 to 2.14.2

The main motivation is to pick up security fixes for:

CVE-2015-8011
CVE-2020-27827
CVE-2020-35498

See https://github.com/openvswitch/ovs/blob/branch-2.14/NEWS
Signed-off-by: Antonin Bas<EMAIL_ADDRESS>

This will be cherry-picked to all maintained release branches, starting with v0.13

Codecov Report

Merging #2451 (6f1bd73) into main (9adf9c9) will decrease coverage by 17.79%.
The diff coverage is n/a.

@@             Coverage Diff             @@
##             main    #2451       +/-   ##
===========================================
- Coverage   59.79%   42.00%   -17.80%
===========================================
  Files         284      146      -138
  Lines       22171    18118     -4053
===========================================
- Hits        13258     7611     -5647
- Misses       7490     9823     +2333
+ Partials     1423      684      -739

Flag
Coverage Δ

kind-e2e-tests
?

unit-tests
42.00% <ø> (-0.02%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

 Is this important to backport this one in particular?

@antoninbas I maintain v0.11 privately. As far as I know there are some old deployments which are using v0.11. If you could update OpenvSwitch for 0.11, it would safe some of my effort. It's OK you skip. I'll do the change.

@antoninbas I notice the window ovs installation is still use 2.14.0 here https://github.com/antrea-io/antrea/blob/9adf9c9120578b50730c7a160fe7d38e09dcda2c/hack/windows/Install-OVS.ps1#L35 I suppose the same CVE will also impact windows?

Ignoring jenkins-windows-networkpolicy failure because this change doesn't affect Windows

@luolanzone It's unclear to me how much these CVEs affect Windows.
For example, this is the description for CVE-2020-35498:

A limitation in the implementation of userspace packet parsing can allow a malicious user to send a specially crafted packet causing the resulting megaflow in the kernel to be too wide, potentially causing a denial of service.

We know that the Windows datapath does not use megaflows at all, so I would say it's very possible that this CVE doesn't affect Windows.
Not sure about CVE-2020-27827.
@lzhecheng to be on the safe side we should consider using a new build for Windows as well, based on 2.14.2. Do you think you can create the build? After that we will need to upload it to downloads.antrea.io and update hack/windows/Install-OVS.ps1 in the appropriate branches (with a separate PR).

@antoninbas That's for sure.


Update MaterialComponents to the latest version(92.0.0)

Should this be merged first? https://github.com/xamarin/XamarinComponents/pull/666

@newky2k Could we have another bump to v92.4.0 before this gets possibly merged?

Hi @newky2k,
First of all thanks for maintaining the MaterialComponents bindings for Xamarin.
Why this pull request hasn't been merged yet? Do you need any help with it?

@mattleibow Could you shine some light on this PR? The material components for iOS got updated to v94.2.0 in the meanwhile.

Why this haven't marged yet?

I am not able to use the actionsheetcontroller with the latest version 92.0.0-rc2. I get the following error:
dyld: Symbol not found: OBJC_CLASS$_MDCActionSheetAction
Referenced from: /var/containers/Bundle/Application/566E3562-7714-44EE-AAB2-D4EF4A487B95/UInterface.iOS.app/UInterface.iOS
Expected in: /private/var/containers/Bundle/Application/566E3562-7714-44EE-AAB2-D4EF4A487B95/UInterface.iOS.app/Frameworks/MaterialComponents.framework/MaterialComponents
in /var/containers/Bundle/Application/566E3562-7714-44EE-AAB2-D4EF4A487B95/UInterface.iOS.app/UInterface.iOS

@newky2k Same issue here. ActionSheetController Class pointer is zero

@gadeweever Use latest stable version. It's fixed there!

@ChasakisD Thanks! For some reason it reproe'd until I cleaned the project and rebuilt. Thinks sheem okay now.


asp.net core web api

Will this library work with a web api implemented in asp.net core ?

absolutely

if i use your below sample code  then in postman  or chrome i see a blank resposne
[HttpGet("api/testStream")]
public async Task<System.Collections.Async.IAsyncEnumerable> testStream()
{
return new AsyncEnumerable(async yield =>
{
            // Just to show that ReturnAsync can be used multiple times
            await yield.ReturnAsync(0);

            for (int number = 1; number <= 10; number++)
                await yield.ReturnAsync(number);

            // You can break the enumeration loop with the following call:
            yield.Break();

            // This won't be executed due to the loop break above
            await yield.ReturnAsync(12345);
        });
    }

This is definitely is not a supported scenario as the ASP.NET framework does not know that the IAsyncEnumerable represents a collection. Your best option is:
[HttpGet("api/testStream")]
public async Task<List<int>> testStream()
{
    var collection = new AsyncEnumerable(async yield =>
    {
         await yield.ReturnAsync(12345);
    });
    return await collection.ToListAsync();
}

Thanks for the response. But if i do it as suggested by you. Then the web api will not return until  the call
return await collection.ToListAsync();.
So the client will not get the values as they are produced, they will get it when the whole list is made.

That's correct, the only other workaround, is to convert an IAsyncEnumerable to IEnumerable, what defeats the whole purpose. This feature most likely will be supported natively by ASP NET Core when C# 8.0 and .NET Core 3.0 are released.

@mj430, I'll have to close this issue as it goes beyond the scope of the current project. Perhaps it is possible to achieve the behavior that you want, but it should be a separate project that binds together ASP.NET Core MVC and AsyncEnumerable library.


Swift 5.10 - Windows toolchain build failed - Error: msiexec.exe exited with code 1619.

https://ci-external.swift.org/job/swift-5.10-windows-toolchain/8/console
swift-ci@EC2AMAZ-JCVOGM7 C:\Users\swift-ci\jenkins\workspace\swift-5.10-windows-toolchain>powershell.exe -ExecutionPolicy RemoteSigned -File C:\Users\swift-ci\jenkins\workspace\swift-5.10-windows-toolchain\swift\utils\build.ps1             -SourceCache C:\Users\swift-ci\jenkins\workspace\swift-5.10-windows-toolchain             -BinaryCache T:             -ImageRoot T:             -BuildType Release                           -Test swift,dispatch,foundation,xctest             -Stage T:\artifacts   || (exit /b 1 )
WiX not found. Downloading from nuget.org...
Extracting WiX...
Swift toolchain not found. Downloading from swift.org...
Extracting Swift toolchain...
T:\toolchains\ : warning WIX8503: Failed to extract attached container. This most often happens when extracting a stripped bundle from the package cache, which is not supported.
Error: msiexec.exe exited with code 1619.
Invocation:
  msiexec.exe /qn /a T:\toolchains\a0 TARGETDIR=T:\toolchains\swift-5.9-RELEASE-windows10
Call stack:
  at Invoke-Program, C:\Users\swift-ci\jenkins\workspace\swift-5.10-windows-toolchain\swift\utils\build.ps1: line 323
  at Ensure-SwiftToolchain, C:\Users\swift-ci\jenkins\workspace\swift-5.10-windows-toolchain\swift\utils\build.ps1:
line 458
  at <ScriptBlock>, C:\Users\swift-ci\jenkins\workspace\swift-5.10-windows-toolchain\swift\utils\build.ps1: line 1624
At C:\Users\swift-ci\jenkins\workspace\swift-5.10-windows-toolchain\swift\utils\build.ps1:327 char:7
+       throw $ErrorMessage
+       ~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : OperationStopped: (Error: msiexec....ps1: line 1624
:String) [], RuntimeException
    + FullyQualifiedErrorId : Error: msiexec.exe exited with code 1619.
Invocation:
  msiexec.exe /qn /a T:\toolchains\a0 TARGETDIR=T:\toolchains\swift-5.9-RELEASE-windows10
Call stack:
      at Invoke-Program, C:\Users\swift-ci\jenkins\workspace\swift-5.10-windows-toolchain\swift\utils\build.ps1: line
   323
      at Ensure-SwiftToolchain, C:\Users\swift-ci\jenkins\workspace\swift-5.10-windows-toolchain\swift\utils\build.ps1
   : line 458
      at <ScriptBlock>, C:\Users\swift-ci\jenkins\workspace\swift-5.10-windows-toolchain\swift\utils\build.ps1: line 1
   624

@shahmishal do we know what is different between https://ci-external.swift.org/job/swift-5.10-windows-toolchain/7/console and https://ci-external.swift.org/job/swift-5.10-windows-toolchain/8/console ?  The previous build seemed to succeed.

We had seen this previously and rebuild helped - https://github.com/apple/swift/issues/69484#issuecomment-1808749695

Going to try to bump up the wix toolset to 4.0.3 as there have been some bugfixes.  Maybe it will help?


[Not an issue but simply confirmation that crystal on bash-on-windows works for me as well] Crystal works on windows bash subsystem

Hello.
I lately noticed:
https://crystal-lang.org/docs/installation/on_bash_on_ubuntu_on_windows.html
I assume that it is a fairly recent addition?
Anyway. About 2 months ago I bought a new laptop and win10 is there, for several reasons. (That investment had to be done either way and I could use linux there; but due to some other reasons, I also have to be able to use windows, e. g. at different work places. At home I use linux exclusively.)
Since then I continued to experiment and am also "partaking", more or less, in the "windows insider builder", which allows for faster updates.
Now to make a long story short, what did I do:

I started to compile stuff from source on my win10 laptop, after setting up ubuntu (the first one in WSL but now there is opensuse too and another one).

After having a build-chain that, surprisingly, worked (using the default from ubuntu as base, but lateron replacing it with source-compiled stuff... binutils, make, sed grep etc...) , yesterday I built LLVM with shared library support. (I followed mostly the LFS/BLFS instructions from https://crystal-lang.org/docs/installation/on_bash_on_ubuntu_on_windows.html; only difference was that I used the prefix /Programs/Llvm/4.0.1 - other than that, I think I sticked to the BLFS way)

Today, I downloaded the crystal binary:

This URL specifically:
https://github.com/crystal-lang/crystal/releases/download/0.23.1/crystal-0.23.1-3-linux-x86_64.tar.gz
Extracted it, moved it about a bit, then looked if the hell world and the class example from the
official docu works. And it does!
I had to use "crystal build foo.cr" to get the executable, which also ran.
Of course this was not on a "windows" host machine per se but rather within the bash-on-windows subsystem which is ... mostly ubuntu. At the least in my case.
But the important thing, why I wanted to keep you guys informed, is that it:
a) indeed works, just as written down at https://crystal-lang.org/docs/installation/on_bash_on_ubuntu_on_windows.html - while it says it is highly experimental, it works quite fine, at the least for basic stuff (I haven't yet tested anything more complicated)
and even more importantly,
b) it worked for me without the "apt-get" way, so compiling from source works too!
Feel free to expand the docu if you want to, or not which is also fine - I only wanted to notify
you. And close this whenever you feel like it - though perhaps one or another one could
also give this a try too; I haven't yet tested to compile a new crystal with crystal yet, for
instance; and I have not yet tested a self-compiled gcc yet either (compiling on bash-on-windows
unfortunately takes quite a bit longer than on linux, from my experience - I assume it may be
because there may have to be some indirect file calls... such as whenever you modify files
and such, which I guess is more costly than natively on linux)
This is of course not full "crystal on windows", but for the time being, I think it may suffice
for a few people who also use windows a lot, for one reason or the other but want to use
crystal as well. I am sure that eventually native windows support will be 100%.
Either way, thanks for reading!

I think this post would have been better on the mailing list than on the issue tracker...


Running with docker compose

I cloned the repository, edited config.yml, and ran docker compose up on an M1 MacBook Pro.
I got this error:
[+] Running 1/0
 ✔ Container texas-dps-scheduler  Created                                                                                0.0s
Attaching to texas-dps-scheduler
texas-dps-scheduler  | node:internal/modules/cjs/loader:1051
texas-dps-scheduler  |   throw err;
texas-dps-scheduler  |   ^
texas-dps-scheduler  |
texas-dps-scheduler  | Error: Cannot find module 'tslib'
texas-dps-scheduler  | Require stack:
texas-dps-scheduler  | - /home/container/index.js
texas-dps-scheduler  |     at Module._resolveFilename (node:internal/modules/cjs/loader:1048:15)
texas-dps-scheduler  |     at Module._load (node:internal/modules/cjs/loader:901:27)
texas-dps-scheduler  |     at Module.require (node:internal/modules/cjs/loader:1115:19)
texas-dps-scheduler  |     at require (node:internal/modules/helpers:130:18)
texas-dps-scheduler  |     at Object.<anonymous> (/home/container/index.js:3:17)
texas-dps-scheduler  |     at Module._compile (node:internal/modules/cjs/loader:1233:14)
texas-dps-scheduler  |     at Module._extensions..js (node:internal/modules/cjs/loader:1287:10)
texas-dps-scheduler  |     at Module.load (node:internal/modules/cjs/loader:1091:32)
texas-dps-scheduler  |     at Module._load (node:internal/modules/cjs/loader:938:12)
texas-dps-scheduler  |     at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:83:12) {
texas-dps-scheduler  |   code: 'MODULE_NOT_FOUND',
texas-dps-scheduler  |   requireStack: [ '/home/container/index.js' ]
texas-dps-scheduler  | }
texas-dps-scheduler  |
texas-dps-scheduler  |
texas-dps-scheduler  | Node.js v20.5.1
texas-dps-scheduler  |
texas-dps-scheduler exited with code 1

Please try pull the new image and see if the issue fixed. Also please make sure pickDPSLocation is set to false if using docker :)

Thanks for the quick update! I was getting this same error as well.
pickDPSLocation is set to false.
New image has been pulled and now fails with the following:
docker logs texas-dps-scheduler
node:internal/modules/cjs/loader:1051
  throw err;
  ^

Error: Cannot find module '../../package.json'
Require stack:
- /home/container/Client/index.js
- /home/container/index.js
    at Module._resolveFilename (node:internal/modules/cjs/loader:1048:15)
    at Module._load (node:internal/modules/cjs/loader:901:27)
    at Module.require (node:internal/modules/cjs/loader:1115:19)
    at require (node:internal/modules/helpers:130:18)
    at Object.<anonymous> (/home/container/Client/index.js:14:21)
    at Module._compile (node:internal/modules/cjs/loader:1241:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1295:10)
    at Module.load (node:internal/modules/cjs/loader:1091:32)
    at Module._load (node:internal/modules/cjs/loader:938:12)
    at Module.require (node:internal/modules/cjs/loader:1115:19) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [ '/home/container/Client/index.js', '/home/container/index.js' ]
}

Node.js v20.6.0

Please try the new image and new docker-compose file to work. Tested on Ubuntu 22.04.2 LTS:
sudo docker-compose up
Recreating texas-dps-scheduler ... done
Attaching to texas-dps-scheduler
texas-dps-scheduler | [09/06/2023 6:46:12] Texas Scheduler v1.6.5 is starting...
texas-dps-scheduler | [09/06/2023 6:46:12] Requesting Avaliable Location....
texas-dps-scheduler | [09/06/2023 6:46:19] Found 5 avaliable location that match your criteria
texas-dps-scheduler | [09/06/2023 6:46:19] Lewisville, Carrollton Mega Center, Flower Mound, Denton, Plano
texas-dps-scheduler | [09/06/2023 6:46:19] Checking Avaliable Location Dates....
texas-dps-scheduler | --------------------------------------------------------------------------------
texas-dps-scheduler | [09/06/2023 6:46:25] Lewisville is not avaliable in around 0-7 days from today!
^CGracefully stopping... (press Ctrl+C again to force)
Stopping texas-dps-scheduler   ... done

This is now running properly for me on my Arch Linux host as well as an Ubuntu 22.04 testing VM.
Thank you!

@camilonova I don't think I did an image for arm64, but can you check if its working and does it running natively with apple silicon?

I got this:

dps-scheduler The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested

try to pull the newest image, i did push a new linux/arm64 image

It pulled ok, but now it gave me this error:

Did you tried the new docker-compose.yml? I updated the file along with the multi-arch image


Blade renderer breaks when mixing 2 @php-syntaxes in the same view

Laravel Version: 8.83.27
PHP Version: 7.4.32
Database Driver & Version: n/a

Description:
When mixing @php(..) and @php/@endphp later in the view the renderer breaks and gives preference to the @php/endphp pair above the parentheses.
Steps To Reproduce:
See https://github.com/rikvdh/laravel-broken-blade-view/blob/master/resources/views/welcome.blade.php
In this repo I've created a test-command to render the view (havent set up a webserver etc). But that doesnt matter.
This view is rendered wrong. You expect absolutely no output. But this breaks.
The rendered view:
<?php($hello = 'welcome')
@php
    $hello = 'goodbye';
?>

<?php echo e($hello); ?>

<?php /**PATH /Users/rikvdh/tmp/broken-view/resources/views/welcome.blade.php ENDPATH**/ ?>%

Hey there,
Unfortunately we don't support this version anymore. Please check out our support policy on which versions we are currently supporting. Can you please try to upgrade to the latest version and see if your problem persists? If so, please open up a new issue and we'll help you out.
Thanks!

@driesvints this also applies to newer versions..

I advice against mixin it like that.


prerelease tags clash

prerelease tags should be include a commit name so they don't clash? when I do git push --tags after a git pull it complains. Maybe I just need to change a git config somewhere.

solved in #5


Feature request: Support OPTIONS requests

Needed to download WordPress REST API JSON Schema files.
Example curl request: curl -X OPTIONS http://localhost/wp-json/wp/v2/posts
More info: https://timothybjacobs.com/2017/05/17/json-schema-and-the-wp-rest-api/

Nevermind, just found out that we can use the _method query parameter to actually use GET but kinda emulate an OPTIONS reuqest instead: curl http://localhost/wp-json/wp/v2/posts?_method=OPTIONS | jq .schema


Preserve filename of the original function submission

Motivation
When we submit a function, currently we don't note down the original filename that was uploaded from the client. For regular java functions this works fine. For python workers we manage by doing tricks wrt tricks like filename using classnames. However when we add things like wheel support, the filename encodes information that cannot be discarded away.
This pr preserves that filename
Modifications
Describe the modifications you've done.
Result
After your change, what will change.


New Queues for Performance Team

The performance team is getting some additional hardware as well as wanting to update their existing systems to newer operating systems. We need to create the following queues to support this

ubuntu.2204.arm64.perf

This queue is to replace the ubuntu.2004.arm64.perf queue

windows.11.amd64.viper.perf.open

Used to replace windows.11.amd64.tiger.perf.open

windows.11.amd64.viper.perf

Used to replace windows.11.amd64.tiger.perf

ubuntu.2204.amd64.viper.perf.open

Used to replace ubuntu.2204.amd64.tiger.perf.open

ubuntu.2204.amd64.viper.perf

Used to replace ubuntu.2204.amd64.tiger.perf.

windows.11.amd64.urchin.perf

Configuration is similar to windows.11.amd64.viper.perf

ubuntu 2204.amd64.urchin.perf

Configuration is similar to ubuntu.2204.amd64.viper.perf

Additionally, we would like to "rename" the windows.11.arm64.perf.surf queue to windows.11.arm64.surf.perf so it aligns with the other queues. There are currently no systems attached to this queue.

Release Note Category

[X] Feature changes/additions
[ ] Bug fixes
[ ] Internal Infrastructure Improvements

Release Note Description
Adding new queues to support PERFURCHIN and PERFVIPER systems.

@ilyas1974 I need some information here.
What are the values for the PreInstalled image for all of these queues? specifically I need to know the version of the linux kernel these machines are running before I can create the queues.

Also, I believe this needs to be done in two steps?

[ ] add the new queues
[ ] At a later date, remove the old queue definitions once machines have been moved to the new queue
Is that correct?

Had some clarification for preinstalled images:

ubuntu.2204.open.perf: We still need the PreinstalledImage information
For everything else, copy the configuration exactly from the previous queues.

Even more clarification. This is a chicken and egg problem, we will use the current information from the previous queues initially. That's going to be wrong. After the machines are set up and in the queue, we will go back and fix the values.

@ilyas1974 I see that we also have some perftiger scouting queues, such as Ubuntu.2204.Amd64.Tiger.Perf.Scouting. Do we need scouting viper and/or urchin queues?

Not at this time - I have a feeling we'll be keeping those queues long term. The goal is to allow the perf team to test an OS update before it is mass deployed to the production environment. The type of machine this is done on doesn't matter.

Working on this in PR https://dnceng.visualstudio.com/internal/_git/dotnet-helix-machines/pullrequest/35033

This has rolled out


Add Support for $locationProvider.html5Mode(true);

For using this feature I needed to add a middleware to browsersync initialization mannually. it would be nice to have it as an option when a module is created

That sounds good to me. If anyone has time for a PR that would be much appreciated.

In case anyone else is looking for how to config browsersync for HTML5 mode, add the middleware by installing this:
npm install connect-history-api-fallback --save
And update gulp/watch.js:
  ...
  gulp.task('browserSync', function() {
    $.browserSync({
      host: config.host,
      open: 'external',
      port: config.port,
      server: {
        verbose: true,
        baseDir: config.buildDir,
        middleware: $.connectHistoryApiFallback()
      }
    });
  });
  ...


libxml2 cannot be imported

import ThreeScalePY
File "/Users/.../.pyenv/versions/2.7.9/lib/python2.7/site-packages/ThreeScalePY.py", line 69, in
import libxml2
Doesn't work on heroku. Maybe switch to lxml?

Issue resolved. Switched to branch lxml


Bug: QA Cert Event Add/Edit Modal

Issue:
Calander text overlaps with other text in the QA Cert Event Modal.
Steps to Recreate:
Log in
Check out Barry 1,2, CS20N for QA Cert Events
Click add cert event
Modal pops up and calendar text is overlapping
Private Zenhub Image

Confirming that appropriate changes have been made and calender text is no longer overlapping
Private Zenhub Image


[WIP] Use build strategy runAs configuration for shipwright-managed steps

Changes
Incomplete

Submitter Checklist

[ ] Includes tests if functionality changed/was added
[ ] Includes docs if changes are user-facing
[ ] Set a kind label on this PR
[ ] Release notes block has been filled in, or marked NONE

See the contributor guide
for details on coding conventions, github and prow interactions, and the code review process.
Release Notes

Opened two separate pull requests for the different parts of this change.


[stacked_generator] Dialog generator ignore stacked configuration

Generated file for the stacked dialogs not reflex to the stacked configuration defined in the stacked.config.json file.
In the generated dialog file is imported incorrect locator dart file, it is imported default name of it, not the name defined in the stacked configuration file.
Trouble is in the implementation in the file dialog_class_generator_helper.dart, as can you see, import is firmly defined at the line 23 of the code: https://github.com/FilledStacks/stacked/blob/8fd219037003c562f99c5bfffc78293ed8d1623f/packages/stacked_generator/lib/src/generators/dialogs/generate/dialog_class_generator_helper.dart#L23
Content of the file stacked.config.json:
{
  "views_path" : "lib/src/ui",
  "services_path" : "lib/src/services",
  "stacked_app_path" : "lib/module.dart",
  "locator_name" : "locator"
}

Example of the file module.dart:
@StackedApp(dialogs: [
  StackedDialog(classType: LoginDialogCupertino),
])
class ModuleCore implements BaseModule {}

Part of the generated file module.dialog.dart:
import 'package:stacked_services/stacked_services.dart';

import 'app.locator.dart';
...

How can look right generated part of the file module.dialog.dart:
import 'package:stacked_services/stacked_services.dart';

import 'module.locator.dart';
...
}

Hey, those two packages are separate. Which is why they won't work together.
It's a good thing you picked up on.
Most likely we'll have to read the stacked.config in the generator as well. That's a new feature that will have to be built.
Thanks for taking the time to write out the details.

OK, thank you.

This feature will not be added to the upcoming improvements.
Reason

Generators get access to the location of the current path that contains the annotation in question. We don't have access to all the files with the current generator implementation.
We don't know which use cases this applies too
We don't have many users looking for this functionality


[AF-572] Implement validation to prevent the creation of v1 apps targeting v2 only locations

:v:
/cc @zendesk/vegemite
Description
Adds validation for v2 only locations in v1 apps
References

JIRA: https://zendesk.atlassian.net/browse/AF-572

does setting iframe URLs in a v1 app also error out?

I might be overengineering here, but having a 'v2' and 'v1' variables seems to smell of repetition a bit to me, would it be worth having some whitelist approach in case we add more locations for future versions?

@zendesk/i18n


Please expand readme with usage

It's not clear on how to use it without reading code. I suggest adding such info into readme.

ok


Windsor Installers added

Now when the Windsor service locator is created, it should scan all the assemblies for types implementing IWindsorInstaller and use the WindsorContainer.Install method to run them.
Also fixed a broken AutoFac test and implementation. At least I think I did. Heck, who knows it's AutoFac. But the tests pass. And that says something. I'm not sure what, but something.

I'm glad I don't use Windsor. :)
I feel like this kinda goes against the theme of MVC Turbine, but can't argue that it's not helpful.

Not sure what the theme of MVC Turbine is then. Originally, it helped you by hiding all the nasty-ness of putting a DI container into MVC and making it easier to compose your application. This just extends the kind of behavior exhibited by the built in auto-registration of Filters, Binders, etc ... Dunno if the other IoC containers have this kind of interface, I think StructureMap does. But for us, using Windsor, it has simplified our container initialization. Now we just write installers that are contained in each project assembly that register the interfaces and components contained within the assembly rather than having them in a single class. It's very similar to the IServiceRegistration interface in Turbine.

Also my Git-fu sucks so I couldn't figure out how to re-write history and remove the commits that aren't really necessary for the final result. :/

But that's what I mean:  MVC Turbine has an IServiceRegistration that seems to do the same as this IWindsorInstaller.  However, MVC Turbine does it in a way that is not dependent on any IoC container.
As an example I've brought up during MVC Turbine presentations:  I can write a MVC component that I use in my Unity application, and then pass it to Javier who can use it in his Windsor applications.
MVC Turbine doesn't just hide all of the nasty-ness of a DI container, it actually makes sharing components possible.  I can write components like MvcTurbine.FluentValidation, MvcTurbine.GoogleSiteMap, MvcTurbine.PrincipalProvider, etc. without restricting my tool to just Windsor.
I'm not arguing that this pull request should not be accepted, it is an enhancement that makes it easier to use for Windsor users.  But there are lots of features in all of the IoC containers that could be brought in, but use of those features is going to keep everybody's .Net applications like they already are:  Segregated into their own IoC container camps.

Right, you still can write a component that uses IServiceRegistration and not have to worry about the component. BUT, and this holds true mostly for Enterprise devs or devs working with shared libraries, there may be shared business libraries in an organization that currently use IWindsorInstaller to register their interfaces and impls. This allows those devs to more easily integrate those libraries into MVCTurbine and does not require them to take a dependency on MVCTurbine to use the IServiceRegistration interface.
I can see your point about using IServiceRegistration, especially if you are creating components for use in Turbine. This is more for the devs that are already using Windsor and have existing libraries that make use of the IWindsorInstaller interface. It's very similar to the Facilities support that is already built into Turbine. :)


Update dependencies for v1.0.0-rc.5 release

Signed-off-by: Nicko Guyer<EMAIL_ADDRESS>

Codecov Report

Merging #742 (19e875a) into main (094dc8a) will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##              main      #742   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files          323       323
  Lines        20075     20075
=========================================
  Hits         20075     20075

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 094dc8a...19e875a. Read the comment docs.


fold set_stopping into create_timeline_struct for Cause::Delete

See https://github.com/neondatabase/neon/pull/4552#discussion_r1258368127 for context.
Decided not to merge branches in one match because we set stopping state only when timeline is constructed, but if we're loading it then it makes more sense to have ancestor check executed first

I wonder if this is a bit sneaky but regardless, seems valid.

Yeah, that was @problame idea. I'm ok with either option, but original one looks a bit better to me, because its explicit


Copies only the directory structure on Windows

I have been completely unable to get this crate to copy files on Windows. I have a program with a number of testers on different Windows versions and with different setups for what the program is copying.
I've tried copy_dir() and CopyBuilder::new().run() (both with and without extra property-adder calls) but have never managed to get it to copy a single file on Windows on any computer it's run on. Even a simple test case of copying C:/src to C:/dst with the only file in C:/src being a text file, and it doesn't copy the text file. It copies any directory structure within the source folder perfectly, and then just calls it a day, without any error or other indication that it hasn't finished.
Am I missing some setting or feature flag? It's a bummer, as this seems to be the only remaining, viable, public implementation for recursive folder copying in Rust. On the plus side, works great on Linux!

Ended up implementing my own simple behavior, as I only needed directory and file copying for absolute paths.
Given the nature of the problem and the lack of other reports of a similar nature, it was probably some part of my code or my build environment anyway.

Thank you so much for the issue!
If you believe this is an actual bug please don't close it - I would like to look into it!

I think it might be, but honestly it's difficult to come up with a minimum reproducible example, as the underlying code is so convoluted.
I realized that my simplest tests were still utilizing the program's task dispatch system. When I made a test of just dircpy and nothing else, it worked fine. So it was probably something to do with how dircpy's file copying interacted with the task dispatch system. Unfortunately, I didn't write the task dispatch system, and I don't understand how it works at all, so I'm having trouble simplifying it to give you something to reproduce with.
I also don't understand how dircpy's file copying works, as I never followed the macro back to its definition, but if it shells out the file copy process to the OS, then it's possible dircpy sends a completion signal back to the dispatch system, which kills the thread and any shell invoked from it. Folders, which are practically trivial to make, could be finished before that happens, while files would get cut off. That's just a complete shot in the dark, though.
I'll try to figure out how to get you a reproducible example.


feat(core): relax reserved property names

Partially addresses Issue #572
This PR relaxes the parsing and core API restrictions on identifier names. The following names are now permitted:

false
true
null

Changes

Adds explicit test cases which give good coverage of valid & invalid identifiers
Changes Parser rules to allow NullLiteral and BooleanLiterals as identifiers
Removes the explicit reserved keyword check in the core API. This was only introduced recently to address a ReDOS limitation when generating sample JSON files regexes (negative lookahead was not supported). #484
Upgrades Unicode character categories to Unicode 15.0.0

Flags

I believe that this should be considered a non-breaking change, as it loosens the constraints on identifiers.

Author Checklist

[x] Ensure you provide a DCO sign-off for your commits using the --signoff option of git commit.
[x] Vital features and changes captured in unit and/or integration tests
[x] Commits messages follow AP format
[ ] Extend the documentation, if necessary
[x] Merging to main from fork:branchname

@Jonathan-Casey

I assume imposing the identifier constraints on the AST will be in a follow-up PR?

Yes, that's right


Error: Please install the R package "vscDebugger"

After installing the latest version with R Debugger > update or install the required R package I get "Please install the R package "vscDebugger".

To Reproduce
Steps to reproduce the behavior:

Install R Debugger from extensions tab
Create test file test.R
Set breakpoint
Debug with "Debug R-File" profile listed below.
See error

Your R code
print("Hello world")
Your Launch config
If applicable, the launch config that causes the bug. E.g.:
        {
            "type": "R-Debugger",
            "request": "launch",
            "name": "Debug R-File",
            "debugMode": "file",
            "workingDirectory": "${workspaceFolder}",
            "file": "${file}",
            "allowGlobalDebugging": true
        }

Desktop (please complete the following information):

OS: Ventura 13.0 (M1 mac)
R Version: 4.2.2
vscDebugger Version: 0.4.7
vscode-r-debugger Version: 0.4.7

Repeating the same steps above but installing with
install.packages("https://github.com/ManuelHentschel/vscDebugger/releases/download/v0.4.7/vscDebugger_0.4.7.tgz", repos = NULL, type = "mac.binary")
devtools::install_github("ManuelHentschel/vscDebugger", force = TRUE)

results in the same error as https://github.com/ManuelHentschel/vscDebugger/issues/131.

There have been some changes to the communication between the vscode-extension and the R package in version v.0.5.0., but also some issues with the Github actions, causing the packages to not be released correctly. To make things worse, there seem to have been some changes to how R handles UTF-8 encoding internally recently (see some of the posts on the R blog). I just released v0.5.1 for both the vscode-Extension and the R package. At least for me, they work fine with R v0.4.1, but sometimes cause issues on R v0.4.2.


Syntax Error on Ubuntu v. 16 (ChakraCore v. 1.3.1)

When executing JS script on Linux we get an error saying "SyntaxError: Invalid character"

Script
Succeeds
comment

$global.data[1];
false
Seems to fail due to semicolon

$global.data[1]
true
Succeeds without semicolon

function foo(){ var a = [true,false]; return a; }
true

function foo(){ var a = [1,2]; return a; }
true
This works, but when it comes to an objects array it again raises an error

function foo(){ var a = [{'b': true}, {'b': false}]; return a; }
false
Again with missing semicolon its working

function foo(){ var a = [{'b': true}, {'b': false}]; return a }
true
this only happens when using an object array

Basically we are using native method: JsRunScriptUtf8 to execute scripts above. Can you please give me a hint what we are doing wrong, since its seems to be a pretty weird. All scripts mentioned do not raise any error when running them on windows machines.
In addtion to that can you please provide in more detailed example on how to get to the exact error from the engine. I already made it to get the error message but the line and column number would be great to include into the error.
Thanks in advance.

@SBuder I suspect a problem with measuring the string length. If you could share some part of your code, that could help us to identify.

[DllImport( "libChakraCore.so" )]
internal static extern JavaScriptErrorCode JsRunScriptUtf8(
      byte[] script,
      JavaScriptSourceContext sourceContext,
      byte[] sourceUrl,
      out JavaScriptValue result );

We use this method of the native library, by calling it like this:
internal static JavaScriptErrorCode JsRunScript( string script,
      JavaScriptSourceContext sourceContext,
      string sourceUrl,
      out JavaScriptValue result ) {
      var scriptUtf8Bytes = Encoding.UTF8.GetBytes( script );
      var sourceUrlUtf8Bytes = Encoding.UTF8.GetBytes( sourceUrl );

return JsRunScriptUtf8(scriptUtf8Bytes, sourceContext, sourceUrlUtf8Bytes, out result );
}

I was able to solve this issue by terminating the string / UTF8 bytes explicitly with a 0
So something like:
 var scriptUtf8Bytes = Encoding.UTF8.GetBytes( script ).ToList();
scriptUtf8Bytes.Add( 0 );

var sourceUrlUtf8Bytes = Encoding.UTF8.GetBytes( sourceUrl ).ToList();
sourceUrlUtf8Bytes.Add( 0 );

Would work.


add buster layer to github runner's cache

This is to add buster image as part of the cache
Related issue:
#1358
Check list

[X] Related issue / work item is attached

Image size is 804 MB. Installation time is 12 seconds.

/azp run ubuntu1604, ubuntu1804, ubuntu2004

@arroyc , could you please confirm that we need to add this image to all ubuntu (16.04, 18.04, 20.04)?

Awesome!. Thanks @alepauly and @maxim-lobanov for the quick response on this. Are you also planning to update the following doc with the SHA digest just as you do for stretch image? (My team relies on this SHA digest)
https://github.com/actions/virtual-environments/blob/main/images/linux/Ubuntu2004-README.md

Yes, this documentation will be auto-updated on next image rollout (about 1 week)

Great. Thanks @maxim-lobanov

Thanks @maxim-lobanov  and @alepauly  .. for some reason I didn't get the notifications of this thread .. so sorry for delayed response ... all looks good so far ...


The mount source path of nfs  is not correct after deploy pod with  using CSI  NFS driver.

Enable CSI by add the below parameters

API Server binary:
--allow-privileged=true
--feature-gates=CSIPersistentVolume=true,MountPropagation=true
--runtime-config=storage.k8s.io/v1alpha1=true

Controller-manager binary:
--feature-gates=CSIPersistentVolume=true

Kubelet:
--allow-privileged=true
--feature-gates=CSIPersistentVolume=true,MountPropagation=true

Deploy NFS driver according to https://github.com/kubernetes-csi/drivers/tree/master/pkg/nfs/deploy/kubernetes

[root@ib17b13 kubernetes]# kubectl  get sts
NAME                     DESIRED   CURRENT   AGE
csi-attacher-nfsplugin   1         1         22d
[root@ib17b13 kubernetes]# kubectl get ds
NAME                       DESIRED   CURRENT   READY     UP-TO-DATE   AVAILABLE   NODE SELECTOR   AGE
csi-nodeplugin-nfsplugin   2         2         2         2            2           <none>          22d

[root@ib17b13 kubernetes]# kubectl  get serviceaccount
NAME             SECRETS   AGE
csi-attacher     1         22d
csi-nodeplugin   1         22d
default          1         24d

[root@ib17b13 kubernetes]# kubectl get clusterrole|egrep "attach|csi"
csi-nodeplugin                                                         22d
external-attacher-runner                                               22d

[root@ib17b13 kubernetes]# kubectl  get clusterrolebinding|grep csi
csi-attacher-role                                      22d
csi-nodeplugin                                         22d

Deploy a Pod using CSI volume. Before starting this example, ensure to install NFS.

[root@ib17b13 kubernetes]# showmount  -e
Export list for ib17b13:
/root/csi/nfs *

Modify pkg/nfs/examples/kubernetes/nginx.yaml, replace
csi.volume.kubernetes.io/volume-attributes: '{"server": "<IP_ADDRESS>", "share": "share "}'  as

csi.volume.kubernetes.io/volume-attributes: '{"server": "<IP_ADDRESS>", "share": "/root/csi/nfs"}'

Create nignx:
# kubectl create -f nginx.yaml
persistentvolume "data-nfsplugin" created
persistentvolumeclaim "data-nfsplugin" created
pod "nginx" created

Check the nignx container nfs path, found mount nfs server source path is not correct.
# kubectl exec nginx -it  -- bash

The nfs mount source path should be <IP_ADDRESS>:/root/csi/nfs not  <IP_ADDRESS>:/

@haoqing0110 can you help check?

@gyliu513 @mengjieli0726
The image docker.io/k8scsi/nfsplugin:v0.1 used in csi-attacher-nfsplugin.yaml and csi-nodeplugin-nfsplugin.yaml is out-of-date, inconsistent with code and example nginx.yaml
Try to build image using Dockerfile with latest nfs driver code, source path defined in "share" will be mounted correctly.
@chakri-nelluri Can you help update official nfsplugin image with latest code? as well as image tag in csi-attacher-nfsplugin.yaml and csi-nodeplugin-nfsplugin.yaml.
Thanks.

@sbezverk  @xing-yang @rootfs Can you help on this update?

@gyliu513  @haoqing0110 I would suggest a couple of things:

Use v0.2.0 images from quay.io for external-provisioner/external-attacher/registrar (no need to build them, just make sure your yaml points to them)
build nfs image from the master of kubernetes-csi/drivers

Hi @haoqing0110,
Can you try image links posted in this PR https://github.com/xing-yang/drivers/commit/dfe7d4e4e68994eabaf9a395cf0f4efd7d32874c?  The NFS plugin image is not in the official repo yet, but @rootfs has verified it works.  Thanks.

The image links under deploy/kubernetes/ are updated to point to v0.2.0 images now.  nginx.yaml is also updated.  Please test again.

@xing-yang one question, does the 0.2.0 means nfs driver version or the CSI version?

0.2.0 is the CSI version.  Our new images for NFS plugin, attacher, registrar are all tagged with 0.2.0 now.

Actually it is both.  NFS plugin version is also 0.2.0 now.


Not able to publish Workbook with Published data source

Tableau Server version is 2020.4.5
tableauserverclient version is 0.15.0
I have two questions
Question 1)
I am able to publish a workbook with a Live connection, but not able to publish a workbook with the published data source.
The main issue is I have two connections one is a live connection and the second is a published data source.
I tried to publish workbooks with a single connection, live is working but not a published data source.
Question 2)
Is there a way to convert the data source (connected to the workbook) from Dev to Prod while publishing the workbook?
analytics.d-datalake.com is like Dev DB server address
analytics.p-datalake.com is like Prod DB server address
User name and password remain the same.
I tried to convert it to Prod, even the workbook with a live connection is not working.
Find the code I am using.
import tableauserverclient as TSC
from tableauserverclient import ConnectionCredentials, ConnectionItem
server = TSC.Server("https://tableau.com/")
tableau_auth = TSC.TableauAuth('username', 'password', 'sitename')
with server.auth.sign_in(tableau_auth):
all_projects, pagination_items = server.projects.get()
default_project = next((project for project in all_projects if project.is_default()), None)
all_connections = []
connection = ConnectionItem()
connection.server_address = 'analytics.d-datalake.com'
connection.connection_credentials = ConnectionCredentials('dbuser', "dbpassword", True)
all_connections.append(connection)
all_connections
new_workbook = TSC.WorkbookItem(default_project.id)
server.workbooks.publish(new_workbook, 'Security.twbx',
TSC.Server.PublishMode.Overwrite
, connections=all_connections)
Thanks in advance

Hi @pavankumartableau
I am not sure as I did not try this on my end, but I have a couple of remarks that maybe help you out:

I can see that you are publishing to the default project, maybe it is worth creating a new one so you can do your tests there. This will also give you the freedom to configure the content_permissions parameter. Maybe something like the code below is useful to always use the same project or create a new one:

def get_or_create_project(server, project_name):
    # Get project
    for project in TSC.Pager(server.projects):
        if project.name == project_name:
            return project.id

    # Create project
    new_project = server.projects.create(
        TSC.ProjectItem(
            name=project_name,
            content_permissions="LockedToProject",
            description="Project created for testing",
        )
    )

    return new_project.id

When publishing your workbook, you are only sending out the WorkbookItem and the file_path. I believe that you should also need to populate the connections via the connections attribute in the WorkbookItem. There you should include the list of data connections for the data sources used by the workbook. In your use case, you should use include de ConnectionItem for your already-published Datasource.

This is new to me, but if you wish to change the datasource for a workbook, the same would apply as previous, as the ConnectionItem list should be updated to the ones needed for that workbook.

I hope it helps!

Hi @pavankumartableau , did you solve your problem here?


feat: Add readOnlyMouseCursor to config mouse cursor type

Description
See #1752.
Related Issues
Fix [Bug] Mouse cursor is still text insertion type when readOnly is false #1752
Checklist

[x] I read the Contributor Guide and followed the process outlined there for submitting PRs.
[x] I titled the PR using Conventional Commits.
[x] I did not modify the CHANGELOG.md nor the plugin version in pubspec.yaml files.
[x] All existing and new tests are passing.
[x] I have run the commands in ./scripts/before_push.sh and it all passed successfully

Breaking Change
Does your PR require plugin users to manually update their apps to accommodate your change?

[ ] Yes, this is a breaking change (please indicate that with a ! in the title as explained in Conventional Commits).
[x] No, this is not a breaking change.

Can you address the failures?

Oh I see those failures are not related to your PR

If you can help address the failures, that would be great

If you can help address the failures, that would be great

I am sorry that I can't figure out what's wrong with CI, I haven't use Linux.

It's related to the color picker package
I will try to temporarily clone it in the project  in lib/packages in flutter_quill
It's already updated in the GitHub repository, but for some reason, it's not published


Gibbon::MailChimpError: the server responded with status 400 @title="Member Exists"<EMAIL_ADDRESS>is already a list member.  Use PUT to insert or update list members.", @body={"type"=>"http://developer.mailchimp.com/documentation/mailchi

View details in Rollbar: https://rollbar.com/trystant/BOW/items/290/

Faraday::ClientError: the server responded with status 400
  File "/Users/dariusgoore/development/blackops/.bundle/gems/ruby/2.2.0/gems/faraday-0.9.1/lib/faraday/response/raise_error.rb", line 13, in on_complete
  File "/Users/dariusgoore/development/blackops/.bundle/gems/ruby/2.2.0/gems/faraday-0.9.1/lib/faraday/response.rb", line 9, in block in call
  File "/Users/dariusgoore/development/blackops/.bundle/gems/ruby/2.2.0/gems/faraday-0.9.1/lib/faraday/response.rb", line 57, in on_complete
  File "/Users/dariusgoore/development/blackops/.bundle/gems/ruby/2.2.0/gems/faraday-0.9.1/lib/faraday/response.rb", line 8, in call
  File "/Users/dariusgoore/development/blackops/.bundle/gems/ruby/2.2.0/gems/faraday-0.9.1/lib/faraday/rack_builder.rb", line 139, in build_response
  File "/Users/dariusgoore/development/blackops/.bundle/gems/ruby/2.2.0/gems/faraday-0.9.1/lib/faraday/connection.rb", line 377, in run_request
  File "/Users/dariusgoore/development/blackops/.bundle/gems/ruby/2.2.0/gems/faraday-0.9.1/lib/faraday/connection.rb", line 177, in post
  File "/Users/dariusgoore/development/blackops/.bundle/gems/ruby/2.2.0/gems/gibbon-2.2.1/lib/gibbon/api_request.rb", line 11, in post
  File "/Users/dariusgoore/development/blackops/.bundle/gems/ruby/2.2.0/gems/gibbon-2.2.1/lib/gibbon/request.rb", line 38, in create
  File "/Users/dariusgoore/development/blackops/app/models/user.rb", line 37, in add_to_mailchimp
  File "/Users/dariusgoore/development/blackops/.bundle/gems/ruby/2.2.0/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb", line 432, in block in make_lambda
  File "/Users/dariusgoore/development/blackops/.bundle/gems/ruby/2.2.0/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb", line 239, in call
  File "/Users/dariusgoore/development/blackops/.bundle/gems/ruby/2.2.0/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb", line 239, in block in halting
  File "/Users/dariusgoore/development/blackops/.bundle/gems/ruby/2.2.0/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb", line 506, in call
  File "/Users/dariusgoore/development/blackops/.bundle/gems/ruby/2.2.0/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb", line 506, in block in call
  File "/Users/dariusgoore/development/blackops/.bundle/gems/ruby/2.2.0/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb", line 506, in each
  File "/Users/dariusgoore/development/blackops/.bundle/gems/ruby/2.2.0/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb", line 506, in call
  File "/Users/dariusgoore/development/blackops/.bundle/gems/ruby/2.2.0/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb", line 92, in __run_callbacks__
  File "/Users/dariusgoore/development/blackops/.bundle/gems/ruby/2.2.0/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb", line 778, in _run_validation_callbacks
  File "/Users/dariusgoore/development/blackops/.bundle/gems/ruby/2.2.0/gems/activemodel-<IP_ADDRESS>/lib/active_model/validations/callbacks.rb", line 113, in run_validations!
  File "/Users/dariusgoore/development/blackops/.bundle/gems/ruby/2.2.0/gems/activemodel-<IP_ADDRESS>/lib/active_model/validations.rb", line 338, in valid?
  File "/Users/dariusgoore/development/blackops/.bundle/gems/ruby/2.2.0/gems/activerecord-<IP_ADDRESS>/lib/active_record/validations.rb", line 58, in valid?
  File "/Users/dariusgoore/development/blackops/.bundle/gems/ruby/2.2.0/gems/activerecord-<IP_ADDRESS>/lib/active_record/validations.rb", line 83, in perform_validations
  File "/Users/dariusgoore/development/blackops/.bundle/gems/ruby/2.2.0/gems/activerecord-<IP_ADDRESS>/lib/active_record/validations.rb", line 37, in save
  File "/Users/dariusgoore/development/blackops/.bundle/gems/ruby/2.2.0/gems/activerecord-<IP_ADDRESS>/lib/active_record/attribute_methods/dirty.rb", line 21, in save
  File "/Users/dariusgoore/development/blackops/.bundle/gems/ruby/2.2.0/gems/activerecord-<IP_ADDRESS>/lib/active_record/transactions.rb", line 286, in block (2 levels) in save
  File "/Users/dariusgoore/development/blackops/.bundle/gems/ruby/2.2.0/gems/activerecord-<IP_ADDRESS>/lib/active_record/transactions.rb", line 351, in block in with_transaction_returning_status
  File "/Users/dariusgoore/development/blackops/.bundle/gems/ruby/2.2.0/gems/activerecord-<IP_ADDRESS>/lib/active_record/connection_adapters/abstract/database_statements.rb", line 211, in transaction
  File "/Users/dariusgoore/development/blackops/.bundle/gems/ruby/2.2.0/gems/activerecord-<IP_ADDRESS>/lib/active_record/transactions.rb", line 220, in transaction
  File "/Users/dariusgoore/development/blackops/.bundle/gems/ruby/2.2.0/gems/activerecord-<IP_ADDRESS>/lib/active_record/transactions.rb", line 348, in with_transaction_returning_status
  File "/Users/dariusgoore/development/blackops/.bundle/gems/ruby/2.2.0/gems/activerecord-<IP_ADDRESS>/lib/active_record/transactions.rb", line 286, in block in save
  File "/Users/dariusgoore/development/blackops/.bundle/gems/ruby/2.2.0/gems/activerecord-<IP_ADDRESS>/lib/active_record/transactions.rb", line 301, in rollback_active_record_state!
  File "/Users/dariusgoore/development/blackops/.bundle/gems/ruby/2.2.0/gems/activerecord-<IP_ADDRESS>/lib/active_record/transactions.rb", line 285, in save
  File "/Users/dariusgoore/development/blackops/.bundle/gems/ruby/2.2.0/gems/activerecord-<IP_ADDRESS>/lib/active_record/persistence.rb", line 252, in block in update
  File "/Users/dariusgoore/development/blackops/.bundle/gems/ruby/2.2.0/gems/activerecord-<IP_ADDRESS>/lib/active_record/transactions.rb", line 351, in block in with_transaction_returning_status
  File "/Users/dariusgoore/development/blackops/.bundle/gems/ruby/2.2.0/gems/activerecord-<IP_ADDRESS>/lib/active_record/connection_adapters/abstract/database_statements.rb", line 213, in block in transaction
  File "/Users/dariusgoore/development/blackops/.bundle/gems/ruby/2.2.0/gems/activerecord-<IP_ADDRESS>/lib/active_record/connection_adapters/abstract/transaction.rb", line 184, in within_new_transaction
  File "/Users/dariusgoore/development/blackops/.bundle/gems/ruby/2.2.0/gems/activerecord-<IP_ADDRESS>/lib/active_record/connection_adapters/abstract/database_statements.rb", line 213, in transaction
  File "/Users/dariusgoore/development/blackops/.bundle/gems/ruby/2.2.0/gems/activerecord-<IP_ADDRESS>/lib/active_record/transactions.rb", line 220, in transaction
  File "/Users/dariusgoore/development/blackops/.bundle/gems/ruby/2.2.0/gems/activerecord-<IP_ADDRESS>/lib/active_record/transactions.rb", line 348, in with_transaction_returning_status
  File "/Users/dariusgoore/development/blackops/.bundle/gems/ruby/2.2.0/gems/activerecord-<IP_ADDRESS>/lib/active_record/persistence.rb", line 250, in update
  File "/Users/dariusgoore/development/blackops/app/controllers/users_controller.rb", line 12, in update
  File "/Users/dariusgoore/development/blackops/.bundle/gems/ruby/2.2.0/gems/actionpack-<IP_ADDRESS>/lib/action_controller/metal/implicit_render.

This occurred while testing the gibbon gem on localhost.  closing.


perf: Use @@toStringTag and Map for better perf

There are 2 macro optimizations:

Use instanceof rather than toString, this is about 100% faster, and makes things more accurate.
Use Map rather than Object, this is about 30% faster.

However, I am not sure about isPromise. According to the name, it only want to judge whether it's a Promise or not. But current logic is to judge whether it's thennable. This might be a breaking change.

instanceof cannot work across <iframe> boundaries. A common alternative is to use Object.prototype.toString to determine the type of an object. This covers more use cases.

@Justineo Thanks. I hardly use iframe and did not think about them...
Use Symbol.toStringTag instead if possible, and it's even faster. However, I found the profiling time vary a lot...

You could consider setting up some benchmarks on platforms like jsbench.me.

Here is my benchmark example: https://jsbench.me/krm45pp7nd/1
And here is the result:

Current: 2266万 ops/s ± 3.28%
instanceof: 2.3亿 ops/s ± 1.22%
@@toStringTag: 2.2亿 ops/s ± 2.1%

Looks good.

/ecosystem-ci run

Use Map rather than Object, this is about 30% faster.

I got different performance data, see https://measurethat.net/Benchmarks/Show/32894/1/test-map-vs-object

@edison1105
here is my 5 cents about benchmark:
https://measurethat.net/Benchmarks/Show/32902/0/test-map-vs-object-with-string-which-use-string-as-key
For set-value, the perf is similar, for get-value, it's 200% faster(wow...) as shown in this case. I think in the real world case, the key should be more frequently different with last time visited?

LGTM


Version 4.0 migration issues

I have issues converting to version 4.0 without data loss. This probably relates to me having used SwiftyUserDefaults in a weird way before 🙃
For version 3, I have the following extension in my code:
extension UserDefaults {
    subscript(key: DefaultsKey<Data?>) -> Data? {
        get { return unarchive(key) }
        set { archive(key, newValue) }
    }
}

To decode into a type, I use the following code in version 3:
if let myModelData = Defaults[.myModel], // DefaultsKey.myModel is of type DefaultsKey<Data?>
   let myModel = try? JSONDecoder().decode(MyModel.self, from: myModelData)
{
    // ...
}

For version 4, I tried to use a DefaultsKey<MyModel?> directly, but Defaults[.myModel] returns nil instead of a MyModel instance. Am I doing something wrong? Or is there no possible way to migrate from my weird use of version 3 SwiftyUserDefaults to a proper use of version 4 SwiftyUserDefaults without data loss? Thanks in advance!

Hey @fredpi! First of all thank you very much for testing the new version even though it's a beta. I've been working on it a long time and any feedback on the current state of the library is priceless.
Coming back to your issue: from what you've written yeah, it seems like your usage is quite unusual, but since version 3 had many (mostly Swift-based) limitations I don't blame you at all.
What I think is happening here is that you were archiving your already-converted Data to Data again (double-archiving basically). Thus, it can't unarchive to given model as it needs to do it twice.
Good news is that you probably can make it work with version 4. In the newest version we use something called "bridges" to save or fetch values from UserDefaults. For all codable-represented types there is a DefaultsCodableBridge that does the transition:
public final class DefaultsCodableBridge<T: Codable>: DefaultsBridge<T> {

    public override func save(key: String, value: T?, userDefaults: UserDefaults) {
        guard let value = value else {
            userDefaults.removeObject(forKey: key)
            return
        }
        userDefaults.set(encodable: value, forKey: key)
    }

    public override func get(key: String, userDefaults: UserDefaults) -> T? {
        guard let data = userDefaults.data(forKey: key) else {
            return nil
        }
        return deserialize(data)
    }

    public override func isSerialized() -> Bool {
        return true
    }

    public override func deserialize(_ object: Any) -> T? {
        guard let data = object as? Data else { return nil }

        return try? JSONDecoder().decode(T.self, from: data)
    }
}

You can customize these bridges in your types however you want, e.g.:
struct MyModel: DefaultsSerializable {

    static var _defaults: DefaultsBridge<MyModel> { return DefaultsCustomBridge() }
    static var _defaultsArray: DefaultsBridge<[MyModel]> { return DefaultsCustomBridge() }

    let name: String
}

So now we would need to make our own custom bridge that would double deserialize your Data into a custom codable-represented model:
public final class DefaultsArchivedCodableBridge<T: Codable>: DefaultsBridge<T> {

    public override func save(key: String, value: T?, userDefaults: UserDefaults) {
        guard let value = value else {
            userDefaults.removeObject(forKey: key)
            return
        }
        userDefaults.set(encodable: value, forKey: key)
    }

    public override func get(key: String, userDefaults: UserDefaults) -> T? {
        guard let data = userDefaults.data(forKey: key) else {
            return nil
        }
        // Let's try deserialize to Codable first
        if let codable = deserialize(data) {
            return codable
        } else if let unarchivedData = NSKeyedUnarchiver.unarchiveObject(with: data) {
            // in case it didn't succeed, let's unarchive the Data again and then deserialize
            return deserialize(unarchivedData)
        } else {
            // okay now I don't know the reason it failed, you would need to debug what's happening here
            return nil
        }
    }

    public override func isSerialized() -> Bool {
        return true
    }

    public override func deserialize(_ object: Any) -> T? {
        guard let data = object as? Data else { return nil }

        return try? JSONDecoder().decode(T.self, from: data)
    }
}

This bridge would not only get back your Data when it's double-serialized, but it would also save it correctly the next time and fetch it correctly the next time again. At least in theory, didn't test it really 😅
So you'd need to use this new bridge in your model now:
struct MyModel: DefaultsSerializable {

    static var _defaults: DefaultsBridge<MyModel> { return DefaultsArchivedCodableBridge() }
    static var _defaultsArray: DefaultsBridge<[MyModel]> { return DefaultsArchivedCodableBridge() }

    let name: String
}

Oh that's a lot of text! Please let me know if this makes sense and if it helped.

Hey @sunshinejr, thanks for this fanatastic support!!! I'm quite overwhelmed to receive such valuable ideas in almost no time. 🚀🚀🚀
You pointed me into the right direction, so I could create the following bridge that's almost identical to what you proposed:
final class UserDefaultsv3v4CompatibleBridge<T: Codable>: DefaultsBridge<T> {
    override func save(key: String, value: T?, userDefaults: UserDefaults) {
        guard let value = value else { return userDefaults.removeObject(forKey: key) }

        let encoder = JSONEncoder()
        if let data = try? encoder.encode(value) {
            userDefaults.set(data, forKey: key)
        } else {
            assertionFailure("Encodable \(T.self) is not _actually_ encodable to any data...Please fix 😭")
        }
    }

    override func get(key: String, userDefaults: UserDefaults) -> T? {
        guard let data = userDefaults.data(forKey: key) else { return nil }

        if let codable = deserialize(data) {
            return codable
        } else if let unarchivedData = NSKeyedUnarchiver.unarchiveObject(with: data) {
            return deserialize(unarchivedData)
        } else {
            return nil
        }
    }

    override func isSerialized() -> Bool {
        return true
    }

    override func deserialize(_ object: Any) -> T? {
        guard let data = object as? Data else { return nil }
        return try? JSONDecoder().decode(T.self, from: data)
    }
}

Unfortunately, the set(encodable:forKey:) interface extending UserDefaults has an internal protection level (see here), so I had to copy its functionality into my bridge. I'd propose to make it public, as it may be useful for custom bridges (such as the one I implemented).
While implementing the "migration" bridge, I noticed it wasn't compatible with arrays of Codable. In Version 3, I encoded each of the items in the array separately, so I had an array of Data. To get my data back in Version 4, I implemented the following bridge that works just as expected:
final class CodableArrayBridge<T: Codable>: DefaultsBridge<[T]> {
    override func save(key: String, value: [T]?, userDefaults: UserDefaults) {
        guard let value = (value.flatMap { serializeArrayItems(of: $0) }) else { return userDefaults.removeObject(forKey: key) }
        userDefaults.set(value, forKey: key)
    }

    override func get(key: String, userDefaults: UserDefaults) -> [T]? {
        guard let array = userDefaults.array(forKey: key) else { return nil }

        if let codableArray = deserializeArrayItems(in: array) {
            return codableArray
        } else {
            return nil
        }
    }

    private func serializeArrayItems(of array: [T]) -> [Data]? {
        let encodedArray = array.compactMap { try? JSONEncoder().encode($0) }
        guard encodedArray.count == array.count else { return nil }
        return encodedArray
    }

    private func deserializeArrayItems(in array: [Any]) -> [T]? {
        let decodedArray = array.compactMap { item -> T? in
            guard let data = item as? Data else { return nil }
            return try? JSONDecoder().decode(T.self, from: data)
        }

        guard decodedArray.count == array.count else { return nil }
        return decodedArray
    }
}

I'm however wondering why such a bridge isn't part of the framework in the first place. There is probably a better way to store Codable arrays in version 4, is it?

Hey @fredpi! Glad you figured it out and that it works as expected - I'm happy that the current system is capable of doing something like that, it feels powerful.
So the Codable array issue. If you look at the array bridge for Codable it's basically the same class as for a single value bridge. This is because an array of Codable values is a Codable itself!
And, as you probably know, Codable is first serialized into Data and then saved to the property list that UserDefaults uses underneath the public API layer. This is the reason why it didn't work for you - you had an array of data values saved into property list instead of one data blob that can be deserialized into an array.
Also, great job on the custom array bridge! One thing to note is that you should implement isSerialized and deserialize methods if you want to use KVO in the future with your types.
And yeah I agree, we could not only make set(encodable:) but also probably make all the bridges we have in the library open instead of public to make it easier to create custom bridges. If you would like to make a PR with given updates I would be really grateful!

@sunshinejr Thanks for your further explanations and suggestions! I adjusted my bridge accordingly, and it still works as expected 👍
I opened https://github.com/radex/SwiftyUserDefaults/issues/186 to track progress on the public / open matter and I will happily create a PR for this soon.
Closing this issue now, as my migration issues are resolved 🎊