Error finding appsource.json

PLEASE DO NOT INCLUDE ANY PASSWORDS OR TOKENS IN YOUR ISSUE!!!
Describe the issue
Getting an error in DevOps-Pipeline.ps1 where it does not find the appsource.json file that is in my repo.  There is one (see image).  I am also getting warnings that I have suppressed in my ruleset.json so it does not seem to be finding that either.  The files are being copied to the container into the C:\source folder.
Scripts used to create container and cause the issue
Param(
    [Parameter(Mandatory=$false)]
    [ValidateSet('AzureDevOps','GithubActions','GitLab')]
    [string] $environment = 'AzureDevOps',
    [Parameter(Mandatory=$true)]
    [string] $version,
    [Parameter(Mandatory=$false)]
    [int] $appBuild = 0,
    [Parameter(Mandatory=$false)]
    [int] $appRevision = 0
)

if ($environment -eq "AzureDevOps") {
    $buildArtifactFolder = $ENV:BUILD_ARTIFACTSTAGINGDIRECTORY
}
elseif ($environment -eq "GitHubActions") {
    $buildArtifactFolder = Join-Path $ENV:GITHUB_WORKSPACE "output"
    New-Item $buildArtifactFolder -ItemType Directory | Out-Null
}

$baseFolder = (Get-Item (Join-Path $PSScriptRoot "..")).FullName
. (Join-Path $PSScriptRoot "Read-Settings.ps1") -environment $environment -version $version
. (Join-Path $PSScriptRoot "Install-BcContainerHelper.ps1") -bcContainerHelperVersion $bcContainerHelperVersion -genericImageName $genericImageName

$params = @{}
$insiderSasToken = "$ENV:insiderSasToken"
$licenseFile = "$ENV:licenseFile"
$codeSigncertPfxFile = "$ENV:CodeSignCertPfxFile"
if (!$doNotSignApps -and $codeSigncertPfxFile) {
    if ("$ENV:CodeSignCertPfxPassword" -ne "") {
        $codeSignCertPfxPassword = try { "$ENV:CodeSignCertPfxPassword" | ConvertTo-SecureString } catch { ConvertTo-SecureString -String "$ENV:CodeSignCertPfxPassword" -AsPlainText -Force }
        $params = @{
            "codeSignCertPfxFile" = $codeSignCertPfxFile
            "codeSignCertPfxPassword" = $codeSignCertPfxPassword
        }
    }
    else {
        $codeSignCertPfxPassword = $null
    }
}

$allTestResults = "testresults*.xml"
$testResultsFile = Join-Path $baseFolder "TestResults.xml"
$testResultsFiles = Join-Path $baseFolder $allTestResults
if (Test-Path $testResultsFiles) {
    Remove-Item $testResultsFiles -Force
}

Run-AlPipeline @params `
    -pipelinename $pipelineName `
    -containerName $containerName `
    -imageName $imageName `
    -artifact $artifact.replace('{INSIDERSASTOKEN}',$insiderSasToken) `
    -memoryLimit $memoryLimit `
    -baseFolder $baseFolder `
    -licenseFile $LicenseFile `
    -installApps $installApps `
    -previousApps $previousApps `
    -appFolders $appFolders `
    -testFolders $testFolders `
    -doNotRunTests:$doNotRunTests `
    -testResultsFile $testResultsFile `
    -testResultsFormat 'JUnit' `
    -installTestFramework:$installTestFramework `
    -installTestLibraries:$installTestLibraries `
    -installPerformanceToolkit:$installPerformanceToolkit `
    -enableCodeCop:$enableCodeCop `
    -enableAppSourceCop:$enableAppSourceCop `
    -enablePerTenantExtensionCop:$enablePerTenantExtensionCop `
    -enableUICop:$enableUICop `
    -azureDevOps:($environment -eq 'AzureDevOps') `
    -gitLab:($environment -eq 'GitLab') `
    -AppSourceCopMandatoryAffixes $appSourceCopMandatoryAffixes `
    -AppSourceCopSupportedCountries $appSourceCopSupportedCountries `
    -additionalCountries $additionalCountries `
    -buildArtifactFolder $buildArtifactFolder `
    -CreateRuntimePackages:$CreateRuntimePackages `
    -appBuild $appBuild -appRevision $appRevision

if ($environment -eq 'AzureDevOps') {
    Write-Host "##vso[task.setvariable variable=TestResults]$allTestResults"
}

Pipeline output attached.
[output.txt](https://github.com/microsoft/navcontainerhelper/files/5607553/output.txt)
**Screenshots**
If applicable, add screenshots to help explain your problem.
![image](https://user-images.githubusercontent.com/4333956/100443864-6444d580-30a2-11eb-98d9-14db9e8ebf70.png)
![image](https://user-images.githubusercontent.com/4333956/100443956-8c343900-30a2-11eb-9a99-3b38ce061cda.png)
![image](https://user-images.githubusercontent.com/4333956/100444174-ee8d3980-30a2-11eb-8116-e983f4b04457.png)

**Additional context**
- does it happen all the time? yes
- did it use to work? No

Looks like the images did not come over.  here they are again.

And here is the file
output.txt

At this time, the appsourcecop.json is generated from the pipeline which means that it is hard to add rulesets.
I have it on my plate to add support for this and rulesets.
Will keep this open and you will be able to see when this will work.

So is there a way to add my prefix to the created appsource.json?

Run-AlPipeline -AppSourceCopMandatoryAffixes @('fab')
Reason for the generation is that it needs to add previous app to enable running the breaking change analyzer.

Will be adding a rulesetfile parameter to run-alpipeline


vertically center text in banner

Took me a while to spot the difference, even when switching tabs from the deployment preview to the deployed homepage 😅


Segfault on Asahi Linux M1 Max

Does this issue occur when all extensions are disabled?: Yes/No

VS Code Version:  current latest aarch64 release on linux
OS Version: Asahi Linux

Steps to Reproduce:

in asahi linux M1 Max, I ran the latest code-insider and the latest code with segfault

victoryang00@victoryang00-mac:~/Downloads/VSCode-linux-arm64/bin -$ ./code-insiders
./code-insiders: line 53: 46797 Segmentation fault      ELECTRON_RUN_AS_NODE=1 "$ELECTRON" "$CLI" --ms-enable-electron-run-as-node "$@"

Can you provide the crash log following the steps at https://github.com/microsoft/vscode/wiki/Native-Crash-Issues#creating-and-symbolicating-local-crash-reports

It sounds like the crash occur before it can output the crash report.
I have other unusable desktop like telegram, I think the problem is because of the allocator don't support 16KB page size.
 telegram-desktop
<jemalloc>: Unsupported system page size
<jemalloc>: Unsupported system page size
GLib (gthread-posix.c): Unexpected error from C library during 'malloc': Cannot allocate memory.  Aborting.
[1]    1788 IOT instruction (core dumped)  telegram-desktop

The runtime uses a custom allocator https://chromium.googlesource.com/chromium/src/+/HEAD/base/allocator/partition_allocator/PartitionAlloc.md and from the log the runtime is incorrectly using jemalloc, please check if you LD_PRELOAD that might have caused this allocator to be loaded.

The same problem with M1 Air (2020) :
VS Code Version: current latest aarch64 release on linux
OS Version: Asahi Linux

LD_PRELOAD is clean

As this page say: https://github.com/AsahiLinux/docs/wiki/Software-known-to-have-issues-with-16k-page-size
You need build electron yourself and backport this patch: https://chromium.googlesource.com/chromium/src/+/a704c3a94179e6f3abb28f090d096ced72657d7c

@12101111 thanks for linking to the chromium CL
I will wait for the upstream issue to be verified before backporting the patch to Electron https://bugs.chromium.org/p/chromium/issues/detail?id=1301788

As this page say: https://github.com/AsahiLinux/docs/wiki/Software-known-to-have-issues-with-16k-page-size
You need build electron yourself and backport this patch: https://chromium.googlesource.com/chromium/src/+/a704c3a94179e6f3abb28f090d096ced72657d7c

Yeah, my solution is to build a 4k page size latest Linux kernel for all the stuff to work.


Suggestion: Link to external sources for Trainer metadata

Trainerdex and silphroad traveler cards are both great hooks for a trainer's profile. It would be cool if a trainer could add links to their profiles on those websites in order to link their metadata to it. Linking to those locations would also allow you to pull metadata from that trainer if you wanted to display their trainer level or their location or their actual Pokedex statistics compared to their photodex statistics.
Will mock-up something basic as an example of how I think it could look and post to this issue in the next day or so.

In the absence of any mock ups from you I am going to close this. Feel free to create a pull request if you ever manage to get something working yourself.


Automated cherry pick of #6403

Cherry pick of #6403 on cloud-beta.
/cc  @fmunshi

Thanks for this fix @fmunshi. Cherry picking for the cloud beta branch.


Allow linkifier tokens to be set (and used) per stream

Linkification is great.  Being able to put an issue number in chat and get the full link to that issue is very convenient.  We used to do it with Redmine and IRC.  We never set that up again after moving to GitLab for project management, but now that we're moving to Zulip and i saw how easy it is to define linkifiers, I went to set it up again— and realized there's a problem.
For GitLab, GitHub, and undoubtedly other project management tools and bug trackers, issue numbers are repository-specific.  This means there's no way to drop an issue number or (easily) a commit hash and get a working link— unless Linkifiers can be context-aware, and we can add tokens to streams to define that context.
For example, an administrator could define two tokens available to streams— organization and repo (and perhaps the administrator could set defaults, say, zulip and zulip.  Then in stream Terminal for discussing the Zulip Terminal, a user could set the repo token to zulip-terminal in the stream settings.  Then the issue number example (hash followed by a number of any length) could look like this:

Pattern: #(?P<id>[0-9]+)
URL format string: https://github.com/$organization/$repo/issues/%(id)s
Original text: #352
Automatically links to: https://github.com/zulip/zulip-terminal/issues/352

For organizations that use one stream per project like we do, allowing the definition of token fields that can be set in streams, and letting linkifiers use these tokens, addresses the use case that inspired #11222 also.  Rather than trying multiple repositories for commit hashes, the linkification pattern could make use of tokens that get different values for each stream.
So this is two separate, but simple, enhancements: organization-wide definition of tokens that can be set per stream (probably called "tokens" rather than "linkifier tokens", as i'm sure per-stream tokens could be useful in other contexts, such as bots) and Linkfier URL format strings being able to interpret these tokens.

@mlncn thanks for the report!  https://github.com/zulip/zulip/issues/4627 is the existing issue for this (which I think @aero31aero had plans to work on soon).
In the meantime, I think you can configure the zulip/zulip#1234 style syntax.

For our needs, this would be a great feature to have!
Another possible implementation would be to provide some environment variables as options to be included in the regex.  If "$stream" and "$topic" could be inserted into the linkifier then it would be easy to contextualize the linkifier.  For example:
https://github.com/myorg/$topic/issues/%(id)s or https://github.com/myorg/$stream/issues/%(id)s or any combination would be possible.
This approach would have the advantage of no more UI or configuration complexity in the settings.  Presumably, just listing these as options in the instructions would be sufficient for adminstrators to understand how to use them.  Of course, the disadvantage is that it is less customizable.

Closing as a duplicate of #4627.


Wrap by default on FlexLayout and tweak layout storybook examples

These changes have been reviewed by Stella. They mostly include small tweaks to the storybook examples but the most important change is allowing the FlexLayout component to wrap by default

Why was the wrap being set by default? This shouldn't be an accessibility decision? We changed to nowrap previously because that is the default value for flex-wrap. This now causes mind bending debug issue when using flex layout...?

Why was the wrap being set by default? This shouldn't be an accessibility decision? We changed to nowrap previously because that is the default value for flex-wrap. This now causes mind bending debug issue when using flex layout...?

Stella's point was around some of the examples having a double scroll bar when zoomed in so her suggestion was that the component should always wrap by default. If we think this causes debugging issues maybe we could revert the change and add an accessibility section to the docs that explains this?

We can make the examples wrap, but i don't think think changing default value of any CSS is a good idea. Personally, i'm expecting flex and other layouts are some drop in replacement for not writing CSS line by line myself, if we enforce different default value than what CSS gives, we're driving into corners.

We can make the examples wrap, but i don't think think changing default value of any CSS is a good idea. Personally, i'm expecting flex and other layouts are some drop in replacement for not writing CSS line by line myself, if we enforce different default value than what CSS gives, we're driving into corners.

Makes sense although I wonder if changing the examples only (particularly the "default" one) would be a bit misleading as well


KOGITO-8303: [SWF Dev UI] Add support for starting workflows using gateway API and cloud events

This is part of an ensemble, please merge with:
https://github.com/kiegroup/kogito-apps/pull/1592
https://github.com/kiegroup/kogito-examples/pull/1538
https://github.com/kiegroup/kogito-docs/pull/291

JIRA:
https://issues.redhat.com/browse/KOGITO-8303

Description:
Modifications on the cloud event form page to enable adding a better cloud event configuration.
On the start workflow form screen, It also adds the ability to show a code editor to manually type the post payload if no form is available.

Please make sure that your PR meets the following requirements:

[ ] You have read the contributions doc
[ ] Pull Request title is properly formatted: KOGITO-XYZ Subject
[ ] Pull Request title contains the target branch if not targeting main: [0.9.x] KOGITO-XYZ Subject
[ ] The nav.adoc file has a link to this guide in the proper category
[ ] The index.adoc file has a card to this guide in the proper category, with a meaningful description

How to backport a pull request to a different branch?

In order to automatically create a backporting pull request please add one or more labels having the following format backport-<branch-name>, where <branch-name> is the name of the branch where the pull request must be backported to (e.g., backport-7.67.x to backport the original PR to the 7.67.x branch).

NOTE: backporting is an action aiming to move a change (usually a commit) from a branch (usually the main one) to another one, which is generally referring to a still maintained release branch. Keeping it simple: it is about to move a specific change or a set of them from one branch to another.

Once the original pull request is successfully merged, the automated action will create one backporting pull request per each label (with the previous format) that has been added.
If something goes wrong, the author will be notified and at this point a manual backporting is needed.

NOTE: this automated backporting is triggered whenever a pull request on main branch is labeled or closed, but both conditions must be satisfied to get the new PR created.

@kaldesai your turn again

what is the status of that PR ?

@pefernan what is the status of this PR ?

@radtriste god I thought it was already merged! let me take a look please.

@pefernan @radtriste I'd like to merge this one if you don't mind reviewing it.

@MarianMacik already approved so I am good with it ;)

Let's go for it!@ricardozanini


I have Python 3.4 but get "gmusicapi-wrapper does not support this version of Python"

In Ubuntu 14.04 I have both python 2.7 and 3.4 installed but when I try to install this with pip I get this "gmusicapi-wrapper does not support this version of Python" error.

Make sure you're running it with 3.4 and not 2.7.

Ok, I think I've got that figured out but it's still giving me an error for the wrong version of setuptools.
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/pkg_resources.py", line 444, in _build_master
    ws.require(__requires__)
  File "/usr/lib/python3/dist-packages/pkg_resources.py", line 725, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/usr/lib/python3/dist-packages/pkg_resources.py", line 632, in resolve
    raise VersionConflict(dist,req) # XXX put more info here
pkg_resources.VersionConflict: (setuptools 3.3 (/usr/lib/python3/dist-packages), Requirement.parse('setuptools>=11.3'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/gmupload", line 5, in <module>
    from pkg_resources import load_entry_point
  File "/usr/lib/python3/dist-packages/pkg_resources.py", line 2749, in <module>
    working_set = WorkingSet._build_master()
  File "/usr/lib/python3/dist-packages/pkg_resources.py", line 446, in _build_master
    return cls._build_from_requirements(__requires__)
  File "/usr/lib/python3/dist-packages/pkg_resources.py", line 459, in _build_from_requirements
    dists = ws.resolve(reqs, Environment())
  File "/usr/lib/python3/dist-packages/pkg_resources.py", line 632, in resolve
    raise VersionConflict(dist,req) # XXX put more info here
pkg_resources.VersionConflict: (setuptools 3.3 (/usr/lib/python3/dist-packages), Requirement.parse('setuptools>=11.3'))

I've tried installing setuptools through pip but for whatever reason is doesn't help. And when I install it through apt-get it installs version 3.3 and gives me this error.


[WIP] Move to stylelint-codeguide

Move to stylelint-codeguide to allow the usage of stylelint 16
Ref: elirasza/stylelint-stylistic#17
Close #232

Hi! I'd advise you to take your time. I've got the renaming and organization move almost ready. Both github and npm addresses will change soon.

Thanks for the heads-up!
Moved this PR draft, will keep an eye on the repo

It was in the plan to publish the package today. But for reasons beyond my control, I had to postpone publication until Wednesday.
But already now you can start preparing for the new package, if you follow the information from this branch.

Thanks. Everything is ready on my side I guess, waiting for the release

Superseded by twbs/stylelint-config-twbs-bootstrap#236


update quickstart docs

What issue type does this pull request address? (keep at least one, remove the others)
/kind documentation
What is this pull request for? Which issues does it resolve? (use resolves #<issue_number> if possible)
resolves #
It addresses three things in the Quickstart doc:

Update the path for starting the example Java app.
Add instruction to use the existing Dockerfile for quickstart projects.
Add instruction to exit the container after testing developerment.

Does this pull request has user-facing changes? (e.g. config changes, new/modified commands, new/modified flags)
Docs changes only.
Does this pull request add new dependencies?
No.
What else do we need to know?

I need to add this on the versioned docs too.

@LukasGentele I made those changes we discussed.


fix(banner): colors

Description

Updated banner documentation to use dt-banner
Fixed Banner colors to match designs

Default Light

Important Light

Default Dark

Important Dark

Is that important warning color intended? seems too off for me, but it's using the right token.
Pull Request Checklist

[x] Ask the contributors if a similar effort is already in process or has been solved.
[x] Review the contribution guidelines.
[x] Use staging as your pull request's base branch. (All PRs using production as its base branch will be declined).
[x] Ensure all gulp scripts successfully compile.
[x] Update, remove, or extend all affected documentation.
[] Ensure no private Dialpad links or info are in the code or pull request description (Dialtone is a public repo!).

Obligatory GIF (super important!)

Is that important warning color intended? seems too off for me, but it's using the right token.

Ping @fede-dp here

Need to also do the same for warning+imporant variants of Notice and Toast

PR preview didn't work. but I check it out the branch and looks good. Approving but would be cool to have a design catch here. Thanks!!

Hmm strange, I'll give it a look, seems like we're having old deploy-previews there and now this is not working.

Need to also do the same for warning+imporant variants of Notice and Toast

All of the use the same base, let me check them and fix in this PR also to avoid more issues on product


Invalid content /en/testnets/cardano/get-started/installing-and-running-the-cardano-node/

Where does the issue occur?
/en/testnets/cardano/get-started/installing-and-running-the-cardano-node/
Describe the issue
Every link on this page points to a "Page not found" page.
Expected changes
Pages should exist...

Thank you for your feedback!

Thank you for your feedback!

Why did you close this...? The links are still broken...

Why did you close this...? The links are still broken...

@thisismytofu, when we visit the following page , all the four links are working:
Building a node using Nix
Building a node from source
Running a node using Docker
Running a node on an AWS instance
The 'supported platforms' link has been fixed a few hours ago, and the PR is merged into staging. These changes will be live and visible by tomorrow.

@thisismytofu, when we visit the following page , all the four links are working:
Building a node using Nix
Building a node from source
Running a node using Docker
Running a node on an AWS instance
The 'supported platforms' link has been fixed a few hours ago, and the PR is merged into staging. These changes will be live and visible by tomorrow.

Ah cool, works now, thanks! (and sorry for having been impatient with my first reply...) :)

Ah cool, works now, thanks! (and sorry for having been impatient with my first reply...) :)

No worries, thanks for your time and input :)

No worries, thanks for your time and input :)


Missing options on Annotations' position documentation

Hello!
Not really a bug, but I've noticed that some options for the yaxis.label.position key for annotations are missing on the "Available Options": https://apexcharts.com/docs/options/annotations/
For Y-Axis it also accepts "right" and "left".
Thank you!

Thanks for pointing it out.
Actually, y-axis annotations just accept left and right positions. top and bottom are for x-axis.
Fixed it on the website.


Update http package

we finally reached version 1.0.0 for the HTTP package.

@patachi I'd love to, but latest http version will create conflict with other packages which are still on 0.13, but I'll do it asap regardless of that

While they upgrade to version 1.0.0 you can use this
dependency_overrides:
    http: ^1.0.0

While they upgrade to version 1.0.0 you can use this
dependency_overrides:
    http: ^1.0.0

Ok Ok, I'll do it soon

Is there still anything preventing the version bump?

Is there still anything preventing the version bump?

Not anymore, I'll update today, Thank You


byte index 4 is out of bounds of

I was playing a bit with this,
But not sure whether it's a bug or I did something wrong.
Describe the problem you are observing.
Steps to reproduce

minimal posssible steps necessary

        use reedline::{
            default_emacs_keybindings, ColumnarMenu, DefaultCompleter, DefaultPrompt, Emacs,
            ExampleHighlighter, FileBackedHistory, KeyCode, KeyModifiers, ListMenu, Reedline,
            ReedlineEvent, ReedlineMenu, Signal,
        };

        let history = Box::new(
            FileBackedHistory::with_file(5, "asda.txt".into())
                .expect("Error configuring history with file"),
        );

        let commands = vec![
            "test".into(),
            "hello world".into(),
            "hello world reedline".into(),
            "this is the reedline crate".into(),
        ];
        let completer = Box::new(DefaultCompleter::new_with_wordlen(commands.clone(), 2));
        // Use the interactive menu to select options from the completer
        let completion_menu = Box::new(ListMenu::default().with_name("completion_menu"));
        // Set up the required keybindings
        let mut keybindings = default_emacs_keybindings();
        keybindings.add_binding(
            KeyModifiers::NONE,
            KeyCode::Tab,
            ReedlineEvent::UntilFound(vec![
                ReedlineEvent::Menu("completion_menu".to_string()),
                ReedlineEvent::MenuNext,
            ]),
        );

        let edit_mode = Box::new(Emacs::new(keybindings));

        let highlighter = Box::new(ExampleHighlighter::new(commands));

        let mut line_editor = Reedline::create()
            .with_completer(completer)
            .with_menu(ReedlineMenu::EngineCompleter(completion_menu))
            .with_edit_mode(edit_mode)
            .with_highlighter(highlighter)
            .with_history(history);

        let prompt = DefaultPrompt::default();

        loop {
            let sig = line_editor.read_line(&prompt);
            match sig {
                Ok(Signal::Success(buffer)) => {
                    println!("We processed: {}", buffer);
                }
                Ok(Signal::CtrlD) | Ok(Signal::CtrlC) => {
                    println!("\nAborted!");
                    break;
                }
                x => {
                    println!("Event: {:?}", x);
                }
            }
        }

to cause the problem

Type test
Press TAB
Press SPACE

Screenshots/Screencaptures
~/bin/nushell? testthread 'main' panicked at 'byte index 4 is out of bounds of ` `', library/core/src/str/mod.rs:107:9                                           11/13/2022 08:30:29 PM
stack backtrace:0 - 0  total: 0
   0: rust_begin_unwind
             at /rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/panicking.rs:584:5
   1: core::panicking::panic_fmt
             at /rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/panicking.rs:142:14
   2: core::str::slice_error_fail_rt
   3: core::ops::function::FnOnce::call_once
             at /rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/ops/function.rs:248:5
   4: core::intrinsics::const_eval_select
             at /rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/intrinsics.rs:2376:5
   5: core::str::slice_error_fail
             at /rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/str/mod.rs:86:9
   6: core::str::traits::<impl core::slice::index::SliceIndex<str> for core::ops::range::Range<usize>>::index
             at /rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/str/traits.rs:218:21
   7: core::str::traits::<impl core::ops::index::Index<I> for str>::index
             at /rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/str/traits.rs:65:9
   8: <reedline::completion::default::DefaultCompleter as reedline::completion::base::Completer>::complete
             at /home/maxim/.cargo/git/checkouts/reedline-e42026a78d91c510/39e6bc8/src/completion/default.rs:75:32
   9: <reedline::menu::list_menu::ListMenu as reedline::menu::Menu>::update_values
             at /home/maxim/.cargo/git/checkouts/reedline-e42026a78d91c510/39e6bc8/src/menu/list_menu.rs:437:13
  10: <reedline::menu::list_menu::ListMenu as reedline::menu::Menu>::update_working_details
             at /home/maxim/.cargo/git/checkouts/reedline-e42026a78d91c510/39e6bc8/src/menu/list_menu.rs:522:21
  11: reedline::menu::ReedlineMenu::update_working_details
             at /home/maxim/.cargo/git/checkouts/reedline-e42026a78d91c510/39e6bc8/src/menu/mod.rs:208:17
  12: reedline::engine::Reedline::buffer_paint
             at /home/maxim/.cargo/git/checkouts/reedline-e42026a78d91c510/39e6bc8/src/engine.rs:1450:17
  13: reedline::engine::Reedline::repaint
             at /home/maxim/.cargo/git/checkouts/reedline-e42026a78d91c510/39e6bc8/src/engine.rs:1226:13
  14: reedline::engine::Reedline::read_line_helper
             at /home/maxim/.cargo/git/checkouts/reedline-e42026a78d91c510/39e6bc8/src/engine.rs:571:29
  15: reedline::engine::Reedline::read_line
             at /home/maxim/.cargo/git/checkouts/reedline-e42026a78d91c510/39e6bc8/src/engine.rs:440:22
  16: <nu_command::viewers::scroll::Scroll as nu_protocol::engine::command::Command>::run
             at ./crates/nu-command/src/viewers/scroll/mod.rs:107:23
  17: nu_engine::eval::eval_call
             at ./crates/nu-engine/src/eval.rs:170:9
  18: nu_engine::eval::eval_expression_with_input
             at ./crates/nu-engine/src/eval.rs:651:25
  19: nu_engine::eval::eval_block
             at ./crates/nu-engine/src/eval.rs:780:31
  20: nu_cli::util::eval_source
             at ./crates/nu-cli/src/util.rs:233:11
  21: nu_cli::repl::evaluate_repl
             at ./crates/nu-cli/src/repl.rs:438:21
  22: nu::main
             at ./src/main.rs:438:31
  23: core::ops::function::FnOnce::call_once
             at /rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/ops/function.rs:248:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

Thanks for the repro, I had a report that was hard to track down previously.
Quick question, the code shown is the minimal example the backtrace is from nu?

Quick question, the code shown is the minimal example the backtrace is from nu?

negative

Any news on this? I'm encoutering the same issue right now :/
On my side it only happens with ListMenu, not ColumnarMenu, for some reason...


Semi-accidental rewrite

I was recently dissecting the styled-system code, trying to understand how it works. I was working on a similar library, basically maintaining a dictionary of method props and configure styles by breakpoint, but at the core its styled-components, so quite similar.
Anyway I thought I should try and apply my recipe on styled-system and see if i've come up with any improvements or if you find it interesting. I basically rewrote styled-system.
The main difference is that i merged the style and space functions and that i'm populating the styles using template literals instead of objects. And i'm using rollup to bundle the code.
Other than that, it's the same logic. I haven't replicated every feature of styled-system yet, but for example, this is what the api looks like.
export const margin = style({
  prop: 'margin',
  shortProp: 'm',
  cssProperty: [ 'margin-top', 'margin-right', 'margin-bottom', 'margin-left' ],
  transformValue: em
});

export const fontSize = style({
  prop: 'fontSize',
  shortProp: 'fs',
  transformValue: rem
});

The benchmark also shows its a bit faster, but not by much and also a bit early to read much into it, so i'm just mentioning it.
fontSizeNew x 802,807 ops/sec ±0.46% (87 runs sampled)
fontSize x 379,590 ops/sec ±0.37% (86 runs sampled)
spaceNew x 1,016,655 ops/sec ±0.85% (83 runs sampled)
space x 502,807 ops/sec ±0.53% (83 runs sampled)

If you have a chance to have a look and let me know what you think, maybe with a bit more work i could come back with a pull request. Or maybe you have other plans, and it's not worth pursuing.
Source
https://github.com/tol-is/styled-system/tree/master/src-new
Example:
https://github.com/tol-is/styled-system/blob/master/examples/new/App.js
Cheers

Thanks for sharing! Will try to dig into this a little deeper later. It might be worth comparing your implementation with the one in #383 to see if there's any performance gains that could be wrapped up there. As for rollup, this repo doesn't gain anything from using it because the source code is in a single file. For such a small library, it's nice to avoid additional build tools.
Also, at first glance, it doesn't look like you're supporting object literal values, but let me know if I missed that.

Could you give me an example of object literal value usage?  Is it like?
const Heading = styled.h1({
   fontSize: 20
})

Also regarding rollup, to be honest, I just really struggle with long files, and there are some benefits, like tree shaking, but indeed less important for a library this size. But that's a separate thread.

Object literal values can be used like e.g. <Box width={{ xs: 1, sm: 1/2, lg: 1/4 }} /> see https://github.com/jxnblk/styled-system/blob/master/docs/responsive-styles.md#using-objects


No tests found in index.js

I have written my first endpoint with micro and endpoint. I can perfectly run and pass my test using npm run test but when I do commit for some reason that same task throws an exception:
✖ No tests found in index.js, make sure to import "ava" at the top of your test file
I have simply an index.js file with my code and then a a test under test/add-key.js

This does not seem like an issue with micro - do your have an example repo?

Yeah, Micro is not a test runner 🤔


⚠ AlphaRatio has degraded performance

In 9c38e0c, AlphaRatio ($AR) experienced degraded performance:

HTTP code: 200
Response time: 2617 ms

Resolved: AlphaRatio performance has improved in 792739c.


Add some quality of life hooks

I've added a couple of hooks which I've found useful while making my Safe Apps. They allow direct access to the most used variables such as the safe's address and the function to send transactions.
I'll need to double check the types I've updated some of them for the new SDK version plus you're initialising the SDK slightly differently to how I do which means its "technically" possible for appsSdk to be undefined.

We will have them as a library ... currently they are still in my private repo, but at some point we will move them.

Ah awesome. Yeah, I've been thinking about how creating something like that would be useful as there's a ton of boilerplate I have to copy from app to app. These 100% make more sense pushed out into a library.


Internal Server Error on using PNG files

500 Internal Server Error in /predict when a PNG file is uploaded.

Reason: PNG files have shape (n, m, 4) whereas in model.py reshape dimensions are (224, 224, 3)
Solution: modify  image = Image.open(pic_path) in model.py to image = Image.open(pic_path).convert('RGB')

Hey @pratik-choudhari, thank you so much for raising the issue and following up on it!
Would you be interested in making a PR to resolve that?

Sure!


CAND support?

Continuously Adaptive Neural Networks (CAND) is not supported yet in CapyMoa and my own implementation using the CapyMoa interface throws the following error:

ai.djl.engine.EngineException: No deep learning engine found.

Here is the code without type checking:
from __future__ import annotations
from capymoa.base import (
    MOAClassifier,
)
from capymoa.stream import Schema
from capymoa._utils import build_cli_str_from_mapping_and_locals
from moa.classifiers.deeplearning import CAND as _MOA_CAND

class ContinuouslyAdaptiveNeuralNetworks(MOAClassifier):
    def __init__(
            self,
            schema: Schema | None = None,
            training_method: str = 'CAND',
            larger_pool: str = 'P10',
            number_of_MLPs_to_train: int = 5,
            number_of_layers_in_each_MLP: int = 4,
            number_of_instances_to_train_all_MLPs_at_start: int = 1000,
            mini_batch_size: int = 32,
            use_one_hot_encode: bool = True,
            use_normalization: bool = True,
            back_prop_loss_threshold: float = 0.3,
            device_type: str = "CPU",
            do_not_train_each_MLP_using_a_separate_thread: bool = False,
            votes_dump_file_name: str = "",
            stats_dump_file_name: str = "",
    ):

        mapping = {
            "larger_pool": "-P",
            "number_of_MLPs_to_train": "-o",
            "number_of_layers_in_each_MLP": "-L",
            "number_of_instances_to_train_all_MLPs_at_start": "-s",
            "mini_batch_size": "-B",
            "use_one_hot_encode": "-h",
            "use_normalization": "-n",
            "back_prop_loss_threshold": "-b",
            "device_type": "-d",
            "do_not_train_each_MLP_using_a_separate_thread": "-t",
            "votes_dump_file_name": "-f",
            "stats_dump_file_name": "-F"
        }

        config_str = build_cli_str_from_mapping_and_locals(mapping, locals())
        super(ContinuouslyAdaptiveNeuralNetworks, self).__init__(
            moa_learner=_MOA_CAND,
            schema=schema,
            CLI=config_str,
        )

Is there any solution for this?

Hi @Kirstenml , thanks for raising this problem and providing your code. I believe this problem is related to DJL. I am going to request assistance from the author of the code in MOA, we had similar issue before. @nuwangunasekara do you know how we could solve it?

Thank you for your answer. This is my code to invoke ContinuouslyAdaptiveNeuralNetworks:
from capymoa.evaluation import prequential_evaluation

data_stream = Electricity()
ob_learner = ContinuouslyAdaptiveNeuralNetworks(schema=data_stream.get_schema())
results = prequential_evaluation(data_stream, ob_learner)
print("Continuously Adaptive Neural Networks: {}".format(results["cumulative"].accuracy()))

Hi @Kirstenml just updating you with the status of the issue.
Looks like initial issue is caused by not having CUDA 10.2 which is required for DJL 0.9 pytorch engine (1.7.0).
conda install pytorch=1.7.1 torchvision  cudatoolkit=10.2 -c pytorch-lts

would take you bit forward on a Linux  system.
But still you might get errors :(
I am looking into a work around for this.
Otherwise best option would be to create a native CapyMOA implementation of CAND using pytorch.
PyTorchClassifier in 03_pytorch.ipynb notebook could be a good starting point.
I am happy to help you in that as well.
Hopefully I could come up with a workaround for your initial issue :)

Hi @Kirstenml,
Temporary workaround on MacOS (Intel) (similar steps might work on Linux):

installed cuda-10.2 from https://developer.nvidia.com/cuda-10.2-download-archive (this is the CUDA version supported in DJL 0.9.0)

mkdir CapyMOA_TEST

cd CapyMOA_TEST

git clone  https://github.com/nuwangunasekara/CapyMOA.git ./  (cloned my fork to avoid latest updates to CapyMOA)

conda create -n capymoa_test python=3.9

conda activate capymoa_test

conda install pytorch=1.7.1 torchvision -c pytorch-lts

pip install --editable ".[dev]"

pip install numpy==1.23

python CAND.py
log4j:WARN No appenders could be found for logger (ai.djl.util.cuda.CudaUtils).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
System GPU count: 0 Model using Device: cpu()
System GPU count: 0 Model using Device: cpu()
System GPU count: 0 Model using Device: cpu()
System GPU count: 0 Model using Device: cpu()
System GPU count: 0 Model using Device: cpu()
System GPU count: 0 Model using Device: cpu()
System GPU count: 0 Model using Device: cpu()
System GPU count: 0 Model using Device: cpu()
System GPU count: 0 Model using Device: cpu()
System GPU count: 0 Model using Device: cpu()
System GPU count: 0 Model using Device: cpu()
System GPU count: 0 Model using Device: cpu()
System GPU count: 0 Model using Device: cpu()
System GPU count: 0 Model using Device: cpu()
System GPU count: 0 Model using Device: cpu()
System GPU count: 0 Model using Device: cpu()
System GPU count: 0 Model using Device: cpu()
System GPU count: 0 Model using Device: cpu()
System GPU count: 0 Model using Device: cpu()
System GPU count: 0 Model using Device: cpu()
System GPU count: 0 Model using Device: cpu()
System GPU count: 0 Model using Device: cpu()
System GPU count: 0 Model using Device: cpu()
System GPU count: 0 Model using Device: cpu()
System GPU count: 0 Model using Device: cpu()
System GPU count: 0 Model using Device: cpu()
System GPU count: 0 Model using Device: cpu()
System GPU count: 0 Model using Device: cpu()
System GPU count: 0 Model using Device: cpu()
System GPU count: 0 Model using Device: cpu()
Number of features before one-hot encode: 6 : Number of features after one-hot encode: 6
Continuously Adaptive Neural Networks: 87.29475635593221

CAND.py:

from __future__ import annotations
from capymoa.base import (
    MOAClassifier,
)
from capymoa.stream import Schema
from capymoa._utils import build_cli_str_from_mapping_and_locals
from moa.classifiers.deeplearning import CAND as _MOA_CAND

class ContinuouslyAdaptiveNeuralNetworks(MOAClassifier):
    def __init__(
            self,
            schema: Schema | None = None,
            training_method: str = 'CAND',
            larger_pool: str = 'P10',
            number_of_MLPs_to_train: int = 5,
            number_of_layers_in_each_MLP: int = 4,
            number_of_instances_to_train_all_MLPs_at_start: int = 1000,
            mini_batch_size: int = 32,
            use_one_hot_encode: bool = True,
            use_normalization: bool = True,
            back_prop_loss_threshold: float = 0.3,
            device_type: str = "CPU",
            do_not_train_each_MLP_using_a_separate_thread: bool = False,
            votes_dump_file_name: str = "",
            stats_dump_file_name: str = "",
    ):

        mapping = {
            "larger_pool": "-P",
            "number_of_MLPs_to_train": "-o",
            "number_of_layers_in_each_MLP": "-L",
            "number_of_instances_to_train_all_MLPs_at_start": "-s",
            "mini_batch_size": "-B",
            "use_one_hot_encode": "-h",
            "use_normalization": "-n",
            "back_prop_loss_threshold": "-b",
            "device_type": "-d",
            "do_not_train_each_MLP_using_a_separate_thread": "-t",
            "votes_dump_file_name": "-f",
            "stats_dump_file_name": "-F"
        }

        config_str = build_cli_str_from_mapping_and_locals(mapping, locals())
        super(ContinuouslyAdaptiveNeuralNetworks, self).__init__(
            moa_learner=_MOA_CAND,
            schema=schema,
            # CLI=config_str,
        )

from capymoa.datasets import Electricity
from capymoa.evaluation import prequential_evaluation

data_stream = Electricity()
ob_learner = ContinuouslyAdaptiveNeuralNetworks(schema=data_stream.get_schema())
results = prequential_evaluation(data_stream, ob_learner)
print("Continuously Adaptive Neural Networks: {}".format(results["cumulative"].accuracy()))

This works, thank you @nuwangunasekara!
For me some more steps were necessary after creating the conda environment:

conda install pip
/anaconda/envs/venv_name/bin/pip install --editable ".[dev]"
/anaconda/envs/venv_name/bin/pip install numpy==1.23
Add cand.py in directory src/ and run python cand.py

Thanks @Kirstenml for sharing those additional steps.
Please feel free to raise a PR, if you managed to get every thing working without the extra steps in the workaround.


Show indication of game selection in join game view

When I click on a game, there should be some sort of indication that the game was selected

Somebody did that already

Oh, yeah, I did that. I didn't know there was a task for it.


New package: NamespacedEnums v0.1.0

Registering package: NamespacedEnums
Repository: https://github.com/christopher-dG/NamespacedEnums.jl
Created by: @christopher-dG
Version: v0.1.0
Commit: 775d00f0259ff05ac5adaf9306a4b2949d4f276a
Reviewed by: @christopher-dG
Reference: https://github.com/christopher-dG/NamespacedEnums.jl/commit/775d00f0259ff05ac5adaf9306a4b2949d4f276a#commitcomment-39406675

Turns out that this is just a worse version of SuperEnum.


adding worker to instance health check

Health check from spawner to instance.  The health is also used in reverse for instance to determine if spawner is still alive

@srkukarni @sijie please review

@srkukarni I have refactored based on our discussion

@srkukarni I have added the check for python as well.  Please review again


how writing a new page ? not post ?

Checklist

[ ] I have read the newlest tutorials and know the correct effect of the functional design.
[ ] There is no similar question on existing issues (including closed ones).
[ ] I have tried to find the answer on Jekyll Forum and StackOverflow.
[ ] My question is based on the latest code of the master branch.

Description

You may consider adding a markdown file in \tabs to simply have a page.

You may consider adding a markdown file in \tabs to simply have a page.

Thank you for your reply .
3 steps:

adding a markdown file in \tabs to simply have a page.
cd _data\locales , and edit some of files .
add new line below this
`# The tabs of sidebar
tabs:

format: <filename_without_extension>: `


[bugfix] Update PropTypes import statements to use wildcard syntax

Switched from default imports to wildcard imports for PropTypes across multiple files. This change ensures consistency in PropTypes usage and can help with tree-shaking during the build process.
Replaced all occurrences of import PropTypes from 'prop-types'; with import * as PropTypes from 'prop-types'; in the entire Material UI codebase.
Bug Report: [Error] Incorrect Import of PropTypes Causes Compilation Error

[ ] I have followed (at least) the PR section of the contributing guide.

@morozow Thanks for submitting the PR! @Janpot What do you think about the proposed fix?

I believe this change should be fine.  I'll open https://github.com/mui/material-ui/pull/43703 to see if we can get rid of esModuleInterop altogether.

Trying the following in Node.js (node index.mjs):
// test.mjs
import * as PropTypes from 'prop-types';
// import PropTypes from 'prop-types';

console.log(PropTypes.any);

This prints undefined. The prop-types package only exports commonjs.
It's a bit strange, but these types don't look correct to me. I'm seeing this change a long time ago. I'm wondering why they did that.
I'm a bit worried making this change because I believe it could result in broken ESM once we move to https://github.com/mui/material-ui/pull/43264

@Janpot I cloned your forked repository https://github.com/Janpot/material-ui, checked out the esm-exports branch.  After running npm run build from the repository’s root, a repo can not be built now. Looks like my PR can't play in your sandbox right now. Give me a shout when your PR is set up and can build the whole app. I'll jump to repo then and see if my PR and your PR can agree

I cloned your forked repository https://github.com/Janpot/material-ui, checked out the esm-exports branch. After running npm run build from the repository’s root, a repo can not be built now.

We use pnpm install and the command is pnpm release:build. The branch definitely builds as you can see from our CI: https://app.circleci.com/pipelines/github/mui/material-ui/138049/workflows/5fec5701-c382-4e1c-a938-a3b6a10803dc
But you can probably spare yourself the effort, try running the following script in Node.js:
// ./test.mjs
import * as PropTypes1 from 'prop-types';
import PropTypes2 from 'prop-types';
import * as React1 from 'react';
import React2 from 'react';

console.log(!!PropTypes1.any, !!PropTypes2.any);
// false true
console.log(!!React1.useState, !!React2.useState);
// true true

I believe import * will only work when there is also esModuleInterop: true available or an equivalent in a bundler.

@Janpot As a result, the build was successful. I must mention that I haven’t had much time to fully set up a Material UI application from the PR branch locally, including configuring all dependencies and executing the pnpm build.
Nevertheless, I see no reason to reject this PR at this point now. Could you perhaps provide a guide on how to quickly install the @mui/material fork repo branch along with all its dependencies built locally as an package.json dependencies item?
I’ve merged your esm-exports branch into my fork merged branches, and it built without any issues.

@Janpot Again my 2 cents, as esModuleInterop: true setting in TS configuration is a more advanced solution that aligns with modern JavaScript standards, facilitating the use of both ES and CommonJS modules seamlessly. It’s important to note that the tsconfig.json at the root of the project dictates how TypeScript handles .d.ts files, influencing module resolution and type definitions across the project.
Adopting esModuleInterop: true can lead to fewer issues when integrating with diverse package ecosystems.
Nevertheless, for supporting existing users who might rely on older configurations, consider employing versioning strategies. This way, newer versions can default to esModuleInterop: true, while older versions remain available to ensure backward compatibility without disrupting dependent projects.

@Janpot

Additionally, it seems the repo includes a webpack setup. The built file should be useful, but I don’t see any issues with the TypeScript *.d.ts files in this PR. Perhaps I’m mistaken there, but regardless, it seems to be fine.

That's a false, the other PR is related to this approach with d.ts files. But nevertheless, the next comment is workable

@Janpot While retesting locally with the TypeScript compiler and tsconfig.json set to "esModuleInterop": false, your examples function correctly as console.log(!!PropTypes.any); > returns true. I understand that every project has its unique philosophy from inception to the present, but using my root tsc config everything works well across different compilers.
Webpack is a different scenario, yet even here approaches like outputModule (which confessedly carries its own risks), libraryTarget: umd or commonjs2 and @babel/plugin-syntax-dynamic-import manage to get the job done.
Does this fit with our current solution?
Just to note: as of now, Material UI v6.0.2 cannot be built with tsc

For https://github.com/mui/material-ui/pull/43264 the build command is
MUI_USE_PACKAGE_EXPORTS=true pnpm release:build

While working on this, the new mode is activated with a MUI_USE_PACKAGE_EXPORTS env var.
This is going to build a new file e.g. packages/mui-material/build/Button/Button.mjs which will be an ES module. This one will be loaded natively in node.js when you run node test.mjs with
// test.mjs
import Button from '@mui/material/Button';

console.log(Button.propTypes);

When you build from import * as PropTypes from 'prop-types'; this will produce an error
file:///Users/janpotoms/Projects/material-ui/packages/mui-material/build/Button/Button.mjs:399
  color: PropTypes /* @typescript-to-proptypes-ignore */.oneOfType([PropTypes.oneOf(['inherit', 'primary', 'secondary', 'success', 'error', 'info', 'warning']), PropTypes.string]),
                                                                              ^

TypeError: PropTypes.oneOf is not a function
    at file:///Users/janpotoms/Projects/material-ui/packages/mui-material/build/Button/Button.mjs:399:79
    at ModuleJob.run (node:internal/modules/esm/module_job:195:25)
    at async ModuleLoader.import (node:internal/modules/esm/loader:337:24)
    at async loadESM (node:internal/process/esm_loader:34:7)
    at async handleMainPromise (node:internal/modules/run_main:106:12)

When you build from import * as PropTypes from 'prop-types'; this will print the proptypes
When you run the commonjs variant, node test.cjs with
const Button = require('@mui/material/Button').default;

console.log(Button.propTypes);

this will also just print the proptypes (because babel added an interopRequireDefault helper)

To test this within the material-ui repo, you can temporarily add publishConfig.linkDirectory: true to the packages/mui-material/package.json and run pnpm install.

This is the hard way of reproducing the problem. My little snippet from above essentially replicates what's happening inside of @mui/material after we make them strict ESM.
It'll be important to be able to load @mui/material natively in node.js because it'll be a requirement for pigment css to e.g. load the theme logic inside of vite.config.ts.

Nevertheless, I see no reason to reject this PR at this point now.

I hope the long explanation makes it clear that the changes are not mergeable as for now. I understand this is frustrating. I believe the best next steps we could take is investigate why named exports in react behave differently from named exports in prop-types as my script above demonstrates and fix the problem first in prop-types so that it also prints true true.

@morozow I think I figured out the problem with prop-types. Node.js uses static analysis to find which names are exported from commonjs modules.
See https://nodejs.org/api/esm.html#commonjs-namespaces and https://github.com/nodejs/cjs-module-lexer/tree/1.2.2
react is statically analyzable so it provides named exports. prop-types is not and therefore it only finds a default export. The prop-types repo is archived since last week, we can't fix this o their end. The only reliable solution that I can think of is to keep with import PropTypes from 'prop-types'.
🤔  Maybe we need to patch the types so that we don't accidentally change this.

@Janpot You've highlighted some valuable ideas. I've tailored the solution to directly address the core issue with the tsc in the upcoming PR https://github.com/mui/material-ui/pull/43736.
We can either keep this current PR open for future reference or close it, depending on your preference. However, we should consider enhancing a codebase with a new PR by incorporating utilities to enable tsc

The problem has been fixed by updating the documentation: https://github.com/mui/material-ui/pull/43747


Fix examples/nomad-consul-ami/nomad-consul-docker.json

Hi! I have no idea what I'm doing, but I had issues using examples/nomad-consul-ami/nomad-consul-docker.json,
and the following made my problems go away:
Find and replace all occurrences of clean_ami_name with clean_resource_name in the file.
(I had a PR ready for this, but I don't feel like reading your CLA at the moment.)

Hi @harryvederci,
The examples you've pointed to haven't been upgraded to use the latest version of Packer. As you mentioned the solution is to replace clean_ami_name with clean_resource_name as the former is now deprecated. I'll leave this issue open for now until somebody submits a PR and signs the CLA.


Introducing master script. Bug fixes. Cleanup

src/main/python/gcloud > python master_mm_hydra_setup.py -i <deployment_id/tag> -f setup_config.ini

Parallel deployments now work for real.
Adding retries in checking the hosts status.
Introducing continue feature.
Bug fixes

Signed-off-by: tahir24434<EMAIL_ADDRESS>

Please change director from gcloud to hydra-deploy


LinearScan.Contravariant missing?

Hello! Trying to build linearscan for a project, and I get this problem with both cabal-install and stack:
Configuring linearscan-0.11...
Building linearscan-0.11...
Preprocessing library linearscan-0.11...

LinearScan/Const.hs:16:18:
    Could not find module ‘LinearScan.Contravariant’
    Use -v to see a list of the files searched for.

Do you know what might be happening? I have also tried going back to version 0.10.2, but I get the same problem as well. Thanks!

Thanks for this report, @Chobbes!  Somehow I'd missing that the Cabal file didn't reference that file.
I've uploaded release 0.11.1, which should build fine for you again.


export also notification component for avoiding registering it globally

it is not only component but a global event bus that component needs and cannot work without plugin


Create shards when creating collections

When pulling specs, collections get automatically initialized without sharding. This should either get removed for now, or have an automatic shard setting passed
resources:

https://www.arangodb.com/2019/01/sharding-freedom-when-you-need-it-least/
https://docs.arangodb.com/3.4/Manual/Architecture/DeploymentModes/Cluster/Architecture.html

For the sake of simplicity, will set the number of shards for new collections based on an environment variable and will fall back to 30.

Closed by #38


Remove region from stack_master.yml

This is a Proposal for the direction of 2.0 of StackMaster. The proposal started here and given there wasn't a great deal of discussion on this I thought I'd throw it up.
I often find the duplication of stacks in stack_master.yml to be tedious and unnecessary. We often duplicate stack configuration many times, especially when we are building multiple environments and multi-region deployments.
Essentially we are duplicating the region parameter twice, once in stack_master.yml and once on the command line. We can remove it from the config file, but it does have some draw backs:

The need for a migration "tool" to upgrade existing configurations to 2.0. I'll add this as part of a future PR
We can no longer have per region per stack configurations for notification_arns, s3_buckets and files. The configuration must be for an entire region or an entire stack. For 2.0 we also plan on supporting per account settings called "environments", so this will likely also become a place to configure these settings (and probably per region per environment).
It breaks existing CI workflows we use where we just run stack_master validate to validate all our stacks. validate, status and diff now must be passed a region.

Alternatively we could move these configurations into the parameter files, but that doesn't feel like where it should go. Other proposals for where this configuration might live are welcome.
Ultimately I think this proposal moves us closer to a better experience for StackMaster users, by making it simpler to manage multiple stacks deployed to multiple regions and environments with little heavy lifting. Eventually we want stack_master.yml file to look something like this:
---
environments:
  production:<PHONE_NUMBER>10
  staging:<PHONE_NUMBER>10
bundles:
  my-web-app:
    vpc:
      template: vpc.rb
    web:
      template: asg.rb
    db:
      template: rds.rb
    redis:
      template: redis.rb
stacks:
  my-bastion:
    template: bastion.rb

Where I can stack_master apply us-east-1 my-web-app and have it come up cleanly.
This PR is to be merged into the version-2.0 branch, which will act as our develop branch until 2.0 is considered ready for a release.

Actually this will break the way we use validate in more ways. Because we don't give our CI environment access to the S3 buckets that stacks are uploaded to and if any stack has to be uploaded to s3 all stacks will be uploaded to s3 and our workflow will be broken.
But we don't really need to validate all stacks just all templates. So I will update this PR to make validate do just that.

I think something along these lines makes sense, though, I feel like region is just a characteristic of another unexpressed concept. Ignoring the migration path for a bit, I feel like stacks (or perhaps bundles from your example above) should be applied into different targets (or deployments or some better name). These could be staging, sandbox, production, production-failover, my-throw-away-test-deployment etc...
Each target would require a region and optionally have other parameters which override things in the stack itself. Something like the following perhaps:
# Stack
- name
- templates
- base parameters

# Each stack has one or more `targets` which has the following

- name
- region
- parameters which override those base ones defined in the stack (optional)

Thoughts?

I very much like the "targets" idea, @grassdog

I think this helps solve another problem, which is how can we ensure you are in the right AWS account for the right "target", by making the targets static (in config), not ephemeral (on the command line).

FWIW this will very much break how my team uses Stack Master. For numerous regions we have the same stacks deployed in multiple accounts and regions, but they have different stack names. It's fine if this is the direction Stack Master is heading, but it will likely be too much work for us to refactor everything (we have hundreds of projects with multiple stack_master.yml files). It would be more helpful to me and my team to merge https://github.com/envato/stack_master/pull/221 as a bug fix for a v1 release (since this seems like a bug and there is nothing saying only a single alias is supported per region).
Take this comment with a grain of salt...we don't really need any other features and if we want more parameter resolvers we can patch them in to our tooling manually. Also we have mitigated the aforementioned bug by just using more stack_master.yml files (although this is more confusing and complicated for CI/CD). So ultimately we will probably just pin stack_master to ~> 1.

Taking @grassdog's target idea and trying to resolve @arecker & @rbayerl's issues above, how about something like:
---
targets:
  production-us:
    account_id:<PHONE_NUMBER>10
    region: us-east-1
  production-eu:
    account_id:<PHONE_NUMBER>10
    region: eu-west-1
stacks:
  us-bastion:
    target: production-us
    region: us-east-1
    template: bastion.rb
  eu-bastion:
    target: production-eu
    region: us-east-1
    template: bastion.rb
    # stack name override (optional, eg supporting existing stacks)
    stack_name: bastion-eu-old

Not sure how this would work w/ bundles, and not sure how it'd work for common parameters between templates that are essentially the same between different targets.

Another potential idea (although I'm a bit worried about indirection / abstraction):
targets:
  production-us:
    account_id:<PHONE_NUMBER>10
    region: us-east-1
  production-eu:
    account_id:<PHONE_NUMBER>10
    region: eu-west-1
stacks:
  bastion:
    template: bastion.rb
    # some config can go here ?
deployments:
  production-us:
    us-bastion:
      stack: bastion
  production-eu:
    eu-bastion:
      stack: bastion
      # override, as above
      stack_name: bastion-eu-old

Thanks a lot for your feedback @rbayerl and @arecker. Can you provide your feedback on @grassdog and @mipearson 's proposals too?
I like @grassdog's proposal a lot and I feel this is a better solution than this PR, but it's going to have to require some more thought on how we model the concept of "stacks" and "targets" or "deployments".

I do find the concept intriguing. It would be nice to have all of this metadata in the config file itself. The idea of targets would help us to decrease the number of config files we have currently too. The only thing I would be worried about is how Stack Master would find credentials - we've seen some pretty good examples of how not to do it 🙂. That's more of an implementation detail though.
I think @arecker said it much more succinctly when he said we don't want to lose any of the existing infrastructure as code.

@rbayerl I generally defer finding credentials to the SDK provider. That has a well defined, well known, lookup order
Tools such as aws-vault help with managing multiple accounts and access keys. They can be used in conjunction with StackMaster without us needing to explicitly support it, it just works.
I'm open to supporting specifying what profile to use to lookup credentials using the CLI configuration, as this is pretty simple and well defined integration.
I don't have any other proposals for how we manage credentials but I'm open to them, please create a Github issue if you want to propose something else.


dx | 1171 replace personalize-url with getter method

replaced personalize-url with getter method

closing this PR as the base branch has too many commits to be merged with feat branch


enhancement(action/linking): support multiple linkDirs

linkDir is converted to linkDirs: [linkDir].
We choose a linkDir by testing if stat.dev if between source path and any linkDir is the same. Hardlink requires this to have a match but symlink will only display and warning and use linkDirs[0].
The only possible issue is with seasonFromEpisodes, these could be sourced from multiple drives. The ones that are will be logged as failures before attempting to link.

The option name change alone is a breaking change....

The option name change alone is a breaking change....

linkDir is converted to linkDirs: [linkDir].

Yes, so what - you're just going to accept either type or option now? I can provide a single string or an array in linkDirs or an array in linkDir or any variation of this?
I don't have any of that nuance from "It's converted"

Yes, so what - you're just going to accept either type or option now? I can provide a single string or an array in linkDirs or an array in linkDir or any variation of this?

It won't allow having both defined. linkDirs must be an array.

Documentaiton needs updating.


Watch shows count when clicked

repo

it shouldn't show the count, that's all

Why? That is how the feature is intended to work.

Oh, maybe an option that does what I'm suggesting? Always hide the count?

You can add some custom CSS to override it

This is a compromise I picked in order to hide the count but still keep it accessible somehow. I’d have done the same for the fork count if the DOM allowed it


Question: TokenRequestValidationLog

The TokenRequestValidator uses a TokenRequestValidationLog directly to control the serialization of the log messages.
I'm interested in overriding this mechanism in order to supply an additional list of fields to be redacted before serialization.
So my question is, before I get into the nitty gritty, is there a way I can achieve using a custom list of field names to be redacted for logging within the the TokenRequestValidator/TokenRequestValidationLog?
On my first pass through the code it doesn't look like this is immediately supported. I'm guessing I'll need to recreate the TokenRequestValidator and pull in the supporting players where necessary and of course drop in my Log scrubber and register it via the dependency injection hooks. That seems like it might be a maintenance issue in the waiting when all I really want is to supply some field names to redact.
Of course a cleaner solution would be a light refactor to supply some interface for field names to be redacted. I'm happy to do that if this is something that would be useful elsewhere. I admit our use cases may not be very typical so maybe there is a likelihood such a feature would be ignored.
Or maybe there is another way?
Cheers,
Neil

Not a bad idea as a feature request. We'll leave it open to discuss.

This would require some cross-cutting re-work of all the logging - which is a good idea, but I don't see that happening very soon.
Would hooking into the ASP.NET Core logging system directly be a much more universal solution?

It would certainly introduce wider scope for customizations and disambiguate some of the logging concern. I can't speak to the level of effort that may incur. However, if the log entries already came in as well defined objects instead of serialized strings I would have been able to introduce a custom formatter already and leave you in peace :)

Let's move that discussion to #1809

Did you look into redacting at the logger factory level? If we would give every data a unique name - e.g. subjectId, clientId etc - then you could probably redact it that way.

I did look into it.
Manipulation is possible for objects of known types either as a culture specific formatter (decided this would be a hack approach as it violates the interface intention) or within a custom template if your logger library supports such a thing. We use Serilog that does support custom templates but the problem for us became the point at which serialization occurs (both as a structured k/v object and as a string within the message). As it stands we'd need to completely re-write our formatter.
I think a named value approach might work well in some situations but a well known, typed, object is a more sustainable solution avoiding naming collisions from other log events. We can certainly attack this at the logger level but right now it's just too much work for us. Deciding how things look vs what things (inside what things) should be displayed at all is a concept our current libraries do not handle and will take time to address correctly.
I then do have to wonder if this is a concern of the logger at all. Sure it's possible (with some work) but is it right? The concept of what is sensitive is pushed to the logger itself where it feels like it's something we want to keep close to IDS.


Missing IndexMut implementation

I hope I'm not blind and missed the reason as for why this is but I noticed that HashMap doesn't seem to implement the IndexMut trait. Is there a specific reason for why this is?

Ah i see, thank you for even linking the issue. Good to know as to why this was done


Add option to specify connection timeout

Description
Currently the default httpx timeout of seconds is set, which means that bigger queries might give a timeout. This PR allows you to specify which timeout you like to use, such that you can set bigger timeouts or even disable the timeout.
Related documentation: https://www.python-httpx.org/advanced/#timeout-configuration

Thank you for the additional pointers! I have added the environment variables, documentation and a test.

Rebased and merged to main


Nimble package?

How can I install this with nimble? Thanks!

nimble install html-dsl
:x:

nimble install nim-html-dsl
:x:

:confused:

Hey @juancarlospaco, I see you never added this to Nimble - is that an oversight? https://github.com/nim-lang/packages/pulls?q=is%3Apr+author%3Ajuancarlospaco

Stuff not in Nimble is not finished.

Oh, my apologies!

I saw a confusing reference to html_dsl here https://framework.embarklabs.io/news/2019/11/18/nim-vs-crystal-part-1-performance-interoperability/ and thought it looked like a very :ok_hand: way of expressing html in nim.
I'll grab the code and try it out myself - thank you!

Nice I didnt know of the blog post.
I think it starts failing when used nested elements, I dont remember exactly, but I am all the time doing things so...


Cut a new release

0.3.4 has been released more than a year ago. I'd be nice to have a new release for dependency upgrades.
Also https://mvnrepository.com/artifact/com.google.protobuf/protobuf-java/3.19.2 has vulnerabilities

Released 0.3.5.


Tracking issue: Update Darwin bootstrap files

I'm opening a tracking issue for updating the Darwin bootstrap files to document pending changes, and to see if/when we have enough reasons to perform an update.
The last update was in 2021 as part of #105026. I've been told updates to bootstrap-files are supposed to be rare, and there should be good rationale. Once we have that, plus some thumbs up from @NixOS/darwin-maintainers, we should be able to find help getting tarballs.nixos.org updated via #infra:nixos.org.
Currently pending changes:

#202347
This PR changed the structure of the bootstrap files. Currently, there is the main bootstrap-tools archive, plus some separate executables required to unpack it (sh, bzip2, mkdir, cpio). The new structure is just an archive, and uses the builtin Nix support for .nar.xz unpacking. The PR contains more rationale, plus there's an example commit.

#166401
Maybe(?) update LLVM and change arch flags for Apple Silicon.

Anything else?

Reasons to delay:

With only the first one above currently on master, the pending changes do not significantly affect the resulting stdenv.

Reasons to move forward:

Maybe we want to resolve the discrepancy in structure (between the bootstrap-tools we build and the bootstrap-tools we fetch) before 23.05?

One reason to update bootstrap tools is it allows the x86_64-darwin stdenv to be bootstrapped with a newer SDK since that needs pbxz and the tbd tools. That was a blocker to moving forward with #180931, which replaced the override-based approach with a fully bootstrapped 11.0 stdenv on x86_64-darwin.
I’ve also started experimenting with exposing an sdkPkgs attribute that would allow you to use packages built against the newer SDK without having to stuff them into the callPackage workaround. That would also depend on newer bootstrap tools for the same reason.

I should add that #180931 needs revised to break out the SDK changes into its own PR since many of the packages have been updated separately.

I believe all the packages in #180931 now build on x86_64-darwin, or at least at some point.
In the meantime, it is currently difficult to build GTK-dependent packages with darwin.apple_sdk_11_0.callPackage without hacks, simply because of the propagatedBuildInputs here:
https://github.com/NixOS/nixpkgs/blob/656670527ad638f22a7ff3ff24da7bc1fc55e422/pkgs/development/libraries/gtk/3.x.nix#L151-L152
And I think bumping the stdenv to 11.0 is the most direct way to solve this...

I can upload new bootstrap tarballs if needed.

One reason to update bootstrap tools is it allows the x86_64-darwin stdenv to be bootstrapped with a newer SDK since that needs pbxz and the tbd tools. That was a blocker to moving forward with #180931, which replaced the override-based approach with a fully bootstrapped 11.0 stdenv on x86_64-darwin.

I'm not sure I follow. By 'override-based approach', do you mean the callPackage solution we have now? Both 10.12 and 11.0 SDKs already use pbzx? Isn't a 11.0 stdenv undesirable at this point, or is the plan to bootstrap a 10.12 SDK with a 11.0-based bootstrap tools?

I can upload new bootstrap tarballs if needed.

Thanks for the offer. I deliberately didn't ping, because I expected some discussion first, but can ping when there is a decision, if you'd rather not stay subscribed. 🙂

I'm not sure I follow. By 'override-based approach', do you mean the callPackage solution we have now? Both 10.12 and 11.0 SDKs already use pbzx? Isn't a 11.0 stdenv undesirable at this point, or is the plan to bootstrap a 10.12 SDK with a 11.0-based bootstrap tools?

I’m referring to the callPackage solution we have now. It relies on being able to override cc, which just happens to work for clang. While working on #180931, I explored bootstrapping the stdenv at @toonn’s suggestion. The direction it was going was to have a private 11.0 pkgs that the callPackage could selectively reference.
The callPackage approach doesn’t compose very well. I’ve been looking at using the 11.0 SDK to build Wine, but it’s going to be pretty messy. I think Qt 5 is another example of a package that’s too complex to build with just the override. It would be much easier if one could just reference the 11.0 SDK pkgs directly.
I have a proof of concept working. It builds Wine just by referencing <11.0 pkgs>.wine64Packages.wine. It’s a bit hacky though and requires the updated bootstrap-tools. From what I understand, while the PR was merged to add pbzx and tbd to x86_64-darwin, the bootstrap tools archive hasn’t been updated. Using a bootstrapped 11.0 SDK stdenv on x86_64-darwin has the same requirements as aarch64-darwin, so it needs the same tools.
As far as whether it’s desirable to build an 11.0 stdenv and expose a pkgs, I don’t know. I think it’s a conversation worth having (though maybe not in this issue). From the perspective of MoltenVK, it’s going to increasingly make use of features introduced in Metal 3, which requires the 13.0 SDK. It can gracefully fall back on older systems, but it can’t use newer features without using the newer SDK.
Hopefully that helps clarify. If not, please let me know.

@reckenrode  I think you explained it well, thanks, and I won't dive in too much further here. Some specific questions for this issue:

What further updates to bootstrap-tools would help along experimentation on those areas, and which are not yet on master?

I understand the pbzx stuff is already in, and it just needs an upload?
Are updates to Clang or the SDK components we package inside bootstrap-tools helpful in any way? (Not sure what the minimum Clang is to bootstrap a 13.0 SDK stdenv, for example.)

Is there any benefit to doing this within the 23.05 release window?

My apologies for the slow reply. I meant to reply earlier but got sidetracked.

What further updates to bootstrap-tools would help along experimentation in those areas, and which are not yet on master?

The current state of master should be sufficient. What I need is parity between aarch64-darwin and x86_64-darwin because I’m trying to build the stdenv on x86_64-darwin using the same SDK that’s used on aarch64-darwin.
One thing I’d like to explore is using the linker in LLVM instead of cctools (since Apple seems to have stopped supplying source updates), but I wouldn’t add it to the bootstrap-tools proactively.

I understand the pbzx stuff is already in, and it just needs an upload?

Correct.

Are updates to Clang or the SDK components we package inside bootstrap-tools helpful in any way? (Not sure what the minimum Clang is to bootstrap a 13.0 SDK stdenv, for example. But maybe we simply don't know yet.)

That would also result in a bump in the clang used in the stdenv in the regular SDKs?
You’re right that we don’t know what clang is needed to bootstrap the 13.0 SDK. I wouldn’t proactively bump clang just for that, though there may be other reasons to do it (that don’t come to my mind but others might have).

Is there any benefit to doing this within the 23.05 release window?

The separate bootstrapped stdenv stuff seems unlikely for 23.05, so I wouldn’t try to make the release window just for that. I also think we’ve usually held off making changes this close to the release window in the past, so it might be something to target for 23.11.

While working on an LLVM bump to LLVM 15 (in support of #229210), I ran into the following issues. They can be worked around, but it would be ideal if the bootstrap tools were updated to make the workarounds unneeded.

sigtools needs updated to 0.1.3. LLVM 15 assumes it can call codesign --sign, but that’s not supported by the version in the bootstrap tools.
sigtools on x86_64-darwin. LLVM 15 assumes that the version of ld64 we have requires manual ad hoc signing regardless of platform.
CoreFoundation has definitions in CFAvailability.h that cause the build to fail. I believe this is related to https://github.com/llvm/llvm-project/issues/48757. Passing the flag to suppress the error does work (for now anyway).

Those are all I have found so far that have been blockers. My modifications for the bump allow it to use a different bootstrap compiler from the final one. It’s also written to play nicely with LLVM 11 even after #166401 is addressed.

After doing some more work, I don’t think any of the changes above are needed. I reworked the stdenv bootstrap to (re)build what it needs in stage 1. It may not even be necessary to include rewrite-tbd or print-exports on x86_64-darwin for newer SDKs, but I haven’t tried doing that (yet?).
Regarding the two bullets in the issue:
I also handle https://github.com/NixOS/nixpkgs/issues/166401 in the bootstrap. If the clang is bootstrap, it assumes the clang version is older than 12 and overrides the build flags. For clang in nixpkgs, it uses the flags defined in platform.nix unless your clang is 11, then it uses a compatible cpu definition.
I don’t think the changes to nar.xz will impact what I’m doing as long as the unpack scripts are updated accordingly. The reworked stdenv bootstrap tries to assume as little as possible about what it gets from bootstrap tools. The goal is to decouple from it, so bumps to LLVM don’t also require bumps to the bootstrap tools.

With the stdenv rework merged, I can confirm no changes or additions are required from the bootstrap tools. I think the place for the platform optimization stuff is in cc-wrapper like it’s done for GCC. Doing it in the stdenv is too brittle and not the correct place anyway.

Should this move forward again? We've waited quite a while for updates to group. Maybe the LLVM bump @reckenrode is working on?

It’s not needed for the LLVM update, so it should probably wait until after #241962 is merged to minimize the risk of breaking something and blocking the PR.

@reckenrode wrong PR link?

@reckenrode wrong PR link?

Thanks, fixed it. I messed up transcribing it while replying on my phone.

I'm working on a couple of changes to update the bootstrap tools. I made some modifications to the llvm toolchain for versions 12+ and thus would like to drop llvm11. also, stdenvBuildTools is now broken on staging-next and unpinning llvm11 will fix.

https://github.com/NixOS/nixpkgs/pull/295557
https://github.com/NixOS/nixpkgs/pull/295558

So far i've only tested on x64 darwin with the sandbox enabled.
things i've done

unpin llvm11 and remove override so the tools will be built with current stdenv
add @loader_path/. to various libs that reference other libs. eg: libc++ references and re-exports libc++abi
add a couple checks to just validate that shared objects with an @rpath  contain the library in lib
use getDev, getLib, getBin when copying out of the packages. hopefully this minimizes breakage in the future
somewhat aligned the darwin make-bootstrap-tools.nix with the linux side
use a nar container to hold the compressed cpio archive and binaries required to unpack. this is done rather than just using a nar file to reduce copying. the expanded archive is about 500M (74 M compressed).

i need to test on an aarch64 machine but once that's done i'd like to move forward on getting the code committed.
thx

closed by #302387

Closing, assumed indeed closed by https://github.com/NixOS/nixpkgs/pull/302387.
If this is inaccurate, please re-open and (or) add a comment detailing why.


Windows CI should also test GCC ELF enclaves

Currently, Windows CI tests only use Clang ELF enclaves and this is a shortcoming. This should be extended to test GCC ELF enclaves.

@johnkord @ionutbalutoiu - We may want to discuss if we really need this.  One configuration - Debug - Linux-Elf-Build may be all we need.

Hello @anitagov!
@dragoshsch tried this configuration and it seems to work on the current master branch. He opened the PR https://github.com/Microsoft/openenclave/pull/1478 adding a new testing stage for Win2016 Ubuntu1804 gcc Debug Linux-Elf-build (see https://github.com/Microsoft/openenclave/pull/1478/commits/8749517e823ce1ee4e592d936b335feb94437275 from the PR).
Regards,
Ionut


fix windows unit test

Fixes #
Checklist

[ ] Added unit / integration tests for windows, macOS and Linux?
[ ] Added a changelog entry in CHANGELOG.md?
[ ] Updated the documentation (README.md, docs)?
[ ] Does your change work on Linux, Windows and macOS?

@SimonBaeumer, something interesting I noticed with this fix. It seems the current executor isn't based on the shell you're running commander from. For example this test failed because I was expected the execution environment to be powershell. It appears to just be CMD on windows.
Should we provide a way to specify the execution env with a flag? Or better yet we interpret he execution environment?

@dylanhitt you are right, sorry about that. The PR looks good to me!
Hm, I would agree but the problem with Windows tests are more the environment it runs in (here CircleCI) and the tooling available.
If you want to add it and need it feel free to do so. 👍


logging-winston: make winston a peerDependency

Users of this library need to depend on winston directly. There are
scenarios where the user, or other libraries may depend on a semver
incompatible version of winston. Not only does it end up installing an
unnecessary copy of winston; it may also lead to our transport not
being visible in winston.transports that the user has visibility to.

This is breaking, so this should bump us the logging-winston module to 0.5.0.

Coverage remained the same at 99.988% when pulling ecb1bf26369b969dcf32399cd7d749bb4d95d1cf on ofrobots:winston-peer into 193e747a6732895ea5052fc323bc27276b397b4c on GoogleCloudPlatform:master.

Need a release now?

Hold off for https://github.com/GoogleCloudPlatform/google-cloud-node/pull/2454.


set MDSD_MSGPACK_SORT_COLUMNS to perf column sorting on MDSD side

Which issue this PR addresses:
Try to avoid hitting max schema count issue on RP logs search by setting MDSD_MSGPACK_SORT_COLUMNS=1 in mdsd env variables
What this PR does / why we need it:
Try to avoid hitting max schema count issue on RP logs search by setting MDSD_MSGPACK_SORT_COLUMNS=1 in mdsd env variables.
Test plan for issue:

Are there unit tests? No
Are there integration/e2e tests? No
If it is not possible to write automated tests, explain why and document how
to manually test and verify the feature. Will need to test in INT

Is there any documentation that needs to be updated for this PR?
N/A

Will test in INT first - stay tuned


Handle SVD files which don't define register reset values

Atmel's ARM chips omit the <resetValue> element for register descriptions. <resetValue> is used to generate a write function which doesn't have to read the initial value from a register in order to preserve it while modifying individual fields.

You can access the bits via modify

I have used @ecstatic-morse's branch to generate the code for an ATtiny2313. The generated code compiles, but doesn't contain .write() and .modify() at all.

@kunerd, I was wrongly using "ocd-rw" to determine whether registers were read only, when really this field tells the on chip debugger not to read or write to this register because of side effects.

@kunerd @ecstatic-morse is this an issue with the ATDF to SVD parser, or with svd2rust?
I'd like to close this if there is nothing to be done at the svd2rust level.

@jamesmunns: Yes this an issue with ATDF to SVD parser, but closing depends on the questions:

Do we want to have a separate atdf2rust executable, or should .atdf support be added to svd2rust directly. And if we choose the separate executable way, should it be part of svd2rust or should it life in it's own repo?

I prefer the former one (for the first question), but I think the questions have not really been answered, yet. See #184

@kunerd is there anything that #184 doesn't cover that this one does? AFAICT, svd2rust does handle registers without reset values (even without <resetValue> elements), as long as they are marked as writable.
I will definitely follow #184 though (no opinions yet :) )

@jamesmunns Ahh, I missed the part with the ocd-rw above. So, I would say, this can be closed, but maybe you want to wait until @ecstatic-morse confirmed that, because he has done the implementation and I only tried to test it.

@ecstatic-morse any feedback?

This should be supported. The SVD spec allows <resetValue> to be missing. In the case that resetValue is missing we should not generate the reset method and we should modify the write method somehow because that makes use of resetValue.

I've come across this recently as I'm trying to use an Atmel provided SVD file which doesn't have <resetValue> for all registers.
I belive that with no reset value the reset and reset_value methods should be omitted and the write method should instead initialise w to 0 as in:
let mut w = W { bits: 0 };

See https://github.com/sjroe/svd2rust/tree/resetvalue for an example implementation.

I've pushed pull request #258 with a rebased version of #175 that also addresses the comments from @sjroe.

Closing due to write_with_zero. Reopen if you have better solution.


STM32L011D4 flash/erase error

[X] Programmer/board type: Stlink/v2,
[X] Programmer firmware version: V2J28S7
[X] Operating system: Linux Mint 17.1, 64b
[X] Stlink tools version and/or git commit hash: v1.3.1-4-g9d08810
[X] Stlink commandline tool name: st-flash
[X] Target chip (and optional board): STM32L011D4 (in TSSOP14 package)

Program st-info detect connected MCU, only RAM size is wrong. After try erase/flashing binary file (with st-flash), there is an error:
st-flash erase
st-flash 1.3.1-4-g9d08810
2017-03-10T10:43:40 INFO src/common.c: Loading device parameters....
2017-03-10T10:43:40 INFO src/common.c: Device connected is: L011 device, id 0x10086457
2017-03-10T10:43:40 INFO src/common.c: SRAM size: 0x4000 bytes (16 KiB), Flash: 0x4000 bytes (16 KiB) in pages of 128 bytes
-> Flash page at     0/  128 erased
2017-03-10T10:43:40 WARN src/common.c: pecr.pelock not clear (0x2030405)
2017-03-10T10:43:40 WARN src/common.c: Failed to erase_flash_page(0x8000080) == -1
stlink_erase_flash_mass() == -1

st-flash write program.bin 0x8000000
st-flash 1.3.1-4-g9d08810
2017-03-10T10:43:25 INFO src/common.c: Loading device parameters....
2017-03-10T10:43:25 INFO src/common.c: Device connected is: L011 device, id 0x10086457
2017-03-10T10:43:25 INFO src/common.c: SRAM size: 0x4000 bytes (16 KiB), Flash: 0x4000 bytes (16 KiB) in pages of 128 bytes
2017-03-10T10:43:25 INFO src/common.c: Attempting to write 7148 (0x1bec) bytes to stm32 address: 134217728 (0x8000000)
Flash page at addr: 0x08000000 erased2017-03-10T10:43:25 WARN src/common.c: pecr.pelock not clear (0x2030405)
2017-03-10T10:43:25 ERROR src/common.c: Failed to erase_flash_page(0x8000080) == -1
stlink_fwrite_flash() == -1

I also try MCU STM32L031K6 (nucleo32 board) and works fine. Is there problem in flashloader or somewhere else?
Please add support for STM32L011xx.

Probably duplicate of #564. I only added preliminary support for STM32L011xx so it would show up in the probe. It indeed seems te flashloader for this version is currently broken.

Could you test against PR #572 (latest git master)?

Hello!
I tested flashing against #572. Chip erase is OK, with flashnig are some problems:
When flashing, in terminal counts of flashed pages. From 1/66 to 65/65. In penultimate page occurs error:
/usr/local/bin/st-flash --reset write build/EMS_tx_C.bin 0x8000000
st-flash 1.3.1-4-g9d08810
2017-03-16T08:23:21 INFO src/common.c: Loading device parameters....
2017-03-16T08:23:21 INFO src/common.c: Device connected is: L011 device, id 0x10086457
2017-03-16T08:23:21 INFO src/common.c: SRAM size: 0x2000 bytes (8 KiB), Flash: 0x4000 bytes (16 KiB) in pages of 128 bytes
2017-03-16T08:23:21 INFO src/common.c: Ignoring 60 bytes of 0x00 at end of file
2017-03-16T08:23:21 INFO src/common.c: Attempting to write 8504 (0x2138) bytes to stm32 address: 134217728 (0x8000000)
Flash page at addr: 0x08002100 erased
2017-03-16T08:23:22 INFO src/common.c: Finished erasing 67 pages of 128 (0x80) bytes
2017-03-16T08:23:22 INFO src/common.c: Starting Half page flash write for STM32L core id
2017-03-16T08:23:22 ERROR src/flash_loader.c: unknown coreid, not sure what flash loader to use, aborting! coreid: bc11477, chipid: 457
2017-03-16T08:23:22 WARN src/flash_loader.c: Failed to write flash loader to sram!
2017-03-16T08:23:22 WARN src/common.c: stlink_flash_loader_init() == -1
2017-03-16T08:23:22 WARN src/common.c:
write_half_pages failed == -1
 65/ 66 pages written
2017-03-16T08:23:30 INFO src/common.c: Starting verification of write complete
2017-03-16T08:23:30 ERROR src/common.c: Verification of flash failed at offset: 0
stlink_fwrite_flash() == -1
make: *** [flash] Chyba 255

Size of flashed program was 8.5k and interesting thing is, that program start after reset.
In #572, is wrong parameter .sram_size, I thing. The amount of sram is only 2kB, so .sram_size=0x800
Next, if the MCU is in boot state (BOOT0 pin in hi state after HW reset), stlink cant connect: unknown chip id! 0

I tried flashing another board and got the same error (3/4 pages written as it was a small test program), so #572 might have some issues. The .sram_size parameter is a bit confusing; the chip only has 2kB of sram but table 3 of RM0377 (pg 52) seems to suggest that the total reserved memory address space is 8k. I'll try changing back to .sram_size=0x800 and see if that fixes the error.

@duro80 I missed adding sl->chip_id == STLINK_CHIPID_STM32_L011 to flash_loader.c. I just made that change and tested flashing an STM32L011G4 and didn't get any errors. See this commit -- can you test to verify that it works on your end before I submit a fresh PR?
Note that I tried changing .sram_size from 0x2000 to 0x800 and I didn't get errors either way, so I left it as-is. I defer to the experts on that one.

Hi @zakqwy .
There is still some error:
st-flash 1.3.1-4-g9d08810
2017-03-17T08:17:48 INFO src/common.c: Loading device parameters....
2017-03-17T08:17:48 INFO src/common.c: Device connected is: L011 device, id 0x10086457
2017-03-17T08:17:48 INFO src/common.c: SRAM size: 0x2000 bytes (8 KiB), Flash: 0x4000 bytes (16 KiB) in pages of 128 bytes
2017-03-17T08:17:48 INFO src/common.c: Ignoring 60 bytes of 0x00 at end of file
2017-03-17T08:17:48 INFO src/common.c: Attempting to write 10228 (0x27f4) bytes to stm32 address: 134217728 (0x8000000)
Flash page at addr: 0x08002780 erased
2017-03-17T08:17:48 INFO src/common.c: Finished erasing 80 pages of 128 (0x80) bytes
2017-03-17T08:17:48 INFO src/common.c: Starting Half page flash write for STM32L core id
2017-03-17T08:17:48 INFO src/flash_loader.c: Successfully loaded flash loader in sram
2017-03-17T08:17:51 ERROR src/flash_loader.c: flash loader run error
2017-03-17T08:17:51 WARN src/common.c: l1_stlink_flash_loader_run(0x8000000) failed! == -1
2017-03-17T08:17:51 WARN src/common.c:
write_half_pages failed == -1
 78/ 79 pages written
2017-03-17T08:18:00 INFO src/common.c: Starting verification of write complete
2017-03-17T08:18:00 ERROR src/common.c: Verification of flash failed at offset: 0
stlink_fwrite_flash() == -1

The change is in line: src/flash_loader.c: Successfully loaded flash loader in sram
but next line is another error.
And still write only (n-1) pages from.

@duro80 does running with --debug give you any more detail on the error? I can't recreate on my end, at least with my STM32L011G4 boards.

I try on another computer, and result is here:
st-flash 1.3.1-9-gc04df7f
2017-03-17T16:53:35 INFO src/common.c: Loading device parameters....
2017-03-17T16:53:35 INFO src/common.c: Device connected is: L011 device, id 0x10086457
2017-03-17T16:53:35 INFO src/common.c: SRAM size: 0x2000 bytes (8 KiB), Flash: 0x4000 bytes (16 KiB) in pages of 128 bytes
2017-03-17T16:53:35 INFO src/common.c: Attempting to write 7124 (0x1bd4) bytes to stm32 address: 134217728 (0x8000000)
Flash page at addr: 0x08001b80 erased
2017-03-17T16:53:35 INFO src/common.c: Finished erasing 56 pages of 128 (0x80) bytes
2017-03-17T16:53:35 INFO src/common.c: Starting Half page flash write for STM32L core id
2017-03-17T16:53:35 ERROR src/flash_loader.c: unknown coreid, not sure what flash loader to use, aborting! coreid: bc11477, chipid: 457
2017-03-17T16:53:35 WARN src/flash_loader.c: Failed to write flash loader to sram!
2017-03-17T16:53:35 WARN src/common.c: stlink_flash_loader_init() == -1
2017-03-17T16:53:35 WARN src/common.c:
write_half_pages failed == -1
 54/ 55 pages written
2017-03-17T16:53:42 INFO src/common.c: Starting verification of write complete
2017-03-17T16:53:42 INFO src/common.c: Flash written and verified! jolly good!

With parameter --debug: http://pastebin.com/uHAtzudm

Problem seems to be in function stlink_flash_loader_run(...), sile flash_loader.c. At the end of function (line 363) is wait cycle:
    for (i = 0; i < WAIT_ROUNDS; i++) {
        usleep(10);
        if (stlink_is_core_halted(sl))
            break;
    }
    if (i >= WAIT_ROUNDS) {
        ELOG("flash loader run error\n");
        return -1;
    }

I think, that core is still RUNNING, because function stlink_flash_loader_run return -1. In my previous comment, I attached listing of st-flash with --debud parameter. The file has 3.5MB and most of the content of log output was:
2017-03-17T17:35:46 DEBUG src/common.c: *** stlink_status ***
2017-03-17T17:35:46 DEBUG src/common.c:   core status: running

But I don't now how to enter in HALT state to correctly exit stlink_flash_loader_run function. May be flashloader is no correct for L0, I don't now.

Hmm. Again, unfortunately I have to defer to the experts on this one. I was able to get the micro flash tying the BOOT0 pin to ground and disconnecting the nRST line from my Stlink/v2. Maybe give that a shot?

Hi. I use ST-link  like http://www.ebay.com/itm/Mini-ST-Link-V2-Stlink-Emulator-Downloader-Programming-STM8-STM32-w-Metal-Shell-/282122691251
When I flashing the code the MCU have to be in normal/run mode. When MCU is in boot mode (running the bootloader) by connect log.0 to NRST while BOOT0 pin is in log.1, the st-flash/st-info cant connect to MCU.
But: interesting is, that st-flash write "Flash written and verified! jolly good!" and the end.

Please verify if the problem still exists in Release v1.6.0.


One To Many Relation

Hello,
I noticed when I add field as Drop Down / Multi-select, then choose values from Table Source. The one to many relationship is implemented by adding a column to the table with type varchar. Then, lara admin generator only fills the cell with array of ids of related fields from foreign table.
I wonder how to normalize this kind of auto generated relation to a new separate table without affecting Lara Admin auto generated validations / views, etc.????

Hi, can you please guide me how to do one to many relation here.
I have courses module and books module, and now I am trying to build relation so I have added in the couse migration file like this.
 ["books", "Books", "Multiselect", false, [0], 0, 0, false, "@books"],

class CreateCoursesTable extends Migration
{
    /**
     * Run the migrations.
     *
     * @return void
     */
    public function up()
    {
        Module::generate("Courses", 'courses', 'name', 'fa-cube', [
            ["name", "Name", "Name", true, "", 5, 250, true],
            ["teacher", "Teacher", "String", false, "", 0, 256, true],
            ["fees", "Fees", "Currency", false, "0", 0, 2, true],
            ["description", "Description", "Textarea", false, "", 0, 1000, false],
            ["books", "Books", "Multiselect", false, [0], 0, 0, false, "@books"],
        ]);

While doing php artisan migrate:refresh
Getting error
Class CreateBooksTable not found


Could you make refactoring for Lua script?

I'd be so happy.

I don’t know Lua at all, so I’m really not the right person to do this. Pull requests are welcome, though!


Protocols can now be defined  in yaml files

This PR moves the protocol definition from the source code to YAML files. That way new protocols can be added by YAML file creation.

The Anchore container scan is not happy with the remote Dockerfile and I have no idea how to fix it at the moment. Can you please have a look ?

The anchore scan fails because it is using the master branch of the main repo https://github.com/Threagile/threagile.git instead of the changes in this pull request. I submitted PR #53 to fix this and recommended some changes in order to be able make the process simpler.


Provide a way to close producers, fixes #42

Codecov Report

Merging #77 into master will decrease coverage by 5.55%.
The diff coverage is 90%.

@@            Coverage Diff             @@
##           master      #77      +/-   ##
==========================================
- Coverage     100%   94.44%   -5.56%
==========================================
  Files           4        4
  Lines          11       18       +7
  Branches        0        1       +1
==========================================
+ Hits           11       17       +6
- Misses          0        1       +1

Impacted Files
Coverage Δ

...nfradet/spark/kafka010/writer/RDDKafkaWriter.scala
100% <ø> (ø)
:arrow_up:

...det/spark/kafka010/writer/DStreamKafkaWriter.scala
100% <ø> (ø)
:arrow_up:

...det/spark/kafka010/writer/KafkaProducerCache.scala
90% <90%> (-10%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f63033a...dd46f31. Read the comment docs.