Create a dedicated repository for binary documentation assets?

In light of issues such as #169, I wonder if we should establish a way to store binary blobs (aka image files) that we want to use in the documentation. Right now, we already use 2-3 images in the README.md and the docs, but each time I consider adding a new one I feel slightly bad about permanently increasing the repo size by 50-100 KiB. This is exacerbated by the fact that images, just like other documentation, often become outdated very fast and need to be replace regularly.
One possible solution could be to have a dedicated repository to store documentation assets such as images or videos, e.g., trixi-framework/documentation-assets, that we allow to grow in size over time. A different approach could be to store binary assets in a dedicated branch within the main repo, which gets regularly purged by deleting its history (however, this would raise the question of if and how docs for older releases should be kept alive).
Another thought (although slightly orthogonal to the main question here), is whether it would be beneficial to have a dedicated repository to host the documentation, such as Oceananigans.jl does it with OceananigansDocumentation.

I like the suggestion to not clutter our main repo with binaries and use a dedicated repo for that :+1:
It could make sense to use another repository for the documentation, but we should definitely figure out how to set up everything such that we still get all the benefits we have right now (automatic creation of docs for all releases so far).

What do you propose as repository names? Maybe

trixi-framework/TrixiDocumentation for the docs, and
trixi-framework/TrixiDocumentationAssets for the assets?
The latter name is not as important as it won't be shown anywhere but in the source code, but the first one will become part of the documentation URL.

I am not sure if it is easy to keep the history for all releases so far. For the future it should be no problem, as it seems like it's only a matter of specifying a different repo in the call to deploydocs here:
https://github.com/trixi-framework/Trixi.jl/blob/645fe4aecf10ba9bc688676d141c7133744a547d/docs/make.jl#L56-L58

trixi-framework/TrixiDocumentation for the docs, and
trixi-framework/TrixiDocumentationAssets for the assets?

Sounds good to me :+1:

I am not sure if it is easy to keep the history for all releases so far.

I think it would also be sufficient to keep the docs for v0.2 (current stable release).

By default, Documenter pushes documentation to the gh-pages branch. If the branch does not exist it will be created automatically by deploydocs. If it does exist then Documenter simply adds an additional commit with the built documentation.

Xref. https://juliadocs.github.io/Documenter.jl/stable/man/hosting/#gh-pages-Branch
So it sounds like we should be able to keep everything. Alright, I've removed the question label and replaced it by good first issue


Manage junk packets being sent when plugging in

When you plug an already-powered (plugged in a battery) ProMotion board in the PC, there might be junk bytes already in the buffer. The program needs be able to manage this.

Only Windows issue, can be seen more easily with CoolTerm?

I think I can fix this bug. The issue happens when the streaming is already ON. We need to close and re-open the COM port at the beginning and flush the UART input buffer. I will create a branch for this.

Go ahead


How setup an Authorization field in HTTPHeader?

Moya: 8.0.0-beta.5
Swift: 3.0
Xcode: 8.1
Hi, everyone
I am using the endpointClosure as follows for my GET requests, but I am getting a 401 response.
let endpointClosure = { (target: MyTarget) -> Endpoint<MyTarget> in
        let defaultEndpoint = MoyaProvider<MyTarget>.defaultEndpointMapping(target)
        switch target{
        case .event:
        return defaultEndpoint.adding(newHTTPHeaderFields: ["Authorization": "Token token=\(access_token)"])
        default:
            return defaultEndpoint
        }
    }

When using the DHC Restless extension I add the header like this and works:

Can someone say what I am doing wrong or a workaround?
My Target:

enum ServiceAPI{
    case event(dateID: Int)
}

extension ServiceAPI: TargetType{
    var baseURL: URL {return URL(string: "http://myapi.com/api")!}

    var path: String{
        switch self{
        case .event(let dateID):
            return "/dates/\(dateID)"
        }
    }

     var method: Moya.Method{
        switch self{
        case .event:
                return .get
        }
    }

     var parameters: [String : Any]?{
        switch self{
        case .event:
            return nil
        }
    }

     var task: Task {
        switch self {
        default:
            return .request
        }
    }

     var sampleData: Data {
        return "Hi".utf8Encoded
    }
}

private extension String {

    var utf8Encoded: Data {
        return self.data(using: .utf8)!
    }
}

Really thanks!

So I've found a workaround for this issue. I added an empty string after the authorization parameter and it worked!
let endpointClosure = { (target: MyTarget) -> Endpoint<MyTarget> in
        let defaultEndpoint = MoyaProvider<MyTarget>.defaultEndpointMapping(target)
        switch target{
        case .event:
        return defaultEndpoint.adding(newHTTPHeaderFields: ["authorization ": "Token token=\(access_token)"])
        default:
            return defaultEndpoint
        }
    }

If there is a better way to do it let me know!
Thanks.

Hey @liliah.
I can't say that I know specifically what is going wrong here, but if I had to guess, I would say that there could be a problem with how the authorization header is being encoded and/or how it is interpreted by your API. If you have the ability to change how authorization headers are used by the API, I think you might have fewer issues if you use the standard approach for access-token-based authentication. This is the standard format:
Authorization: Bearer <access_token>

If you have to use the current approach, I would note that the capitalization of Authorization also changed. That may be the source of the different behavior as well.
For debugging purposes, you can also utilize the NetworkLoggerPlugin. Turn on the curl output and then you can try the generated requests via CLI for another perspective on what might be going on.

Were you able to get this sorted out @liliah?

The workaround that I posted before is actually working. I contacted the API's developer and said about this issue but he told me that it's working normally on Postman and on the Android client so I don't know what exacly caused it. You can close the issue and thanks!

Glad to hear it is working for you! 👍

I've encountered this issue using AccessTokenAuthorizable, by removing it and using endpointClosure "Authorization " instead of "Authorization" it worked

That is interesting @mihailstumkins. Would you mind opening up an issue to track that? AccessTokenPlugin and AccessTokenAuthorizable are new additions, so there is a possibility that they contain a bug that is not being tested for.

parameter

wished to work for me but it doesn't

I've encountered this issue using AccessTokenAuthorizable, by removing it and using endpointClosure with "Authorization " string instead of "Authorization" it worked

I love you !!!!! <3


Observe allocated deltas in system docs

Description
A new parameter has been added in the SHOW STORAGE INFO that tracks the total unreleased amount of Delta objects in the system.
Pull request type
Please check what kind of PR this is:

[X] Fix or improvement of an existing page
[ ] New documentation page, release related

Related PRs and issues
PR this doc page is related to:
(especially necessary if the PR is related to a release)

Closes:
X
Checklist:

[X] Check all content with Grammarly
[X] Perform a self-review of my code
[X] Make corresponding changes to the rest of the documentation (consult with the DX team)
[X] The build passes locally
[X] My changes generate no new warnings or errors
[X] Add a corresponding label
[X] If release-related, add a product and version label
[X] If release-related, add release note on product PR

@kgolubic Please don't merge into memgraph-2-18, we'll wait for 2.19 for this.


Add new basic tests TestDocsGenerateDatabricks and TestDocsGenReferencesDatabricks

Description
Adds new basic tests TestDocsGenerateDatabricks and TestDocsGenReferencesDatabricks based on BaseDocsGenerate and BaseDocsGenReferences respectively as per an item from #127.

Thanks! merging.


rpm-packages sometimes contains string with exception

When plugin PostBuildRPMqaPlugin doesn't finish successfully (raise an exception) rpm-packages label contain string describing this exception. Because both proper output of that plugin and this error are both strings it is hard to determine if the content of the label is valid.
Please provide a way to determine if the run was successful. E.g. content of rpm-packages would be either None or empty string and the reason why the plugin failed would be available somewhere else.
The content of rpm-packages was something like this (it was reconstructed from some split() which ate spaces):
plugin 'all_rpm_packages' raised an exception: 'ReadTimeout(ReadTimeoutError("UnixHTTPConnectionPool(host='localhost', port=None): Readtimedout.(readtimeout=60)"))'

Note: I'm using BuildResponse().get_rpm_packages() call of osbs-client to get this key.

So, the all_rpm_packages plugin raised an exception, the plugins runner stored a failure message in plugin_results, and the store_metadata_in_osv3 dutifully copied that message into the metadata.
I think PluginsRunner.run() needs to store plugin results more carefully. Perhaps it could store the actual exception rather than a description of it. Then the metadata plugin can skip it.

Should we increase the timeout? Do you know why it took docker to run the command so long?

I don't have more data about this.

Since #240 was merged, can we close this?

Yes.


added version command

looked like:
$ lavatory version
0.0.3.dev1+dirty

Looks good to me, passing version command is giving a version.


Update e3dc-rscp to 1.2.2

Please update my adapter ioBroker.e3dc-rscp to version 1.2.2.
This pull request was created by https://www.iobroker.dev c0726ff.

RE-CHECK!

Automated adapter checker
ioBroker.e3dc-rscp

:thumbsup: No errors found

[ ] :eyes: [W171] "common.title" is deprecated in io-package.json
[ ] :eyes: [W105] "common.titleLang" should be translated into all supported languages (en, de, ru, pt, nl, fr, it, es, pl, uk, zh-cn)
[ ] :eyes: [W109] "common.desc" should be translated into all supported languages (en, de, ru, pt, nl, fr, it, es, pl, uk, zh-cn)
[ ] :eyes: [W156] Adapter should support admin 5 UI (jsonConfig) if you do not use a React based UI
[ ] :eyes: [W145] Each "common.news" should be translated into all supported languages (en, de, ru, pt, nl, fr, it, es, pl, uk, zh-cn)
[ ] :eyes: [W145] Each "common.news" should be translated into all supported languages (en, de, ru, pt, nl, fr, it, es, pl, uk, zh-cn)
[ ] :eyes: [W145] Each "common.news" should be translated into all supported languages (en, de, ru, pt, nl, fr, it, es, pl, uk, zh-cn)
[ ] :eyes: [W145] Each "common.news" should be translated into all supported languages (en, de, ru, pt, nl, fr, it, es, pl, uk, zh-cn)
[ ] :eyes: [W145] Each "common.news" should be translated into all supported languages (en, de, ru, pt, nl, fr, it, es, pl, uk, zh-cn)
[ ] :eyes: [W145] Each "common.news" should be translated into all supported languages (en, de, ru, pt, nl, fr, it, es, pl, uk, zh-cn)
[ ] :eyes: [W145] Each "common.news" should be translated into all supported languages (en, de, ru, pt, nl, fr, it, es, pl, uk, zh-cn)
[ ] :eyes: [W145] Each "common.news" should be translated into all supported languages (en, de, ru, pt, nl, fr, it, es, pl, uk, zh-cn)
[ ] :eyes: [W145] Each "common.news" should be translated into all supported languages (en, de, ru, pt, nl, fr, it, es, pl, uk, zh-cn)
[ ] :eyes: [W145] Each "common.news" should be translated into all supported languages (en, de, ru, pt, nl, fr, it, es, pl, uk, zh-cn)
[ ] :eyes: [W145] Each "common.news" should be translated into all supported languages (en, de, ru, pt, nl, fr, it, es, pl, uk, zh-cn)
[ ] :eyes: [W145] Each "common.news" should be translated into all supported languages (en, de, ru, pt, nl, fr, it, es, pl, uk, zh-cn)
[ ] :eyes: [W145] Each "common.news" should be translated into all supported languages (en, de, ru, pt, nl, fr, it, es, pl, uk, zh-cn)
[ ] :eyes: [W145] Each "common.news" should be translated into all supported languages (en, de, ru, pt, nl, fr, it, es, pl, uk, zh-cn)
[ ] :eyes: [W145] Each "common.news" should be translated into all supported languages (en, de, ru, pt, nl, fr, it, es, pl, uk, zh-cn)
[ ] :eyes: [W513] "gulpfile.js" found in repo! Think about migrating to @iobroker/adapter-dev package

Add comment "RE-CHECK!" to start check anew

This release has been created at 25.05.2023, 18:10:00 and is now not even available at latest repository.
Release must be available at latest for one or two weeks and/or have a significant number of users and have no major issues.
Exceptions exists of course, ie. if this is a hiot fix for an urgent problem.
Please indicate why this release should be released immidiatly - otherwise I will keep it in evidence and release in some time.

This release has been created at 25.05.2023, 18:10:00 and is now not even available at latest repository.
Release must be available at latest for one or two weeks and/or have a significant number of users and have no major issues. Exceptions exists of course, ie. if this is a hiot fix for an urgent problem.
Please indicate why this release should be released immidiatly - otherwise I will keep it in evidence and release in some time.

Thank you for the heads-up. In fact, 1.2.2 is kind of an urgent fix for 1.2.1 (see Issue#160) - this is why I intended to skip 1.2.1 in stable repo.
But it is not crucial: the bug/fix does not affect the majority of users, so I expect no big impact when waiting 2 weeks before v1.2.2 is declared "stable".

RE-CHECK!

Automated adapter checker
ioBroker.e3dc-rscp

:thumbsup: No errors found

[ ] :eyes: [W156] Adapter should support admin 5 UI (jsonConfig) if you do not use a React based UI
[ ] :eyes: [W145] Each "common.news" should be translated into all supported languages (en, de, ru, pt, nl, fr, it, es, pl, uk, zh-cn)
[ ] :eyes: [W202] Version of package.json (1.2.3) doesn't match latest version on NPM (1.2.2)

Add comment "RE-CHECK!" to start check anew

@git-kick
Thanks for fixing the adapter checker issues so fast.
I'll release the 1.2.2 after the holidays if no major issue arises and some users from latest have used as you note an important fix.
I would ask to fix two other problems:

This adapter seems to use sentry, but no sentry info is added to Readme.md
This adapter seems to use sentry for error logging. Please add the suggested sentry information at the top of Readme.md
**This adapter uses Sentry libraries to automatically report exceptions and code errors to the developers.** For more details and for information how to disable the error reporting see [Sentry-Plugin Documentation](https://github.com/ioBroker/plugin-sentry#plugin-sentry)! Sentry reporting is used starting with js-controller 3.0.

Please add the sentry info to README.md. See other adapters if you need more info.
A new release is NOT necessary for this change as github always displays the head revision anyway.

The PR has (now) conflicts cause by the fact that you created PRs for 1.2.1 and 1.2.2 from the same base while 1.2.1 is now released. I can fix this druing merge, but maybe it's easier if you rebose the PR yourself.

Note that the warnings reported by re-check (https://github.com/ioBroker/ioBroker.repositories/pull/2310#issuecomment-1564646541) can be ignored. Form my side we are done as soon as you add the sentry info and the requested release is active at lasted for some days.
Thanks for effort and for maintaining thsi adapter.

Please add the sentry info to README.md

done (master branch).

maybe it's easier if you rebose the PR yourself.

I'm afraid I do not really understand what I should do.
I tried to repeat "Set as stable" in the ioBroker Developer Portal, but failed:

Update ioBroker.e3dc-rscp to 1.2.2 in stable repository
Connecting to GitHub...
Connected to GitHub as git-kick
Found fork git-kick/ioBroker.repositories
Cloning repository from GitHub
Creating branch e3dc-rscp-1.2.2
Preparing ioBroker.repositories (this may take a while)
Updating e3dc-rscp to 1.2.2 locally
Pushing changes to git-kick/ioBroker.repositories
Error: Process exited with 1
Failed

How can I (re-)trigger the v1.2.2 release for the stable repo?

maybe it's easier if you rebose the PR yourself.

I'm afraid I do not really understand what I should do. I tried to repeat "Set as stable" in the ioBroker Developer Portal, but failed:
How can I (re-)trigger the v1.2.2 release for the stable repo?

OK, sorry for the typ.
rebose should read REBASE, that a git functionality
The "problem" is that there is a merge conflict currently.
I'm not a spcialist with github so I cannot answer hwo to effectly rebase at your side. Sorry.
I will fix this during merge - so do not bother further with this detail.
For the future - please ensure that thers only ONE open PR for a single adapter in parallel.
Thanks for processing my suggestions.

I will fix this during merge - so do not bother further with this detail.
For the future - please ensure that thers only ONE open PR for a single adapter in parallel.

Many thanks. I rely on you this time and next time, I will do my best to avoid two concurrent PRs.

1.2.2 25.5. user 52 (5%)
sentry has been added to readme.md
OK wg. wichtigem Fix


stereo and surround sound audio

done when outputting array or object
[frontLeft, frontRight, center, sideLeft, sideRight, rearLeft, rearRight, subwoofer]
{frontLeft, frontRight, center, sideLeft, sideRight, rearLeft, rearRight, subwoofer}
colors match ANSI/CEA-863-A

surround sound is an interesting thing to support but would probably cause issues for many soundsystems, so only stereo should be added
[left, right], no object support
left is green and right is magenta, when overlapping they become white
this is chosen for a few reasons:

green is the most visible of the primary colors and magenta is the two others combined which balances luminosity better than other single / dual primary color pairs
clearly the two best colors
this color order matches green magenta 3d glasses
doesn't conflict with error colors like red and yellow


remove dollar sign in line 1 in openDAO Overview

I suspect this might be an issue with overview page crashing

Probably not because we don’t use GitHub overview now, overview is from arweave -  sudheer


need to add preload attribute for mozilla

I find a little bug in Mozilla FF (41-50 versions). if Audio tag have no preload attribute audio events lifecycle are broken. Html5 will not trigger canplay event while javascript trigger 'play' event programmaticly.
so i think we must add workaround for this situation.

If they don't have a preload we can add it by default to none; would that work?

I tested this on FF 50 using Browserstack and I couldn't find any issues. Is this something related to the Flash shims only?

Preload none will works. We found this bug in 43 version. Some users says that it can be in 50-51. It's not reliable information.

Ok I'll add it soon. Thanks for reporting this.

The feature was added. Thanks for reporting this. Closing issue


No integrity hash in yarn.lock?

I'm currently working with reproducible build systems. For whatever reason, whatwg-mimetype is the only package in the yarn.lock file that doesn't have an integrity sha1 hash, which seems to correlate with whatwg-mimetype producing a different hash on each compilation (and then for other reasons I don't understand, nix is then saying that it failed to match its hash. Where is it getting its hash? I think I've found the script that does it, but I haven't looked deeper).
So I guess I'm just here to ask if you know why this might be.

Ah, sorry, I've noticed whatwg is the only package being referenced through its github link, that's probably what it is, anomaly is not on your end, closing.


Can not use beta version @anthropic-ai/vertex-sdk

I used this package, and it referred to this repository

I tried using the beta version, where the interface already defines messages as BetaContentBlockParam[].

However, when I attempted to send a PDF file with type = 'document', it returned an error stating:

Does anyone have any ideas on how to resolve this issue? Thank you

source
{
  type: "image",
  source: {
    type: "base64",
    media_type: "image/png",
    data: "your_base64_raw"
  }
}

@x-ray-s Thank you. But I want to send pdf file. Is it possible with type is "image" ?

@thuynt46 try use beta namespace and BetaBase64PDFBlock describe the types.

@x-ray-s  Thank you. I did this, which is why I created this issue. Even when I use the model version that supports PDF vision. It is possible that the beta version of the Vertex API does not support all endpoints of the Anthropic beta. I am not sure exactly what the issue is. Anyway, thank you for supporting me.


Add support for thread commissioning

Problem
Add and enable zcl command does not support Thread
Change overview
Align Add and enable zcl command to support Thread
Testing
Tested with https://github.com/chip-csg/chip-certification-tool-backend/pull/100.
Test: test_commissioning_thread

@wahajsyed can you look at this PR ?

@doublemis1 - Added a comment


Add method normalizeWebsiteOfUrl to format the website of url to lower case

This PR adds a new method normalizeWebsiteOfUrl in str.js to format the website of url to lower case. The new method is used in both link and autolink rule replacement to format the href property of an anchor tag. It can also be used in the App codebase when handling edited link.
Fixed Issues
$ https://github.com/Expensify/App/issues/18514
Tests
See unit tests
QA
N/A

Looks good, but there are some conflicts!

@fedirjh Could you review this as well?

@fedirjh @MariaHCD Updated App test steps and attached demo video in the PR description part, thank you.

@MariaHCD Friendly bump! Would you like to have a look at this PR? Thank you!


Ignoring all images when 2 or more tags are present

What steps will reproduce the problem?

Donwload all images with the "yang-do" tag from danbooru and gelbooru.
Download all images with the "yang-do" + "ponytail" tags from danbooru and gelbooru in a separate folder.

What is the expected behavior? What do you get instead?

All images are saved in the "yang-do" folder.
All images with the "yang-do" + "ponytail" tags are ignored. Nothing is downloaded.

How often does this problem occur?
Almost all the time.
Seems to happen when I have a folder with all images from an artist downloaded and try to download images from same artist with additional tags in a separate folder.
What version of the program are you using? On what operating system?
v5.4.2 on Windows 7 Ultimate 64bit SP1
Please provide any additional information below
Tried changing download location. Still ignored.
Managed to download after changing the original "yang-do" folder name to something else.

It must come from the duplicate prevention using the MD5 list.
Could it be that it is set to "Don't save" in your settings?

It was actually set to "Don't save". I actually  thought that option was there so it doesn't download an existing file a second time and overwrite it.
But should it still refuse to download files if the original folder and the second folder are not in the same location?

If a file already exists with the same name and directory, it won't be downloaded even if this setting is set to "Save".
This setting only relates to not having duplicates among all downloaded files, even if the filename and output directory are different.

And actually, since you already have the file downloaded, you can simply use "Copy" instead of "Save" to not re-download it. :smile:

I suppose I misunderstood the purpose of that setting entirely.
That could reduce my download times.

I didn't realize that this setting was for all files, although, I should have since I understand that there's a file with all the MD5 data. I've never tested it, though, as all my files are stored in one folder.

I guess this setting should be made clearer indeed, by renaming it to something like "global duplicates" and adding a second one "same-folder duplicates".


Tags (admin panel)

Tags should be unique, we should not have the permission to define repeated tags.
admin panel/tags

@aghpr Done

Done @ehsan-g


Commands & keybindings features

These features are needed to use commands and keybindings (see way-cooler/tree/registry-rejson):

[X] Keybindings use commands from the registry
[ ] Key presses can be sent through IPC
[ ] Lua can register key handlers in init script
[ ] Reconfigure keybindings at runtime

Updated to reflect changes in Lua thread design.

I'm going to go ahead and close this, so that we can chunk this up into individual issues to deal with and delegate them better.


(WIP) Handle TiDB write-write conflict when optimistic transaction mo…

…de is on.
When optimistic transaction mode is on TiDB detects write-write conflicts on commit. These exceptions must be mapped to Hibernate and retried.
https://docs.pingcap.com/tidb/stable/optimistic-transaction
https://docs.pingcap.com/tidb/stable/optimistic-transaction

I really like this!

Failed unrelated SQS tests, so just gonna merge


move viv debug messages to a TRACE log level

https://bugs.python.org/issue31732
things like: instruction stepping messages, etc.

maybe we can use -dd for "more verbose" debugging messages

levels:

error: the program is exiting with the given message, e.g. incorrect arguments
warning: this doesn't look right but we're going to try to continue, e.g. analyzing a PE as shellcode
info: status updates, e.g. progress bars etc
debug: additional relevant information, e.g. which functions look like decoding routines

proposed:

trace: fine grained and extremely verbose messages, e.g. every instruction emulated by vivisect


FEATURE: Provide also breakpoint-prev

When you use Bootstrap as a layout tool, with all it mixins, I come often in the situation where I want to select the previous breakpoint. For example, I set a variable for the breakpoint of a navigation and want to use the media-breakpoint-down mixin. Without this function, I have to set a second variable with the lower breakpoint. With this function, I can set it dynamically:
$variable: lg;

@include media-breakpoint-down(breakpoint-prev($variable)) {
   // Content
}

Hi @jonnitto.
Thanks for this PR, but I'm not really convinced of the added value of this. The function will not be used by Bootstrap by default and imo this is not something a lot of developers will use frequently. We'll need to draw the line somewhere to prevent a proliferation of functions.


haskell: Split boot libraries into individual outputs

PoC fix for #42069.
Applied only to GHC 8.2.2 because, well, it is just a quick PoC. I would really love to know what you all think.

@angerman @mpickering

@GrahamcOfBorg eval

I'm generally super duper happy about these changes, except for the pkgs/top-level/haskell-packages.nix ones. The idea with there is that instead of doing buildCompiler everywhere, we can just do buildHaskellPackages.compiler, buildHaskellPackages.hscolour, etc. This is just like how we do buildPackages.stdenv.cc instead of a hypothetical stdenv.buildCC or something. bootPkgs i think also should be kept, and instead the ghcs should do bootPkgs.hscolour instead of taking a seprate hscolour argument.

@Ericson2314 Thanks for the comments! I think I am starting to understand what is going on, but I still have so many questions!

The idea with there is that instead of doing buildCompiler everywhere, we can just do buildHaskellPackages.compiler

This sounds to me more or less like passing pkgs and buildPackages explicitly and then letting the expr get dependencies it wants manually instead of using callPackage 🤔. I can’t say that I find this idea bad, it’s quite the opposite, I think I like it, but it seems to contradict current nixpkgs practices. Don’t you think so?

The generic builder builds both the package and its Setup.hs. In order to build the package it needs libs from pkgs and build tools from pkgs.buildPackages. In order to build Setup.hs it needs libs from pkgs.buildPackages and build tools from pkgs.buildPackages.buildPackages. I don’t think it currently has access to the latter, but, if I understand correctly, it works this around by using those six attributes of mkDerivation. So, it takes nativeGhc from buildHaskellPackages, which means it is shifted one back and then puts it into pkgsBuildBuild which, iiuc, shifts it two more back, which gives three, doesn’t it? And we wanted just two. Why even bother getting nativeGhc from buildHaskellPackages if it is the same ghc that we were given anyway?
buildHaskellPackages = bh.packages.ghc822;
ghc = bh.compiler.ghc822;

What also makes me worry is that we use various nativeGhc attributes (assuming, I guess, that, in theory it can be a separate compiler than ghc, even though it can’t as the code above shows), but then we use these attributes from a “-1-compiler” with a “-3-compiler” when what we actually want is a “-2-compiler”. I have a feeling that I am missing something very important.

I completely don’t understand the different use of buildHaskellPackages. In GHC package sets it is just a build version of the same set. In GHCJS package sets it is a build version of, mmm, the set of packages used to build a matching(?) version of GHC? I don’t get it at all. :(

Given that I don’t really understand what is going on, I’ll just revert those changes that are irrelevant, as, I think, we can do well with having a heavily overloaded ghc attribute in the package sets, at least for some time.

@kirelagin I'll post more when I'm not on my phone (happy to also chat on IRC today), but the short answer to all of those is https://github.com/NixOS/nixpkgs/blob/master/pkgs/top-level/splice.nix hasn't been done for Haskell packages sets, so all the usual magic of getting the right version doesn't work. I do hate splicing, but maybe I just need to make a newScopeWithSlicing so Haskell and all the other users of newScope will not wet the bed with cross.
CC @matthewbauer

IRC sounds great, unfortunately, I’m afraid I won’t be available today, we can probably try tomorrow.

If splicing is not implemented for haskell packages, it means that a wrong compiler is used for Setup.hs as it is shifted only 1 back, not 2. nativeGhc should be coming from bootPackages.bootPackages.

@kirelagin I wish your second comment was correct! The problem is haskellPackageSet.ghc today is a lie, it is not the GHC built in that package set, as would be consistent with everything else, but the GHC used to build that package set.
stdenv has the same issue: stdenv.cc is not the gcc or clang from pkgs, but the gcc or clang from buildPackages. But least stdenv.cchas a "meta-package"-sounding name (to use the Debian term). That's one of the reasons I really like yourghc-to-compilerrename: it also gives it a "meta-package"-sounding name for consistency withstdenv`.

This is part of the reason why I changed it to passing the compiler explicitly – this way it is easier to track what is going on instead of trying to figure out which ghc comes from which package set and what it means.
Regarding the name, I would really prefer those “meta” names to have something special in them, e.g. _compiler (or just not be there and be passed explicitly), because it works only until someone uploads a package called compiler to Hackage 🤷♂.
Another option would be not to pass the compiler explicitly and not put it into the package set, but instead pass a package-set-selector (for example just the name, e.g. "ghc822") and pkgs to let the expression do pkgs.buildPackage.haskell.{compiler,packages}."${selector}" whenever it needs a package or the compiler. What do you think?

@kirelagin I tried not adding buildHaskellPackages to the set for similar reasons, but decided that was kind of too annoying, so I think I lean towards the _compiler option.

@kirelagin Are you / Serokell still interested in this sort of thing? I lay out in https://gitlab.haskell.org/ghc/ghc/issues/16503 how we're pretty close to being able to build a stage1 GHC that doesn't assume any bootstrapping plan, rather reading everything it needs from the settings file. That + some jerry-rigging in Nix along these lines would allow cross compiling to any platform. A build = host "cross compile" then in turn allows building another "stage 1" compiler (as far as GHC's own build system is concerned) that when told in settings to enable GHCi, TH, etc won't crash but work properly; i.e. is as good as a normal stage 2 proper native GHC build.

@Ericson2314 I, personally, am very interested and Serokell, in principle, too, however I’ll need to check whether we can actually devote any effort to this at the moment.
My current view on the subject is rather pessimistic: I feel that it is easier to rewrite all of nixpkgs Haskell infrastructure form scratch, migrating it to cabal-new and resolving other issues at the same time. However, I’ll take a look at the issue you mentioned and see if we can fix boot libraries in a less intrusive way...

Yes this is in part why I was bring up the GHC side of things. The more GHC can optionally "delegate" bootstrapping to the package manager (i.e. the Nix haskell infra), the better job the package manager can do.
If you are referring to https://github.com/input-output-hk/nix-tools, I agree we should be using that :). I guess I might be a bit more optimistic in that I think once GHC has its stuff order even the current infra could also take advantage of it too, but regardless, GHC is the choke-point here.

The documentation for haskell.nix (used to be referred to as nix-tools) is at https://input-output-hk.github.io/haskell.nix/
Notably we are using it mostly with a somewhat crude hack to make lib:ghc reinstallable (https://gitlab.haskell.org/ghc/ghc/merge_requests/490). I'm waiting for @Ericson2314 to clean up his settings file managing and then rebase it ontop of that. It should make the reinstallable lib:ghc even easier.

I've got the vast majority of the multi-target work done, and the rest open PRs. That's meant I've been able to start shuffling the build system to insure not regressions by construction, and also make building things separately more robust: https://gitlab.haskell.org/ghc/ghc/merge_requests/1897/diffs


track first open time across devices

i am doing some of these at home and some in my lunch break on my work pc, it would be great if the first opened time was saved to the chrome sync storage and tracked across devices.
i have done some work with chrome sync storage before so would be happy to take a look if you are open to the idea.
i believe firefox has a similar system but i have never used it so probably cant help much there

Hey there, sorry for the late reply. That sounds great, and I'm fully open to the idea. I haven't had much time to dedicate to the extension (I've only just today upgraded the manifest file that Google has been hounding me about), so if you'd like to submit a PR to add this feature, please do!

cool, ill take a look. (and dont worry, i have 2 extentions and a theme i need to update to v3)


Adding debug suspend mode

This is the first step for: https://github.com/fabric8io/fabric8-maven-plugin/issues/1003
@jstrachan can you have a look?
A new tag should be created in order for the downstream images to be able to pick the change.

LGTM though I don't appear to have karma to merge ;) @rhuss looks like you're the only one?

@jimmidyson do you have karma?


Support non-square tile geometries

Currently Wangscape only supports writing square tiles.

[x] Support rectangular tiles.
[ ] Support non-rectangular four-cornered tiles (using custom replacement for the corner-combiner and edge-favouring masks). Rhombic tiles will be the priority here, but an ideal solution would support any base tile mask that tessellates like a parallelogram.
[ ] Support hexagonal tiles. The output corner Wang tiles will be triangles, but they must be built for 2 different orientations. The tile count per clique will be 2*n^3 (rather than n^4 for rectangular tiles). This may require the support for user-defined masks for the base tile shape, corner-combiners, edge-favouring masks, etc.
[ ] Support non-hexagonal six-cornered tiles. Probably using the same framework as the non-rectangular four-cornered tiles.
[ ] Support triangular and three-cornered non-triangular tiles. The output corner Wang tiles will be hexagons, and the tile count per clique will be n^6. This may be impractical, especially since games with triangular tiles are so rare. As such, this is a low-priority task.

Some notes about how to add support for non-rectangular four-cornered tiles (with a view to implementing three- and six-cornered tiles in the future):

Tile resolution becomes a bounding box rather than defining tile geometry.
Tile geometry should be specified using a bitmap mask (two masks for triangular tiles). This mask should have a black or transparent background, and the region associated with each corner should have a distinct colour.
An "anchor" pixel should be specified (the centre or top-left of a display tile.
Two offset vectors should be specified, defining how the tile tessellates (in six-cornered tiles the third offset vector will be the sum or difference of the other two).
Test that the mask(s) has/have no overlaps or gaps when tessellated according to the offset vectors. The mask and each of its corner regions should be connected, to make this test easier.
The dual mask (two tiles for hex tiles) is/are found by recombining the corner regions according to the offset vectors.
The dual mask(s) and the original mask(s) may have different bounding boxes. The output tilesets will be based on the dual mask(s).
Generating corner masks for output tiles:

Quadrant selector module translations must be based on the offset vectors.
Three of the modules modules in the default combiner module group must be replaced:

The corner combiner (LL27-45)
The corner distance and peaks (LL51-99)
The edge-favouring mask (LL105-130)

They all assume square geometry, which won't work with arbitrary four-cornered masks. Instead they should be based on relative discrete distances to corners and edges.
To avoid recalculating these distances in every tile, they should be stored in a new bitmap module. This could be initialised manually in a function (like a Curve module) or automatically (like a Cache module, as a wrapper for a module which calculates these distance values).

Here's a draft of the tile mask rearrangement technique in Python: https://gist.github.com/serin-delaunay/a04417c5bd2ef9dad64d01f867c2ee19.
It uses some numpy.ndarray features heavily, not all of which translate well into SFML sprite methods.

Updated to identify edge and corner pixels and calculate distance fields: https://gist.github.com/serin-delaunay/3da9c4838e9e489d7ea1c4b22373c8cc.
Operations used:

Blitting with translation and clipping.
Connected component labelling.
Binary erosion.
2D array arithmetic (and, or, xor).
Nonzero index list (optional).
Padding, unpadding.
Breadth-first search with multiple targets. Replaceable with fast marching method for Euclidean distance.

These all translate pretty easily to C++; the only complex algorithm is fast marching, which is optional. Boost.MultiArray may help.
The existing Python script could be used directly (these operations are not in a performance hotspot), but would again require packaging. Unless called from Boost.Python, it would also affect how Wangscape is invoked.

Updated roadmap for resolving this issue:

Support arbitrary 4-cornered tiles:

Process tile partitions:

Calculate dual tile masks based on partitioned base tiles. (#163)
Calculate corners and edges of dual tile masks. (#163)
Calculate distance fields from corners and edges of dual tile masks. (#163)
Add BaseTilePartition and TilingOffsets fields to Options. Process the base partition with 1. Add a square partition to examples.

Split the Bitmap noise module into a base class, and make a new derived class which holds the distance field data from 1.iii.
Update the example module groups to use distance fields:

Update tile generation and partition code to use clockwise numbering of corners, rather than binary numbering.
Replace the QuadrantSelector module with a CornerSelector module. It has 4 source modules, and returns the value from the one specified by an integer parameter.
Alter example combiner module group to use the new distance field modules instead of NormLPQ and CornerCombinerBase modules.

Alter RasterBase so that build() restricts evaluation of noise modules to the pixels of the dual tile mask.
Add isometric tileset example:

Add rhombic base tile partition and terrain textures, use non-axis-aligned offsets.
Include tiling offsets in metaoutput.
Update Wangview to use offsets specified in metaoutput.
Update Wangview to generate maps which fill the display area.

Support 3- and 6-cornered tiles:

Update all data structures which hold a base or dual tile to hold one or two tiles, and process both if there are two. This includes:

Tile rearrangements.
Source terrain textures.
Output tiles/tilesets.
Image sizes.
Filenames.

Specify in Options how many corners the base tile has.
Update tile rearranger:

Convert a base hexagon to two dual triangles, or two base triangles to a dual hexagon. (The third offset for a hexagon is the difference between the first and second offsets)
Tessellate hexagons and triangle pairs correctly.

Update CornerSelector module from 1.iii.b. to allow 3, 4, or 6 source modules.
Update TilesetGenerator to iterate over the correct number of corner terrains.
Add hexagonal and triangular tileset examples:

Add hexagonal and triangular base tile partitions and terrain textures.
Include tiling offsets in metaoutput.
Update Wangview to draw hex tiles in the correct position (test using a fixed map).
Update Wangview to use both dual triangles in hexagonal tilings.
Update Wangview to generate maps correctly for hexagonal and triangular tilings.


fix: Handle streaming for not chunked request

when body is received in same chunk as the rest of the request we don't handle it well.
this makes streaming agnostic and we are able to handle both chunked and not chunked request

Regression test/tests would be important here.

I would also like to see tests 🙏


Failing checks '422 Unprocessable Entity [{Resource: Field: Code: Message:Body is too long (maximum is 65536 characters)}]'

I am getting failures when people PR code that has too many problems and flake8 hits the maximum character limit. Is there some way I can set a soft stop before it hits the absolute cap and dies? Due to sensitive nature of code cannot share too much debug info.
reviewdog: POST https://api.github.com/repos/[REDACTED]: 422 Unprocessable Entity [{Resource: Field: Code: Message:Body is too long (maximum is 65536 characters)}]
[action-flake8] Clean up reviewdog...
Error: Process completed with exit code 1.

I'm running into the same problem with pyright.

Hi ! I'm running into the same problem with eslint

Running into this with mypy

Running into the same issue with rubocop.
There is an on-going discussion but when it will be actioned is unclear.

I started using reviewdog and now have exact the same problem.
Is there any chance, to upload in chunks or filter the reported errors. In my current setup, i have one error which is reported may times due to having multiple unittests failing with this.

It would be great if this reporter dumped the remaining content into a comment like github-pr-review does.

Hey! I am also facing a similar issue , how you guys implemented the solution to this ? would love to discuss your approaches on this problem. @brianpaden289 @tino @wallforfry @4dahalibut @istisiki @sidey79 @carmanchris31


Bucket manager also dispatches directly.

@vekterli PR
Just review, I will coordinate with #13766

That was my conclusion too, so I think this one should be fine.


Add Interval.md to archive for older versions

I'm hoping that this fixes the Github Action issue! However, if not, I'll need some help to troubleshoot further! It seems to work on my local Jekyll feed...

Maybe @Franz-Kafka can have a look?

This is no longer required, this problem has been fixed by d296d7c104dc51b857bf83bde6a45c6efd3fa9e9


dev -> master PR

PR checklist

[x] This comment contains a description of changes (with reason).
[ ] If you've fixed a bug or added code that should be tested, add tests!

[ ] If you've added a new tool - add to the software_versions process and a regex to scrape_software_versions.py
[ ] If you've added a new tool - have you followed the pipeline conventions in the contribution docs
[ ] If necessary, also make a PR on the nf-core/circrna branch on the nf-core/test-datasets repository.

[x] Make sure your code lints (nf-core lint .).
[x] Ensure the test suite passes (nextflow run . -profile test,docker).
[x] Usage Documentation in docs/usage.md is updated.
[x] Output Documentation in docs/output.md is updated.
[ ] CHANGELOG.md is updated.
[ ] README.md is updated (including new tool citations and authors/contributors).

#26

Hi @BarryDigby  - I think this needs a couple of more things to be able to merge what is on dev to the master branch:

Latest nf-core template (2.5.1 is current now, that should also fix some of the issues with linting)
One round of reviews from @nf-core/core (2 required for making a release on master), probably with some more fixes required but looking at what has been done already not likely to be that much anymore

Hi @apeltzer,
I recently deleted the nf-core template branches when cleaning up my repo, can I generate the latest template branch locally using nf-core?
Best,
Barry

Hi @BarryDigby  - yes you can: https://nf-co.re/developers/sync#manual-synchronisation
Might be a bit more cumbersome than if the TEMPLATE would still be available, but still feasible I think. The merge into dev then takes a bit more work but should still fix all of the issues I think - @nf-core/core anything I've overlooked?

If the pipeline was created using nf-core create in the first place then there should be no need. Just go back to the first commit, where all code is vanilla template without customisation, and make the new branch from there.
git checkout master
git log --reverse
git checkout <first commit hash>
git branch TEMPLATE
git push

Note that if you can resurrect the deleted branch, that's better. Every time you've done a template sync you've solved merge conflicts and doing them cumulatively should be easier than coming from the first template version. However, the approach above ☝🏻 should still be mich easier than a fully manual sync.
If you have local copies you can push them, if you deleted locally you can try git reflog, if you have old PRs you can sometimes resurrect stuff. Git is quite good at keeping things around..

Ah, looks like you have a TEMPLATE branch on your fork, so don't even need to do the git commands above, can just push that branch to the nf-core repo: https://github.com/BarryDigby/circrna/tree/TEMPLATE
There's also this fork which has a bunch of the automated sync commits in. That's what we want ideally, as if those same commits (with commit hashes) are in dev or master then that's the previous resolved PRs there.
If this isn't making sense, ping me on Slack 😁

ok I should have looked around before posting. Just found this closed PR which still had the option to un-delete the automated sync branch. I restored the nf-core-template-merge-2.4 branch and that should basically be identical to TEMPLATE.
In fact I'll just rename it. https://github.com/nf-core/circrna/tree/TEMPLATE 👈🏻 there all done 😅

@nf-core/core maybe ? 😉😬


Add derive4j HKT interfaces to cyclops-react datastructures

Make integration with any libraries that support them (such as HighJ) simpler.
Also in cyclops we could create a more general integration library that maps between HKT witness types and the underlying type it witnesses (e.g. from List.µ to List). Which would allow us to retrofit HigherKindedType support to any library - even those without inner witness types defined on their datastructures (i.e. Functional Java, Javaslang etc), and provide implementations of HKT based Type Classes by plumbing the mappings between unimplemented methods and the appropriate method references (for map / bind / unit etc).

See https://github.com/aol/cyclops/issues/228, moved out derive4j HKT integration until 1.1.0 - to make sure there are no unforeseen impacts of introducing the annotation processor.
Will introduce a less strict 'Higher' and Higher2 type in 1.0.2 which can be used in cyclops-typeclasses to integrate 3rd party libraries such as Functional Java, Javaslang, Reactor & RxJava.

The infrastructure has been put in place in MI1. There seems to be some conflict between derive4j and the Lombok pluin in IntelliJ will log an issue and work with the derive4j team to resolve for MI2.

Closing as non-ascii class names cause issues in AWS.

That's interesting... Do you have a link to a description of the issue with non-ascii class name?

Hey @jbgi - here is the issue logged by one of our Engineers (https://github.com/aol/cyclops-react/issues/641) . The inner class µ doesn't get loaded into the JVM in AWS. I think this is the same bug that Javaslang users experienced with a class named λ.
We've mostly been using Docker so didn't see (it only occurs on bare metal EC2 instances).
If you switch to a different naming convention we can reconsider (also would need to get smooth IntelliJ / lombok integration - I'm not sure which one is causing the need to use clean - this will affect application engineers using cyclops.)


Updated Tantivy version for slop queries

Updated tantivy version for phrase query with slop
Updated the docs

Closes https://github.com/quickwit-oss/quickwit/issues/1716

@guilload is this good to go?

Yes. Thanks, @evanxg852000, for adding examples.


feat(windows) allow custom System.AppUserModel.ID

@hoodie Let me know if there's something you want me to change here.

looking pretty good, thanks for your contribution


Build breaks when UHDM includes change

Files included in UHDM model headers changed in PR chipsalliance/UHDM#635. That change itself shouldn't have had any impact on Surelog whatsoever but still Surelog build broke. Surelog isn't including the files what it uses in both headers and source files. External library changes of this nature shouldn't have an impact (especially, given that UHDM itself compiled and all of its own test cases passed).
What's the approach going forward? Is Surelog meant to depend on public header i.e. uhdm.h or is it expected to cherry-pick individual files that it actually needs. And what about use of Surelog.h within the library implementation itself? Also, headers are including a lot of other files that it could potentially forward declare. Is this by design?
Given the big impact noticed in UHDM (library file size reduced from 3.4 GB to 210 MB, and build times dropped to one-third) merely by optimizing the header includes, I think Surelog deserves a similar update.
@alaindargelas @hzeller Thoughts?

Personally, I like CI telling me I did something wrong. I don't want it to fix it for me.
That said, I think, Surelog needs a pass to fix the include headers - regardless of whether it is part of CI or not. With code already cleaned up it will easier to enable checks on CI.
Are we open to taking a large change just to fix the header includes? I can do that over the weekend.
Heads up - chipsalliance/UHDM#635 currently cannot be merged into Surelog. Build fails with lot of compile errors.

Yes, the CI will only complain, no fix. So we do need first to do the cleanup.
I often create large cleanup changes and @alaindargelas is typically fine with it. In particular if the cleanup is mostly pretty mechanical changes.

The break in Surelog is pretty trivial, it looks like UHDM::VectorOfany definition disappeared from the include file Surelog includes from UHDM

There are tons of changes in that changelist which are not just files moving around, or forward declaration changing. One cannot presume anything from this changelist.
That statement " That change itself shouldn't have had any impact on Surelog whatsoever but still Surelog build broke. " is incorrect.

#2644 Fixes this issue.


SQL compatibility to MySQL 5

Hi,
I'm evaluating on the sqlboiler project, and it's really inspiring to me.
I found sqlboiler generates SQLs not working on MySQL 5, which is still the main database server in the company I serve.
i.e.
-- SQL generated by sqlboiler, not working on MySQL 5
DELETE FROM mytable as t1 WHERE ...

-- compatible SQL would be
DELETE t1 FROM mytable as t1 WHERE ...

See the quote from this page: https://www.mysql.com/support/eol-notice.html

January 9, 2012
MySQL 5.0 covered under Oracle Lifetime Sustaining Support
Per Oracle's Lifetime Support policy, MySQL 5.0 is now covered under Oracle Sustaining Support.
Users are encouraged to upgrade to MySQL 5.5.
December 31, 2009
MySQL 5.0 Active Support ended on December 31, 2009 - Now in Extended Support Phase
Per the MySQL Support Lifecycle policy, active support for MySQL 5.0 ended on December 31, 2009. MySQL 5.0 is now in the Extended support phase.

Unfortunately MySQL 5.0 has been dead for a long time, I highly encourage you to upgrade your version of MySQL if it's still in production use at your company. There have been a LOT of security fixes in this time, and there are probably known security vulnerabilities for this version. This is actually fairly dangerous to continue to run this version of MySQL.
Now that I've said that, let's talk about the issue. The syntax you're saying is "compatible" is a syntax that's still valid today in MySQL 5.7, it's called multi-table delete. This is the same as it was in MySQL 5.0 from what this documentation suggests: http://downloads.mysql.com/docs/refman-5.0-en.a4.pdf
The SQLBoiler generated SQL statement should work looking at this documentation. Can you show the error message you're receiving or provide any more information, since what you're claiming according to the documentation doesn't appear to be true.

Hi,
Sorry for the confusing question.
I'm working with MySQL 5.5, and it simply throws "Syntax error ..."
A work around is modify "buildDeleteQuery":
  113 func buildDeleteQuery(q *Query) (*bytes.Buffer, []interface{}) {
  114     var args []interface{}
  115     buf := strmangle.GetBuffer()
  116
! 117     buf.WriteString("DELETE `a` FROM ")
  118     buf.WriteString(strings.Join(strmangle.IdentQuoteSlice(q.dialect.LQ, q.dialect.RQ, q.from), ", "))

and the sql will be
DELETE a FROM t1 as a ...

Do you think this is the correct way to fix this issue? I'll be happy to submit PRs on MySQL.

Can you turn on debugging mode and show the full query? From what you're telling me it doesn't look like a syntax error for MySQL 5.5 since even 5.0 doesn't have a problem with this syntax.
Instructions are here: https://github.com/vattle/sqlboiler#diagnosing-problems

For errors with other causes, it may be simple to debug yourself by looking at the generated code. Setting boil.DebugMode to true can help with this. You can change the output using boil.DebugWriter (defaults to os.Stdout).

Hi,
Below are some screenshots to run the delete queries against MySQL 5.5.44
The Table:

The Query:

After I update the query from "DELETE FROM ..." to "DELETE a FROM ...",  it works

Your problem is you're mixing syntax. The syntax you're showing that's "broken" is called a multi-table delete
that has incorrect syntax in it.
Multi-Table deletion syntax
Correct syntax is below. But keep in mind that there is NO reason to do this for a single table, please see the Single-Table deletion syntax below.
mysql> DELETE a FROM apps as a where id = 50;
Query OK, 0 rows affected (0.00 sec)

Mistaken syntax:
mysql> DELETE FROM apps AS a WHERE id = 50;
ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'AS a WHERE id = 50' at line 1

Single-Table deletion syntax
To delete records from a single table, you simply do not use the "as" statement.
mysql> DELETE FROM apps WHERE id = 50;
Query OK, 0 rows affected (0.00 sec)

This is not an incompatibility with MySQL and is indeed standards compliant with the SQL Standard as well:
<delete statement: searched> ::=
  DELETE FROM <table name>
    [ WHERE <search condition> ]

I don't believe this is a version problem, as MySQL has had this since 5.0. I highly recommend you try to use the Single-Table Deletion Syntax and remove the "AS" keyword
completely. Then this will work without any problems.

Sound good, and yes, it works to avoid Multi-Table deletion~
Thank you!


Doesn't extract for multiple files using --output flag. Only for first of them.

Step to reproduce:

run one of the commands:

ngx-translate-extract --input ./src --output ./src/i18n/{es,en}.json
ngx-translate-extract --input ./src --output ./src/i18n/es.json ./src/i18n/en.json

Actual result: Changes saved only for first specified locale file
More technical results:
OS: Mac OS
Output:

ng2-translate-extract --dir ./src  -s --output ./src/assets/i18n/{en,es}.json
INFO: Extracting strings from '/src'
OK: * Extracted 11 strings
OK: * Merged with 10 existing strings
OK: * Saved to '/src/assets/i18n/en.json'

Please do npm install @biesbjerg/ngx-translate-extract --save-dev
Looks like you have ng2-translate-extract installed which is an old version.

Cool. Thanks a lot.
I get this instruction from really old topic)

Thanks. I've deprecated the package :-)


wasm: update V8 to v<IP_ADDRESS>.

Signed-off-by: Piotr Sikora<EMAIL_ADDRESS>

cc @moderation

/lgtm deps

/retest

@PiotrSikora we Ok merging this just before the release?  wasm isn't GA and it's not a huge delta so I defer to your preference here (once we have deps approval)

@alyssawilk yeah, I think so.
cc @moderation for deps


docs: Recommend node18

Also, remove unneeded yarn installation as it is included in the SDK

if we are going to recommend node18, we need to add that to our CI as well.

I think we can drop 12.x too?


Adapt the Prometheus Quickstart to the breaking chance in 0.14.1

Adapts the demo to 0.14.1 according to https://keptn.sh/docs/0.14.x/operate/upgrade/#upgrade-from-keptn-0-13-x-to-keptn-0-14-x
CC @stacscribe @agardnerIT @henrikrexed

outdated


Bug in loading of pretrained BERT weights

If I load a pretrained BERT model from HF like this
import torch

from transformers import BertTokenizer, BertModel, BertConfig

tokenizer = BertTokenizer.from_pretrained('google-bert/bert-base-uncased')
config = BertConfig.from_pretrained('google-bert/bert-base-uncased')
model = BertModel.from_pretrained("google-bert/bert-base-uncased", config=config)
model.eval()
model.cuda()

input_ids = torch.randint(0, tokenizer.vocab_size, (1, 512)).cuda()

logits = model(input_ids)['last_hidden_state']

I would expect to get approximately the same result as from a FlashAttn BertModel loaded with the same weights:
from flash_attn.models.bert import BertModel

config.use_flash_attn = True

model = BertModel.from_pretrained('google-bert/bert-base-uncased', config=config)
model.eval()
model.cuda()

with torch.autocast('cuda'):
    logits_fa = model(input_ids)['last_hidden_state']

But torch.allclose gives False;
>>> torch.allclose(logits_fa, logits, atol=1e-4)
False

Upon close inspection, it turns even out that the difference in outputs is non-deterministic with respect to model initializations:
for i in range(10):
    model = BertModel.from_pretrained('google-bert/bert-base-uncased', config=config).cuda()
    model.eval()

    with torch.autocast('cuda'):
        logits_fa = model(input_ids)['last_hidden_state']

    print(float((logits_fa - logits).abs().sum()))

gives
340247.3125
336003.46875
335443.8125
342978.3125
348175.34375
336785.28125
338270.5625
339818.8125
339333.34375
340444.90625

This suggests that some weights are probably not loaded from the pretrained weights and thus randomly initialized.
If I construct and load the pretrained weights manually, as in https://github.com/Dao-AILab/flash-attention/blob/main/flash_attn/models/bert.py#L332 but with strict=False, the problem becomes apparent:
from flash_attn.models.bert import BertModel, BertPreTrainedModel, remap_state_dict
from flash_attn.utils.pretrained import state_dict_from_pretrained

model = BertModel(config=config)

state_dict = remap_state_dict(
    state_dict_from_pretrained('google-bert/bert-base-uncased'),
    config
)

model.load_state_dict(state_dict)

gives
RuntimeError: Error(s) in loading state_dict for BertModel:
	Missing key(s) in state_dict: "embeddings.word_embeddings.weight", "embeddings.position_embeddings.weight", ...
        Unexpected key(s) in state_dict: "bert.embeddings.word_embeddings.weight", "bert.embeddings.position_embeddings.weight", ...

It seems like the loading and mapping function was written for the BertForPreTraining class where the BertModel instance is present as bert attribute. It could therefore be seen as expected behavior. However, since BertModel also inherits from BertPreTrainedModel and provides a from_pretrained() method, I assumed that it can be used in the same way. As strict=False is used for loading weights, this doesn't result in an exception if the weights are not properly mapped and loaded.
I would either expect to have an error raised in this case or have the mapping of the weights handled transparently. The current behavior can easily lead to unexpected behavior that is notoriously difficult to debug.
Update:
Apparently, the problem even occurs when I load the weights to BertForPreTraining:

Hello @kklemon how did you solve this problem?


python36Packages.qtpy: 1.4.2 -> 1.5.2

Semi-automatic update generated by https://github.com/ryantm/nixpkgs-update tools. This update was made based on information from https://repology.org/metapackage/python3.6-qtpy/versions.
meta.description for python36Packages.qtpy is: '"Abstraction layer for PyQt5/PyQt4/PySide2/PySide"'.

Checks done (click to expand)

built on NixOS

0 of 0 passed binary check by having a zero exit code.

0 of 0 passed binary check by having the new version present in output.

found 1.5.2 with grep in /nix/store/sk6rpjclkcqkfa037qwkfa6kligplax1-python3.6-QtPy-1.5.2

directory tree listing: https://gist.github.com/d1fa2ca205254bd3e2fa7535f232cf3f

du listing: https://gist.github.com/ad624724d091750187aaf9af202d57a2

Outpath report (click to expand)

Outpath difference report
19 total rebuild paths
5 package rebuilds
5 x86_64-linux rebuilds
5 i686-linux rebuilds
4 x86_64-darwin rebuilds
5 aarch64-linux rebuilds
First ten rebuilds by attrpath
python27Packages.qtawesome
python27Packages.qtpy
python36Packages.qtawesome
python36Packages.qtpy
spyder

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/sk6rpjclkcqkfa037qwkfa6kligplax1-python3.6-QtPy-1.5.2 \
  --option binary-caches 'https://cache.nixos.org/ https://r-ryantm.cachix.org/' \
  --option trusted-public-keys '
  r-ryantm.cachix.org-1:gkUbLkouDAyvBdpBX0JOdIiD2/DP1ldF3Z3Y6Gqcc4c=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(r-ryantm's Cachix cache is only trusted for this store-path realization.)
Or, build yourself:
nix-build -A python36Packages.qtpy https://github.com/r-ryantm/nixpkgs/archive/75cd9a3b9db3d0a6825ea540cc3588980acc5dda.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/sk6rpjclkcqkfa037qwkfa6kligplax1-python3.6-QtPy-1.5.2
ls -la /nix/store/sk6rpjclkcqkfa037qwkfa6kligplax1-python3.6-QtPy-1.5.2/bin

Success on aarch64-linux (full log)
Attempted: python36Packages.qtpy
Partial log (click to expand)
Installing collected packages: QtPy
Successfully installed QtPy-1.5.2
/build/QtPy-1.5.2
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/0ij5mxvjwc09zss9liyi7mbrdy77dvqy-python3.6-QtPy-1.5.2
strip is /nix/store/p9akxn2sfy4wkhqdqa3li97pc6jaz3r1-binutils-2.30/bin/strip
stripping (with command strip and flags -S) in /nix/store/0ij5mxvjwc09zss9liyi7mbrdy77dvqy-python3.6-QtPy-1.5.2/lib
patching script interpreter paths in /nix/store/0ij5mxvjwc09zss9liyi7mbrdy77dvqy-python3.6-QtPy-1.5.2
checking for references to /build in /nix/store/0ij5mxvjwc09zss9liyi7mbrdy77dvqy-python3.6-QtPy-1.5.2...
/nix/store/0ij5mxvjwc09zss9liyi7mbrdy77dvqy-python3.6-QtPy-1.5.2

Success on x86_64-linux (full log)
Attempted: python36Packages.qtpy
Partial log (click to expand)
Installing collected packages: QtPy
Successfully installed QtPy-1.5.2
/build/QtPy-1.5.2
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/sk6rpjclkcqkfa037qwkfa6kligplax1-python3.6-QtPy-1.5.2
strip is /nix/store/vcc4svb8gy29g4pam2zja6llkbcwsyiq-binutils-2.30/bin/strip
stripping (with command strip and flags -S) in /nix/store/sk6rpjclkcqkfa037qwkfa6kligplax1-python3.6-QtPy-1.5.2/lib
patching script interpreter paths in /nix/store/sk6rpjclkcqkfa037qwkfa6kligplax1-python3.6-QtPy-1.5.2
checking for references to /build in /nix/store/sk6rpjclkcqkfa037qwkfa6kligplax1-python3.6-QtPy-1.5.2...
/nix/store/sk6rpjclkcqkfa037qwkfa6kligplax1-python3.6-QtPy-1.5.2

Success on x86_64-darwin (full log)
Attempted: python36Packages.qtpy
Partial log (click to expand)
/private/tmp/nix-build-python3.6-QtPy-1.5.2.drv-0/QtPy-1.5.2/dist /private/tmp/nix-build-python3.6-QtPy-1.5.2.drv-0/QtPy-1.5.2
Processing ./QtPy-1.5.2-py2.py3-none-any.whl
Installing collected packages: QtPy
Successfully installed QtPy-1.5.2
/private/tmp/nix-build-python3.6-QtPy-1.5.2.drv-0/QtPy-1.5.2
post-installation fixup
strip is /nix/store/g5r4apl0za012ffs6ladinwa5w0m1l3k-cctools-binutils-darwin/bin/strip
stripping (with command strip and flags -S) in /nix/store/m63grqd3ihapndg8v8l0r9s4z6bf6imk-python3.6-QtPy-1.5.2/lib
patching script interpreter paths in /nix/store/m63grqd3ihapndg8v8l0r9s4z6bf6imk-python3.6-QtPy-1.5.2
/nix/store/m63grqd3ihapndg8v8l0r9s4z6bf6imk-python3.6-QtPy-1.5.2


Using continueAsNew in cron workflow should allow the new run runs with the cron schedule

#2753

Not valid any more


Confusing rssi values

I run your latest code (dec 6, 24) on a heltec v3 and tlora v1.6.1  I am observing some rather odd results in the reported rssi values as displayed in the web and syslog data.  For instance, a constant remote station may report -30 as rssi using a stubby antenna that came with the board and is probably not the best choice.  Connecting a better antenna (5.8dbi gain) results in a rssi value -129. Per definition of rssi, a worse signal.
according to this article https://forum.arduino.cc/t/negative-rssi-issue-with-esp32-and-lora-ra-02-sx1278-module/1248199/9 the real rssi value should be calculated for the lf port like
Rssi(dbm) = -164 + (Lora rssi)
so in my case that would add with the stubby antenna being -164 + 30 =-134  dbm mor the better antenna -164 + 129 =-35 dbm
Any thoughts

That report references the sx1278 chip and the heltec uses sx1262. Assuming it may be afflicted by the same assumption

Also looking at the code include files, the rssi value is defined as "int" and could not be negative. Whereas the snr is defined as float and supports negative values. If these definitions are from Lora designs then this would suggest those formulas above being meaningful

Thanks for pointing this out
I will try to validate all of this real soon (29 dec as I return home)


Improve Nomad integration and non-persistent behavior

Motivation was starting to work with Nomad batch jobs and hitting a
couple of problems that made them not so useful, especially:

Not exit code propagation, so batch jobs were also marked as
successful.
Once exit code propagation worked, unnecessary delays due to
the existing start-up procedure.
Automatic restarts causing all sorts of problems (life-time
of pots would overlap and cause issues).

This commit corrects pot itself, another review will be opened
to adapt the nomad-pot-driver making use of these new features.
PR for nomad-pot-driver changes: trivago/nomad-pot-driver#30
Changes in detail:

start sleep command will now be killed as soon as the actual
jail command was started, reducing the minimal jail life-time
(sleep 1234 taken to have a good pkill target).
once the jail command finishes, its exit_code is written to
$confdir/$pname/conf/.last-run-stats in JSON format.
This could be extended to be more useful in the future.
In case of non-persistent jails, pot start returns exit code
125 if the jail command returned an exit code != 0.
The content of .last-run-state can be retrieved using the
new command pot last-run-state -p potname.
A new flag "-g" (for grace period) was added to pot prune.
If set, pots will be pruned only if their .last-run-state was
written more than a minute ago. This gives processes the
time to call pot last-run-stats.
Non-persistent jails will also be stopped after the jail command
returned (will not keep jails alive accidentally like it was
happening before by left behind processes or by someone
poking into the jail).
While there, fixed paths executing prestart and poststart
scripts.

Thanks for the submit, I'll give it a thorough look.
In the meanwhile, could you please also provide the needed modification in the CHANGELOG?

I added an extra sleep, as - due to optimizing the startup process - hitting an epair race is more likely (basically, adding the same sleep that is used in pot-stop). FreeBSD 13.1 should - hopefully - eliminate that race.
For the records, this is what the epair race looks like in reality:
Unread portion of the kernel message buffer:
<6>epair7b: link state changed to DOWN
<6>epair7a: link state changed to DOWN

Fatal trap 9: general protection fault while in kernel mode
cpuid = 6; apic id = 06
instruction pointer     = 0x20:0xffffffff8298512d
stack pointer           = 0x28:0xfffffe02127495d8
frame pointer           = 0x28:0xfffffe0212749600
code segment            = base 0x0, limit 0xfffff, type 0x1b
                        = DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags        = interrupt enabled, resume, IOPL = 0
current process         = 85669 (ifconfig)
trap number             = 9
panic: general protection fault
cpuid = 6
time =<PHONE_NUMBER>
KDB: stack backtrace:
#0 0xffffffff80c40295 at kdb_backtrace+0x65
#1 0xffffffff80bf5d91 at vpanic+0x181
#2 0xffffffff80bf5b63 at panic+0x43
#3 0xffffffff810878f7 at trap_fatal+0x387
#4 0xffffffff81086d84 at trap+0xa4
#5 0xffffffff8105b808 at calltrap+0x8
#6 0xffffffff80d0709e at if_detach_internal+0x12e
#7 0xffffffff80d0d2ac at if_vmove+0x3c
#8 0xffffffff80d09fce at ifhwioctl+0x111e
#9 0xffffffff80d0afdf at ifioctl+0x37f
#10 0xffffffff80c61819 at kern_ioctl+0x289
#11 0xffffffff80c614da at sys_ioctl+0x12a
#12 0xffffffff8108827e at amd64_syscall+0x12e
#13 0xffffffff8105c12e at fast_syscall_common+0xf8
Uptime: 31d22h50m36s
Dumping 7523 out of 65271 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91%

__curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:55
55              __asm("movq %%gs:%P1,%0" : "=r" (td) : "n" (offsetof(struct pcpu,
(kgdb) #0  __curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:55
#1  doadump (textdump=<optimized out>) at /usr/src/sys/kern/kern_shutdown.c:399
#2  0xffffffff80bf59bb in kern_reboot (howto=260)
    at /usr/src/sys/kern/kern_shutdown.c:486
#3  0xffffffff80bf5e00 in vpanic (fmt=<optimized out>, ap=<optimized out>)
    at /usr/src/sys/kern/kern_shutdown.c:919
#4  0xffffffff80bf5b63 in panic (fmt=<unavailable>)
    at /usr/src/sys/kern/kern_shutdown.c:843
#5  0xffffffff810878f7 in trap_fatal (frame=0xfffffe0212749510, eva=0)
    at /usr/src/sys/amd64/amd64/trap.c:915
#6  0xffffffff81086d84 in trap (frame=0xfffffe0212749510)
    at /usr/src/sys/amd64/amd64/trap.c:212
#7  <signal handler called>
#8  0xffffffff8298512d in epair_qflush (ifp=<optimized out>)


docs(upgrade): restrict component directive to elements

The heroDetail directive is shown as an example of a component directive, such as those built using the .component() helper in v1.5+. In order for the directive to be functionaly equivalent with the component directive created with .component, it needs to be restricted to elements only. (If the restrict is omitted, it defaults to 'AE', which would also match attributes.)
/cc @teropa

From here I can see they are restricted to E by default.

Components are restricted to 'E'. This PR is about changing a directive (created via .directive()) - by default restricted to 'EA'.

Merged. No harm in being clear. Thanks.


Use ngx.timer.every() for the AR cache update

The lua code has been using ngx.timer.at() for the AR cache update. This can lead to the problem described in the JIRA ticket below. This PR replaces ngx.timer.at() with a more robust alternative of  ngx.timer.every(), which also fixes the problem described in the JIRA ticket below.
Corresponding DC/OS tickets (obligatory)
These DC/OS JIRA ticket(s) must be updated (ideally closed) in the moment this PR lands:
https://jira.mesosphere.com/browse/DCOS-38248
Checklist for all PRs

[ ] Added a comprehensible changelog entry to CHANGES.md or explain why this is not a user-facing change:
[ ] Included a test which will fail if code is reverted but test is not. If there is no test please explain here:
[x] Read the DC/OS contributing guidelines
[x] Followed relevant code rules Rules for Packages and Systemd

Checklist for component/package updates:
If you are changing components or packages in DC/OS (e.g. you are bumping the sha or ref of anything underneath packages), then in addition to the above please also include:

[ ] Change log from the last version integrated (this should be a link to commits for easy verification and review): example
[ ] Test Results: [link to CI job test results for component]
[ ] Code Coverage (if available): [link to code coverage report]

PLEASE FILL IN THE TEMPLATE ABOVE / DO NOT REMOVE ANY SECTIONS ABOVE THIS LINE
Instructions and review process
What is the review process and when will my changes land?
All PRs require 2 approvals using GitHub's pull request reviews.
Reviewers should be:

Developers who understand the code being modified.
Developers responsible for code that interacts with or depends on the code being modified.

It is best to proactively ask for 2 reviews by @mentioning the candidate reviewers in the PR comments area. The responsibility is on the developer submitting the PR to follow-up with reviewers and make sure a PR is reviewed in a timely manner. Once a PR has 2 ship-it's, no red reviews, and all tests are green it will be included in the next train.

This repo has @mesosphere-mergebot integration. You can interact with the following commands.
@mesosphere-mergebot bump-ee
@mesosphere-mergebot merge-it
@mesosphere-mergebot label [Request For Comment |Work In Progress |Holding |Ship It |Ready For Review]
@mesosphere-mergebot override-status

PR creators can apply one of  [Ready For Review |Work In Progress]. Owners can apply any label.

@mesosphere-mergebot bump-ee

@mesosphere-mergebot label Ready For Review

@vespian @gpaul @jgehrcke please review this PR

:wave: This PR has been inactive for 3 days. Please review the status checks to see what needs to be done to move it forward or change the label to Work in Progress. Thank you!

:wave: This PR has been inactive for 3 days. Please review the status checks to see what needs to be done to move it forward or change the label to Work in Progress. Thank you!

In https://github.com/dcos/dcos/pull/2989#discussion_r199490006 I have mentioned a race condition and I also mentioned that we have code in place that takes care of handling it. This is the code I was referring to:
    -- Handle the special case of very early request - before the first
    -- timer-based cache refresh
    local entry_last_refresh = cache:get(name_last_refresh)
    if entry_last_refresh == nil then
        refresh_cache(false, auth_token)

        entry_last_refresh = cache:get(name .. "_last_refresh")
        if entry_last_refresh == nil then
            -- Something is really broken, abort!
            ngx.log(ngx.ERR, "Could not retrieve last refresh time for `" .. name .. "` cache entry")
            return nil
        end
    end

Assuming that timer.every() only calls the callback after the initial period has passed (and not immediately and then every period) we actually make sure that this piece of code is regularly exercised: it is exercised for all HTTP requests that

arrive at Admin Router after worker process initialization and before the first timer-triggered cache refresh
require a populated cache

For instance, I expect both to be true for our integration test suite.
It is important that we keep exercising this code path in our various test approaches.
When this part is broken / does not work as expected then we will run into issues like the following: https://jira.mesosphere.com/browse/DCOS-5916 (very hard to debug).
Please also see https://github.com/mesosphere/adminrouter/pull/59 for reference how I once stress-tested the startup phase of Admin Router.

Thanks for the useful info, @jgehrcke ! I will also repeat the test https://github.com/mesosphere/adminrouter/pull/59

@mesosphere-mergebot bump-ee

sdjlfadjflsajf

After all, let's please have the behavior change and the test change in two different commits, and the test change commit message should briefly explain why it was OK to change the test.

After all, let's please have the behavior change and the test change in two different commits, and the test change commit message should briefly explain why it was OK to change the test.

We paired with @ahairape on this for 1.5 hours. The commit message that you pointed out is just a quick dump of our work and @ahairape is going to polish it soon I think.

yes, I'm going to split the commit above into two and explain the changes

@mesosphere-mergebot bump-ee

@mesosphere-mergebot label Work In Progress

After discussing it with @vespian we came to a conclusion that it is simpler to keep close to the the initial business logic of the code and keep the initial delay of 2s (CACHE_FIRST_POLL_DELAY) intact. The first cache update with ngx.timer.every() is then triggered every CACHE_POLL_PERIOD (25s) after the "AR start". This is different from the previous version where the first recursive cache update was triggered CACHE_FIRST_POLL_DELAY + CACHE_POLL_PERIOD = 27s  after the AR start. The modification affected the test test_cache.py, that's why the tests there have been changed as well (the timeouts adjusted) to reflect the modified logic.

@mesosphere-mergebot label Ready For Review

@mesosphere-mergebot bump-ee

Didn't we decide not to have an initial update? @jgehrcke

Yes.

Also, rewriting the tests to match new logic would take some time extra time

This is a bit sad :-). Because otherwise we actually agreed that it's technically meaningful to remove the initial update after 2 seconds. To quote from above:

To conclude the discussion: it looks like we can totally remove the CACHE_FIRST_POLL_DELAY (as done by the current version of the patch)

Let's also have the report from the manual test from https://github.com/mesosphere/adminrouter/pull/59, please.

@gpaul I had a quick discussion with @vespian and now I think we should keep the "Initial timer-triggered cache update early after nginx startup:" we had indeed one argument not stressed enough in this current discussion about the purpose of this early update: because the cache update might actually take long we want to minimize the likelihood for an incoming HTTP request to be slowed down (we talk of a slowdown of potentially multiple seconds).
I suggest adding a comment to the code, like this:
Initial timer-triggered cache update early after nginx startup:

Triggering a cache update roughly 2 seconds after the
startup of the worker process is a trade-off between the
following two aspects:

- It makes sense to have this initial timer-triggered cache update _early_ after
  nginx startup at all, and it makes sense to make it very early, so that we
  reduce the likelihood for an HTTP request to be slowed down when it is
  incoming _before_ the normally scheduled periodic cache update (example: the
  HTTP request comes in 15 seconds after nginx startup, and the first regular
  timer-triggered cache update is triggered only 25 seconds after nginx
  startup).

- It makes sense to have this time window not be too narrow, especially not
  close to 0 seconds: under a lot of load there *will* be HTTP requests incoming
  before the initial timer-triggered update, even if the first timer callback is
  scheduled to be executed after 0 seconds. There is code in place for handling
  these HTTP requests, and that code path must be kept explicit, regularly
  exercised, and well-tested. That is easier when the initial update is
  scheduled after e.g. 2 seconds, as opposed to 0 seconds.

@mesosphere-mergebot bump-ee

Thanks for performing the manual test.

@mhrabovcin @gpaul can you help pushing this across the finish line? Goal is to land this by EOD.

Someone needs to investigate the one failed CI check.

teamcity/dcos/build/dcos/windows is broken due to some dcos-checks-runner on windows issue:
https://teamcity.mesosphere.io/viewLog.html?buildId=1121238&buildTypeId=DcOs_Windows_Build_BuildDcOs
@branden is this something you can fix? @vespian do you know who is responsible for the "DC/OS on Windows" effort? If building on windows is not REQUIRED then it is a matter of time until it breaks.

test_service_discovery.test_service_discovery_mesos_host failed in teamcity/dcos/test/aws/cloudformation/simple and it appears to be a real error, or at least an extremely rare flake.

The test failure is a known issue https://jira.mesosphere.com/browse/DCOS_OSS-3539 which has already been fixed on master by https://github.com/dcos/dcos/pull/2987.

@mesosphere-mergebot override-status teamcity/dcos/test/aws/cloudformation/simple https://jira.mesosphere.com/browse/DCOS_OSS-3539

@vespian do you know who is responsible for the "DC/OS on Windows" effort?

@branden?

@mesosphere-mergebot override-status teamcity/dcos/test/aws/cloudformation/simple
https://jira.mesosphere.com/browse/DCOS-22377

@mesosphere-mergebot override-status teamcity/dcos/test/aws/cloudformation/simple
https://jira.mesosphere.com/browse/DCOS-22377

@mesosphere-mergebot override-status teamcity/dcos/build/dcos/windows
https://jira.mesosphere.com/browse/DCOS-39083

@mesosphere-mergebot bump-ee

@mesosphere-mergebot bump-ee

Closing this PR in favour of https://github.com/dcos/dcos/pull/3064


Can't invoke MIS

Hello
we want to try MIS.
But when we invoke MIS with curl, it ends up kubernetes crashed.
It seems monai-pod have issues with writing files to volume.
minikube version: v1.28.0
commit: 986b1ebd987211ed16f8cc10aed7d2c42fc8392f
WARNING: This version information is deprecated and will be replaced with the output from kubectl version --short.  Use --output=yaml|json to get the full version.
Client Version: version.Info{Major:"1", Minor:"25", GitVersion:"v1.25.4", GitCommit:"872a965c6c6526caa949f0c6ac028ef7aff3fb78", GitTreeState:"clean", BuildDate:"2022-11-09T13:36:36Z", GoVersion:"go1.19.3", Compiler:"gc", Platform:"linux/amd64"}
Kustomize Version: v4.5.7
Server Version: version.Info{Major:"1", Minor:"25", GitVersion:"v1.25.3", GitCommit:"434bfd82814af038ad94d62ebe59b133fcb50506", GitTreeState:"clean", BuildDate:"2022-10-12T10:49:09Z", GoVersion:"go1.19.2", Compiler:"gc", Platform:"linux/amd64"}
version.BuildInfo{Version:"v3.10.2", GitCommit:"50f003e5ee8704ec937a756c646870227d7c8b58", GitTreeState:"clean", GoVersion:"go1.18.8"}

Hi @batbilguun44 ! Thanks for the issue. MIS has been deprecated and not recommended to run, test or validate MAPs.
Instead, we recommend you using https://github.com/Project-MONAI/monai-deploy/tree/main/deploy/monai-deploy-express
But it depends on your use case. If you can explain a bit more, we will point you in the right direction.
Thanks!


vkEnumerateInstanceExtensionProperties crashes in Nvidia driver when LunarG SDK installed

Since I installed the <IP_ADDRESS> version of the LunarG SDK with the validation layers I am experiencing a bug which makes calling vkEnumerateInstanceExtensionProperties() with a non-null pProperties crash in Nvidia's nvoglv64.dll driver DLL. The error in the <IP_ADDRESS> SDK occurred only in around 1 out of 10 runs of the program.
Now that I installed the <IP_ADDRESS> SDK, the error occurs almost always, so now it just does not occur in 1 out of 10 runs.
The symptoms are that either nvoglv64.dll hangs somewhere within the vkEnumerateInstanceExtensionProperties() (checked with attaching a native platform debugger, suspending the threads and seeing where they resided) or - what happens most of the time - that it crashes the process with an access violation.
The issue is very undeterministic and only happens when the LunarG SDK is installed. I have tried about 100 runs of 20 different Vulkan programs which all ran fine when the LunarG SDK was uninstalled.
There seems to be some really nasty concurrency problem there, I guess. Also, the issue does not exist when running the native program under a native platform debugger from the start and stepping through the code instruction by instruction to find the offending one.
I checked that it was indeed vkEnumerateInstanceExtensionProperties() crashing by having fprintf/fflush before and after each Vulkan function call.
The very first Vulkan functions being called that lead to the crash described are:

vkEnumerateInstanceExtensionProperties(NULL, &count, NULL)
vkEnumerateInstanceExtensionProperties(NULL, &count, ep)

The second call crashes. No other Vulkan calls were being made before. Just LoadLibrary("vulkan-1.dll") and GetProcAddress("vkGetInstanceProcAddr")
System specifications:

OS: Windows 7 x64 SP1 (all recent updates installed)
GPU: Nvidia Quadro K2000M
Driver: Nvidia 364.72 WHQL
LunarG SDK: <IP_ADDRESS>

This issue also makes any GLFW application not working, because they call vkEnumerateInstanceExtensionProperties to detect the instance extensions like VK_KHR_win32_surface and VK_KHR_surface to know whether Vulkan is enabled.

Any chance the memory pointed to by "ep" is not large enough? How is it allocated?

Sadly no. I am exactly using GLFW from its current head.
See: https://github.com/glfw/glfw/blob/master/src/vulkan.c#L81

I wasn't able to reproduce this locally with SDK <IP_ADDRESS> installed and the "examples" that glfw builds like "wave" and "particles". Are there any extra repro steps I should know about.
Since you mention it's related to the SDK, can you try uninstalled all SDKs and Vulkan Run Time Libraries from the system and reinstall <IP_ADDRESS> and the runtime library. After doing the uninstall, make sure that HKEY_LOCAL_MACHINE\SOFTWARE\Khronos\Vulkan\ExplicitLayers and HKEY_LOCAL_MACHINE\SOFTWARE\Khronos\Vulkan\ImplicitLayers are empty and also there are no VulkanSDK or VulkanRT entries in HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall. If there are, you'll need to rerun the "UninstallString" for each entry until their "InstallCount" goes to 0 and the entry disappears.
Perhaps there is some layer that is modifying the parameters so that when they finally get to the driver there is something wrong with them.

Thanks for taking care of it!
I have created the following simple C demo to reproduce. The error happens about 95% of the program run in non-debug mode:
#include <Windows.h>
#include <stdio.h>
#include "vulkan.h"
int main(int argc, char** argv) {
	HMODULE mod = LoadLibraryA("vulkan-1.dll");
	PFN_vkGetInstanceProcAddr vkGetInstanceProcAddr = (PFN_vkGetInstanceProcAddr) GetProcAddress(mod, "vkGetInstanceProcAddr");
	PFN_vkEnumerateInstanceExtensionProperties vkEnumerateInstanceExtensionProperties = (PFN_vkEnumerateInstanceExtensionProperties) vkGetInstanceProcAddr(NULL, "vkEnumerateInstanceExtensionProperties");
	uint32_t count;
	VkResult result = vkEnumerateInstanceExtensionProperties(NULL, &count, NULL);
	VkExtensionProperties* props;
	props = (VkExtensionProperties*) calloc(count, sizeof(VkExtensionProperties));
	fprintf(stderr, "%s\n", "Before..."); fflush(stderr);
	result = vkEnumerateInstanceExtensionProperties(NULL, &count, props); // <- crashes right here!
	fprintf(stderr, "%s\n", "After..."); fflush(stderr);
	return 0;
}

Okay, I uninstalled the Vulkan SDK <IP_ADDRESS> and also the Vulkan Runtime Library <IP_ADDRESS> and Vulkan Runtiem Library <IP_ADDRESS>, which were present as installed programs.
But now that also uninstalled the vulkan-1.dll. So I cannot run any Vulkan programs anymore.
Aren't the graphics card drivers supposed to provide that dll?

I just uninstalled everything. All Vulkan Runtime Libraries and the SDK and also the graphics driver and reinstalled only the graphics driver. That fixed vulkan-1.dll being around again.
Now, all Vulkan apps work just fine (that is without having the SDK installed).
I will now reinstall the SDK.

Thanks again! After manually deleting the registry keys and reinstalling the <IP_ADDRESS> SDK, everything seems to work fine now. Also actually enabling layers works now.
Sadly I did not remember what the 11 values in the HKEY_LOCAL_MACHINE\SOFTWARE\Khronos\Vulkan\ExplicitLayers and the 1 value in HKEY_LOCAL_MACHINE\SOFTWARE\Khronos\Vulkan\ImplicitLayers contained exactly.
Now, HKEY_LOCAL_MACHINE\SOFTWARE\Khronos\Vulkan\ExplicitLayers contains 11 values such as "C:\VulkanSDK\<IP_ADDRESS>\Bin\VkLayer_image.json".
For me, this issue is now solved.

Awesome, I'm glad the issue has been resolved. There may be a bug with the SDK uninstall or overinstall not removing old incompatible layers. If you see it again see if you can file an issue against the Vulkan-LoaderAndTools repo.

@pdaniell-nv @httpdigest
Sorry for reopening an old thread, but I seem to be having the same issue.
Excuse me if I sound like a doob, but I'm not entirely sure what HKEY_LOCAL_MACHINE\SOFTWARE\Khronos\Vulkan\ExplicitLayers and HKEY_LOCAL_MACHINE\SOFTWARE\Khronos\Vulkan\ImplicitLayers refer to. Are they file paths? Because I can't cd to it.
Thanks.

@romanlarionov, those paths are referring to locations in the Windows Registry.  If you open RegEdit, you should be able to find them. If you're having a similar problem, you should try running the VIA application that comes in the new LunarG SDK and it will likely give you much of the information you're interested in since it extracts this information for you.

Thanks for the help @MarkMLunarG. Sorry for the late response. Finals took priority.
I tried what you suggested but I had an issue with the new 1.0.33 LunarG SDK. It only worked after I installed the SDK with administrative privileges.
You can see an ongoing log on this issue: https://vulkan.lunarg.com/issue/view/584544cb9ab0fa4b673614b2


error: WebSocket was closed before the connection was established

I am cretaing a function to send the wscat to background and redirecting a FIFO file (mkfifo ppp) to drop the data to send by the streaming.
**sendToBackground**() {
wscat -c wss://example.url -w 86400 < **ppp**
sleep 20
}

sendToBackgound &

echo -n '{"method": "login", "params": {"algo": "BASIC","pKey": "ABCD","sKey": "ABCD"}}' > **ppp**

After execute it, I get the next error:
error: WebSocket was closed before the connection was established
What can I do to drop data to sedn by the wss ?
Thanks.

Did you manage to figure out a solution or workaround?

Did you manage to figure out a solution or workaround?

I've seen this happen with other programs, such as when trying to pipe cat and grep in a shell script, and the cause is typically that the program is expecting an attached TTY and an stdin channel, when actually these are not available, causing an immediately closed pipe, and both sides to close abruptly.
It's trivial to reproduce this issue with Docker, because the docker run command can be given -t (attach TTY), -i (stdin), or none of them. With none of them, wscat prints the "error: WebSocket was closed before the connection was established" message and it doesn't work.
Related:

https://github.com/websockets/wscat/issues/118

https://github.com/websockets/wscat/issues/130

wscat commands from script

[wscat] isn't written to follow conventions that make it suitable for scripted use (such as keeping prompts on stderr rather than stdout; or suppressing prompts when output is not to a TTY; or treating an EOF as a signal to close a connection). Consider websocat instead.

This tool simply seems exclusively designed with interactive use in mind. Which is not bad at all, especially given the money users paid for it :-) But given that a common usage pattern for CLI tools is to end up scripting them, I guess the minimum that could be done is warning in the README about not to do so with this one...


Implement net/http.Handler for local development

This commit modifies the lmdrouter.Router type to implement the
net/http.Handler interface (meaning the ServeHTTP method is now
available on *lmdrouter.Route objects).
This allows usage of applications using lmdrouter in environments other
than AWS Lambda, but is mostly useful for local development purposes.
It should be noted that this means applications will now have to use the
netgo build tag (go build -tags netgo) to make sure binaries are
still statically compiled.
Resolves: 1

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.Ido Perlmuter seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.


Integrate ng2-bootstrap module with ALMighty UI

ng2-bootstrap module consists of native Angular2 components and directives, no jQuery or Bootstrap javascript

It has been added.


Example app extremely slow in windows

I follow the example app steps, and when I load the /hello route, the request takes 4 seconds. I don't see any errors or fails
Env:
Windows 10
Julia: v1.0.0

Thanks - every time? Or the first time?

Every time.

I tried in another computer, and takes 500ms. Better but to much anyway. I tested Mux to discard a julia problem and it seems correctly.

@DavidBernal Thanks for the update - can you please share how you benchmarked so I can replicate?

There doesn't seem to be any problem with Genie itself - the response time on my mac is 79ms. I'll try it now in a VM with Win 10, but that will affect the performance as the VM is considerably slower. So I'm not sure how relevant will that be, but let's see.

Yes, very bad on Win 10 in my VM - around 5s! :-O

Ah no actually, false alarm. Turned out to be the VM was using up all the resources to perform a windows update. After the update, the response time is around 650ms. So yes, that's a few times slower compared to the mac. I'll see what the profiler has to say.

I'm out until friday so I can't do test now. Anyway, 650ms for a text in local network is a lot.

Yes, something's off on Windows. HTTP.jl responds fast, so it's something else in the stack which is fast on Mac but slow on Win.

@DavidBernal Closing this as the performance has improved massively due to latest tweaks, 16ms now:

Nice, I will try again.


update certs package

rm mage, replace with make
update gRPC dependencies top v1.65.0

Pull Request Test Coverage Report for Build<PHONE_NUMBER>6
Details

0 of 17 (0.0%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 0.0%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

certs.go
0
17
0.0%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
0

Relevant Lines:
246

💛  - Coveralls


avoid requeuing not found pods

Issue #, if available:
#128
Description of changes:
If the pod is not found, the call returns k8s API errors NotFound. If the job doesn't find the pod, worker will forget the item. If the pod has update, cache client will fetch the event and create another job for queuing.
Test result
{"level":"error","timestamp":"2022-10-05T00:52:26.642Z","logger":"vpc.amazonaws.com/pod-eni-worker","msg":"won't requeue a not found errored job","job":{"Operation":"Create","UID":"","PodName":"hello-world-5d97946fc7-8sgvz","PodNamespace":"default","RequestCount":1,"NodeName":""},"job":{"Operation":"Create","UID":"","PodName":"hello-world-5d97946fc7-8sgvz","PodNamespace":"default","RequestCount":1,"NodeName":""},"error":" \"default/hello-world-5d97946fc7-8sgvz\" not found","stacktrace":"github.com/aws/amazon-vpc-resource-controller-k8s/pkg/worker.(*worker).processNextItem\n\t/workspace/pkg/worker/worker.go:164\ngithub.com/aws/amazon-vpc-resource-controller-k8s/pkg/worker.(*worker).runWorker\n\t/workspace/pkg/worker/worker.go:141"}

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Change LGTM, but just so I understand, if the job was not found, we previously requeued the job and it just spun indefinitely?

yes,,in some cases, the pod was queued without DELETE event. Since our old logic only checks if the node object is cached and not check if the node is deleted and removed from cache, we think the node is not ready yet and indefinitely requeue the pod to wait the node being ready.


Thanks for the project + question

Hi,
Thanks for the awesome project, really neat.
I picked it up from a this Talos blogpost , but when searched for it the result was two repos with identical name and almost same functionalities baked in:

obviously this repo
and this one reverse-ssh

So... I suppose it's a coincidence, but still the projects are very similar and I got confused.
Cheers,
PY

Ah interesting, always interesting to see almost your project be used for malware. Thanks for sharing this!
It would be nice to differentiate the two projects, just happens to be luck of the draw. I started this a few months before the other project haha.