Jp1/dmverityboot

NOTE: CI is failing on all branches with the same error, including main, see the latest run on main here.
The effect of this PR is to reduce SNP mode boot time from ~20 seconds to ~4 seconds.
This PR also adds linux kernel 6.1 support (the changes to pkg/amdsevsnp/report.go ripped from PR1886) which supersedes PR-1886.
I'm happy to talk through these changes if it would be useful.
Summary:

Overall moving SNP mode from booting an initramfs (initrd) in memory, to booting only the kernel + kernel commandline (e.g. the .vmgs file) followed by the kernel directly mounting a DM-Verity protected rootfilesystem on disk rather than in memory. Because the PSP now only needs to measure and attest to the smaller vmgs file, this reduces boot time from ~20 seconds to ~4 seconds.
Adds optional debug builds to the Makefile.
Adds annotations for dmverity boot options and an annotation to add debug ports on debug builds.
Adds PR-1886 linux 6.1 report fetching.

To help reviewers, here's a more detailed summary of the changes :

The new boot directory contains the SNP mode startup script (startup.sh) and two debugging startup scripts for the debug builds.
blob2vhd.go is a tool to convert devices to VHDs.
debuginit.c is an debug init used in the debug builds.
changes to both annotations.go and uvm.go are about plumbing new annotations through.
In create_lcow.go:

in SNP/DM-Verity mode we add two scsi devices, the dmverity protected rootfs and another device containing the dm-verity merkle tree (hashes of all the data in the rootfs).
The rest of the changes to create_lcow.go are plumbing annotations through.

Changes to report.go are independent, see PR-1886 Testing this PR required those changes so they are here. (If preferred to keep a cleaner history, reviewers can merge that PR quickly now and then those changes don't need to be present here.)

Can you please update the PR title to a short description of the changes?
We should also either rebase the commits into a logical set, or ensure we squash commits when merging.

also seems like some of the changes are from the PR that has been merged. Can you rebase instead?


Skip configuration for certain paths only

Is your feature request related to a problem? Please describe.
I'm a bit annoyed by the B101 assert_used warnings in my testing code, as pytest recommends (and I follow) using assert statements.  It's confusing that bandit complains about them.  On the other hand, I want to be warned about using assert in my non-test code.
Describe the solution you'd like
I'd like to be able to ignore pattern B101 for the /tests folder only.

sounds rationale to me @andreas-h , is this a patch you would like to work on?

Alternatively, can we make Bandit a bit smarter in a way that it recognizes when assert is used in a test module, and not warn about it?

See also #346 - B101 in tests is a big pain.

Hi, the following may be helpful to configure bandit to avoid raising B101 assert_used warnings on python tests.

https://github.com/PyCQA/bandit/issues/603#issuecomment-971057519

I am facing the same issue, is there a way to exclude B101 in the /tests folder? I tried to use the assert_used but this does not seem to work.


train_model.py plots fail with RuntimeError: main thread is not in main loop

Occasionally when running models back-to-back I will get the below error at the end of train_model.py when the  plotcomp_n_getiou fn is called
Exception ignored in: <function Image.__del__ at 0x7f096d0b74c0>
Traceback (most recent call last):
  File "/usr/lib/python3.8/tkinter/__init__.py", line 4017, in __del__
    self.tk.call('image', 'delete', self.name)
RuntimeError: main thread is not in main loop

The result is that not all plots are rendered and saved
this seems to be a matplotlib issue.. i.e.
https://stackoverflow.com/questions/27147300/matplotlib-tcl-asyncdelete-async-handler-deleted-by-the-wrong-thread

src\imports.py now uses
import matplotlib
matplotlib.use('Agg')
from matplotlib import pyplot

instead
 import matplotlib.pyplot as plt

I also added K.clear_session() at the bottom of the counter for-loop in plotcomp_n_getiou, which should help
you'll see changes in next commit

works!


Case resource should be downloadable

The content as XML containing CMMN model should be downloadable from the Case Definition view.

Is part of cockpit.


Suspected slowdown of code.

Need to investigate
Reported by mtwomey in post

Yes sir, that fixes it for me as well. I do notice, however, that it now takes significantly longer to return from GetVJDStatus (used to be instant, now takes 1-2 seconds the first time I call it). That may be unrelated and just something to do with new features / items in 2.7 - but just thought I'd mention it.

Could not reproduce.
Used profiler for x86/x64


Fix TA tests build after service update and cleanup code

Service now returns exception when more than 5 documents are sent. Before we had to "hack" it in the SDK, so cleaning up that code.
Fixes: https://github.com/Azure/azure-sdk-for-net/issues/11946

That cleanup surfaced that we needed to implement ClientDiagnostics.ExtractFailureContent in order to get the ErroCode and Message into RequestFailedException.

The service updated the model version for their endpoints and with it we starting getting different results in the Detect Language endpoint. While This in English used to return English as the language, now we get Portuguese or Hungarian.
The service has been notified, so in this PR I am making the single document test validate the returned type has content on it and for batch, I am setting the model version to 2019-10-01 which we know returns the languages we expect and are not going to change.
Fixes: https://github.com/Azure/azure-sdk-for-net/issues/13947

/azp run net - textanalytics - tests

/azp run net - textanalytics - tests

/azp run net - textanalytics - tests


build: set GitSHA version in cmd directory

Fix https://github.com/coreos/etcd/issues/5255.

/cc @heyitsanthony @xiang90
I am not sure if this is the right way. Since we use symlinks, we need to set the GitSHA variable inside our own vendor directory. Otherwise, it ignores the value from the flag.

lgtm


Examples link in Homepage leads to nowhere

https://github.com/danielrearden/sqlmancer/src/__tests__

Thanks! Fixed.


Create a simple file uploader in Meltano UI

Migrated from GitLab: https://gitlab.com/meltano/meltano/-/issues/1329
Originally created by @dmor on 2019-11-27 15:52:11

This will:

Allow us to serve customers who have files (like CSV of API export from Shopify)
Allow users to give us repeating files (like a daily report)
Set credentials files that we might need (like secrets)
Side benefit: Ability to upload custom transforms and models
Side benefit: Makes all our tutorials work with DigitalOcean

Implementation concerns:

Let's try to use a plugin to do this, not build from scratch
Doesn't have to be pretty
Recommend not restricting the file types (until we know the full list we want to support)
It is totally ok to make this hacky, we are the ones who will use this
Dummy route like /upload with a file selector, no UI beyond a single field
We are knowingly making this less secure, since it is bounded to the single user's instance

Documentation:

Don't document on the public website, this is an internal tool for now and we can share the link with users selectively

Timeline:

Ideally will have this working for users on Monday/Tuesday calls next week

View 24 previous comments from the original issue on GitLab


Option to close connections when deleting database on Sql Server

Is your feature request related to a problem? Please describe.
If you try to delete database that is in use, it will fail.
Describe the solution you'd like
Add option

[ ] close existing connections

This is something that MS Managment Studio has:

Describe alternatives you've considered
There are no alternatives except doing it manually which defeats the purpose of menu item and bulk delete

Thank you for the idea.
As a workaround you can remove default status from the database (select another one as a default via database context menu) and delete it.

That is not workaround. Another systems are using the database, not dbeaver.

This option is enabled by default now.
After implementing #10958, we will add checkbox to toggle this option inside object delete dialog.

This option is enabled by default now.
After implementing #10958, we will add checkbox to toggle this option inside object delete dialog.

verified

verified

Thank you for quick response. You are awesome.

Thank you for quick response. You are awesome.


Tutorial provisions an expensive resource

A user following the steps of this tutorial, but stopping after the "Get Job Output" step (not doing the cleanup steps) will be surprised when they later get a bill for the provisioned NC6 cluster.
For this reason, I strong suggest that you instead change this "hello Azure Batch API" tutorial to NOT use a dedicated cluster, but use the Batch API option to allocated a machine just for the duration of the job being run.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: fb33bcc8-340f-698e-cd60-a762113975d6
Version Independent ID: ffc3a55c-ee6c-b05a-4496-3fa6e203d14b
Content: Azure Quickstart - Deep learning training - Azure CLI
Content Source: articles/batch-ai/quickstart-tensorflow-training-cli.md
Service: batch-ai
GitHub Login: @dlepow
Microsoft Alias: danlep

@MicahMcKittrick-MSFT We don't need to do anything else for this issue? Thanks.

@YutongTie-MSFT. Nope. User reached out offline and is working with author directly.


error calling listing api with client package

Error: cannot estimate gas; transaction may fail or may require manual gas limit [ See: https://links.ethers.org/v5-errors-UNPREDICTABLE_GAS_LIMIT ] (error={"reason":"cannot estimate gas; transaction may fail or may require manual gas limit","code":"UNPREDICTABLE_GAS_LIMIT","error":{"code":-32000,"message":"gas required exceeds allowance (1541)"},
const  { createClient} = require("@reservoir0x/reservoir-kit-client");
const HDWalletProvider = require("@truffle/hdwallet-provider")
const  { ethers } = require("ethers");
const web3provider = new HDWalletProvider(privateKey, url);
var provider = new ethers.providers.Web3Provider(web3provider)
var wallet = new ethers.Wallet(privateKey);
singer = wallet.connect(provider)
const client = createClient({
    apiBase: "https://api.reservoir.tools",
    apiKey: apiKey,
    source: source
  });

const result = await client.actions.listToken({
    listings: listing,
    signer: signer,
    // options: options,
  })

Can you send us some more info on the token that you tried to list? Seems like there wasn't enough ETH in the wallet to approve the transaction. Can you confirm you had enough ETH in the wallet?

I'm sure the account has enough eth. I also encountered the same error when calling buyToken api, I added options to solve the problem, listToken has no options parameter
  const options = {maxFeePerGas: "25000000000", maxPriorityFeePerGas: "1000000000"}
const result = await client.actions.buyToken({
    tokens: [{ tokenId: tokenId, contract: contract}],
    signer: signer,
    options: options,
    onProgress: (steps) => {
      console.log(`account ${account} buy ${contract} ${tokenId} process`)
    }
  })

Hey @infra403 if you could provide the full request details, that would be great! Otherwise, it's quite tricky to debug this without having a reproducible example.

Closing due to inactivity.


Use go-encrypt for AES

Hey man,
Welcome to the golang community :) I thought I'd send this along, I did this work for other projects and thought it'd make things simpler. You should probably take what I've done here and make it more idiomatic for your use case, along with allowing different length keys...or just don't merge this at all, haha. But thought you might want to check it out.

Hi Peter,
Thanks for the PR! One thing that stands out is that the secret is randomly generated - whilst this in theory is a good idea, the secret on the work server and worker need to be the same and this acts as a form of authentication. Forcing the user to somehow transfer the secret to "legal" workers seems a bit too complicated for the average use case. If you modify your PR to use a predefined secret instead of a randomly generated one then i'll pull it in :)

Yep, that is true - how I envision this being used is that on a separate call, you generate a secret, and then later you run with it, something along the lines of (assuming you put main.go in cmd/gowork):
gowork generate > ~/.gowork.secret
gowork run --secret-path ~/.gowork.secret

Using the rand package means in practice, when people share this secret, it's less likely to be something like abcd1234. What do you think?

s/abcd1234/somethingthatis32charactersbutequallyobvious/

Alternatively, see the last commit, you can do either now haha.

Good idea, will take a look later :)

Thanks for updating the PR to the latest codebase, have you tested that your changes work?

How should I test it/are there tests available?
On Wednesday, July 15, 2015, Ryan Skidmore<EMAIL_ADDRESS>wrote:

Thanks for updating the PR to the latest codebase, have you tested that
your changes work?
—
Reply to this email directly or view it on GitHub
https://github.com/ryanskidmore/GoWork/pull/2#issuecomment-121722336.

If tests are added and/or something as simple as scripted integration
testing is provided, I'd be happy to make sure this complies, but I saw no
tests here so assumed this was in very early alpha stages? :)
On Wednesday, July 15, 2015, Peter Edge<EMAIL_ADDRESS>wrote:

How should I test it/are there tests available?
On Wednesday, July 15, 2015, Ryan Skidmore<EMAIL_ADDRESS><javascript:_e(%7B%7D,'cvml','notifications@github.com');>> wrote:

Thanks for updating the PR to the latest codebase, have you tested that
your changes work?
—
Reply to this email directly or view it on GitHub
https://github.com/ryanskidmore/GoWork/pull/2#issuecomment-121722336.

My current method of testing is just building a sample application and seeing if it works (until I add tests!). I've got a sample application i'm building at the moment but it's not really complete, pushed it up to the repo anyway (feel free to PR the sample application too if you need to change anything etc!)
Cheers

Hi Peter - could you merge your branch with the latest changes?
Cheers

OK, updated with the mustNewTransformer function, and removed crypto_test.go

Merged, cheers!

:)


Color not change

We have implemented Color change functionality. its working fine on yesterday but today i have checked i have experience below error
Uncaught TypeError: material.pbrMetallicRoughness[channel].texture.source.setURI is not a function

Indeed, this is from the v2.0 upgrade. You may want to pin to an earlier version range until you have a chance to update to createTexture, though that shouldn't be too hard. You can follow the examples here.


WordPress Site Health reports 'Should be improved' on default Bedrock install

Description
On a default Bedrock install, the Site Health tool reports 'Should be improved'
Steps to reproduce

Install Bedrock
Visit /wp/wp-admin/site-health.php

Expected behavior
The Site Health tool shouldn't consider some checks to be relevant on a Bedrock install
Actual behavior
The Site Health tool complains about:
2 critical issues

Your site is set to display errors to site visitors (on development)
Background updates are not working as expected

1 recommended improvements

You should remove inactive themes (see https://github.com/roots/wordpress/issues/3)

When I do a default Bedrock install, instead of

You should remove inactive themes

I get

Have a default theme available

which gets listed under "critical" and is equally annoying :D

Is a MU plugin the way to go that hooks into site_status_tests action and disables two tests?
function disable_status_tests($tests) {    unset($tests['direct']['theme_version']); // disable theme update, theme inactive and default theme checks.    unset($tests['async']['background_updates']); // disabled the background updates check.    return $tests; }

Sharing here incase it helps anyone, I have made a PR with a solution to the themes problem — https://github.com/roots/bedrock/pull/555

@christopherdarling Your solution should be included in the next release, having a ton of old themes on the server is a waste of resources and not secure.

https://packagist.org/packages/roots/wordpress-no-content is now available to use, fyi!
props to @LeoColomb and @swalkinshaw for their work on the new composer wordpress packages via https://github.com/roots/wordpress-packager


Nodejs sample does not build

Using Windows 10 19042.572 VS Code 1.501 trying to develop Nodejs edge module.
Installed all prereqs and Docker Desktop 20.10.0
Setup script always seem to end with:
=> [5/5] COPY app.js ./                                                                                0.2s
 => exporting to image                                                                                0.7s
 => => exporting layers                                                                                0.7s
 => => writing image sha256:3629444b5f58924004d46d8170214a8c02b73108f5fd6e7cc527d2ad5e89103e                                                          0.0s
 => => naming to localhost:5000/samplemodule:0.0.1-amd64.debug                                                                                0.0s
Traceback (most recent call last):
  File "lib\site-packages\PyInstaller\loader\rthooks\pyi_rth_pkgres.py", line 11, in <module>
  File "d:\a\1\s\venv\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 627, in exec_module
  File "lib\site-packages\pkg_resources\__init__.py", line 86, in <module>
ModuleNotFoundError: No module named 'pkg_resources.py2_warn'
[3744] Failed to execute script pyi_rth_pkgres

C:\Users\dvesc\Documents\vs code edge solution\1\EdgeSolution>

I tried different docker node images and few other things but can not clear this issue.
start up:
Executing docker version
Client: Docker Engine - Community
 Cloud integration: 0.1.22
 Version:           20.10.0-beta1
 API version:       1.41
 Go version:        go1.13.15
 Git commit:        ac365d7
 Built:             Tue Oct 13 18:13:24 2020
 OS/Arch:           windows/amd64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.0-beta1
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.13.15
  Git commit:       9c15e82
  Built:            Tue Oct 13 18:17:18 2020
  OS/Arch:          linux/amd64
  Experimental:     true
 containerd:
  Version:          v1.4.1
  GitCommit:        c623d1b36f09f8ef6536a057bd658b3aa8632828
 runc:
  Version:          1.0.0-rc92
  GitCommit:        ff819c7e9184c13b7c2607fe6c30ae19403a7aff
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 5a3f2f86-40bf-19f2-4f56-9dc71ed47fa2
Version Independent ID: 20e8912c-4a10-68b9-c46a-e945a62421d6
Content: Develop and debug modules for Azure IoT Edge
Content Source: articles/iot-edge/how-to-vs-code-develop-module.md
Service: iot-edge
GitHub Login: @kgremban
Microsoft Alias: kgremban

Hello @dvescovi1
Thanks for the feedback! We are currently investigating and will update you shortly.

Just tried with C# module also ... ends same way.

Sorry for the inconvenience caused, we are actively working on this issue, will get back soon.

Is this then a known issue?   i.e. not something I did?

Is this then a known issue? i.e. not something I did?

We can't confirm it as an issue until we complete our investigation on the complete document section. Please stay tuned for further updates.

seems this is the issue 288
as everyone that uses vs 2019 or vs code to develop iot edge module is hit by this I hope they fix it soon.

Please try this  suggestion,
We are actively investigating this issue,
workaround by installing https://pypi.org/project/iotedgehubdev/ into the default python env.
Steps

pip install --upgrade iotedgehubdev (installs https://pypi.org/project/iotedgehubdev/ in the default python env)
notice the iotedgehubdev.exe in the install log
copy that exe into C:\Users\Documents\Visual Studio 2019\iotedgehubdev\default_python
set version to "default_python" in C:\Users\Documents\Visual Studio 2019\iotedgehubdev\simulator.config
vs will now use this install instead of C:\Users\Documents\Visual Studio 2019\iotedgehubdev\v0.14.2

@dvescovi1
Just checking back! Please let us know if you need further help.
Please also do visit Microsoft Q&A the home for technical questions and answers at Microsoft.

@dvescovi1
Please refer to the response posted over here on Microsoft Q&A.

We will now proceed to close this thread. If there are further questions regarding this matter, please tag me in your reply. We will gladly continue the discussion and we will reopen the issue.
Please also do visit Microsoft Q&A the home for technical questions and answers at Microsoft.


Question: How to obtain a JWT token to send to backend?

I've attempted to scan the codebase, but did not see anything obvious about obtaining a JWT token for authorizing calls to my backend API. In React, this would be like using the useAuth().getToken() function. Does this capability exists yet? If so, please provide documentation. Thanks

You can get the token by accessing clerk.session.getToken() the method is mentioned here: https://swiftpackageindex.com/clerk/clerk-ios/main/documentation/clerksdk/session

Thanks @semirteskeredzic.
@tuantvu one clarifying note, you don't necessarily need to use @ObservedObject unless you are using SwiftUI or want to observe the lifecycle state stored for you by the ClerkSDK. You could also just manually fetch a token like so:
let token = Clerk.shared.session?.getToken()

@mikepitre @semirteskeredzic Two follow up questions:
I saw how startSessionTokenPolling is continuously getting a new token every 50 seconds and caching it. There is documentation saying the token only lives for 60 seconds. This is fine with me, but I am wondering why the TTL is not determined by the settings a client would configure in the Clerk Dashboard for JWT tokens?
2nd question, I'm curious how refresh tokens work? I was unable to find storing or refresh token exchange endpoint calls. We want to implement the ability for the user to authenticate with their face or fingerprint id on subsequent log ins after initial log in via social provider like Google or through email/password.

All Clerk short lived sessions tokens have a TTL of 60 seconds. The lifetime you're able to set via the dashboard is for the long lived refresh token.
The long lived token is included with every subsequent request once it is acquired on the creation of a client (device). It is used to obtain the short lived refresh tokens via the session.getToken() which is what is called via the polling you mentioned.

Thanks @mikepitre. Do you have any advice on how to implement my requirement above for face id/fingerprint log in? Does Clerk offer a solution for the requirement out of the box via configs somehow? My thought right now is to implement biometric checks in my app and showing it after confirming that the client and session exists after calling Clerk.load().

The SDK is still in alpha so all of this is subject to change, but yes the ClerkSDK can handle this for you.
When a user signs up or signs in with identifier + password, you can save their credentials behind biometry access with:
try Clerk.LocalAuth.setLocalAuthCredentials(identifier: identifer, password: password)

and then on subsequent sign ins, you can access those credentials via:
let creds = try Clerk.LocalAuth.getLocalAuthCredentials()
try await SignIn.create(strategy: .identifier(creds.identifier, password: creds.password))


Text module doesn´t load

Describe the bug
In version 5.0.0  the text module does not load.
Screenshots

please complete the following information:
"@_sh/strapi-plugin-ckeditor": "5.0.0",
"@sklinet/strapi-plugin-video-field": "^1.1.0",
"@strapi/plugin-cloud": "5.6.0",
"@strapi/plugin-users-permissions": "5.6.0",
"@strapi/provider-upload-cloudinary": "5.6.0",
"@strapi/strapi": "5.6.0",
"pg": "8.13.1",
"react": "^18.3.1",
"react-dom": "^18.3.1",
"react-router-dom": "^6.28.0",
"styled-components": "^6.1.13"

Did you update to version 5 and encounter this issue? Have you followed the migration guide?

I get the same result with "@_sh/strapi-plugin-ckeditor": "^5.0.0".  I followed the migration guide with no luck.

I realized I had to select a default editor type in the content type config.


FAQs documentation missing

Documentation needed.. exomiser?

The Phenopacket Schema is an emerging GLobal Alliance for Genomics and Health (GA4GH) standard for sharing disease and phenotype information that characterizes an individual person, linking that individual to detailed phenotypic descriptions, genetic information, diagnoses, and treatments.  The Phenopacket Schema, together with other GA4GH data and technical standards, will enable data exchange and provide a foundation for the computational analysis of disease and phenotype information to improve our ability to diagnose and conduct research on all types of disorders, including cancer and rare diseases. Specifically, Phenopackets are designed to be both human and machine‐interpretable, enabling computing operations and validation on the basis of defined relationships between diagnoses, lab measurements, and genotypic information. Phenopackets can be used for a growing number of computational tasks, including serving as input for genomic analysis software such as Exomiser.
@belkassaby could we add this to the FAQs ?

... such as Exomiser, LIRICAL, SvAnna?
@pnrobinson how about a little bit of shameless self-promotion?

I added Daniel suggestion of links and put some paragraphs so it is more readable.

@ielis I would be the last person to experience a sense of shame about self promotion... @belkassaby  -- not sure if the paragraphs about LIRICAL etc were added already?

@pnrobinson i meant I split the text description you provided into 3 paragraph for more readibility. and added at the end the links to Exomiser, LIRICAL and Svanna. its already deployed on https://phenopacketlab-dev.jax.org/help . let me know if it looks like you expected or if anything needs to be added/changed.


Mark as read option missing

Prerequisites

[X] Put an X between the brackets on this line if you have done all of the following:

Checked that your issue isn't already filed: https://github.com/GetStream/Winds/issues

Description
I am missing a «Mark all as read» option somewhere (global or per feed)
Also If I go to an article and press back, it doesnt remove the green dot which I suppose is the read indicator

Thank you @borisrorsvort! We'll add this to our backlog and address it as soon as possible! Keep the feedback coming :) 👍

@AminMahboubi We have an endpoint for handling this. Ping me if you need direction.

@nparsons08 Could you send me the endpoint?

Did this get added? Or was there a decision to not have a mark for read option?

Currently the green indicator is set for recent articles, it's not actually a not yet read sign. however we probably will add this feature to the frontend as Stream already support this

Currently the green indicator is set for recent articles, it's not actually a not yet read sign. however we probably will add this feature to the frontend as Stream already support this

That would be awesome! I just want to be able to see which ones are new and which one I just didn't want to read. Because it seems like the green dot never goes away.

Is there any status on this?

@TobiasDev unfortunately no dev yet.

I really like Winds. I like the snap app and I like my subscriptions synced to the cloud.
The only thing stopping me from using it over other podcast/rss app is the "Mark as read" option missing.
Is there anyway I can mark my articles or podcasts as read?

Yes, I accidentally discovered this app, after searching for Feedly open source server alternatives.  Auto mark as read (while scrolling) or similar is definitely #1 on my Wishlist, for me I can't actually use an app that doesn't have this otherwise I just end up looking at the same stuff over and over.  Unless, there's some other way of handling that?  Could do some kind of a 'viewed to here' as an option.


Extend e2e test coverage for user-provided tolerations

Implement these e2e tests to increase the coverage for the user-provided tolerations scenario:
https://docs.google.com/document/d/1Nh2wuDfWmhCBS-Jy1vGGe-5knjy9FM08GaxBTdHWwVA/edit

/assign @shajmakh

solved by #900 and #910


Incorrect string replacement more than one source

@VFK
I have two html files that needs to be processed with similar blocks.
My gulpfile task:
//process view files - fix paths
gulp.task('process-view', function () {
    return gulp.src([
        './index.html',
        './admin.html'
        ])
        .pipe(htmlreplace({
        css: 'assets/style.min.css',
        js: {
            src: [['js/context.min.js', 'js/require.js']],
            tpl: '<script data-main="%s" src="%s"></script>'
        }
    }))
        .pipe(gulp.dest('./dist/views'));
});

My HTML(s) all look like:
<!DOCTYPE html>
<html>
<head>
    <title></title>
    <!-- build:css -->
    <link href="../assets/font-awesome.css" rel="stylesheet" />
    <link href="../assets/pnotify.custom.min.css" rel="stylesheet" />
    <!-- endbuild -->

    <!-- build:js -->
    <script src="../lib/jquery-2.1.3.min.js"></script>
    <script data-main="../js/bootstrapper" src="../lib/require.js"></script>
    <!-- endbuild -->
</head>
<body>
</body>
</html>

The problem lies with the second source file (./admin.html).
The output (NOTE: line-breaks were removed):
<!DOCTYPE html>
<html>
<head>
    <title></title>
    <link rel="stylesheet" href="assets/style.min.css">
    <script data-main="<script data-main="%s" src="%s"></script>" src="js/context.min.js"></script> js/require.js
</head>
<body>
</body>
</html>

Please suggest a workaround for this.

Hi @arpanmukherjee1 this is actually a very good find, sorry for the delay in fixing it. Please try v1.5.5


Choose k8s version

Great tool.  It would be nice if the version of k8s was configurable by way of a variable.
I guess this could be done with a modification to the aws_instance.master resource, with an addition to the kubeadm init command.  Something like;
  %{if var.kubernetes_version != null~}
    --kubernetes-version "${var.kubernetes_version}" \
  %{endif~}

Hi @hyweldaniels, great idea! I will schedule this for the next minor release.


PeerJ invitation to be a reviewer

Copy of invitation:

<EMAIL_ADDRESS><EMAIL_ADDRESS>Mon, Jan 14, 9:16 PM

<EMAIL_ADDRESS>

Dear Dr Murray-Rust,
I am the co-founder of PeerJ, an open access journal publisher which originally started in the biosciences. We are launching 5 new peer-reviewed journals covering all of Chemistry and so I am writing to invite you to join the Editorial Board of ‘PeerJ Organic Chemistry’ specifically to cover the area of Crsystallography.
I know you get a lot of email like this, so I want to assure you that we are a legitimate and well respected publisher - our Advisory Board includes 5 Nobel Laureates (including the chemistry laureate Kurt Wüthrich); we are indexed in the Web of Science; and we have won several industry awards for innovation. Many of the world’s most prestigious universities support PeerJ by covering the publication fees for their faculty (the full list is at https://peerj.com/edu/). You can see our current journals in biology & environmental sciences, and computer science, at https://peerj.com.
At PeerJ, we publish ‘broad topic’ journals, and we peer-review only for scientific soundness (not ‘impact’). Since launching our initial titles in biology, medicine and computer science, we have received feedback from chemists requesting that we launch journals in their fields as well. With our experience in publishing high quality, rapid, open access journals, combined with our Mission to drive down the costs of open access to authors, we believe we are uniquely placed to serve the chemistry community. As a result, we are pleased to be launching ‘PeerJ Physical Chemistry’; ‘PeerJ Organic Chemistry’; ‘PeerJ Inorganic Chemistry’; ‘PeerJ Analytical Chemistry’; and ‘PeerJ Materials Science’. As you can see from our announcement blog post, we will open for submissions this week.
As a result, I am actively recruiting leading academics to join the Editorial Boards of these 5 titles. Based on your publication history I believe you would make an ideal Editorial Board member and so I would be honored if you would consider joining us.
To reassure you, the workload is entirely determined by yourself (editors pick which papers to take on from a pool of ‘available’ submissions) and can be as much or as little as you wish. We have no minimum expectation, and many of our current Academic Editors handle less than 4 papers a year.
I hope you are interested in taking on this Editor role and I am happy to answer any questions you might have, or to explain how the process works.
For more information about us, please see: https://peerj.com/about/publications/  I look forward to your reply and to welcoming you onto the Editorial Board!
Yours,
Peter Binfield
. . .
Peter Binfield, PhD
Co-Founder and Publisher, PeerJ
Email<EMAIL_ADDRESS>Web: PeerJ.com
Office: + 1<PHONE_NUMBER>
PO Box 910224, San Diego, CA 92191, USA

I know Pete Binfield. He shows no indication of knowing who I am and I assume the mail has been created automatically and sent out by robots.
I tweeted about this and my reply including @thePeerJ twitter handle. There was no reply from the journal or "Pete Binfield".

My reply

Peter Murray-Rust<EMAIL_ADDRESS>Peter Murray-Rust<EMAIL_ADDRESS>Jan 16, 2019, 4:45 PM

Peter Murray-Rust<EMAIL_ADDRESS>

On Mon, Jan 14, 2019 at 9:16 PM<EMAIL_ADDRESS>wrote:

Dear Dr Murray-Rust,

PMR>> We have actually met several times and I was originally a public supporter of PeerJ so you can use first names.
I am an advocate of new types of scholarly communication including emphasis on the global south, the reduction of unnecessary costs, of unjustified profit margins, of semantic communication , of reducing the complexity of access . In my view the scholarly publishing industry has played a major role in reducing access to knowledge except for rich Universities.
 Note that there are highly quality journals with no author or reader fees , such as Beilstein J Chem. I am seriously considering my positions on editorial boards in this light. I'd be grateful to know:

what the new PeerJ set of journals does differently from anyone else?
what technology they use. For example does it have chemical substructure search?
has PeerJ invested in any public technology such as Open Source chemical authoring tools?
does it publish everything in XML as well as PDF?
does it require authors to deposit data (spectra, syntheses, structures) in machine-readable form (not PDF)?
how it adds value to preprints which cost authors and readers nothing?
what are the APCs? are they based on actual costs? are these costs transparent?
what are the APCs for a Brasilian chemist? for a Zimbabwean?
is any emphasis given to increasing the involvement of non-academic readers and authors?
what emphasis is given to early-career researchers in the editorial process?

These are reasonable questions for a prospective member of an editorial board to ask and I look forward to answers. I reserve the right to make these answers public.
Looking forward to hearing why PeerJ chemistry is radically different.
Cheers

I have heard nothing from PeerJ or "Pete Binfield" after nearly 2 months.

APCs:
https://peerj.com/blog/post/115284878682/new-publication-prices-at-peerj/
by Jason Hoyt · Published February 15, 2016 · Updated July 25, 2018
PeerJ has just passed its third year of publishing high quality science in the most efficient, cost effective, and open way possible. In that time we have published more than 1,700 peer-reviewed articles and 2,100 preprints; we have launched a new journal; we have released open source software to support the community; we have supported and advocated for many good causes; we have gained the support of over 160 major international institutions; and we have innovated in almost every aspect of academic publishing. We are proud of what we have achieved and we look forward to the future and to continuing to deliver on our mission of affordable, quick, and high quality Open Access scholarly publishing.
We want to take this opportunity to sincerely thank our early supporters – all those people who have believed in PeerJ since its launch. We pursued the vision of PeerJ with the belief that academics were looking for a more reasonable Open Access pricing option. That belief has been fulfilled. We have enjoyed extremely strong support from many important advocates; we have been able to attract a world-class Editorial Board and world-class authors; we have developed a highly respected publishing system; and we have made a real difference in the world. We have been able to do this because we are pursuing a mission to deliver a publication experience that surpasses all other major journals, but at a fraction of the normal cost to authors.
Our plans for the future are to continue investing in significant internal development, which will allow us to deliver an even better author experience, but still at a rock bottom price. We’re also extremely mindful of PeerJ’s founding principles on affordable high-quality Open Access for all. That’s still our goal, and so we’ve strived to ensure that at the end of the day, the value authors get from PeerJ surpasses all other major Open Access options. To succeed in our mission we’re making a few changes to how we price things.
With current and future benefits in mind, some prices are going up, some are staying the same, and some things are improving straight away.
What stays the same?
The introductory per article publication fee of $695 is not changing. Also, the Unlimited Membership price stays the same at $399, allowing authors to publish an unlimited number of papers, for free, for life.
The institutional pricing (i.e. the membership price paid by universities which make a pre-payment arrangement with PeerJ) also stays the same for the Basic Membership at $139.
What is changing?
Going forward, the Basic Membership (1 publication per year) will be $199, and the Enhanced Membership (2 publications per year) will be $299. At the same time, we are doing away with the extra charge that was applied when memberships were purchased after an editorial ‘accept’ decision. That had been an additional $50 on top of our ‘base prices’, but now you can pay for memberships before or after an editorial accept decision and pay the same price, so there are no additional charges or penalties for waiting until an article is accepted.
Note: these prices are subject to change – please refer to the Pricing page for the most up-to-date info on fees.
Does that mean I will end up paying more?
Not necessarily. One reason we introduced the APC of $695 was so that authors could save money. As each author needed a membership, it meant that some papers (i.e. those with several co-authors) were actually paying a lot more than the APC price before it was introduced.
 Overtime, it is still the best value and considerably cheaper than the APC option.
When do these changes take effect and what if I’ve already submitted?
The change will start from Monday, February 22nd at midnight UTC (that’s 00:00 hours on the 22nd). Any submissions prior to that time will still qualify for the previous membership pricing, even if paid after February 22nd. Those “legacy” prices for submissions prior to February 22, 2016 are:
Basic $99 (if paid after accept decision add $50)
Enhanced $199 (if paid after accept decision add $50)
Unlimited $399 (if paid after accept decision add $50)
APC $695 (same price)
NOTE INCREASE:
https://peerj.com/pricing/ (retrieved 2019-03-08)
Open Access Publication Prices
Free publishing at 170+ institutions. $200 reviewer discounts.
PeerJ
£835*
€945* / $1,095
Peer-reviewed in biology, medicine, and the life sciences
Author page proofs before publication
Median first-decision time of ~27 days
Indexed in PubMed, Web of Science, Google Scholar, Scopus, ...
Citation and other article-level reports

Waivers
We hope you agree that PeerJ's publication fees are already great value with the low APC and lifetime memberships! That said, we do recognize that some people are unable to pay this amount. Therefore, we offer a no questions asked fee waiver, on request, to anyone from countries that are classified by the World Bank as Low-income economies. The waiver simply applies to the publication in question, and is not a waiver for a full membership plan. We only allow one waiver per person per year.
In addition, any co-author who was an undergraduate at the time of the research may request a membership waiver (provided the paper has senior co-author(s) who have at least a Basic publishing plan, and provided the article passes peer review as normal). This is valid if paying through the Membership route rather than APC. Read more about this policy.
Countries in Middle Income: (2019-03-08)

Tim,
When PeerJ came out I was enthusiastic and said so. I supported the model, particularly because you were an investor.
Recently I was asked by "Pete Binfield" (who I know as a human) to be one the editorial board of a new PeerJ chemistry journal. I wrote back asking a number of questions about PeerJ. I haven't received an answer after 2 months and assume the message was a robotic one. Given that I am well known as an innovator in chemistry - and have made many contributions to chemical publishing systems - I am disappointed.
I am putting my correspondence at:
https://github.com/petermr/correspondence/issues/1
I am more generally concerned about the divisive effect of western publishing on knowledge inequality and I consider fees of 1095 USD per paper (hiked by 40% over 2 years) to be indefensible. Arxiv manages on ca 10 USD per preprint and I see no reason to have much higher prices. Review management has costs, but typesetting could be an optional add-on (arxiv authors do fine) and most of the other costs are wildly inefficient. Moreover profits run at 30%.
I've just come back from India. PeerJ charges the full amount. There is no way I can recommend to Indian scientists that they publish in any journal with such APCs. I'd see 250USD (Cameron Neylon's figure for the cost) as the max.


Issue  installing "OMERO.server installation on CentOS 7"

I was doing the install via this link, https://omero.readthedocs.io/en/stable/sysadmins/unix/server-centos7-ice36.html
All was going well until this section:
Running OMERO.server
The following steps are run as the omero-server system user. (su - omero-server)

OMERO should now be set up. To start the server run:

omero admin start

This is the logging from my install:
[omero-server@ip-172-30-4-46 ~]$  omero admin start
WARNING: Your server has not been configured for production use.
See https://docs.openmicroscopy.org/omero/latest/sysadmins/server-performance.html?highlight=poolsize
for more information.
No descriptor given. Using etc/grid/default.xml
Waiting on startup. Use CTRL-C to exit
.............................
Failed to startup some components after 300 seconds
Calling "stop" on remaining components
Waiting on shutdown. Use CTRL-C to exit
......[omero-server@ip-172-30-4-46 ~]$

Thoughts?
Thanks!

What does omero admin diagnostics show?

Thanks for all the info, @scuniff. The fact that Processor and Tables tried to start implies to me that the main Blitz process began but then shutback down. Could you also share master.out, master.err and Blitz-0.log? Thanks.


Dx

No description provided.

Nice issue, I completely agree with you. But how can one fix it ? That's a great question.


May I know how to generate confusion matrix?

🚀 Feature
A clear and concise description of the feature proposal.
Motivation & Examples
Tell us why the feature is useful.
Describe what the feature would look like, if it is implemented.
Best demonstrated using code examples in addition to words.
Note
We only consider adding new features if they are relevant to many users.
If you request implementation of research papers --
we only consider papers that have enough significance and prevalance.
We do not take requests for most projects in the projects/ directory,
because they are research code release that is mainly for other researchers to reproduce results.
Instead of adding features inside easydet,

Please describe your problem in detail. What do you mainly use for.

@Lornatang I could generate mAP and IOU but, I need a matrix to visualize how many true positives are detected.
So I can have it in my final report of my master thesis.
for example in my case, I have three classes to detect from a single image.
such as company name, date, and amount from the invoice image.
Such that I can an overview of how many times the company name, date, and amount is been detected in the test/valid data set.

Hello @Lornatang, as it is a bit urgent may I know how to generate it?

I'm sorry, I'm busy at the moment. I can't help you at the moment


Add option outlineColor

Sometimes it would be nice to not just have the infoColor option, it could be useful to have an outlineColor that would change:

The color of the outline of the symbol frame
The color of the direction arrow
The color of the echelon indicator
the color of the mobility indicators

This would be outside of the standards, but I still think it would be nice to make the symbols stand out more against satellite imagery.

Oh, I forgot, this is now implemented as frameColor.


CIP-0030 Suggestions

Paginate
...
if a wallet is modified between paginated calls that this will change the pagination, e.g. some results skipped or showing up multiple times but otherwise the wallet must respect the pagination order.

To avoid this issue we could change API functions with pagination to return an async iterator instead of Promise.

PaginateError
{maxSize} is the maximum size for pagination and if the dApp tries to request pages outside of this boundary this error is thrown.

Is the dApp supposed to try to call the method with {limit: Number.POSITIVE_INFINITY} to figure out max size? I think it's an odd API and if we keep the concept of pagination there should be a way to get the limit without an error.

type TransactionUnspentOutput = cbor<transaction_unspent_output>
...
api.getUtxos(amount: cbor<value> = undefined, paginate: Paginate = undefined): Promise<TransactionUnspentOutput[] | undefined>

I initially thought that TransactionUnspentOutput wasn't cbor, the definition could use better consistency: I think we should either define all types as cbor<[type]> or list all type definitions under Data Types.

api.getBalance(): Promise<cbor<value>>

Does it include rewards and deposit? Need to make this explicit, as using these funds requires certificates in the transaction.

api.getUsedAddresses(paginate: Paginate = undefined): Promise<cbor<address>[]>
api.getUnusedAddresses(): Promise<cbor<address>[]>

There will be different types of wallets. Some might be single address and not have any unused addresses. Given this API, dApp will probably expect to always get some unused addresses, which might not be the case. The dApp itself can figure out if it was used or not. I suggest to consolidate these to api.getAddresses(paginate: Paginate = undefined): Promise<cbor<address>[]>.

The wallet is free to return thereject with TxSendError

@rooooooooob

To avoid this issue we could change API functions with pagination to return an async iterator instead of Promise.

Would the wallet do something like ordering all utxos and then have the returned generator remember the last returned one and query the wallet for the next one in whatever arbitrary ordering they chose yielding that until it's there are none? I'm not primarily a javascript dev (more C++/Rust) and haven't worked with this feature before so I just want to be sure this would be easily doable and solve the issues with the current API.

Is the dApp supposed to try to call the method with {limit: Number.POSITIVE_INFINITY} to figure out max size? I think it's an odd API and if we keep the concept of pagination there should be a way to get the limit without an error.

The idea was they would pick a limit based on their own need like 10 or 50 or whatever and repeatedly call it with increasing page numbers until PaginateError is returned. I guess if you wanted to know how many pages there would be you could use infinity. To be honest I'm not really familiar with typical paging APIs so I had hoped someone would point out better ways during the original PR review if possible. I guess the above generator approach wouldn't let you figure out how many it would return in total either though?

I initially thought that TransactionUnspentOutput wasn't cbor, the definition could use better consistency: I think we should either define all types as cbor<[type]> or list all type definitions under Data Types.

Do you mean including all the types that are defined via cbor<T> explicitly? The problem with this is that they are just composed of other types and including those too would be a lot of types. Plus I think it's best to refer to directly to the binary spec. All other types are listed under Data Types so I don't think it's a problem.

Does it include rewards and deposit? Need to make this explicit, as using these funds requires certificates in the transaction.

This was discussed in #88 here but we never finalized any changes. Perhaps it's time to continue that discussion that was started there and get to a final solution. @KtorZ @SebastienGllmt @alessandrokonrad

There will be different types of wallets. Some might be single address and not have any unused addresses. Given this API, dApp will probably expect to always get some unused addresses, which might not be the case. The dApp itself can figure out if it was used or not. I suggest to consolidate these to api.getAddresses(paginate: Paginate = undefined): Promise<cbor[]>.

Do we expect the dApp to filter them all to see if they've ever been used on-chain before? I think people would be using the two endpoints (used and unused) for very different purposes. We could change the API description to mention that they shouldn't expect there to always be addresses if that would help. We should see what others think though.

Would the wallet do something like ordering all utxos

I think we should be thinking in terms of sets of utxo (unordered).

and then have the returned generator remember the last returned one and query the wallet for the next one in whatever arbitrary ordering they chose yielding that until it's there are none?

I think we shouldn't assume any specific wallet implementation or data source. It could already have all utxo in memory, could be fetching it from some server, or loading from indexeddb. Generator implementation would depend on wallet internals, but from API user point of view it would be a generator of the most recent snapshot of the utxo set.

I'm not primarily a javascript dev (more C++/Rust) and haven't worked with this feature before so I just want to be sure this would be easily doable and solve the issues with the current API.

Not yielding duplicates should be trivial to implement regardless of where the wallet is sourcing the utxo (checking by tx id + index). The wallets can also not yield a utxo once it's spent, even if it's spent after returning the generator. There might still be a situation where a utxo that was yielded is spent before the generator completes. There could be an error for that.
For this specific function of api.getUtxos, I actually suggest just api.getUtxos(amount: cbor<value> = undefined): Promise<TransactionUnspentOutput[]>. What was the idea behind pagination for this one in the first place?

The idea was they would pick a limit based on their own need like 10 or 50 or whatever and repeatedly call it with increasing page numbers until PaginateError is returned. I guess if you wanted to know how many pages there would be you could use infinity.

I thought the PaginateError error was for a too large max page size. I don't think there's much value in having an error for when there are no more pages, as it can be inferred from an empty array result. I actually suggest to just remove this error.

To be honest I'm not really familiar with typical paging APIs so I had hoped someone would point out better ways during the original PR review if possible.

A common pattern is to return something like {items: Item[], numTotalItems: number}.

I guess the above generator approach wouldn't let you figure out how many it would return in total either though?

Yes, you wouldn't know the total number of items, but you can stop taking them when you got enough.

Do you mean including all the types that are defined via cbor explicitly? The problem with this is that they are just composed of other types and including those too would be a lot of types. Plus I think it's best to refer to directly to the binary spec. All other types are listed under Data Types so I don't think it's a problem.

That's one option, but I suggest getUtxos is declared as: api.getUtxos(amount: cbor<value> = undefined, paginate: Paginate = undefined): Promise<cbor<transaction_unspent_output>[] | undefined>, not using the TransactionUnspentOutput type declared above for consistency (it's the only type declared this way).

This was discussed in #88 here but we never finalized any changes. Perhaps it's time to continue that discussion that was started there and get to a final solution. @KtorZ @SebastienGllmt @alessandrokonrad

The discussion seems to be about total vs available balance/utxo. I think it's ok for the connector to always return available balance/utxo (would be great to document it though). My question was about what this balance consists of. I suggest returning something like that: { utxo: cbor<value>; rewards: Lovelace; deposit: Lovelace; }

Do we expect the dApp to filter them all to see if they've ever been used on-chain before? I think people would be using the two endpoints (used and unused) for very different purposes. We could change the API description to mention that they shouldn't expect there to always be addresses if that would help. We should see what others think though.

Yes I was thinking the dApp would filter them, but now that you mention very different purposes I no longer think that consolidating them is a good idea. Changing the description would be a good improvement.

Thanks for all the feedback.

I think we should be thinking in terms of sets of utxo (unordered).

I wasn't saying it should be in the spec, I was just thinking of a possible way to implement it that would solve both issues with the previous approach, those being that you might skip utxos if the set changed mid-iteration (e.g. you are on page 5 and something in pages 1-4 was spent so now everything shifts and you might miss a utxo) and that maybe new utxos will be skipped in a similar way. Having some kind of ordering was just the first idea that came to mind for me in order to implement it in a way that solves both of those issues - I didn't mean to have it sound like I was mandating it as the only way.

For this specific function of api.getUtxos, I actually suggest just api.getUtxos(amount: cbor = undefined): Promise<TransactionUnspentOutput[]>. What was the idea behind pagination for this one in the first place?

We might have been overly conservative of which endpoints might need pagination since it is optional functionality anyway. If you don't pass in a Paginate instance (it default to undefined) then you get back everything.

That's one option, but I suggest getUtxos is declared as: api.getUtxos(amount: cbor = undefined, paginate: Paginate = undefined): Promise<cbor<transaction_unspent_output>[] | undefined>, not using the TransactionUnspentOutput type declared above for consistency (it's the only type declared this way).

The problem is that transaction_unspent_output isn't in the binary spec. It's just a convenience type that we've made up since using just the spec's input type doesn't give you the value which makes it useless for figuring out input selection without consulting the ledger. Or are you talking about adding it in the cbor<T> header as a raw cddl definition and then referring to it with cbor<transaction_unspent_output> directly everywhere else?

Thanks for all the feedback.

Thanks for considering my suggestions.

Having some kind of ordering was just the first idea that came to mind for me in order to implement it in a way that solves both of those issues

If we think about the issues separately, duplicate items issue is really just some complexity that needs to be handled either by the wallets or by the dapps. Both can do it, but I think this API should aim to make dapp development easy.

We might have been overly conservative of which endpoints might need pagination since it is optional functionality anyway.

I have an alternative suggestion. Instead of optional pagination argument, how about a separate API method that you would use like this:
const pageSize = 10;
// not sure if numPages is a good idea
const { numPages, getPage } = await api.paginateUtxo(pageSize);
for (let page=0; page<numPages; page++) {
  const utxoPage = await getPage(page); // an array of utxo with length up to pageSize
}

The problem is that transaction_unspent_output isn't in the binary spec.

Oh, I see. I misunderstood the purpose of that type def. Thanks for an explanation.


Update golangci-lint and fix linting issues

Update golangci-lint from to latest version:
Update golangci-lint from version 1.43.0 to the latest linter version 1.46.2.
This is the latest version of golangci-lint which finds new (small) errors in the source code. This linting issues have been fixed in this merge request, too.
Fixed linting issues:
Remove newline from error string in internal/test/test.go
and remove redundant nil / len(0) check from asMap() function
in pkg/chartutil/values.go. Guard this function by a unit test.
Special notes for your reviewer:
Only linting issues fixed, added unit tests for changed logic to ensure nothing breaks.
If applicable:

[x] this PR contains unit tests
[x] this PR has been tested for backwards compatibility

Ping @devfbe


Flickering textures

https://share.getcloudapp.com/geuoNYPp

This has nothing to do with Cull Leaves, that's Round Trees

This has nothing to do with Cull Leaves, that's Round Trees


Turbolinks: Uncaught Error: unmountComponentAtNode

I recently updated to the lattest React on Rails v6.10.0, which did fixed the blinking issue #706 of turbolinks reload. But the problem is that every time I navigate to any link react raises a Uncaught Error: unmountComponentAtNode(...): Target container is not a DOM element..
@szyablitsky @justin808  Is this a know issue?

@3den Please review all issues and PRs regarding Turbolinks and let me know your thoughts.
Even better is to submit a PR that addresses the issue.
Thanks,
Justin

@justin808 seems like none of the issues address this problem, if I disable serverside rendering the pages wont even render. Below is the code of my application.js:
//= require turbolinks
//= require webpack-bundle

ReactOnRails.setOptions({
  traceTurbolinks: <%= Rails.env.development? %>
});

this is what i get on the logs
TURBO: USING TURBOLINKS 5: document added event listeners  turbolinks:before-visit and turbolinks:load.
TURBO: reactOnRailsPageLoaded
Clicked link
TURBO: reactOnRailsPageUnloaded
Uncaught Error: unmountComponentAtNode(...): Target container is not a DOM element.
...

I would not turn on and off turbolinks
https://github.com/shakacode/react_on_rails/blob/master/spec/dummy/app/assets/javascripts/application_static.js
https://github.com/shakacode/react_on_rails/blob/d5d579e7c1a24667eb5ccfbb34c17c0e898d6bf9/spec/dummy/client/app/startup/clientRegistration.jsx#L31
you cannot turn turbolinks on for server rendering, which is what you are doing by putting that JS code in your application.js
https://github.com/shakacode/react_on_rails/search?utf8=✓&q=traceTurbolinks&type=

@justin808 we are using node server, which will only load the webpack file that does not have turbolinks anywhere.
Is it posible to get turbolinks to work with server side rendering? I saw on the dummy app but is a bit  hard to understand how/if you guys got turbolinks working there.

Also even removing turbolinks the components don't render when serverside rendering is disabled, it was working fine on version 6.5.1 tho.
Was there any breaking change on the api?

@3den Hi! I'd love to get your feedback on the Node server for rendering! I sent you an invite to our slack room.
Turbolinks is a client side technology. @3den, how does it relate to server rendering for you?
Thanks,
Justin

The turbolinks problem has nothing todo with the node server. The node server works fine but I did some changes on it https://gist.github.com/3den/1416fc25a47208f73be8dbbc7e98ca52 .
The problems i have with the nodeserver is that it does not pick changes on the on the webpack file as it should, and I have to manually delete the node.sock.
// client/package.json
{
  ...
  scripts: {
    ...
    node-server: rm -f node/node.sock && node ./server.js,
  }
}

thanks for the help @justin808, every think seems to be working fine now, the problem was because I forgot to update the node_module after updating the gem.
I noticed that the you replaced npm with yarn and that broke my tests, but i fixed it with this:
class NodeTester
  attr_reader :config

  def initialize(config)
    @config = config
  end

  def compile_and_launch
    compile_assets
    launch_node
  end

  def compile_assets
    ReactOnRails::TestHelper.configure_rspec_to_compile_assets(config)
  end

  def launch_node
    path = Rails.root.join('client', 'node')
    puts "Launching NodeJS server at #{path}"
    system("cd #{path} && npm start &")
    sleep(1)
  end
end

@3den @alleycat-at-git Should this get merged back into the main project? or maybe a file in /docs/additional-reading? Such as here:
https://github.com/shakacode/react_on_rails/blob/master/docs/additional-reading/node-server-rendering.md


Spelling error on the website Home page

The spelling of benefit is misspelled.

@abhay-raj19 Request you to assign this issue to me under GSSOC'24.

@Ayushhgupta39 Add relevant gssoc level label plz

@Ayushhgupta39 Please add the gssoc label and levels to this PR as it was not counted on the leaderboard.


[BUG] {No available devices after SIGTERM}

Hi @Serafadam ,
while using OAK-FFC-4P custom camera module I occasionally face the issue when the camera is not being found by the host PC. It regularly happens when the camera was previously killed by SIGTERM. Do you know what might be the case? Please, see the output log:

SUMMARY
========

PARAMETERS
 * /oak/camera_i_base_frame: oak-d_frame
 * /oak/camera_i_cam_pitch: 0.0
 * /oak/camera_i_cam_pos_x: 0.0
 * /oak/camera_i_cam_pos_y: 0.0
 * /oak/camera_i_cam_pos_z: 0.0
 * /oak/camera_i_cam_roll: 0.0
 * /oak/camera_i_cam_yaw: 0.0
 * /oak/camera_i_enable_imu: True
 * /oak/camera_i_enable_ir: True
 * /oak/camera_i_floodlight_brightness: 0.0
 * /oak/camera_i_imu_from_descr: False
 * /oak/camera_i_ip:
 * /oak/camera_i_laser_dot_brightness: 800.0
 * /oak/camera_i_mx_id:
 * /oak/camera_i_nn_type: spatial
 * /oak/camera_i_parent_frame: oak-d-base-frame
 * /oak/camera_i_pipeline_type: stereo
 * /oak/camera_i_publish_tf_from_calibration: False
 * /oak/camera_i_usb_port_id:
 * /oak/camera_i_usb_speed: SUPER_PLUS
 * /oak/nn_i_nn_config_path: depthai_ros_drive...
 * /oak/stereo_i_publish_left_rect: True
 * /oak/stereo_i_publish_right_rect: True
 * /robot_description: <?xml version="1....
 * /rosdistro: noetic
 * /rosversion: 1.16.0

NODES
  /
    oak (nodelet/nodelet)
    oak_nodelet_manager (nodelet/nodelet)
    oak_state_publisher (robot_state_publisher/robot_state_publisher)
    set_log_level (rosservice/rosservice)

ROS_MASTER_URI=http://localhost:11311

process[set_log_level-1]: started with pid [115394]
process[oak_state_publisher-2]: started with pid [115395]
process[oak_nodelet_manager-3]: started with pid [115396]
process[oak-4]: started with pid [115397]
[ INFO] [1695047161.437818122]: Loading nodelet /oak of type depthai_ros_driver/Camera to manager oak_nodelet_manager with the following remappings:
[ INFO] [1695047161.442854749]: waitForService: Service [/oak_nodelet_manager/load_nodelet] has not been advertised, waiting...
[ INFO] [1695047161.456336435]: Initializing nodelet with 12 worker threads.
[ INFO] [1695047161.463674415]: waitForService: Service [/oak_nodelet_manager/load_nodelet] is now available.
[2023-09-18 16:26:01.695] [depthai] [info] DEPTHAI_DEBUG enabled, lowered DEPTHAI_LEVEL to 'debug'
[2023-09-18 16:26:01.695] [depthai] [debug] Library information - version: 2.22.0, commit:  from , build: 2023-06-20 18:09:51 +0000
[2023-09-18 16:26:01.696] [depthai] [debug] Initialize - finished
[ INFO] [1695047161.697871563]: No ip/mxid/usb_id specified, connecting to the next available device.
[2023-09-18 16:26:01.809] [depthai] [debug] Resources - Archive 'depthai-bootloader-fwp-0.0.26.tar.xz' open: 3ms, archive read: 107ms
[2023-09-18 16:26:02.166] [depthai] [debug] Resources - Archive 'depthai-device-fwp-f033fd9c7eb0b3578d12f90302e87759c78cfb36.tar.xz' open: 3ms, archive read: 463ms
[set_log_level-1] process has finished cleanly
log file: /home/martinlucan/.ros/log/298f4a44-5627-11ee-91e7-958d63f75c32/set_log_level-1*.log
[ERROR] [1695047165.747481996]: No available devices
[DEBUG] [1695047165.750262705]: Param camera_i_mx_id with value
[DEBUG] [1695047165.752705438]: Param camera_i_ip with value
[DEBUG] [1695047165.755099328]: Param camera_i_usb_port_id with value

Hi, sorry for the delay as I was on vacation. Could you check if there are any leftover ROS processes after killing the camera?

Hi @Serafadam yes, there was a camera-related node leftover. Even though I killed it manually, I did not manage to launch the camera afterward.

Hi, in new release we have fixed an issue with powercycling PoE cameras, this might also help here, it will be available with the next release (you can test it by using DepthAI 2.26.0 from ros-testing PPA)

Hello @Serafadam!
Our team is encountering the same issue with this board, even when using the latest bootloader. We performed some experiments, and even with a minimal pipeline and no cameras attached, the issue still occasionally occurs.
#include <rclcpp/rclcpp.hpp>

#include "depthai_plugins/depthai_custom_pipeline_plugin.hpp"

#include <depthai_ros_driver/pipeline/base_pipeline.hpp>
#include <depthai_ros_driver/dai_nodes/base_node.hpp>
#include <depthai_ros_driver/dai_nodes/sensors/rgb.hpp>
#include <depthai_ros_driver/dai_nodes/sensors/mono.hpp>
#include <depthai/pipeline/Pipeline.hpp>

std::vector<std::unique_ptr<depthai_ros_driver::dai_nodes::BaseNode>>
depthai_plugins::CustomPipeline::createPipeline(
  std::shared_ptr<rclcpp::Node>/*node*/,
  std::shared_ptr<dai::Device>/*device*/,
  std::shared_ptr<dai::Pipeline>/*pipeline*/,
  const std::string & /*nnType*/)
{
  std::vector<std::unique_ptr<depthai_ros_driver::dai_nodes::BaseNode>> daiNodes;

  return daiNodes;
}

#include <pluginlib/class_list_macros.hpp>
PLUGINLIB_EXPORT_CLASS(
  depthai_plugins::CustomPipeline,
  depthai_ros_driver::pipeline_gen::BasePipeline)

camera:
  oak_top:
    ros__parameters:
      camera:
        i_calibration_dump: false
        i_enable_imu: false
        i_enable_ir: false
        i_nn_type: none
        i_pipeline_dump: false
        i_pipeline_type: depthai_plugins::CustomPipeline
        i_tf_camera_model: OAK-FFC-4P
        i_publish_tf_from_calibration: false

We called SIGTERM repeatedly on ROS (humble), and on approximately every 10th to 15th attempt, we completely lost the device until a full power cycle was performed (lsusb also did't list it).
Thanks in advance for your help!

Hi, @RBT22, does the issue also occur when restarting stereo_inertial_node/ a pure python/c++ code?

Thank you for your response! I performed additional tests using the pure C++/Python libraries, and I was unable to reproduce the issue.

Thanks for verifying, does the issue also occur on different DDS implementations?

Hi, @Serafadam, I am working on debugging our cameras with @RBT22 . We mainly use rmw_cyclonedds_cpp where the aforementioned bug occurred. I have tested the depthai_ros_driver::Camera composed node in a container with rmw_fastrtps_cpp and after 3-14 launch and SIGTERM cycles the OAK-FFC-4P drops and lsusb is not able to list it (unless I disconnect-reconnect it manually). (Also tried with SIGKILL and it is the same.)


Add static_argnums to nn.checkpoint

What does this PR do?
Fixes #2452. Adds a static_argnums to nn.checkpoint and pipes it to jax.remat, copies argument docstring from JAX.

I want to add another test, lets wait before merging.

I discussed this PR with @jheek, final behaviour:

subtracts -1 from each static_argnum in transforms.checkpoint because self (the Module) is never passed to the lifted transformation.
adds +2 to each static_argnum in lift.checkpoint because 2 arguments (variable_groups, rng_groups) are always passed to the inner function in the lifted transformation.

Added an additional test. Should be ready for a final review + internal merge.

Codecov Report

Merging #2457 (f384b33) into main (e320e11) will increase coverage by 0.00%.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##             main    #2457   +/-   ##
=======================================
  Coverage   79.66%   79.67%
=======================================
  Files          49       49
  Lines        4982     4984    +2
=======================================
+ Hits         3969     3971    +2
  Misses       1013     1013

Impacted Files
Coverage Δ

flax/core/lift.py
95.81% <100.00%> (+<0.01%)
:arrow_up:

flax/linen/transforms.py
94.06% <100.00%> (+0.02%)
:arrow_up:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


Use ZIP file size metadata to allocate string

Summary
We spend about 1.4% in VendoredFileSystem::read_to_string. Let's pre-allocate the string with the right size by using the zip file metadata
as a heuristic (don't trust it!)
Test Plan
cargo test

Hmm. In https://github.com/astral-sh/ruff/pull/11863#discussion_r1639337339, you argued against doing this

Yeah, I now saw in the benchmarks (still need to verify that it helped) that I was wrong.
Reading through the code:
    fn read_to_string(&mut self, buf: &mut String) -> Result<usize> {
        default_read_to_string(self, buf, None)
    }

the last argument is the size hint... Oh well, it is None


SharpDX.Direct3D11.ShaderResourceViewDescription.BufferResource ElementCount and ElementWidth members alias

Setting one member changes also the other. At least in the NuGet packet version 4.1
Sorry, I'm quite new to SharpDX, I couldn't manage to build it from source on my own. Ohterwise I would have checked where this behaviour is coming from.

Thats correct as the are basically union, even in C/C++:
typedef struct D3D11_BUFFER_SRV {   union {     UINT FirstElement;     UINT ElementOffset;   };   union {     UINT NumElements;     UINT ElementWidth;   }; } D3D11_BUFFER_SRV;
`
[StructLayout(LayoutKind.Explicit)]
public struct BufferResource
{
///
///   Number of bytes between the beginning of the buffer and the first element to access.
///
/// ff476094
/// FirstElement
/// FirstElement
[FieldOffset(0)]
public int FirstElement;
/// <summary>
/// <dd> <p> The offset of the first element in the view to access, relative to element 0. </p> </dd>
/// </summary>
/// <msdn-id>ff476094</msdn-id>
/// <unmanaged>ElementOffset</unmanaged>
/// <unmanaged-short>ElementOffset</unmanaged-short>
[FieldOffset(0)]
public int ElementOffset;

/// <summary>
/// <dd> <p> The total number of elements in the view. </p> </dd>
/// </summary>
/// <msdn-id>ff476094</msdn-id>
/// <unmanaged>NumElements</unmanaged>
/// <unmanaged-short>NumElements</unmanaged-short>
[FieldOffset(4)]
public int ElementCount;

/// <summary>
/// <dd> <p> The width of each element (in bytes). This can be determined from the format stored in the shader-resource-view description. </p> </dd>
/// </summary>
/// <msdn-id>ff476094</msdn-id>
/// <unmanaged>ElementWidth</unmanaged>
/// <unmanaged-short>ElementWidth</unmanaged-short>
[FieldOffset(4)]
public int ElementWidth;

}
`

That makes totally sense to me now. I should have checked the d3d11 documentation before posting. Apologies for bothering you with that (, but at least I know now that I did the SRV setup in my code incorrect). Thank you very much!

No problem :) Glad to help out.


transitionTo parsing route name instead of URL first

Given
this.transitionTo('http://xxx.local/downtown/');
Results in
Uncaught Error: Invariant Violation: Cannot find a route named "http://xxx.local/downtown/"
But should navigate client-side and attempt to match a route. How can I fix this?
J

You can't use full URLs with transitionTo or replaceWith.
Change it to this.transitionTo('/downtown/'); .

Thanks.
Might want to change this in the docs:
Examples
this.transitionTo('user', {id: 10}, {showAge: true});
this.transitionTo('about');
this.transitionTo('/users/10?showAge=true');
this.transitionTo('http://example.com/users/10?showAge=true');

new docs don't have this, thanks :)


GHC 8.6 compatibility

We'd like to compile with 8.6, especially since this the major version planned to be used for the Shelly release.
However, there is at least one blocker, namely we are affected by https://ghc.haskell.org/trac/ghc/ticket/16104.

Done.


unix: Add readv/writev for illumos

iovec read/writes (readv, writev, preadv, pwritev) are present on illumos,
but they weren't added to the package. This commit added those missing
features.
Updates golang/go#36201

@googlebot I signed it!

@googlebot I fixed it..


Deconvolution is super slow

Hello all.
Thank you for a great tool.
I wonder, is there any way to speed up the computations? Layer deconvolution takes ~7 seconds in my case and I want to calculate deconvolutions for around ~10k examples, the network is of 7 layers.
BR,
Alexander

You could run the experiment in batches. Using a large batch size would be useful here to speed up the computations. We haven't experimented extensively with this, but the outputs of a batch would get written to disk together in an image.
Let us know if this helps.

Hmm, when I feed a batch with multiple inputs, the tool still generates a single image for each layer... I don't understand the principle here.

Hi @kitovyj
We are working on this. We'll roll out a version where you should be able to insert multiple images simultaneously. You are right - we revisited the code and in its current state it would return just the first image output.


[Agent] Modify l7 rrt cache capacity

This PR is for:

Agent

Modify l7 rrt cache capacity
Checklist

[ ] Added unit test.

Backport to branches

main
6.2
6.1

close #2620


assignment.gr.md

assignment lesson 1 greek translation

Wow! Would you like to create one big draft PR and progressively add the translations to that ( see the other PRs here for an example) thank you 🙏!

Wow! Would you like to create one big draft PR and progressively add the translations to that ( see the other PRs here for an example) thank you 🙏!

Yes, i think i could help with that. Althought i'm busy this period and my PR propably won't be  very consistent, at least for the moment.
I would also like to make a question(i'm brand new in Git and GitHub). Do i have to do/change something for the unsuccessful checks(Azure Static Web Apps CI/CD / Build and Deploy Job (pull_request))? Is it my fault? Thanks a lot :)

Yes, i think i could help with that. Althought i'm busy this period and my PR propably won't be  very consistent, at least for the moment.
I would also like to make a question(i'm brand new in Git and GitHub). Do i have to do/change something for the unsuccessful checks(Azure Static Web Apps CI/CD / Build and Deploy Job (pull_request))? Is it my fault? Thanks a lot :)

Wow! Would you like to create one big draft PR and progressively add the translations to that ( see the other PRs here for an example) thank you 🙏!

Yes, i think i could help with that. Althought i'm busy this period and my PR propably won't be  very consistent, at least for the moment.
I would also like to make a question(i'm brand new in Git and GitHub). Do i have to do/change something for the unsuccessful checks(Azure Static Web Apps CI/CD / Build and Deploy Job (pull_request))? Is it my fault? Thanks a lot :)

Wow! Would you like to create one big draft PR and progressively add the translations to that ( see the other PRs here for an example) thank you 🙏!

Yes, i think i could help with that. Althought i'm busy this period and my PR propably won't be  very consistent, at least for the moment.
I would also like to make a question(i'm brand new in Git and GitHub). Do i have to do/change something for the unsuccessful checks(Azure Static Web Apps CI/CD / Build and Deploy Job (pull_request))? Is it my fault? Thanks a lot :)

hi, in that case I will approve this PR and the other one, and you can restart a new draft PR and add to it progressively for the lessons that need to be translated to Greek. Thank you!
Don't worry about the CI/CD job failing; that's the quiz-app build process that fails to build when external PRs occur.
cheers!
Jen

hi, in that case I will approve this PR and the other one, and you can restart a new draft PR and add to it progressively for the lessons that need to be translated to Greek. Thank you!
Don't worry about the CI/CD job failing; that's the quiz-app build process that fails to build when external PRs occur.
cheers!
Jen


Faker::Commerce.promotion_code Missing Translation

Getting a missing translation data error on the latest commit from master.
[2] pry(main)> Faker::Commerce.promotion_code
I18n::MissingTranslationData: translation missing: en.faker.commerce.promotion_code.adjective

@smythey21 Could you please check you are using git revision number 2c0e655b3b8800188a72e3168d66202ae04aa0c6 ? because I was trying to reproduce this issue but it is working for me
[1] pry(main)> Faker::Commerce.promotion_code
=> "SpecialSavings895157"

And which locale you are using?  I am using default.

Strange, it's totally working now. Possible I was referencing the wrong revision. Sorry about that!


refactor: adjust hash for metadata

Thank you for your contribution to the Koda - Generative Art Marketplace.
👇 __ Let's make a quick check before the contribution.
PR Type

[x] Bugfix

Context

[x] Related: https://github.com/kodadot/ops-internal/issues/1659
[x] Need workers deployment: https://github.com/kodadot/private-workers/pull/165

Did your issue had any of the "$" label on it?

[x] My DOT address: Payout

what about this one?

Oh, sorry, I missed the notification about this. will update

PaidMint.vue missing close emit
HolderOfGenerative.vue also needs to be updated like PaidGenerative.vue did

Updated description: I slightly changed the PR.

I reverted the PaidMint.vue; it turns out I got a random reload on a random page, not just a drops page.
CollectionDropModalPaidMint on HolderOfGenerative.vue seems like already the same with PaidGenerative.vue

This PR removes the hash while minting the drop. It means the preview image on the drop page would not be the same as the item he minted finally. There should be something else needs to be implemented on the image preview component. Correct me if I'm wrong.

This PR removes the hash while minting the drop. It means the preview image on the drop page would not be the same as the item he minted finally. There should be something else needs to be implemented on the image preview component. Correct me if I'm wrong.

Correct 👍🏻
I think we can simplify the "Mint Drop" modal. It would reduce some of the code, and the user would not need to wait for the preview to render (especially for massmint).

This PR removes the hash while minting the drop. It means the preview image on the drop page would not be the same as the item he minted finally. There should be something else needs to be implemented on the image preview component. Correct me if I'm wrong.

Correct 👍🏻 I think we can simplify the "Mint Drop" modal. It would reduce some of the code, and the user would not need to wait for the preview to render (especially for massmint).
I am planning to do that in a separate branch (branching from this: #10336)

Great! Let's leave it to separate PR.

i understand that we still got to adjust the modal flow after that change, but the nft i minted using this PR has no image:
https://deploy-preview-10148--koda-canary.netlify.app/ahk/gallery/379-2405285173
Screencast from 2024-05-24 11-02-42.webm

i don't think its working properly @preschian
Screencast from 2024-05-27 11-03-20.webm
its generating images without sn or proper variation:

its generating images without sn or proper variation:
(the one that worked is from canary)

Need to wait it for a moment. Currently, the blockchain itself is unable to mint and set metadata at the same time. Currently, we solve it by updating the metadata (image preview, name, sn, animationUrl, etc.) synchronously. refer: https://github.com/kodadot/private-workers/issues/87#issuecomment-2006716343

its generating images without sn or proper variation:
(the one that worked is from canary)

Need to wait it for a moment. Currently, the blockchain itself is unable to mint and set metadata at the same time. Currently, we solve it by updating the metadata (image preview, name, sn, animationUrl, etc.) synchronously.

oh, cool, ok!


[3.5] Add NavigationAgent2D/3D set_navigation_map() function

Backport of #62044.
Add NavigationAgent2D/3D set_navigation_map() function.
Also fixes some bugs from leftover code before the 3.5 backport update.
Fixes #62103

Thanks!


Carla 0.9.9 crashing way more than previous version

Hello, I am doing Reinforcement Learning in CARLA for more than a year and I have always encountered issues with CARLA crashing time to time (with a Segmentation Fault as mentionned in #1627 or #1227). But generally this was happening really rarely. Usually I have 5/6 instance of CARLA running independently during around 4/5 days and sometimes one of this instance of CARLA would crash and I had to relaunch it.
I recently moved from CARLA 0.9.6 to CARLA 0.9.9 and I discovered that the crashs were happening much more, basically I have trouble to run a CARLA for more than a day without crash. Particularly my script (a slight modification to the spawn_npc.py script) spawning/deleting/respawning all the surrounding autopilot vehicles and pedestrians seems really unstable throwing this exception many time:
$ what(): length > 2.0f * std::numeric_limits::epsilon()
Segmentation fault (core dumped)
As in the script spawn_npc.py I set the traffic manager with those commands:

$ traffic_manager.set_global_distance_to_leading_vehicle(2.0)
$ traffic_manager.set_hybrid_physics_mode(True)

Hi,
Sorry for this issue! Could you provide us some extra information? If possible, running some debugger (like gdb). Or maybe the script to test this?

Hello,
I just made a short version of my script to delete and respawn vehicle and pedestrian every 10 seconds and after just some minutes, CARLA crashs.
I am launching CARLA with this command line:
$ ./CarlaUE4.sh -quality-level=Epic -world-port=2000 -resx=100 -resy=100

@jackbart94 please give high priority to this issue.
@bernatx keep an eye on this.

Hi,
I've been running it for about 30 minutes and can't seem to get any sort of error. I'm on Ubuntu 16.04 with Python 3.7.2 and CARLA <IP_ADDRESS>. Maybe you could try getting this version of CARLA and let me know if you still experience it, or as I said send me the report by using the gdb debugger.

Hi,
Could you detail a bit more what you mean by "using the gdb debugger"?
And this kind or error is hard to detect/reproduce, sometimes it doesn't crash for multiple hours, sometimes it crashs after just 20/30 minutes.
When I am launching my whole reinforcement learning training process, CARLA usually crash after about 20 hours.
But with my minimal script if I let it 2 minutes, then kill and restart it 3 or 4 times every 2 minutes I consistently reach this kind of crash after only 10/15 minutes which reveal in my opinion the non-stability of the process.

Could you detail a bit more what you mean by "using the gdb debugger"?
Sure, please run in the following order:

 $ ./CarlaUE4.sh -quality-level=Epic -world-port=2000 -resx=100 -resy=100
(in another terminal) gdb python
(in the same terminal as command number 2) run rl/rainbow_ign_apex/issue_carla.py
Then, once it crashes, in the same terminal as command number 2 just write bt and it will print the stack trace of the error.

Also, I've been running it and still it won't crash.. and your seg fault in this screenshot doesn't show the error you mentioned.

For exemple, this time I got the issue after only 30 seconds...

And yes this is not the exact same error... When I am launching the RL training I get the error I mentioned (but I don't want to run it for multiple days, I just wanted to give one exemple of possible crash...).
With my minimal script however, I succeed to get a Segmentation Fault consistently and I assume this is related to the same "unstability" problems.
My main question actually is why this doesn't give any Python exception? Like this we could catch each time there is a crash and restart everything automatically and properly. In the current state I just can't do anything from Python side...

Did you try switching to CARLA <IP_ADDRESS> by any chance? Or are you still on 0.9.9?

My main question actually is why this doesn't give any Python exception?
Because it's an error in the C++ API somewhere.

Let me know when it's done, hopefully it was enough to solve this.
Also regarding <IP_ADDRESS>, it's still worth keeping it, even though we hadn't seen this error yet.

I encounters the error $ what(): length > 2.0f * std::numeric_limits::epsilon() as well from time to time on 0.9.9 release. I quickly checked the relevant code and it seems to be used to computer the forward_vector, as a sanity check for zero dividing before making a unit vector. I'm not sure if that is the cause of Segmentation fault (core dumped), but instead of sanity check an addition of eps seems to be more sensible in this case.

Hi all,
I encountered a similar issue on both 0.9.9 and on the latest docker image. I'm running long simulations to collect data from several sensors. To give you an idea, here you have some of the settings:

FPS: 60
Synchronous mode
4 RGB cameras
3 LiDARs
1 segmentation camera
Duration (simulation time): 60 seconds
Simulation runtime: ~1h30min on an NVIDIA 2080 Ti
This is for a single scene, which consists of 3600 frames.
As I am trying to get diverse scenes, I loop over the available maps, changing the map with client.load_world() while keeping the same Carla server running on the background and the same traffic manager and Carla client.
After a while (it's not deterministic as far as I have observed) I get a Segmentation fault      (core dumped).

I am not getting the $ what(): length > 2.0f * std::numeric_limits::epsilon() error log so I'm not sure it's the same issue. It seems to be linked with lengthy simulations (possibly something to do with the frame ID which keeps increasing? - would be nice to have a way to reset the server from the client-side). Please let me know if you want me to open another issue, or if I missed a more relevant one.
I can provide you additional information or a sample script in the next days if needed. (I have very limited C++ knowledge)
Thanks for the amazing work!

Hi all,
I encountered a similar issue on both 0.9.9 and on the latest docker image. I'm running long simulations to collect data from several sensors. To give you an idea, here you have some of the settings:

FPS: 60
Synchronous mode
4 RGB cameras
3 LiDARs
1 segmentation camera
Duration (simulation time): 60 seconds
Simulation runtime: ~1h30min on an NVIDIA 2080 Ti

This is for a single scene, which consists of 3600 frames.
As I am trying to get diverse scenes, I loop over the available maps, changing the map with client.load_world() while keeping the same Carla server running on the background and the same traffic manager and Carla client.
After a while (it's not deterministic as far as I have observed) I get a Segmentation fault (core dumped).
I am not getting the $ what(): length > 2.0f * std::numeric_limits::epsilon() error log so I'm not sure it's the same issue. It seems to be linked with lengthy simulations (possibly something to do with the frame ID which keeps increasing? - would be nice to have a way to reset the server from the client-side). Please let me know if you want me to open another issue, or if I missed a more relevant one.
I can provide you additional information or a sample script in the next days if needed. (I have very limited C++ knowledge)
Thanks for the amazing work!

Hi,
it's probably not the same issue, but I encourage you to either open a new GitHub issue with more details, or directly contact us on the CARLA Discord server, so that we can look a bit more into the problem!

@marcgpuig I believe we have encounter the " what(): length > 2.0f * std::numeric_limits::epsilon()" issue before. Do you have anything to add?

Hi, I am also doing RL on Carla and have been experiencing the same issue. I am running the same scripts all the time and sometimes it crashes after 5-6 hours, sometimes after 10 hours and so on!! I have no idea why.
I tried to restart Carla every 100-200 episodes which is like almost 1 hour of running but after doing this several times, the simulation crashes with a segmentation error. Therefore, I can't kill and restart Carla lets say more than 10 times. I have to restart Ubuntu and run it again which is so annoying.  Any ideas would be appreciated at this point. I can send my code here if you guys need it. Thanks!

Btw, I mostly used Carla 0.9.8.

@zhejun-zhang can you link me the line of code of the length > 2.0f * std::numeric_limits::epsilon()? I'm not able to find it. Maybe it's Unreal's code?

@marcgpuig I found this line could be relevant. But since I switched to <IP_ADDRESS> I haven't seen any crash due to this issue.

I still got this issue time to time and I switched to CARLA <IP_ADDRESS>.

Hi,
So yeah

@marcgpuig I found this line could be relevant. But since I switched to <IP_ADDRESS> I haven't seen any crash due to this issue.

this line might be what's causing it, however it's because you should be using MakeSafeUnitVector. We'll take a look to see if there is something that still might use the other function.

Hi @jackbart94 ,
I'm not sure if that is exactly where the error is thrown. What I did is just searching for length > 2.0f * std::numeric_limits<float>::epsilon() int this repository (GitHub search which you find on the top left corner).
And regarding usage. I use only PythonAPI and I don't know which function call leads to this error

@marintoro , hi! Do you solve this problem? I countered the same problem when training my RL agent with carla 0.9.9(docker).

@marintoro , hi! Do you solve this problem? I countered the same problem when training my RL agent with carla 0.9.9(docker).


Draft ROpenSci Checklist

Below is the demo ROpenSci issue list. Let me know what looks incorrect. The 3 items I'm unsure on are:

Package categories,
The technical checkbox for continuous integration (CI)? We didn't reference the usethis package at all, or Travis--but my understanding in this area is limited.
Confirmation that we intend for this package to go onto CRAN afterwards? I ask because the testing section on the ROpenSci dev guide talks about a skip_on_cran() command that we'll seem to need if we are to submit there...

Submitting Author: Austin Koontz (@akoontz11 )
Repository: https://github.com/pearselab/SymbiotaR2
Version submitted: 0.0.1
Editor: TBD
Reviewer 1: TBD
Reviewer 2: TBD
Archive: TBD
Version accepted: TBD

Paste the full DESCRIPTION file inside a code block below:

Package: SymbiotaR2
Type: Package
Title: Downloading data from Symbiota2 portals into R
Version: 0.0-1
Authors@R: c(person("Austin", "Koontz", email =<EMAIL_ADDRESS>role = c("aut", "cre")),
    person("William D.", "Pearse", email =<EMAIL_ADDRESS>role ="aut", comment=c(ORCID="0000-0002-6241-3164")))
Maintainer: Austin Koontz<EMAIL_ADDRESS>Description: Download data from Symbiota2 portals using Symbiota's
    API. Covers the Checklists, Collections, Crowdsource, Exsiccati,
    Glossary, ImageProcessor, Key, Media, Occurrence, Reference, Taxa,
    Traits, and UserRoles API families. Each Symbiota2 portal owner
    can load their own plugins (and modified code), and so this
    package may not cover every possible API endpoint from a given
    Symbiota2 instance.
License: MIT + file LICENSE
URL: https://github.com/pearselab/SymbiotaR2
BugReports: https://github.com/pearselab/SymbiotaR2/issues
VignetteBuilder: knitr
LazyLoad: yes
Suggests:
    knitr (>= 1.23),
    testthat (>= 2.0.0),
    vcr (>= 0.3.0)
Imports:
    httr (>= 1.0.0),
    xml2 (>= 1.2.0),
    rjson (>= 0.2.20),
    fs (>= 1.2.6)
Encoding: UTF-8
RoxygenNote: 7.0.2

Scope

Please indicate which category or categories from our package fit policies this package falls under: (Please check an appropriate box below. If you are unsure, we suggest you make a pre-submission inquiry.):

[x] data retrieval
[ ] data extraction
[x] database access
[ ] data munging
[ ] data deposition
[ ] workflow automataion
[ ] version control
[ ] citation management and bibliometrics
[x] scientific software wrappers
[ ] database software bindings
[ ] geospatial data
[ ] text analysis

Explain how and why the package falls under these categories (briefly, 1-2 sentences):

The primary purpose of SymbiotaR2 is to allow users to pull objects from the specified Symbiota2 database portal into an R environment. It does this by wrapping each of the default API calls made available on the Symbiota2 portal of interest.

Who is the target audience and what are scientific applications of this package?

The target audiences includes anyone interested in using R to analyze biological specimen data made available through the Symbiota2 content management system. Potential scientific applications include comparing biological trait data between different lineages, analyzing phenological data, and biodiversity research generally.

Are there other R packages that accomplish the same thing? If so, how does yours differ or meet our criteria for best-in-category?

No R packages have been developed for the updated code structure provided by Symbiota2.

If you made a pre-submission enquiry, please paste the link to the corresponding issue, forum post, or other discussion, or @tag the editor you contacted.

Technical checks
Confirm each of the following by checking the box.  This package:

[x] does not violate the Terms of Service of any service it interacts with.
[x] has a CRAN and OSI accepted license.
[x] contains a README with instructions for installing the development version.
[x] includes documentation with examples for all functions.
[x] contains a vignette with examples of its essential functions and uses.
[x] has a test suite.
[ ] has continuous integration, including reporting of test coverage using services such as Travis CI, Coveralls and/or CodeCov.

Publication options

[x] Do you intend for this package to go on CRAN?
[x] Do you wish to automatically submit to the Journal of Open Source Software? If so:

JOSS Options

[x] The package has an obvious research application according to JOSS's definition.

[x] The package contains a paper.md matching JOSS's requirements with a high-level description in the package root or in inst/.
[ ] The package is deposited in a long-term repository with the DOI:
(Do not submit your package separately to JOSS)

[ ] Do you wish to submit an Applications Article about your package to Methods in Ecology and Evolution? If so:

MEE Options

[ ] The package is novel and will be of interest to the broad readership of the journal.
[ ] The manuscript describing the package is no longer than 3000 words.
[ ] You intend to archive the code for the package in a long-term repository which meets the requirements of the journal (see MEE's Policy on Publishing Code)
(Scope: Do consider MEE's Aims and Scope for your manuscript. We make no guarantee that your manuscript will be within MEE scope.)
(Although not required, we strongly recommend having a full manuscript prepared when you submit here.)
(Please do not submit your package separately to Methods in Ecology and Evolution)

Code of conduct

[x] I agree to abide by rOpenSci's Code of Conduct during the review process and in maintaining my package should it be accepted.

I've added "data retrieval" to the categories. We will have Travis-CI (running now, although it will fail because not all the tests work). Once that's up and running, I'll setup Coveralls. I'll show you how I do that once it's done; it's fiddly... We don't need to worry about the skip_on_cran stuff because we're using vcr to circumvent that.