tests.trivial is always a changed path

There is an issue in the test added by #123111. This line
https://github.com/NixOS/nixpkgs/blob/cc60f81e69154789ad3e23f0ec60110ed3f7aece/pkgs/build-support/trivial-builders/test.nix#L16
introduces a dependency on the contents of nixpkgs, making every change evaluate with a different hash.
cc @roberth @Profpatsch

Steps to reproduce:
# nix-instantiate https://github.com/NixOS/nixpkgs/archive/de8dbb21fbf740927ad2888bac59215a92a64ee0.tar.gz -A tests.trivial
warning: you did not specify '--add-root'; the result might be removed by the garbage collector
/nix/store/3kllg4j6rxjbsd4fdii7cg143wm5x8bq-vm-test-run-nixpkgs-trivial-builders.drv
# nix-instantiate https://github.com/NixOS/nixpkgs/archive/56f0c92fadf0aa839ecc75db738d2e1f3ffb1e14.tar.gz -A tests.trivial
warning: you did not specify '--add-root'; the result might be removed by the garbage collector
/nix/store/3737mgd1drcid4va2df3cs8w0rdqdi73-vm-test-run-nixpkgs-trivial-builders.drv
# nix-diff /nix/store/3kllg4j6rxjbsd4fdii7cg143wm5x8bq-vm-test-run-nixpkgs-trivial-builders.drv /nix/store/3737mgd1drcid4va2df3cs8w0rdqdi73-vm-test-run-nixpkgs-trivial-builders.drv
- /nix/store/3kllg4j6rxjbsd4fdii7cg143wm5x8bq-vm-test-run-nixpkgs-trivial-builders.drv:{out}
+ /nix/store/3737mgd1drcid4va2df3cs8w0rdqdi73-vm-test-run-nixpkgs-trivial-builders.drv:{out}
• The input derivation named `nixos-test-driver-nixpkgs-trivial-builders` differs
  - /nix/store/qwdqd88wrr3irnrihxmn69l4x6m29nfm-nixos-test-driver-nixpkgs-trivial-builders.drv:{out}
  + /nix/store/r81nhqj6rai1m1qs4flw04f54djwj455-nixos-test-driver-nixpkgs-trivial-builders.drv:{out}
  • The input source named `source` differs
  • The environments do not match:
      testScript=''
      machine.succeed("""
        cd /nix/store/qgfcfww929y9s5527xixhlhfs4xxbwxr-source/nix/store/6sr5a0lgaqgp94vava5g6dh85ghjn3h0-source
        ./pkgs/build-support/trivial-builders/test.sh 2>/dev/console
      """)

  ''
• Skipping environment comparison

You're absolutely right, but I don't think I see the impact of this fact.
It's not recursed into (no recurseIntoAttrs/recurseForDerivations) and it's not even built by ofborg. #123369
The test needs to compare nix-store -q results, but perhaps the actual evaluation can be removed from it; injected as store paths by the test runner instead. It won't make the test easier to read :(

@roberth This currently impacts every ofborg evaluation. For example: https://gist.github.com/GrahamcOfBorg/da36c4dce51ab35a635f77a9becf364f

I have no idea how to solve this. Why would ofborg have a different hash for multiple evaluations? Cause it puts stuff in the nixpkgs path?

@Profpatsch pkgs.path is the whole Nixpkgs source tree, which kinda always changes. Here's a solution #123597. Not pretty but still quick to run outside the sandbox and solves this issue.

Still an issue https://gist.github.com/GrahamcOfBorg/e62104c50cacbf1a22b807a05beba770 ?

Ofborg doesn't rebase PRs so if the PR is based on a master that's of the wrong age, you'll still see this happen. Please check that the PR includes the improved test.

@roberth I did, and it's been more than 12 hours since the commit landed.
At the same time my original repro doesn't see the issue:
# nix-instantiate https://github.com/NixOS/nixpkgs/archive/8e9d803bac9faaa6fcd619ab2131375b7e2be64d.tar.gz -A tests.trivial
unpacking 'https://github.com/NixOS/nixpkgs/archive/8e9d803bac9faaa6fcd619ab2131375b7e2be64d.tar.gz'...
warning: you did not specify '--add-root'; the result might be removed by the garbage collector
/nix/store/xy4x1frfxfxzbxw3kf5d4zcpn5pr4f4s-vm-test-run-nixpkgs-trivial-builders.drv
# nix-instantiate https://github.com/NixOS/nixpkgs/archive/68c618cba3855568e9a3194823354e049635666d.tar.gz -A tests.trivial
unpacking 'https://github.com/NixOS/nixpkgs/archive/68c618cba3855568e9a3194823354e049635666d.tar.gz'...
warning: you did not specify '--add-root'; the result might be removed by the garbage collector
/nix/store/xy4x1frfxfxzbxw3kf5d4zcpn5pr4f4s-vm-test-run-nixpkgs-trivial-builders.drv

It's a broader issue that applies to all NixOS tests, https://github.com/NixOS/nixpkgs/pull/117275, but it went under the radar.

This seems to be fixed now. @roberth Thanks for looking into this!


Backup instructions

Whenever clicking on a "backup" button or warning message, we always start with this page as the first page of the backup flow.
There are two pages depending on whether or not the wallet has a passphrase.
Text:
(No passphrase)

If you lose your 12 word backup, you will not be able to recover access to the Bitcoin Wallet.
Without a backup, if you lose or break your phone, or if you uninstall the Bull Bitcoin app, your bitcoins will be lost forever.
Anybody with access to your 12 word backup can steal your bitcoins. Hide it well.
Do not make digital copies of your backup. Write it down on a piece of paper, or engraved in metal.
Your backup is not protected by passphrase. Add a passphrase to your backup later by creating a new wallet.

(passphrase)

Your backup is protected by passphrase.
Without a backup, if you lose or break your phone, or if you uninstall the Bull Bitcoin app, your bitcoins will be lost forever.
Anybody with access to both your 12 word backup and your passphrase can steal your bitcoins. Hide them separately.
If you lose your 12 word backup or your passphrase, you will not be able to recover access to the Bitcoin Wallet. Both the 12 words and the passphrase are required.
Do not make digital copies of your backup and passprhase. Write them down separately on a piece of paper, or engraved in metal.

@mocodesmo the text does not entire match and does not change based on whether passphrase exists or not.

@i5hi
this is working fine pls check again
the text is copied from the bullet points on top not the screenshot

@mocodesmo no passphrase wallet backup text is fine now.
Passphrase wallet backup content does not match.
For both cases, the checkbox does not work.

Passphrase wallet unable to view backup.
No Key Error.

Passphrase bug fixed.

Content matches.


Not scrolling to selected time when portalId set

Describe the bug
For a DatePicker instance where showTimeSelect=true, the selected time is automatically scrolled to/focused except when portalId is supplied.
To Reproduce
https://codesandbox.io/s/patient-glade-vhvqq1?file=/src/App.js
Steps to reproduce the behavior:

Go to codesandbox example.
Open the DatePicker instance without portalId set and see that the menu opens with 12:00PM scrolled to/focused.
Open the DatePicker instance with portalId set and see that the menu does not open with 12:00PM scrolled to/focused.

Screenshots

I've been hit with this issue too. I have found a temporary workaround...
The reason we need the portalId, is to circumvent layout issues (where the popper is partially covered in many cases) because of the styling used by parent containers. I found a way of getting around these issues without depending on the portalId using css. I am concerned it might have other side effects, but as a temporary work around it appears to be working well.
I have created a working stackblitz example where you can enable/disable the portalId and the fix
https://stackblitz.com/edit/react-fnkalf?file=src%2FApp.js

Fix above unfortunately is not ideal. The popper fails to flip the positioning when there isn't enough space below the component.


chore(tutone): update generated code

Update generated code using tutone

make generate
PR generated by create-pull-request

DO NOT MERGE

DO NOT MERGE


Please provide NuGet packages from free components

We were trying to use a component ( https://components.xamarin.com/view/jsqmessagesviewcontroller ) but some of the developers in our team have issues with authentication in to Xamarin Components Store in Visual Studio. I understand that components is more of a deprecated way of publishing packages. Is it possible to publish this component and maybe others as a NuGet package?

hi @SamuelDebruyn,
We have moved JSQMessagesViewController to the public repo and published it to nuget
https://www.nuget.org/packages/Xamarin.JSQMessagesViewController/
You can see, and contribute to the code, here
https://github.com/xamarin/XamarinComponents/tree/master/iOS/JSQMessagesViewController
Thanks


retry connection patch

This patch adds several retries to connect before giving up. This came up because when switching from wifi to 3G, it takes some time for the 3G network to come up, and the first reconnect attempt fails with the "host unreachable" error. With this patch connectbot becomes quite robust with regards to connections.
I'm not sure if this patch has any undesirable side effects, but in such a case I would be happy to add it as a configuration option to be enabled only by those who need it.
And lastly, this is my first time writing java, and only after looking at the SDK for a few days, so I apologize if I made any glaring mistakes, and would be happy to hear any criticism and imporve.
Oh, and thanks for connectbot!
Shai

I understand this is a 5 year old pull request, however the issue still exists on 1.8.6 (latest version available on the Play Store) and this pull request might solve it. Could somebody pick this up and merge it? It would be really useful.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Create poc for distributed control

@mamueluth remember to only add the created node once if use_multiple_nodes = false


Should windows archives be something other than .tar.gz

It seems like using 7-zip is the reliable way to expand a .tar.gz archive on windows. However, it looks as though tools like winrar appear to make a best attempt that produces unusable binaries. In order to avoid confusion we should either consider distributing .zip archives for the windows distributions or add some documentation somewhere explaining the recommended install steps on windows.

Looks like Windows 10 with some updates (or newer) comes with tar installed, so this might just work cmd.exe -> tar xf benthos_x.y.z_windows_amd64.tar.gz but, I must admit, I'd be shocked if people won't have to click some buttons first to get that to work. Would be great if someone can test this, since I only have an old Windows VM.

Looks like Windows 10 with some updates (or newer) comes with tar installed, so this might just work cmd.exe -> tar xf benthos_x.y.z_windows_amd64.tar.gz but, I must admit, I'd be shocked if people won't have to click some buttons first to get that to work. Would be great if someone can test this, since I only have an old Windows VM.


IsMenuCurrent() returns false if menu item defined inside config file

It appears that IsMenuCurrent or HasMenuCurrent do not return true everytime it should.

It seems to work for parent/children.
It seems to always work on homepage.
It seems to work on menu items defined inside the page/post's frontmatter.

But it seems not to work on menu items defined inside the main configuration file.
Is this a confirmed bug? At least this is the case for me.
Reference of discussions this is based upon:

#2556
[discuss] Current navigation
(amongst others)

Is this a confirmed bug? At least this is the case for me.

No, but as it depends on the URL you set in config to do the matching, it may be hard to get right. Unless you provide a complete and failing example, there is nothing to do here.

I just uploaded a working site where menus do not work: https://github.com/lebarde/apo
I generated the content using hugo --buildDrafts.
As you can see here, the home button is active on the homepage. But on the biographie/ page the biographie menu link is not marked as active.
Note that this is the case if I create the menu in the pages' front matters.

One nice resolution would be the ability to define menu items by their identifiers or their routes (I assume that identifiers is easier to implement). In that way you do
menu:
  main:
    - Name: Home
      Title: My home title
      URL: /
      Weight: 10
    - Name: "Biographie"
      Identifier: "biographie"
      Weight: 20

Because IMO the page file has to define its identifier (or it is defined automatically, which is the same for me), and then you define the menu globally. This is for me a good way to have an overview of the whole menu.

Hi all!
I keep experiencing the same problem. Don’t know whether it is in issue or a problem of understanding.

Unless you provide a complete and failing example, there is nothing to do here.

Attached you find an example. “Home“ and “blog” are provided with a cur class (-> bold text) when active; contact is not.
Thanks!
zipped.zip

I am seeing the same issue. While my code isn't the exact same as @Brixy - I have a similar setup. Menu is defined in the config file - renders out just fine but IsMenuCurrent fails to work.

I'm facing the same issue with Hugo 0.47.1

Solved this for myself just now by adding trailing slashes to my menu items in the config! I didn't realize it was matching by URL, but using url = "/issues/" instead of url = "/issues" did the trick.

When defining multilingual menus the IsMenuCurrent function does not work as expected.
Expected result: Both menus get the "active" class when navigating to the respective page.
Result: Only the defaultContentLanguage menu gets the active class.
Temporary fix: set the "ja" menus URL to url    = "/ja/"
config.toml:
defaultContentLanguage = "en"

[languages.en]
weight = 0
languageName = "English"

[[languages.en.menu.main]]
identifier = "home"
url    = "/"
title   = "Home"
weight = 0

[languages.ja]
weight = 10
languageName = "日本語"

[[languages.ja.menu.main]]
identifier = "home"
url    = "/"
title   = "ホーム"
weight = 0

menu partial:

    {{ $currentPage := . }}
    {{ range .Site.Menus.main }}

        	{{ .Name }}

    {{- end }}

Please don't comment on old, closed issues.  Use the discussion forums or open a new issue.


Display Total Account Value

Display total account value (Lending, Margin, Exchange) in BTC and Output currency
Description
Implements issue https://github.com/Mikadily/poloniexlendingbot/issues/240
TESTING STAGE
In progress
Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[X] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

[X] I have read CONTRIBUTING.md
[X] I fully understand Github Flow.
[x] My code adheres to the code style of this project.
[ ] I have updated the documentation in /docs if I have changed the config, arguments, logic in how the bot works, or anything that understandably needs a documentation change.
[ ] I have updated the config file accordingly if my change requires a new configuration setting or changes an existing one.
[X] I have tested the bot with no issues for 24 continuous hours. If issues were experienced, they have been patched and tested again.

TODO:
Add configuration option to enable/disable the bot from adding this information to the output.

Looks good so far, will test and review when you are done.

Being over a year old.... is this still in progress or is it time to close it?

@rnevet Missing Bitfinex support

Tried writing a quick patch for Bitfinex. Seems more trouble than it's worth. You'd have to add any/all coins back into all_currencies if you want the values calculated because Bitfinex does not return the estimated BTC value like polo does. So you have to first fetch all coin balances from each account type, fetch individual tickers for each coin (or use v2 API which allows requesting multiple), then calculate each coin's value. I had this somewhat working but my patch caused lending to stop so I gave up.


Inventory overlay extension function

What has been done?

Extension functions add to simplify the opening and sending of inventory overlay containers
Price guide has been mildly refactored to use new extension function

Will do something different


converts

Description
I've found that <dom-bind> converts <template is="dom-if"> to <dom-if>; That messes up the table's cells.
Live Demo
The example is here: https://jsfiddle.net/qjgxkyj2 -- please notice that  does not span over 2 cells correctly. -- if we inspect the table cells, there are a bunch of <dom-if>.
However, if we use <template is="dom-bind">, everything works perfectly. The example is here: https://jsfiddle.net/5an34xz0/ -- if we inspect the table cells, there is no <dom-if>.
Browsers Affected

[x] Chrome
[ ] Firefox
[ ] Edge
[ ] Safari 9
[ ] Safari 8
[ ] IE 11

Versions

Polymer: v2.0.2
webcomponents: v1.0.8

See (#4704 (comment))[https://github.com/Polymer/polymer/issues/4704#issuecomment-310155087]

The comment doesn't work for me. In the example (https://jsfiddle.net/qjgxkyj2/), <template is="dom-if"> is used... though its parent is <dom-bind>.
I have to use <dom-bind> because it's a glue between the backend and Polymer's elements. Here's how I use it:
<dom-bind id="page">
  <template>
    <our-page data="[[data]]"></our-page>
  </template
</dom-bind>

Please let me know if there's a workaround.

Your example is slightly incorrect: The <template is="dom-if"> elements do not actually work in your first example. Here's a fork of your first example where the firstCell and thirdCell properties are defined and set to true, yet never appear: https://jsfiddle.net/qfjq8rvg/
In the second example you have operational dom-if elements, which is why they appear in the element. Dom-if works by inserting the contents of the template it is given above itself, and therefore will always show the behavior you are seeing.
You should probably use a binding to the hidden attribute instead.

Hmm... this is strange. firstCell and thirdCell do show up when they are true. Here's the example (https://jsfiddle.net/9vv4t9j9). Here's the snippet:
<base href="https://polygit.org/polymer+polymer+v2.0.2/webcomponentsjs+webcomponents+v1.0.8/components/">

<link rel="import" href="polymer/polymer.html">

<dom-bind id="page">
  <template>

Span over 2 cells

      <template is="dom-if" if="[[firstCell]]">
        First cell
      </template>
      Second cell
      <template is="dom-if" if="[[thirdCell]]">
        Third cell
      </template>
      Fourth cell

</template>
</dom-bind>

Here's the screenshot:

You should probably use a binding to the hidden attribute instead.

This workaround works. Example: https://jsfiddle.net/p5bxwe51/  --- thank you for suggesting it.

I think in this case there's not a lot we can do, as
 is a somewhat unique element in the parser requiring certain types of elements to be children.
In that case the hidden attribute should always be the preferred way to control visibility.
dom-if is better suited to larger sections of elements that can avoid being fully set up on the first boot (where the dom-if is false by default).

Just want to note that there is a legit issue here, but we don't have a good solution right now so your best bet is what @azakus suggested.
Specifically, we do inject the dom-if element into the table and this is basically ok since we do this after parsing (where it is disallowed), but it does not interact with colspan correctly. You can see this here: http://jsbin.com/sevofuteru/edit?html,output

I want to note that it works with PolymerJS 1. I encountered this issue because I was migrating to PolymerJS 2.

The colspan issue should be fixed on master, so please try to load master in your application and see if it is fixed.

Verified. It works. See: https://jsfiddle.net/9rtq7czd
Thanks @TimvdLippe!


added check for environment variables

Added check for environment variables in the init so that it raises an exception when the RMW_IMPLEMENTATION or CYCLONEDDS_URI environment variables are not set before running launch files.

Changed the if statement to two assertions

Fixes #40 and uses assert to error out if either RMW_IMPLEMENTATION or CYCLONEDDS_URI is not set.


featdisable join reorder if any table row count is not available #43000 (branch-2.0）

What problem does this PR solve?
pick #43000

Issue Number: close #xxx

Related PR: #xxx
Problem Summary:
Check List (For Committer)

Test

[ ] Regression test
[ ] Unit Test
[ ] Manual test (add detailed scripts or steps below)
[ ] No need to test or manual test. Explain why:

[ ] This is a refactor/code format and no logic has been changed.
[ ] Previous test can cover this change.
[ ] No colde files have been changed.
[ ] Other reason

Behavior changed:

[ ] No.
[ ] Yes.

Does this need documentation?

[ ] No.
[ ] Yes.

Release note

None

Check List (For Reviewer who merge this PR)

[ ] Confirm the release note
[ ] Confirm test cases
[ ] Confirm document
[ ] Add branch pick label

run buildall


HDDS-3647. NPE while open datanode page since a pipeline no leader

What changes were proposed in this pull request?
If there is a pipeline have no leader on a datanode, it will through NPE.
What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-3647
How was this patch tested?
Create STAND_ALONE pipeline, and start recon server, open the datanode page, no 500 return.

@vivekratnavel Hi, Please take a look at this PR about NPE.

@maobaolong Thanks for the PR! Merged it.

@vivekratnavel Thank you.


Upgrading to the latest stable Node.js version

This is to address https://github.com/umbraco/Umbraco-CMS/issues/3905

@zpqrtbnk here's a node.js version bump 😃


Fail the computation participant in case of PERMANENT errors.

This change is

This PR will be merged to main after PR#110 is in.


Not calling RestAPI

Version: 8.31.2

Target:

I'm trying to call RestAPI data, using httpclient library in nuget.

It works fine in the browser, but if I build it with electron.net in the same environment and click it,
there is no response.
github code link is below
https://github.com/graylobo/BlazorTron
Can I fix this issue? thanks in advance

Your ElectronNET port does not match the port of the RestAPI call.
Add following line to your Program.cs:
webBuilder.UseStartup();
webBuilder.UseUrls("http://*:8001"); //e.g. 8001
RestAPI.razor:
Students = await http.GetJsonAsync<List>("http://localhost:8001/api/students");

Thank you. It works well thanks to your advice.

However, it does not move to the json page when i click the button

Try this but I don't know if this is best practice:
new MenuItem { Label = "Home", Click = () => Electron.WindowManager.BrowserWindows.First().LoadURL("http://localhost:8001/")

The safest way would be:
new MenuItem { Label = "Home", Click = () => Electron.WindowManager.BrowserWindows.First().LoadURL($"http://localhost:{BridgeSettings.WebPort}/")
Did we solve your problem?

Yes, it works perfect. Thanks a bunch!


DBContext is using JetBrains.Annotations.NotNullAttribute

DBContext is using JetBrains.Annotations.NotNullAttribute
If I disassemble the DBContext class with Visual Studio 2019 (Go to definition) the Code File shows me the Usage of JetBrains
public virtual EntityEntry<TEntity> Add<TEntity>([JetBrains.Annotations.NotNull] TEntity entity) where TEntity : class

Is this a Bug in Visual Studio, or is it really a reference to JetVrains Assembly?

It is referencing JetBrains. Link to source code.
https://github.com/aspnet/EntityFrameworkCore/blob/b9c9b57de1ba41f75427db0a51b2e897d0354656/src/EFCore/DbContext.cs#L773-L774
Is there any issue with that?

Note that there isn't an assembly reference here. It's just a by-convention type defined in that namespace so that tooling can pick it up.

This does pose one problem - when I use Visual Studio's Quick Actions to generate a constructor, it includes a [NotNullAttribute] annotation which is an error.

@jhorv-bvb See #462 and https://github.com/dotnet/roslyn/issues/5646


FunctionExpression name binding should be immutable

Related issue: https://github.com/Samsung/jerryscript/issues/1292
Please tell me if there's better way to find whether CBC_INITIALIZE_VAR is initializing function expression name binding or not.

Personally I would just capture the self-reference, since self-reference is only possible with function expressions. So just decode the literal with ECMA_GET_NON_NULL_POINTER, and if it is the same as the byte code you get a match. Perhaps we could extract this decoding out from the actual function call.

Let me help a bit. In the beginning of vm_init_loop() create a compressed pointer from byte_code_p. If literal_start_p[literal_index] is the same as this compressed pointer, you got a self reference, and create an immutable binding from it.

Thanks for your help, @zherczeg. If I understand correctly, I think you mean bytecode_header_p instead of bytecode_p, and literal_start_p[value_index] instead of literal_start_p[literal_index]. I uploaded a new commit.

Yes I think something like this.
It is worth noting that the original code seems inefficient though, since we create a binding, then resolve it and set its value. This should be reworked in a follow-up patch to allow create and initialize a binding (add an extra default value parameter to ecma_op_create_immutable_binding ecma_op_create_mutable_binding).

Moved the initialization of self_reference into the beginning of the function, and rebased the commit to the latest master.

LGTM

LGTM


Error: Generator at prisma-json-schema-generator could not start

Hi I keep getting the following error when attempting to run 'prisma generator'
`Environment variables loaded from .env
Prisma schema loaded from prisma/schema.prisma
Error: Generator at prisma-json-schema-generator could not start:
/bin/sh: prisma-json-schema-generator: command not found
`
Here's my schema
`
// This is your Prisma schema file,
// learn more about it in the docs: https://pris.ly/d/prisma-schema
// Make sure you run 'prisma generate' in the CLI to generate the schema. note prisma CLI needed
datasource db {
provider = "mongodb"
url      = env("DATABASE_URL")
}
generator jsonSchema {
provider = "prisma-json-schema-generator"
}
generator client {
provider = "prisma-client-js"
previewFeatures = ["mongodb"]
}
model Post {
id String @id @default(dbgenerated()) @map("_id") @db.ObjectId
slug String
title String
body String
}
`

Can you tell me which prisma and node version do you use?

@MarleyDabb Please reopen the issue and add further information, if you still have any problems.

@MarleyDabb I'm facing the error. Did you fix it?


Swift 5 and Xcode 10.2

Can you please update this library for swift 5 and Xcode 10.2?

It should work with Swift 5. If you are using Carthage probably just need to rebuild the framework: #327


api: async mode in ext_proc

API changes   to ext_proc async mode. Implementation will be followed once consensus is reached on the api changes/design.
Highlighted Changes:

For the body, async mode is restricted to STREAMED processing mode only. Alternatively, we can add a new mode (e.g., ASYNC_STREAMED mode). I don't feel strong either way, please let me know your thoughts.
Specify that ext_proc server' response is not expected in this mode and will be ignored .
Add warning section to state that flow control is not ready yet.

/retest

/assign @yanavlasov @htuch @rshriram

/wait-any

/retest

close in favor of https://github.com/envoyproxy/envoy/pull/32696


Section title with accent leads to MD051

A section title with accented character leads to MD051 but it should be valid.
# Title with à accented character

[Link](#title-with-à-accented-character)

3 - MD051 / link-fragments Link fragments should be valid [Context: "[Link](#title-with-à-accented-character)"]

demo

GitHub's implementation translates accented characters into their URI encoded form - in this case "%C3%A0". (You can confirm this by looking at what's generated by the little anchor tag for that link on GitHub.) If you use that expanded form for your link, no warning is generated. This rule looks for that canonical form to be used and is behaving correctly here.
https://dlaa.me/markdownlint/#%25m%23 Title with à accented character

[Link](%23title-with-%25C3%25A0-accented-character)

Documentation (see last paragraph): https://github.com/DavidAnson/markdownlint/blob/main/doc/md051.md
Relevant test cases: https://github.com/DavidAnson/markdownlint/blob/c2c4a44931287c1420d937c38e85719a24c9c6a6/test/link-fragments.md?plain=1#L126-L128

FYI on why this is newly reported from v0.29.0 to v0.30.0: https://github.com/DavidAnson/markdownlint/issues/954#issuecomment-1707634031


Implement setFreeMonitoring stub

Closes #744.
Checklist

[x] Tests are added for new functionality or fixed bugs.
[x] task all passes.

See CONTRIBUTING.md for more details.

@rumyantseva @seeforschauer PTAL


Cow: Watch optimiseaccountants.co.uk

Cow requests the watch of the watch_keyword optimiseaccountants\.co\.uk. See the MS search here and the Stack Exchange search in text, in URLs, and in code.

optimiseaccountants\.co\.uk has been seen in 1 true positive, 0 false positives, and 0 NAAs.

Approved by VLAZ in Charcoal HQ


Custom Receipt

Hello,
we want to print a custom receipt by getting the data from SumUp Rest API
Could somebody explain us in which sequence SumUp is printing the EMV Data on their receipts? We want to print the identical section emv data on our receipts like sumup. By requesting the receipt over the api we receive emv data like TVR TSI CVR ... How to get them in a right sorting?

Hello,
Our apologies for the inconvenience.
We only send raw data with the Receipt API. Unfortunately at the moment you are unable to configure the receipt via API.

I would also like to know, in what order the EMV Data should be displayed. I understand that we can only request raw data, but can you inform us about the order @Mike-Lopez ?


Repeatly create management cluster VM when deploying TCE in vSphere

Hi all, I’ve tried to deploy Tanzu Community Edition (TCE) on vSphere without Avi LB and the problem is the vcenter will repeatly create the same master node (Single Control Plane Node) and worker node during the phase of “Create management cluster”. I’m using Ubuntu 20.04 as my bootstrap machine and I found that this repeated behavior will be stopped until I poweroff the bootstrap machine. Any thoughts on this problem? Thanks!

Additional discussion in Slack can be found here: https://kubernetes.slack.com/archives/C02GY94A8KT/p1636561590442800
We need to get the capv-controller logs to see why it is failing and retrying.

Assuming this has been resolved. This behavior sounds like it was trying to reconcile the desired state, but hitting some sort of error. Slack thread is unresolved, but it was left at the need to get the logs of the capv-controller. We've since added some debug steps here that can be used to find these logs to troubleshoot:
https://tanzucommunityedition.io/docs/latest/tsg-bootstrap/#troubleshooting-manually
Please feel free to reopen with additional information if this is still an issue.


Renamed the asynchronous version of device_copy_n

Fixes issue #541

Can one of the admins verify this patch?

Can one of the admins verify this patch?

add to allowlist

add to allowlist

add to allowlist

add to allowlist


Failing to read empty FlatGeobuf with Java

Given an empty FlatGeobuf file, as per ogrinfo output:
INFO: Open of `/...emptyFlatGeoBuf.fgb'
      using driver `FlatGeobuf' successful.

Layer name: SELECT
Geometry: Point
Feature Count: 0
Layer SRS WKT:
GEOGCRS["WGS 84",
    DATUM["unknown",
        ELLIPSOID["WGS84",6378137,298.257223563,
            LENGTHUNIT["metre",1,
                ID["EPSG",9001]]]],
    PRIMEM["Greenwich",0,
        ANGLEUNIT["degree",0.0174532925199433]],
    CS[ellipsoidal,2],
        AXIS["longitude",east,
            ORDER[1],
            ANGLEUNIT["degree",0.0174532925199433]],
        AXIS["latitude",north,
            ORDER[2],
            ANGLEUNIT["degree",0.0174532925199433]]]
Data axis to CRS axis mapping: 1,2
wind_speed: Real (0.0)
wind_direc: Real (0.0)
backscatte: Real (0.0)

The Java library would fail to read it instead:
Caused by: java.lang.RuntimeException: Node size must be at least 2
	at org.wololo.flatgeobuf.PackedRTree.generateLevelEnds(PackedRTree.java:199)
	at org.wololo.flatgeobuf.PackedRTree.search(PackedRTree.java:337)
	at org.geotools.data.flatgeobuf.FeatureCollectionConversions.deserialize(FeatureCollectionConversions.java:139)
	at org.geotools.data.flatgeobuf.FlatGeobufFeatureReader.<init>(FlatGeobufFeatureReader.java:104)

An empty FGB does not make much sense, but it can happen while bulk processing data (splitting it over attributes to work around the lack of alphanumeric indexes).

Agreed this is a bug. The likely reason for it is because initially I thought to forbid empty FlatGeobuf but have since eased up on that. :)

But hmm, not sure the bug is here. It shouldn't attempt to use PackedRTree in this case.

Yep, logic at https://github.com/geotools/geotools/blob/ffdf418ad09dc20eb8a15caa709c72d4c409017f/modules/unsupported/flatgeobuf/src/main/java/org/geotools/data/flatgeobuf/FeatureCollectionConversions.java#L124 is too naive. Optimally it should handle featuresCount = 0 as potentially (0 = unknown) empty file and return an empty iterator if there is no data after the header. If there is data it is an unindexed file and must iterate the whole file and apply rect filter as secondary. It can also be featuresCount > 0 but indexNodeSize == 0 which means it is an unindexed file with known feature count and must also be treated as the unindexed case.

I've made https://osgeo-org.atlassian.net/browse/GEOT-7457 to track this in geotools


Correct Reminder Naming Convention

Currently we don't differentiate between the different reminder action
rules.
This will add a "+1", "+2", or "+3" to the action rules name for
the reminder, reminder2, and reminder3 event tags.
mvn test

 TBL123: Link action rules to action plans at create time (Create)


i can't push rtsp url and local mp4 video to aws

Hello, I am a developer from China who uses the Hong Kong node in China. I have configured the AWS service based on this video url https://www.youtube.com/watch?v=nVxwX7Q9nPU&t=172s and can only push videos in mkv format to play on AWS, but cannot use rtsp streaming or local mp4 video playback
My local computer configuration is ubutnu20.04, Hong Kong, China IP
你好，我是来自一名中国的开发者，使用中国香港节点，我按照这个视频配置了aws服务，仅仅能推送mkv格式的视频在aws上面播放，而无法使用rtsp流媒体播放和本地的mp4视频播放
我的本地电脑配置是ubutnu20.04 ,中国香港ip

@sxj731533730 ,
Please provide the following details:

What version of the SDK is being used?
When using RTSP URL, did you try a pipeline with another sink to ensure the source works with other sinks?
When using local mp4, did you try a pipeline with another sink to ensure the source works with other sinks?

It looks like you are using kvs_gstreamer_sample.
Could you try using this .mp4 file to confirm that .mp4 files are able to be uploaded using the sample?
https://catalog.us-east-1.prod.workshops.aws/kinesis-video-streams/en-US/3-stream/2-sample-app#download-the-video-for-playback
Could you also verify that the demo.mp4 you are attempting to upload is a valid playable .mp4 file? Does ffprobe give any errors?

Are you able to construct a gstreamer pipeline to pull from the rtsp source and put into a file sink?
Our sample gives a demonstration of how to pull from an RTSP camera but it's using a specific h264 depayer and pad that may not be applicable to all RTSP cameras.


Customisable path in URL

I'd like to use tabula under nginx with path "/tabula/" or simething like this. Currently this is not configurable, only port can be configured.
Can you add a parameter to set root path in URL?

Timothy,
If you're using nginx to proxy requests to Tabula, you don't need to change
its URL.

Can you recommend how to set tabula to be under reverse proxy?
I'd like to have it something like http://mylovelysite.com/tabula/.
What should I configure in tabula to work in such cases?

https://www.nginx.com/resources/admin-guide/reverse-proxy/

Are you kidding?! This is exactly what I'm doing for 10 years. However, tabula does not respect URL provided by nginx and produces HTML which includes incorrect URLs. That's why I created this issue.

No need to be rude.  It would have helped if you provided with specific error messages, that can help us determine where the bug is. Better yet, you can provide us with a patch.
Have you not learned how to properly report an issue in your ten years of experience?

Sorry. I was not so polite. However, the issue is still exist. Feature is not implemented.
Usually, I use my linux machine as a local web-server.
I downloaded windows version of nginx and configured it as a reverse proxy for tabula. Now technical details:

nginx 1.10.1 windows
in http section I added

 location /some/path/ {
    proxy_set_header Accept-Encoding "";
    proxy_pass http://localhost:8000;
}

I started tabula with java -Dfile.encoding=utf-8 -Xms256M -Xmx1024M -Dwarbler.port=8080 -Dtabula.openBrowser=false -jar tabula.jar
I went to http://localhost/tabula/

Results: tabula html page uses absolute paths in all elements referencing other resources like js or CSS. None of the links work. For example, http://localhost/tabula/about is missing (error 404).
Tabula supports only root path and can't be configured for subpaths on the server.
Is there a way to configure tabula to support work with non-root path under reverse proxy?

Hi @theta682,
You're right, we have some absolute URLs in the code (see #110).
@jeremybmerrill, can you take a look at this when you have the chance?

Hi @jazzido,
Thanks. I have only little experience with ruby, I can't create proper fix.
I'll be waiting for this feature.

Please try pull request #586 to see if it works for you.

Hi,
Sorry, I stopped using tabula some time ago and I can't test it. You can close this issue if you think that it is fixed.


Removes note about an old MSL dependency

Since Fluid.Dissipation is part of MSL this is no longer necesssary.

See #2340 for the general issue on the maintenance and update strategy of Fluid.Dissipation.


Do doc validation based on database file

Replaces #1223
This kills the old doc generation script and details md file and updates the doc validation flow to use the stats script and database file instead.
Currently there are two error cases:

If a check is claimed to be implemented in the database but the ID is not found in the source files
If a test is claimed in the database but the testname is not found in the source files

Note that I just kept the existing scripts for doc validation as that's the path of least resistance to get these changes in.

This all looks great.  I am unable to run the test or vk_validation_stats.py on Windows -- anybody else having this issue?

I did test on Windows but I was using python2.7 on my windows system. I know that with python 3 you'll see an error.

Ah, 'python' = python 3.5.1, 'py' = ptyhon 2.7.11 (cygwin) do not work, but 'python2.7' = 2.7.10 works fine.  Thanks.

Replaced by #1248


Hipchat notification

Hi !
First, thanks for Searchlight, it the kind of tool that I missed in my cluster.
I try to enable Hipchat notifications but I can't make it work,
here is my ClusterAlert.yaml :
apiVersion: monitoring.appscode.com/v1alpha1
kind: ClusterAlert
metadata:
  name: component-status
  namespace: default
spec:
  check: component_status
  checkInterval: 30s
  alertInterval: 2m
  notifierSecretName: notifier-config
  receivers:
  - notifier: hipchat
    state: CRITICAL
    to: ["#DevOps"]

(#DevOps) is my room name.
I have created the secret notifier-config with this command :
kubectl create secret generic notifier-config -n kube-system \
    --from-file=./HIPCHAT_AUTH_TOKEN

The file HIPCHAT_AUTH_TOKEN contain the token (write with echo -n).
It's a room token, not a Global token. (I tried both).
When I stop kube-controller-manager for example, Icinga detect it quickly, got a CRITICAL alert, but not any Hipchat message anywhere. (and no logs at all.)
Thanks for your help.

Hi, the ClusterAlert object and notifier Secret notifier-config needs to be in the same namespace. From your description, ClusterAlert is in default namespace and notifier-config is in kube-system namespace.

Yeah, thanks for your answer, I have created the secret in kube-systemand default and it seem to have to same behavior.

Let me replicate the issue at my end and get back to you. If you check the logs for icinga container, do you see any error?

@tamalsaha  Thanks for your investigations. There isn't any error on icinga containers.

@valentin2105 , we have pushed a new fix 3.0.1 to address this issue. https://github.com/appscode/searchlight/releases/tag/3.0.1 . Please upgrade the image tags for operator container to appscode/searchlight:3.0.1 and icinga container to appscode/icinga:3.0.1-k8s .
Let me know how it goes.

It's solved, Thanks for your reactivity,
Any way to tweak message sended to Hipchat ? via Icinga or the ressource ?
Thanks ago ! :)

Any way to tweak message sended to Hipchat ? via Icinga or the ressource ?

Not possible now. What do you want to see?

 I will try more and propose feature if I find good idea :)
Ty


Cross-job cache not working properly

I have a workflow with two jobs, the first one is to build my app, and the second deploy it to the cloud, at the end of my first job I have this step:
- name: Cache prod stagged
  id: cache-prod-stagged
  uses: actions/cache@v3
  env:
    cache-name: cache-prod-stagged
  with:
    key: ${{ runner.os }}-${{ env.cache-name }}-${{ github.run_id }}
    path: |
      ${{github.workspace}}/target/prod-stagged

The post cache job of this step shows it's stored successfully as you can see in the logs:
/usr/bin/tar --posix --use-compress-program zstdmt -cf cache.tzst --exclude cache.tzst -P -C /home/runner/work/xxxxxxx/xxxxxxx --files-from manifest.txt
Cache Size: ~156 MB (163765834 B)
Cache saved successfully
Cache saved with key: Linux-cache-prod-stagged-3210643458

And also when I call https://api.github.com/repos/xxxxxx/yyyyyy/actions/caches I see a lot of entries, and one matches with the cache created
{
    "id": 54,
    "ref": "refs/pull/778/merge",
    "key": "Linux-cache-prod-stagged-3210643458",
    "version": "27af84acdd918c6d16e9df466b0fea6b8e2c47ff238b629dfefeec79336ee494",
    "last_accessed_at": "2022-10-08T13:58:54.243333300Z",
    "created_at": "2022-10-08T13:58:54.243333300Z",
    "size_in_bytes": 163765834
},

So far, so good, the cache is stored and the key matches. Now, i have this as second step of my second job, still on the same workflow.
- name: Recover cached compilation
  uses: actions/cache@v3
  env:
    cache-name: ${{ matrix.stagged_key }}
  with:
    key: ${{ runner.os }}-${{ env.cache-name }}-${{ github.run_id }}
    path: |
      ${{ matrix.stagged_path }}

In this case, matrix.stagged_key is cache-prod-stagged and matrix.stagged_path is ${{github.workspace}}/target/prod-stagged
When I run this job with debug logs enabled, the logs of this steps are the following:
##[debug]Evaluating: matrix.stagged_key
##[debug]Evaluating Index:
##[debug]..Evaluating matrix:
##[debug]..=> Object
##[debug]..Evaluating String:
##[debug]..=> 'stagged_key'
##[debug]=> 'cache-prod-stagged'
##[debug]Result: 'cache-prod-stagged'
##[debug]Evaluating condition for step: 'Recover cached compilation'
##[debug]Evaluating: success()
##[debug]Evaluating success:
##[debug]=> true
##[debug]Result: true
##[debug]Starting: Recover cached compilation
##[debug]Register post job cleanup for action: actions/cache@v3
##[debug]Loading inputs
##[debug]Evaluating: format('{0}-{1}-{2}', runner.os, env.cache-name, github.run_id)
##[debug]Evaluating format:
##[debug]..Evaluating String:
##[debug]..=> '{0}-{1}-{2}'
##[debug]..Evaluating Index:
##[debug]....Evaluating runner:
##[debug]....=> Object
##[debug]....Evaluating String:
##[debug]....=> 'os'
##[debug]..=> 'Linux'
##[debug]..Evaluating Index:
##[debug]....Evaluating env:
##[debug]....=> Object
##[debug]....Evaluating String:
##[debug]....=> 'cache-name'
##[debug]..=> 'cache-prod-stagged'
##[debug]..Evaluating Index:
##[debug]....Evaluating github:
##[debug]....=> Object
##[debug]....Evaluating String:
##[debug]....=> 'run_id'
##[debug]..=> '3210643458'
##[debug]=> 'Linux-cache-prod-stagged-3210643458'
##[debug]Result: 'Linux-cache-prod-stagged-3210643458'
##[debug]Evaluating: format('{0}
##[debug]', matrix.stagged_path)
##[debug]Evaluating format:
##[debug]..Evaluating String:
##[debug]..=> '{0}
##[debug]'
##[debug]..Evaluating Index:
##[debug]....Evaluating matrix:
##[debug]....=> Object
##[debug]....Evaluating String:
##[debug]....=> 'stagged_path'
##[debug]..=> '/target/prod-stagged'
##[debug]=> '/target/prod-stagged
##[debug]'
##[debug]Result: '/target/prod-stagged
##[debug]'
##[debug]Loading env
Run actions/cache@v3
::save-state name=CACHE_KEY::Linux-cache-prod-stagged-3210643458
##[debug]Save intra-action state CACHE_KEY = Linux-cache-prod-stagged-3210643458
##[debug]Resolved Keys:
##[debug]["Linux-cache-prod-stagged-3210643458"]
##[debug]Checking zstd --version
##[debug]*** zstd command line interface 64-bits v1.5.2, by Yann Collet ***
##[debug]Resource Url: https://artifactcache.actions.githubusercontent.com/cHJ0NMbKyvmrTh2U94l0qTwlUdSW0ZAoJzw6DLf2qMy6KGMNlL/_apis/artifactcache/cache?keys=Linux-cache-prod-stagged-3210643458&version=64b8b74310a55182bff7220b1967579395639dae56d787240cb1b6bd6acd1c94
##[debug]Failed to delete archive: Error: ENOENT: no such file or directory, unlink ''
Cache not found for input keys: Linux-cache-prod-stagged-3210643458
##[debug]Node Action run completed with exit code 0
##[debug]Finishing: Recover cached compilation

So, as you can see for some reason the key name matches on all the files (in this example Linux-cache-prod-stagged-3210643458) and they are inside the same workflow but the second job can not find the cached files

Hi @lvalladares-isp, can you please provide the workflow definition or debug logs for the first job, so that we can take a look at this further?
I think this is most likely because of version mismatch between the two caches. One of the params cache-version depends on is the path, so can you please confirm is the path is the same? Reference docs can be found here.


Expression data

Hello
I am learning to use Pathvisio. I have a problem to integrate the expression data to the pathway and there's no tutorial available. Could you please advise me on this?
Thank you
-Alya

Hello Alya,
The tutorial available here https://laurendupuis.github.io/Helis_Academy_April_2020/tutorials/Pathway_Analysis.html can guide you through the process of importing expression data in PathVisio. If this does not help, can you provide more details on where the integration fails and the error message you receive?
Best,
Lauren

Thank you for your prompt response. I will give it a try and let you know
how it goes.
Kind regards,
Alya
On Thu, 6 Aug 2020 at 15:29, LaurenDupuis<EMAIL_ADDRESS>wrote:

Hello Alya,
The tutorial available here
https://laurendupuis.github.io/Helis_Academy_April_2020/tutorials/Pathway_Analysis.html
can guide you through the process of importing expression data in
PathVisio. If this does not help, can you provide more details on where the
integration fails and the error message you receive?
Best,
Lauren
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/PathVisio/pathvisio/issues/161#issuecomment-669758748,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AQQUMVRQII6RSAQ2RXJUVG3R7JLV5ANCNFSM4PWF42BQ
.

Hi Lauren,
I still encountered problems when I tried to use EntrezID as the identifier
column although I have mapped it to EntrezGene as the system code
(DataImport5.png). I've also tried to use Ensembl ID and mapped it to
Ensembl Human (DataImport4.png). Both resulted in errors
(DataImport6.png):-
Line 2: L:"1" Error: Could not look up this identifier in the identifier
mapping database
Line 3: L:"503538" Error: Could not look up this identifier in the
identifier mapping database
Line 4: L:"29974" Error: Could not look up this identifier in the
identifier mapping database
Line 5: L:"2" Error: Could not look up this identifier in the identifier
mapping database
Line 6: L:"144571" Error: Could not look up this identifier in the
identifier mapping database
Have I chosen the wrong system code's database?
On Thu, 6 Aug 2020 at 15:29, LaurenDupuis<EMAIL_ADDRESS>wrote:

Hello Alya,
The tutorial available here
https://laurendupuis.github.io/Helis_Academy_April_2020/tutorials/Pathway_Analysis.html
can guide you through the process of importing expression data in
PathVisio. If this does not help, can you provide more details on where the
integration fails and the error message you receive?
Best,
Lauren
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/PathVisio/pathvisio/issues/161#issuecomment-669758748,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AQQUMVRQII6RSAQ2RXJUVG3R7JLV5ANCNFSM4PWF42BQ
.

Hi Alya,
For some reason I'm not able to see the images. Can you try attaching them again?
Best,
Lauren

Hi Lauren,
Could you view it now?
On Thu, 6 Aug 2020 at 17:43, LaurenDupuis<EMAIL_ADDRESS>wrote:

Hi Alya,
For some reason I'm not able to see the images. Can you try attaching them
again?
Best,
Lauren
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/PathVisio/pathvisio/issues/161#issuecomment-669827734,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AQQUMVRPWZ3SX7K67UN4AJ3R7J3LJANCNFSM4PWF42BQ
.

Hi Lauren,
Sorry, I was replying from the email, thus included the pictures as attachments. Hope you could see it now.

A side note, when I used the Ensembl ID as the primary identifier and set the Ensembl as the database, it worked with 1000+ genes unrecognised. Would this be recommended as my subjects are human so I thought it's best to use Ensembl human as database (but didn't recognise any genes)?

Hi Alya,
Are your files tab delimited or comma delimited? We typically use tab delimited input files, so just make sure that your input file is indeed consistent with what you choose. As for the identifier, Ensembl human won't work unfortunately, so it's best just to use Ensembl because that database includes all of the human identifiers.
Does it work when you try that?

Hi Lauren,
My fikes are in CSV format. I haven't tried using TXT for pathvisio but will give it a try :)
I see -  thank you for explaining that to me. I will then be able to explain this in my write up. Yes, it does work when I use Ensembl.
Thanks a lot, Lauren for your help.

Hi Lauren,
My files are in CSV format. I haven't tried using TXT for pathvisio but will give it a try :)
I see - thank you for explaining that to me. I will then be able to explain this in my write up. Yes, it does work when I use Ensembl.
Thanks a lot, Lauren for your help.

I've tried using TXT files and it worked for all primary identifiers - Ensemble, Entrez and HGNC symbol.
Thank you so much for your suggestions! Really appreciate it.

No problem, and don't hesitate to contact us with any other questions!


[MXNET-319] Add Autocomplete Macros in Scala

Description
Add SymbolBaseMacros File: It will generate a SymbolBase trait with full documentation on the Symbol APIs. Also enable the "return_type" Parameters that will be helpful to generate the docs.
@nswamy @yzhliu @Roshrini
Checklist
Essentials
Please feel free to remove inapplicable items for your PR.

[x] The PR title starts with [MXNET-$JIRA_ID], where $JIRA_ID refers to the relevant JIRA issue created (except PRs with tiny changes)
[x] Changes are complete (i.e. I finished coding on this PR)
[x] All changes have test coverage:
Unit tests are added for small changes to verify correctness (e.g. adding a new operator)
Nightly tests are added for complicated/long-running ones (e.g. changing distributed kvstore)
Build tests will be added for build configuration changes (e.g. adding a new build option with NCCL)
[x] Code is well-documented:
For user-facing API changes, API doc string has been updated.
For new C++ functions in header files, their functionalities and arguments are documented.
For new examples, README.md is added to explain the what the example does, the source of the dataset, expected performance on test set and reference to the original paper if applicable
Check the API doc at http://mxnet-ci-doc.s3-accelerate.dualstack.amazonaws.com/PR-$PR_ID/$BUILD_ID/index.html
[x] To the my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change

Changes

[ ] Feature1, tests, (and when applicable, API doc)
[ ] Feature2, tests, (and when applicable, API doc)

Tested with make docs. The Function definitions were successfully generated on Scala docs


istanbul-init does not work for variable nodes

System information
Geth version: 1.9.7 stable
OS & Version: Ubuntu 16.04
Quorum version: 2.6
Expected behavior
I tried to modify the 7-nodes example in Istanbul-init.sh using the following command

./istanbul-init.sh --numNodes 8

Actual behavior
Unfortunately it does not work for 8th node. Even though my permissioned-nodes.json file contain 8 nodes information. It always shows error

'can not stat raft/nodekey8' : no such file or directory

Why does it show this error?
Please help me to resolve this.

The --numNodes parameter only allows you to reduce the number of nodes (as per the docs).
If you want to increase the number of nodes above 7 then you would need to generate extra key files yourself, and also modify the permissioned-nodes.json file.
The quorum-examples is capped at 7 nodes as that is sufficient to get most people up and running and experimenting with quorum. Also running more than 7 nodes on one machine is likely to exhaust your resources.

Thanks for you reply.
The extra key files are generated using https://github.com/jpmorganchase/istanbul-tools
However, in this 7-node example, it is not explained how the extra keys are to be placed in which directories. Let's assume we need to place the keys in the keys folder. Yet, the generated keys from istanbul tools does not help in producing the contract address when the **eth.getTransactionReceipt ** is issued. Besides, the 7-node example should be able to be reconfigured about how to distributedly deploy blockchain in different nodes. Otherwise, what is the point of having blockchain in single node!


use jupyter kernelspec install when available

and use ipython kernelspec install for IPython 3, rather than attempting to reimplement the install process.
I didn't change the ipython2, ipython3 checks, but I'm not sure how one can get into a state where ipython2/3 exists but ipython does not.
closes #351

Changes you might want to make:

make --user install optional (allow system-wide install)

Looks good, thanks!  I think the --user is right for the Pkg.build script; if someone wants a system-wide install, they should probably do that manually.

Anaconda has a program called jupyter-kernelspec .... is that related?

jupyter-kernelspec is jupyter kernelspec. jupyter dispatches to subcommands like git, so jupyter foo calls jupyter-foo, etc.

The odd thing is that I got ipython and jupyter-kernelspec from ipython-notebook, but not jupyter unless I explicitly told conda to install jupyter.   Does ipython kernelspec dispatch to jupyter-kernelspec?


📝 Documentation(i18n): Translate the FAQ page to French

Bug Report Checklist

[X] I have pulled the latest main branch of the repository.
[X] I have searched for related issues and found none that matched my issue.

Overview
Translate the FAQ Page to French; go to faq.fr-FR.mdx and make the translations from there, you will have to step into the Components folder to allow for there to be french version needed.
@GuiBibeau has an implementation that he has recommended to make this work.

Additional Info

You can come into the docs channel in the Discord Server for further information. Do translate the front-matter as well.

Use an informal tone, not too informal though

Also, checkout the Contriibuting Guide incase you want to set up Million locally

@tobySolutions Please, I would like to work on this. I've gone through @guigui64 PR. I'll start working on mine when his PR is merged to use the same flow he used.

@tobySolutions Please, I would like to work on this. I've gone through @guigui64 PR. I'll start working on mine when his PR is merged to use the same flow he used.

Perfect! Thanks @preshpi! I'm assigning this to you now.


Sensu-Go-Agent support for windows platform

Pull Request Checklist
#46 - Add support for installing Sensu Agent MSI on Windows
General

[X] Update Changelog following the conventions laid out here

[X] Update README with any necessary configuration snippets

[X] Cookstyle (rubocop) passes

[X] Foodcritic passes

[X] Rspec (unit tests) passes

[X] Inspec (integration tests) passes

New Features

[X] Added a Testing Artifact as either an automated test or a manual artifact on the PR.

[X] Added documentation for it to the README.md

Purpose
Allows cookbook to handle installation and configuration of the Windows (5.10.0) MSI.
Known Compatibility Issues
Long term implementation does not support using the latest version, until a package mechanism is created. This will most likely occur via Chocolatey in the future.

Thanks for adding this @derekgroh Windows support is important here. I’ll look at the options for ci testing and review a little more in detail tomorrow.

Anything I can do to further assist?

@webframp travis supports windows in limited capacity I think it might be easier/better to simply have it handled by appveyor or similar with more broad windows support, here is an example setup: https://github.com/chef-cookbooks/windows/blob/master/appveyor.yml

@majormoses I’m a fan of adding .gitattributes for cross platform repo work, sounds reasonable for this case.

I’m interested in any input from @cwjohnston or @amdprophet  for how they want to handle windows versions and the release process there.

Added support for :remove action
Added appveyor files (not sure how to get this into a pipeline)

I’d be ok with appveyor and could set it up, but would want to check with any preference from Sensu org in general for Windows ci first

@webframp they use it in core for building their packages: https://github.com/sensu/sensu-go/blob/5.10.1/.appveyor.yml

I've added this project to our AppVeyor account; I'm thinking the next commit to this branch should kick off a test there.

Thanks @cwjohnston !

@cwjohnston Who's the goto to ask about appveyor setup? I seem to be missing something on the lint tests in the test environment, but they are successful locally.

Looks like appveyor and travis have different cookstyle versions:

https://travis-ci.org/sensu/sensu-go-chef/jobs/575918922#L505
https://ci.appveyor.com/project/majormoses/sensu-go-chef/builds/26918896#L67

Will be good to keep those consistent.

Should we favor TravisCI linting and just use appveyor for integration testing?

Should we favor TravisCI linting and just use appveyor for integration testing?

I think that makes sense


don't set far expiry if hash is '000'

This will close #1581.
I couldn't find any unit tests related to the testing hashed urls, and I know that you want to break that code out of the core application (#1561), so I'm not quite sure what you would like me to for testing.

this has been obviated by the datasette-hashed-urls plugin


Update Credits to include use of OctoKit

While you are there...
also include InspectCode and DupFinder

And MyGet...


Plugin working on basic villagers

About
Plugin working on basic villagers, only should work with merchants.

Fixed on version 1.2.3


Help display not showing the project url

An ellipsis is shown instead of the project url.

I will work on solving this issue


Check if encoding profile exist for imageToVideo Operation

This Pullrequest log a proper ERROR if the Encoding Profile is missing in imageToVideo Operation.
This fixes #3465
Your pull request should…

[ x] have a concise title
[ x] close an accompanying issue if one exists
[x ] be against the correct branch
[ x] pass automated tests
[ x] have a clean commit history
[ x] have proper commit messages (title and body) for all commits

I hope i have done the Opencast "Exception thing" the right way here.
Any hints if not are very welcome.

This looks good and can be merged, but in #3465, you say this effects Opencast 11. Is this intended to go into r/11.x? Or is develop correct?

Yes right. This is also a Problem for OC 11.
Waht would be the right way here.
Repoint this to 11, or close this PR and reopen a new one againts 11?

If you click on Edit right next to the title, you can change the target branch of this pull request. If you already based this on r/11.x, that's all you need to do. If not, git rebase -i r/11.x is an easy way of basing this on r/11.x. Let me know if you need help with that.


Remove the raise error.

I removed the raise from the script so it should work now. :dart: #74
Closes #1

Alyson is seriously cute.


Applications with third-party embedded libraries cannot be submitted to the App Store

You can't submit iOS frameworks with x86 architecture.
See:

https://github.com/Carthage/Carthage/issues/188
https://github.com/Carthage/Carthage/pull/208

Related to: https://github.com/NativeScript/nativescript-cli/issues/383


Feature/load env from file

This MR introduces:

New option environment-file, which can be specified in fmf or via cli --environment-file added. It is merged with the existing environment option (environment has precedence in case of conflict).

    /with_variables:
        summary: This is a plan with variables
        discover+:
            filter:
                - 'tag: with_variables'
        environment:
            STR: L
            INT: 2
        environment-file:
            - ".env"
            - ".env_in.yaml"
            - ".env_in.yml"
            - "https://some-url-to/.env"

Now we can generalize the logic of how tmt work with environment variables:

env vars which sent as VAR=value tmt takes precedence over cli options --environment or --environment-file
cli options --environment or --environment-file takes precedence over fmf options
environment overwrites environment-file vars on conflict

Why:
In convert2rhel we use .env file to deal with the secrets. This .env file contains secrets in a form of env vars. This file should be a source of secrets for all subsystems of the testing framework -> tmt, ansible, pytest, and even tft.
Resolves #840

@psss Just to mark sure if it is all in need to do to submit the MR for a review:) maybe you are waiting for something

@ZhukovGreen I would rather call the attribute environment_files ... tried to find a better solution, not needing another attribute,
but did not find it. @psss any ideas on this one?
execute:
    how: tmt
discover:
    how: fmf
prepare:
    how: ansible
    playbook:
        - tests/ansible_collections/main.yml
environment:
    foo: bar
environment_files:
    - .env
    - .foo
    - .bar

Why: In convert2rhel we use .env file to deal with the secrets.

Let's first clarify the use case: So the expectation is that the file .env contains secret information in the form of variables and this information is loaded into the tmt process so that the secrets can be used for further actions, right? Currently the environment is openly revealed in the logs:
> tmt init -t base
> tmt run -e PASSWORD=secret -avvvddd
/plans/example
summary: Basic smoke test
info
    environment: {'PASSWORD': 'secret'}
    context: {}
...
    execute
        workdir: /var/tmp/tmt/run-012/plans/example/execute
        how: tmt
        order: 50
            Execute '/tests/example' as a 'beakerlib' test.
            Run command 'podman exec tmt-var-tmp-tmt-run-012-plans-example sh -c cd '/var/tmp/tmt/run-012/plans/example/discover/default/tests/tests/example'; export PASSWORD=secret BEAKERLIB_DIR=/var/tmp/tmt/run-012/plans/example/execute/data/tests/example; ./test.sh'.

Do you expect these variables to be available on the guest where tests are executed as well? Do you expect that these sensitive variables are not included in the debug logs or on the console output?

@psss ah missed the point with secrets, for that, we need some flag to mark env var as secret I guess. Maybe learn from gitlab how it does it (it actually uses regex match to hide the secret from logs, and does not accept non-regex strings). I believe the safest would be to do the same.

Why .env?
We use .env as a config, in a format of env vars. Some of vars are secrets, some are just normal vars (i.e. DEBUG).
The config should be separated from the codebase (https://12factor.net/config). So I can easily modify it, without modifying the code. The declaration of the test plans and tests is a part of a code. That's why we use .env file and we need to point tmt to it.
How .env is assumed to be used in tmt
Yes, @psss , you're right. I'd like to have these vars available in the tmt process and all its subprocesses. However, it is not crucial for us to have a feature in tmt to load those vars into the testing env - we're doing it ourself when starting the pytest session (https://github.com/oamg/convert2rhel/blob/main/tests/integration/conftest.py#L20). But, maybe, this could be a consistent solution to make tmt take care of this as well (but we may postpone this feature).
Hiding the secrets
I believe tmt is not the right place to solve the problem of hiding secrets and especially now. I'd wait until the complete solution for convert2rhel and based on experience decided do we need to hide secrets at all. For example, if the logs will be hidden behind the RH VPN, then probably we don't need to take care of hiding the secrets at all.
@thrix I also like the way it is solved by gitlab, and in this case, the hiding secrets should be done by the one who rendering logs (tft or packit). However, let's wait a bit.
What I am trying to achieve in a very nearest future
cli tool for convert2rhel (https://github.com/oamg/convert2rhel/pull/297/files#diff-db8b466799091482f8237075eb4177067a2a12cea3a24f791ac0a995ffc9cff7) which submits tests to tft. Use this tool in non-verbose mode for github action (the same with packit), which is triggered only by convert2rhel project maintainers. In this case, the gh action will be triggered but we'll see only very simple logs (i.e. plan A,B submitted, plan A,B finished with status: PASS). If the plan failed, then the developer can run it locally (via this cli utility or current libvirt provisioner) in the verbose mode and get access to all needed logs to troubleshoot.

@thrix why do you think it is needed to have multiple env files instead of just one?

I would rather call the attribute environment_files ... tried to find a better solution, not needing another attribute,
but did not find it. @psss any ideas on this one?

No quick or elegant solution at the first/second thought. On the command line we already use the @file notation for reading a context definition from a file. Perhaps, something like this could be applied for the environment dictionary as well:
environment:
    key: val
    secrets: "@file"

The key would be ignored and the @ needs quotes as this is a special character for yaml. Not sure if this would be a nicer solution.

Maybe we can use the same like in docker-compose:
https://docs.docker.com/compose/environment-variables/#the-env_file-configuration-option
version: '3'
services:
  api:
    image: 'node:6-alpine'
    env_file:
     - .env
    environment:
     - NODE_ENV=production

Today accidentally faced also one more example of the dot env naming:
deta update --env env-file

Guys pls guide what should I do in this MR, use '@' or env_file or leave it as it is?

Sorry for the late reply. The following two points are important from my point of view:

keep the approach consistent with what we already have
do not introduce (many) new names unless necessary

So to summarize what we currently have: For both --context and --environment options we support reading the context dimensions and environment variables from a file using the @file syntax. The expected format of the file is yaml.
key: val

In this pull request we're introducing parsing environment variables from a shell-like syntax.
key=val

@ZhukovGreen, just to clarify: You need the shell-like syntax in order to share the same .env file with other parts of the tooling, right? Or would the support for reading environment variables from yaml files be enough for your use case?
Regarding the naming/syntax: Currently the following is possible to do from command line:
tmt run --environment key=val
tmt run --environment @file.yaml

In order to keep command line options in sync with config keys we could do this:
environment:
    key: val
    "@": file.yaml

This is a slightly modified version of my proposal above, seems a bit more close to the command line syntax. If the file does not have a yaml or yml extension it would be parsed using shlex.split().
environment:
    key: val
    "@": .env

Multiple files could be provided as well:
environment:
    key: val
    "@":
      - file.yaml
      - .env

@ZhukovGreen, @lukaszachy, @FrNecas, @thrix, what do you think?

Hm, personally I don't like introducing tmt unique way of writing ( "@": name ) and adding new attribute "env_file" (or long name "environment_file") seems to be a better way.
Compare reading this in the plan (or test metadata)
prepare:
   env_file: fname

with
prepare:
  "@": fname

I support the only one and a good way to the things. So if we choose env_file, then we could have:
--environment A=B
--environment .env    # it is not hard to distinguish where it is A=B mapping or file, right?

In this case the fmf file will look like this:
execute:
    how: tmt
discover:
    how: fmf
prepare:
    how: ansible
    playbook:
        - tests/ansible_collections/main.yml
environment:
    foo: bar
    .env

not sure here. --env-file could be a separate argument in tmt, which extends the --environment. But in this case something like --context-file should be added:)
But I definitely support the position of having one good way of doing things.
Let's go already existing path with files (@) and add a ticket to consider env_file and maybe context_file to replace @.
One more point here, that I don't see any need to have multiple files as a source of the environment vars.

I see @lukaszachy's point: The readability is not that good. Just tried to be consistent with the @ notation we already have. Btw, this was brought, I believe, by @happz from testing farm. He might provide some more context about where this notation comes from.

not sure here. --env-file could be a separate argument in tmt, which extends the --environment. But in this case something like --context-file should be added:)

Agree to keep the consistency, so if we go this way we should have --environment-file as well as --context-file added. If possible, I would discourage abbreviations like --env-file.

Let's go already existing path with files (@) and add a ticket to consider env_file and maybe context_file to replace @.

Plus one for this. Just to keep consistency with other options/config values (such as modified-only) the naming should be environment-file and context-file.

I see @lukaszachy's point: The readability is not that good. Just tried to be consistent with the @ notation we already have. Btw, this was brought, I believe, by @happz from testing farm. He might provide some more context about where this notation comes from.

Well, yeah, but not as a standalone token but rather as a string prefix, to announce "this string is not the value but rather a file containing the value" :)
My 2 cents, after reading the various examples above:
# load key:var mappings from a given file
environment<EMAIL_ADDRESS>
# key1 specified here, value of key2 comes from a file
environment:
    key1: val
    key2: @file.txt

But this hits the wall when it comes to mixing with variables in YAML.
environment:
    key1: val
    # how to "load" N variables from @file.yaml into this mapping?

Or multiple files:
environment:
    key1: val
    _: @foo.yaml
    _: @bar.yaml

I'm not sure how to write the case for "merge keys N to M from file foo.yaml with what's already in environment mapping" in a readable form...
With a dedicated option like --environment-file (and environment_files key in YAML) using @ as a prefix wouldn't be needed, it makes sense only when all you have is --environment, to allow both kinds of inputs (and with multiple --environment the merging of multiple files is trivial - writing it down in YAML is the issue).
@ as a standalone key looks pretty weird to me, like shoehorning files into a place where they have no place. Adding environment-file + environment_file sounds to me like the most promising way forward, leading to a more consistent interface.
environment:
    key: val
    "@": file.yaml  # I'd read this as "create envvar<EMAIL_ADDRESS>not "read vars from foo.yaml" since "foo.yaml" does not start with `@`

With --environment-file and environment_file, I'd say things would resolve quite nicely:
environment:
  key1: val
  key2: val

environment_file:
  - .env
  - foo.yaml

# even when complemented with command-line
tmt ... --environment-file bar.yaml (or even --environment @bar.yaml)

Both --environment @bar.yaml and --environment-file could be supported, for the simple use cases, but for more complex, I'd stay away from forcing environment_file-like content into environment under some special key (@).
I'd say @<filepath> still has its use cases, but it was never supposed to solve everything - it's a nice shortcut to have when one wants to load something from a file, a set of variables, or a value of a given variable, but it's probably not well suited to replace environment-files-like feature since @<filepath> is hard to use when it comes to multiple files and mixing in-spec variables with those coming from a file. Having a dedicated section for files looks like the best option to me, while keeping @ around for the limited set of cases where it does actually help.

@happz, thanks a lot for providing a detailed reasoning. Looking at further examples you've brainstormed and points mentioned reminded me one of the zens of python

Explicit is better than implicit.

And the environment-file notation goes this way. Let's take this approach to prevent possible @ sign confusions. Finally, regarding the file format: Do you agree what I proposed above?

If the file does not have a yaml or yml extension it would be parsed using shlex.split().

@ZhukovGreen, do you need the shell key=val format? Or would you be ok with yaml support only?

@psss, unfortunately, .env format should be only like KEY=VALUE. There is tooling we already use and it relies on it. Moreover, it is a standard for dot env

yes, dot env is nicely parsable by shlex.split (https://github.com/psss/tmt/pull/795/files#diff-ccf2e34f34654a98eb09412ae8a65d9c8b1650c7697189603998f54a34e906b9R1072)

@happz, thanks a lot for providing a detailed reasoning. Looking at further examples you've brainstormed and points mentioned reminded me one of the zens of python

Explicit is better than implicit.

And the environment-file notation goes this way. Let's take this approach to prevent possible @ sign confusions. Finally, regarding the file format: Do you agree what I proposed above?

If the file does not have a yaml or yml extension it would be parsed using shlex.split().

Probably safe, I guess, I'd just be worried if this would be the rule in the following use case:
key1=val1 key2=val2

environment:
  FOO: bar
  BAZ: @foo.txt  # the file above

Instead of BAZ="key1=val1 key2=val2" I'd get, hm, not sure what I would get since BARwouldn't be spawned at all, and I'd getkey1=val1andkey2=val2` envvars?
In the case of environment_file, this seems to be safe and acceptable behavior:
BAZ="key1=val1 key2=val2" SOME_OTHER_VARIABLE=...

environment_file:
  - foo.txt

But if the would-be applied to all places where stuff is loaded from a file, then it would prevent me from loading a variable value from a file.
Maybe it's not a bag thing, TBH, in that case, I'd maybe disallow key: @filepath in the specification entirely... The more I think about it, there are conflicts popping up:

(CLI) --environment @foo.yaml - nice to have, splits the content of the file
(CLI) --environment @foo.txt - same, but with shlex.split()
(in spec) environment: key: @foo.yaml - does not split, loads as-is into key envvar - or not? It can't split the content into distinct variables, since that would mean merging the content with environment content, and you have environment_files for loading variables from files... But then it's going to behave in a different way when used here than when used with --environment :/

Maybe prohibiting @ use in the specification, in the YAML file, is the right way, disregarding this idea of mine. Seems like it'd open too many holes for inconsistent behavior. On command-line, it always splits the given file into variables, done. When used under environment, not so nice to use anymore :/
After all this self-torture, --environment @foo.not-yaml + shlex.split seems as valid as --environment @foo.yaml, the same rule would be applied for files under environment_files and --environment-files, but there should be no @ under environment key. To load a large value from a file, constructing a file for --environment-files looks like a more consistent way.

@ZhukovGreen, @happz, it's true that in Convert2RHEL we use the KEY=VALUE format of the .env file. But it would not be a big deal for us to change it to yaml.

@bocekm I disagree, to keep env vars in simple yaml format is not as simple as seems to. There are existing tooling around dot env format as well as a lot of integrations with this format. Dot env allows to keep config which includes secrets, however, yaml is for open configs, so dot env is a more flexible solution for us.

After all this self-torture, --environment @foo.not-yaml + shlex.split seems as valid as --environment @foo.yaml, the same rule would be applied for files under environment_files and --environment-files, but there should be no @ under environment key. To load a large value from a file, constructing a file for --environment-files looks like a more consistent way.

Agreed. This makes sense to me as well. So, to sum up, the plan is to support the following combinations:
For config files:
environment-file: variables.yaml    # parse from a yaml dictionary
environment-file: variables.yml     # parse from a yaml dictionary
environment-file: variables.other   # parse using shlex.split()

For command line:
--environment-file variables.yaml   # parse from a yaml dictionary
--environment-file variables.yml    # parse from a yaml dictionary
--environment-file variables.other  # parse using shlex.split()

And backward-compatible shorcuts:
--environment @variables.yaml       # parse from a yaml dictionary
--environment @variables.yml        # parse from a yaml dictionary
--environment @variables.other      # parse using shlex.split()

Multiple values possible for both command line and config:
--environment-file variables.yaml --environment-file variables.sh

environment-file:
  - variables.yaml
  - variables.sh
  - .env

That's it. No special @ handling in configs. Anything else to be clarified?

The proposal summary has been reviewed on the last hacking session with no additional comments, so it seems we're good to go this way. @ZhukovGreen do you plan to finish the implementation? One note to the pull request: It would be nice to extend the /tests/core/env test a bit to add coverage for the newly introduced functionality.

@psss yes I will finish it

@psss what did you mean by variables.sh in
environment-file:
  - variables.yaml
  - variables.sh    <<<<<<<<<<<<<<<<<<<<<<<<<
  - .env

kind of:
#! /bin/bash

export A=B
export C=D

?

Just shlex.split() as the .env file. The logic should be if yaml or yml extension detected parse as a yaml document otherwise consider the basic key=val format.

@psss @thrix @lukaszachy @happz ready for a review

Updated MR descritption

Added note to MR description about a little bit modified way of writing tests

@thrix any idea why this test:
➞  tmt tests show environment-file                                                 [git:feature/load-env-from-file] ✖
/tests/core/environment-file
     summary Check the work with env_file
 description Verify that - variables from env_file gets propagated to
             the environment - env_file type could be .env, yaml or sh
             formats (test each case) - conflict handling (env_file) and
             environment has the same vars with different values - cli
             option --environment-file env_file takes precedence
     contact Petr Šplíchal<EMAIL_ADDRESS>        test ./test.sh
        path /tests/core/environment-file
   framework beakerlib
      manual false
     require python3-pytest, python3-click and beakerlib
    duration 5m
     enabled true
      result respect
         tag container and virtual
        tier 1

fails on tft with the following log:
Warning: Permanently added '<IP_ADDRESS>' (ECDSA) to the list of known hosts.

::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
::   Run pytest session
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

:: [ 11:44:10 ] :: [  BEGIN   ] :: Running 'pytest -svv'
/usr/share/beakerlib/testing.sh: line 891: pytest: command not found
:: [ 11:44:10 ] :: [   FAIL   ] :: Command 'pytest -svv' (Expected 0, got 127)
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
::   Duration: 0s
::   Assertions: 0 good, 1 bad
::   RESULT: FAIL (Run pytest session)

::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
::   /tests/core/environment-file
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

:: [ 11:44:10 ] :: [   LOG    ] :: JOURNAL XML: /var/ARTIFACTS/work-coreixYjLw/plans/features/core/execute/data/tests/core/environment-file/journal.xml
:: [ 11:44:10 ] :: [   LOG    ] :: JOURNAL TXT: /var/ARTIFACTS/work-coreixYjLw/plans/features/core/execute/data/tests/core/environment-file/journal.txt
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
::   Duration: 1s
::   Phases: 0 good, 1 bad
::   OVERALL RESULT: FAIL (/tests/core/environment-file)

@psss @lukaszachy @happz need your help guys with the 1 CI job fails (see https://github.com/psss/tmt/pull/795#issuecomment-882502083)

@ZhukovGreen RHEL8 version of python3-pytest doesn't provide 'pytest'. There are only -3 and -3.6 suffixed binaries

after https://github.com/psss/tmt/pull/795/commits/69ec3d830a2c6de644f14b8b2f4a5af82e699a54 it is still failing wihtout any change. Need your support here

@ZhukovGreen Looking at other pytest based tests in tmt I see they use python3 -m pytest instead of pytest binary directly. Which should work regardless the binary name (fedora vs epel-8)

Hurray!!! @lukaszachy thanks for your help

Feature tests written in a bit different way - all the testing scenarios moved to pytest, so beakerlib is just run pytest -svv. I believe by using this approach, the burden of writing tmt tests for external devs will be significantly reduced

I've reviewed the different test approach and consulted this on the tmt hacking session today. The way of using a beakerlib test to run a single pytest command does not make much sense. It could be converted to a single line shell test. But furthermore, the way of wrapping shell commands to be tested into python code to check for exit code and output seems much more confusing and less readable than two lines of beakerlib code.
To demonstrate this, the following code:
# check if all tmt tests defined in ./data passed
res = runner.invoke(
    tmt.cli.main,
    ["run", "-vvvddd"],
    catch_exceptions=False,
    )
assert res.exit_code == 0
assert "total: 2 tests passed" in res.output

Could be written in this way using beakerlib:
rlRun -s "tmt run -vvvddd" 0 "check if all tmt tests defined in ./data passed"
rlAssertGrep "total: 2 tests passed" $rlRun_LOG

Would you mind rewriting the test to beakerlib to make it consistent with the rest of the test suite? Or explain what the benefits of wrapping shell commands into python code should be?

@thrix Miro, could you help to troubleshoot the packit/testing-farm-epel-8-x86_64 pipeline?

@psss thank you Petr, your changes looks good to me

Before merging, one more security-related question related to reading variables from files: Let's have the following simple plan, which could be hidden among let's say other 10-20 plans in the repository:
execute:
    script: send-environment-variable
environment-file:
  - /safe/place/env

Anyone who runs tmt run provision --how virtual on their laptop, feeling safe that the tests will be executed under vm, could unintentionally send a secret content... If one comes to an unknown repository, it will be no more safe to execute tests from your laptop, even if virtual provision is used.
@ZhukovGreen, @lukaszachy, @thrix, @FrNecas, what do you think? Does it look dangerous to you?

Before merging, one more security-related question related to reading variables from files: Let's have the following simple plan, which could be hidden among let's say other 10-20 plans in the repository:
execute:
    script: send-environment-variable
environment-file:
  - /safe/place/env

Anyone who runs tmt run provision --how virtual on their laptop, feeling safe that the tests will be executed under vm, could unintentionally send a secret content... If one comes to an unknown repository, it will be no more safe to execute tests from your laptop, even if virtual provision is used.
@ZhukovGreen, @lukaszachy, @thrix, @FrNecas, what do you think? Does it look dangerous to you?

How is that different to what can happen NOW with environment attribute?
IMHO we need to make environment/environment-file applicable to selected steps only (e.g. I imagine that secrets will be mostly used in provision, maybe prepare and possibly reporting.

How is that different to what can happen NOW with environment attribute?

The environment key is part of the tmt config, which is in the git repo, while loading from a file can reveal content from user's laptop.

How is that different to what can happen NOW with environment attribute?

The environment key is part of the tmt config, which is in the git repo, while loading from a file can reveal content from user's laptop.

Ah, I see. Maybe disable accessing local files? So you can only  use files from git repo or http(s)?

Maybe disable accessing local files?

Hmmm, but I believe that was @ZhukovGreen's main use case: To be able to read secrets from an env file so that they can be used for testing. The feature itself seems to be a bit dangerous...

@lukaszachy @psss what if we'll restrict the environment-file to relative path only (relative to the repo root). URL option should stay as is

@psss @lukaszachy Also this comment should be considered https://github.com/psss/tmt/pull/795#discussion_r679976942

@lukaszachy @psss what if we'll restrict the environment-file to relative path only (relative to the repo root). URL option should stay as is

If it works for your use case then +1. We'll just need to make sure to check for 'escaping from repo roote.g.../../.../../etc/...`

Relative path with preventing escaping could possibly be a way. In chat @lukaszachy also suggested introducing a new option tmt --allow-local-access or something similar. This also sounds good to me.

In the end I think allowing only files from repo or remote is better for what is needed right now. In the future, IF someone needs access to local files we could implement that + option.

@psss @lukaszachy with the relative path I think the resolution is good here https://github.com/psss/tmt/pull/795/commits/fc2fc26b1ddce0f145530cc81ae9b98483bca1ff.
But I am quite against the logic with URLs of the same repository. This approach destroys the whole logic of env vars from URL. What is the sense of it in this case? You already have the repo cloned, so you can have vars via a local path.
I think URL shouldn't be restricted, because otherwise, there is no sense to have it.

I found some little issues with the relative path. Previous solution wasn't solve the problem of escaping from the cwd (..)

Now works as expected

@psss I am on it

Thanks for the update. During review I've identified several problems:

Using current working directory as a base for reading environment files
Missing file path when when reporting error messages
Equal sign on the right side of the dotenv line

All of them should be now addressed in af814e6. @ZhukovGreen, @bocekm, please have a look to confirm everything's ok.

/packit test

@psss thx for helping me out. Your solution is more bulletproof. I reviewed it - looks good to me

Thanks for the quick review, @ZhukovGreen. Rebased on latest master to bring the ruamel implementation of yaml parsing.

/packit test

/packit test


Implement physical plan serialization for COPY plans CsvLogicalExtensionCodec

Is your feature request related to a problem or challenge?
As part of https://github.com/apache/datafusion/pull/11060, @devinjdangelo made file format support into a Trait which is good!
However the code to serialize these new (dynamic) structures is not yet implemented
As @devinjdangelo  says https://github.com/apache/datafusion/pull/11060/files#r1650268578

Users depending on the ability to serialize COPY plans (e.g. ballista) will need this TODO to be completed before upgrading to any version of datafusion including this change.

It turns out there are no unit tests for them either so no tests failed
Describe the solution you'd like
Implement the named codec for serializing plans and a test for it
Describe alternatives you've considered
The code is here: datafusion/proto/src/logical_plan/file_formats.rs
The test would go here: https://github.com/apache/datafusion/blob/main/datafusion/proto/tests/cases/roundtrip_physical_plan.rs
Note there is already coverage for LogicalPlans here:
https://github.com/apache/datafusion/blob/d2ff2189dfb8b4624ae2c08846cd713871b37d8c/datafusion/proto/tests/cases/roundtrip_logical_plan.rs#L325-L346
Additional context
There are several other codecs needed:

JsonLogicalExtensionCodec
ParquetLogicalExtensionCodec
ArrowLogicalExtensionCodec
AvroLogicalExtensionCodec

However, I think we need to get one example done and then we can file tickets to fill out the others
Maybe this is what @lewiszlw was getting at with https://github.com/apache/datafusion/pull/11095 / https://github.com/apache/datafusion/blob/main/datafusion-examples/examples/composed_extension_codec.rs

take

starts to work on this one, was delayed by the substrait one.

When I was doing this one, I added a test like
#[tokio::test]
async fn roundtrip_physical_plan_copy_to_sql_options() -> Result<()> {
    // Create a new SessionContext
    let ctx = SessionContext::new();
    let file_type = format_as_file_type(Arc::new(CsvFormatFactory::new()));
    // Create a CSV scan as input
    let input = create_csv_scan(&ctx).await?;
    let plan = LogicalPlan::Copy(CopyTo {
        input: Arc::new(input),
        output_url: "test.csv".to_string(),
        partition_by: vec!["a".to_string(), "b".to_string(), "c".to_string()],
        file_type,
        options: Default::default(),
    });

    // Convert the logical plan to a physical plan
    let planner = DefaultPhysicalPlanner::default();
    let physical_plan = planner.create_physical_plan(&plan, &ctx.state()).await?;
    roundtrip_test(physical_plan)
}

I find that it seems like the roundtrip in physical plan test should be ok since the fileformatfactory is translated into Fileformat automaticly here
LogicalPlan::Copy(CopyTo {
                input,
                output_url,
                file_type,
                partition_by,
                options: source_option_tuples,
            }) => {
                let input_exec = children.one()?;
                let parsed_url = ListingTableUrl::parse(output_url)?;
                let object_store_url = parsed_url.object_store();

                let schema: Schema = (**input.schema()).clone().into();

                // Note: the DataType passed here is ignored for the purposes of writing and inferred instead
                // from the schema of the RecordBatch being written. This allows COPY statements to specify only
                // the column name rather than column name + explicit data type.
                let table_partition_cols = partition_by
                    .iter()
                    .map(|s| (s.to_string(), arrow_schema::DataType::Null))
                    .collect::<Vec<_>>();

                let keep_partition_by_columns = match source_option_tuples
                    .get("execution.keep_partition_by_columns")
                    .map(|v| v.trim()) {
                    None => session_state.config().options().execution.keep_partition_by_columns,
                    Some("true") => true,
                    Some("false") => false,
                    Some(value) =>
                        return Err(DataFusionError::Configuration(format!("provided value for 'execution.keep_partition_by_columns' was not recognized: \"{}\"", value))),
                };

                // Set file sink related options
                let config = FileSinkConfig {
                    object_store_url,
                    table_paths: vec![parsed_url],
                    file_groups: vec![],
                    output_schema: Arc::new(schema),
                    table_partition_cols,
                    overwrite: false,
                    keep_partition_by_columns,
                };

                let sink_format = file_type_to_format(file_type)?
                    .create(session_state, source_option_tuples)?;

                sink_format
                    .create_writer_physical_plan(input_exec, session_state, config, None)
                    .await?
            }

So it causes my confusion here how to actually add a test here, appreciate your guidence @alamb

So it causes my confusion here how to actually add a test here, appreciate your guidence @alamb

Do you mean that the test you wrote above passes without any code changes? Maybe @devinjdangelo has some idea

In the original PR I added a CsvLogicalExtensionCodec which ignores all file format specific options and simply deserializes to the default options. In your test you are also using the default options, so it will pass with the existing extension codec. If you change it to anything other than defaults, it should fail.
let plan = LogicalPlan::Copy(CopyTo {
        input: Arc::new(input),
        output_url: "test.csv".to_string(),
        partition_by: vec!["a".to_string(), "b".to_string(), "c".to_string()],
        file_type,
        options: HashMap::from_iter(vec![("format.delimiter", ";")]),
    });

So it causes my confusion here how to actually add a test here, appreciate your guidence @alamb

Do you mean that the test you wrote above passes without any code changes? Maybe @devinjdangelo has some idea

yes, that's why I am confused

The format factories can optionally carry modified default write options. This is only used for DataFrame::write_* methods, so that they do not need to pass strings to modify write behavior.
https://github.com/apache/datafusion/blob/827d0e3a29c0ea34bafbf03f5102407bd8e9b826/datafusion/core/src/datasource/file_format/csv.rs#L60-L62
The existing SerDe implementation does not handle the case when the format factory carries options itself. This will prevent systems like Ballista from correctly passing around options in its DataFrame::write_* methods. Here are the lines that are not fully implemented:
https://github.com/apache/datafusion/blob/827d0e3a29c0ea34bafbf03f5102407bd8e9b826/datafusion/proto/src/logical_plan/file_formats.rs#L74-L89
You can see that the encoding method does not actually write any bytes, and the decoding method just initializes a default struct, ignoring any written bytes.

let me take a look and

The format factories can optionally carry modified default write options. This is only used for DataFrame::write_* methods, so that they do not need to pass strings to modify write behavior.
https://github.com/apache/datafusion/blob/827d0e3a29c0ea34bafbf03f5102407bd8e9b826/datafusion/core/src/datasource/file_format/csv.rs#L60-L62
The existing SerDe implementation does not handle the case when the format factory carries options itself. This will prevent systems like Ballista from correctly passing around options in its DataFrame::write_* methods. Here are the lines that are not fully implemented:
https://github.com/apache/datafusion/blob/827d0e3a29c0ea34bafbf03f5102407bd8e9b826/datafusion/proto/src/logical_plan/file_formats.rs#L74-L89
You can see that the encoding method does not actually write any bytes, and the decoding method just initializes a default struct, ignoring any written bytes.

sure, I'll take a look then


Ignore kafka-streams namespace by default

According to https://github.com/quarkusio/quarkusio.github.io/blame/develop/_guides/kafka-streams.adoc#L469:

All the properties within the kafka-streams namespace are passed through as-is to the Kafka Streams engine.

So the kafka-streams.* namespace should be ignored by default.
Check https://github.com/quarkusio/quarkus-quickstarts/blob/master/kafka-streams-quickstart/aggregator/src/main/resources/application.properties
When manually excluding "kafka-streams.*", %test.kafka-streams.state.dir still shows up as unknown property

Regarding the profile properties not being excluded from diagnostics : This is due to MicroProfileValidator not excluding the profile segment of the property when matching against the exclude list pattern.
I'll create change against lsp4mp to deal with this.


Is Reentrance to a constructor an issue?

Remix warns about reentrance from the constructor.  Is that an issue?

If the constructor is running for an external account's contract creation or CREATE instruction,

hitting the same address with CALL would invoke the old code that existed before the creation (that has to be empty).  So the call immediately returns.
hitting the same address with CREATE is impossible.  Hitting the same address with CREATE2 is also impossible because of the padding 0xff in the address calculation.

If the constructor is running for a CREATE2 instruction, hitting the same address with CREATE2 is possible.  In that case, the storage is shared among the two execution contexts?

Actually, prior to init_code execution, the nonse of the account should become non-zero, so reentrance with CREATE2 just aborts.