Fix metrics performance regression

Codecov Report

Merging #13536 (7fc431e) into master (3b19fb7) will decrease coverage by 26.15%.
The diff coverage is 56.00%.

@@              Coverage Diff              @@
##             master   #13536       +/-   ##
=============================================
- Coverage     43.76%   17.60%   -26.16%
+ Complexity     9049     2666     -6383
=============================================
  Files          1377     1378        +1
  Lines         79867    79860        -7
  Branches       9708     9708
=============================================
- Hits          34952    14061    -20891
- Misses        41977    64559    +22582
+ Partials       2938     1240     -1698

Impacted Files
Coverage Δ

.../main/java/alluxio/master/file/meta/InodeTree.java
9.07% <ø> (-64.63%)
:arrow_down:

...xio/master/file/meta/InodeTreePersistentState.java
11.01% <40.00%> (-49.57%)
:arrow_down:

...mmon/src/main/java/alluxio/util/BucketCounter.java
55.55% <55.55%> (ø)

...n/java/alluxio/master/metastore/caching/Cache.java
30.10% <61.90%> (-64.58%)
:arrow_down:

...a/alluxio/master/file/DefaultFileSystemMaster.java
15.86% <100.00%> (-48.88%)
:arrow_down:

...mon/src/main/java/alluxio/shell/CommandReturn.java
0.00% <0.00%> (-100.00%)
:arrow_down:

...mon/src/main/java/alluxio/util/ExceptionUtils.java
0.00% <0.00%> (-100.00%)
:arrow_down:

...n/src/main/java/alluxio/wire/AlluxioProxyInfo.java
0.00% <0.00%> (-100.00%)
:arrow_down:

.../src/main/java/alluxio/wire/AlluxioMasterInfo.java
0.00% <0.00%> (-100.00%)
:arrow_down:

...src/main/java/alluxio/job/meta/JobIdGenerator.java
0.00% <0.00%> (-100.00%)
:arrow_down:

... and 664 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3b19fb7...7fc431e. Read the comment docs.

@ggezer  addressed comments, PTAL

alluxio-bot, merge this please

@HelloHorizon Please cherry-pick to branch-2.6.

alluxio-bot, cherry-pick this to branch-2.6 please

alluxio-bot, cherry-pick this to branch-2.6 please


Fix path when using bundled tigervnc

Fixes #43
Tested with local build/run per issue with turbovnc install commented out

Thanks, @pnasrat!


Adds support for Elasticsearch 5

This PR adds support to Elasticsearch 5, according to issue #1410
It has been tested on Elasticsearch 2 and on Elasticsearch 5 and works as expected.
There are three major changes:

Changes the referenced library from elastic.v3 to elastic.v5
Adds the context on validation functions (Do(context.Background()), as this is requested by the new library.
Changes the Godeps, removing old vendor and adding the new one.

This change is

I signed it

Now I really signed it :P

@AlmogBaku / @huangyuqi / @andyxning please review

@DirectXMan12 Thanks for the answer! Should I 'git revert' this, or open another PR?
Thanks!

@rikatz please tag me there

@AlmogBaku sure, I'm just testing stuff here and will open the new PR ASAP


make prefix configurable

make the _discord_ prefix for users and channels configurable

This becomes dramatically easier with #454, which allows you to craft intents from the suffix


Inverse maps

We will want to provide a generalized mechanism for inverse maps. What I propose is to have a boolean flag isInverted in each map object. When that flag is set, a different group of evaluation routines is called that (among other things) invoke inverse mapping functions of the map class. The class can choose to provide them, but they are set to throw errors by default.

Agreed. The isInverted property and default inversion error methods should be implemented in the conformalmap class.


(Minor) Typo: s/githib/github/

Summary
Fix a typo.
Requirements (place an x in each [ ])

[ ] I've read and understood the Contributing Guidelines and have done my best effort to follow them.

No contributing file

[x] I've read and agree to the Code of Conduct.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.Logan Kennelly seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.

Thanks! Merging this now. :)


producers::add zmq

Add a zmq producer.
The zmq producer passes messages through unprocessed so that further clients may process them. It does so by creating a zmq.PUB socket which binds to a preconfigured address.
By default it will publish messages under the topic "vin.txType", though this can be somewhat configured via:

A prefix
A suffix
A topic separator character (defaults to ".")
Connections are secured through ZMQ curve (the server must generate a public/secret keypair. client must know the server's public key prior to connecting.) The server may optionally allow only certain clients to connect via a config file with their public keys.

Metrics are collected similar to the kinesis producer (code for metrics copied and modified).
Building the ZMQ producer requires that libzmq is installed on the machine, which is annoying. To combat this I have separated out the zmq code into a file which is conditionally built. The code will only be built if the build flag has_zmq is defined, which by default it isn't. This means most people can still build without needing to install any C libraries but those who wish to use ZMQ have some hoops to jump through. If this is not an acceptable compromise I'm happy to discuss alternatives, but I definitely don't want to complicate building the build process for the 99% use case (no ZMQ). Tested in the golang:1.21 docker base image, built fine w/o needing to install zmq

@jordan-bonecutter  looking pretty good! mind squashing your commits into one?

@jordan-bonecutter looking pretty good! mind squashing your commits into one?

Done!

Working on this again, will be in a review-able state later today

Hopefully github action passes this time, I ran a (hopefully) equivalent docker and was able to get tests to pass.

can you update the PR description to match latest changes? I don't the topic separator logic, etc is relevant anymore?

Sorry, not sure what's going on. This works locally. Will need some time to look into it

@agbpatro we're looking into this and think we've found a solution. will push soon.

Merged. Thank you @jordan-bonecutter  good work!

Awesome! Thanks all, especially @agbpatro for the help!


Problems with Avast virus protection

I have to add NEM as an exclusion when installing.
Once NEM is running about every 30 seconds I get this annoying pop-up. http://postimg.org/image/ul2st4l6z/

duplicate #300


How to integrate this ticket system with existing project?

Hi,
I want to execute the ticket system on my existing project, is there any documentation or guidance are there?

I need the integration of the ticket creation interface with my existing project

When I use this package on Django 3.1 it throws an error and not running  and also it degrades the Django from 3.1 to 2.1

django-helpdesk does not currently support Django 3
On Fri, Oct 2, 2020 at 2:17 AM BAVASHIEKBAREETH<EMAIL_ADDRESS>wrote:

When I use this package on Django 3.1 it throws an error and not running
and also it degrades the Django from 3.1 to 2.1
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/django-helpdesk/django-helpdesk/issues/872#issuecomment-702547599,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABEKHU2D66E2Q6ULYRBRXLDSIVV55ANCNFSM4SAO7BVQ
.

Thanks @gwasser

There is now good iframe support https://django-helpdesk.readthedocs.io/en/0.5.0/integration.html


Regression/Optimization Checking

Hi there,
I've really enjoyed testing out this package and considering using it for a project that's going to rely heavily on benchmarking. I'm interested if there is any plan for the reporter to report on the diff between the last result.txt saved and the current benchmark run. It would be really useful to monitor regressions and optimizations during development.
It would also be really useful for CI pipeline.

No plan for regression monitoring yet. But a poor man's implementation might be to just output the final report to a text file that you can keep in version control with Git. Would that suffice for your purpose?

No plan for regression monitoring yet. But a poor man's implementation might be to just output the final report to a text file that you can keep in version control with Git. Would that suffice for your purpose?

Hi @pckhoi, sorry I didn't see your reply. Thanks very much - yeah I was thinking of doing something like what you suggest.


The integrated terminal uses fish shell and it lacks the auto completion. Also the working directory doesn't mounts by itself ,have to cd into the directory.

VSCode Version: Code 1.8.1 (ee428b0eead68bf0fb99ab5fdc4439be227b6281, 2016-12-19T14:49:23.350Z)
OS Version: Windows_NT ia32 10.0.15014
The integrated terminal uses fish shell and it lacks the auto completion. Sometimes the terminal hangs for few seconds on tab completion. Also, the working directory doesn't mount by itself , have to cd into the directory.

Hey @krtkeyan, thanks for reporting! Unfortunately I can't reproduce the issue. A few Qs to help narrow it down:

when yous say it "uses fish shell", do you mean you've configured it to use fish on bash on Windows? Or something else?
When you say the working directory doesn't automount itself - what's the directory that it starts you in? And what's the directory that you expect to start in?
When you say it "lacks auto completion" - is this for everything you try, or only some things? Does it include tab completion as well?
When you say "the terminal hangs for a few seconds on tab completion", what happens after the hang? Does what you were typing complete correctly? Is this behavior reproducible, or somewhat flaky?

Thanks in advance!


Parameters seem not to work with Azure Functions

I am currently passing one parameter into the workflow that contains the full resourceId path, but when trying to use that parameter on the id property of the Azure function I am getting the following error:
2020-07-15T05:50:50.0487640Z ##[error]LinkedInvalidPropertyId: Property id '@parameters('AzureFunction').id' at path 'properties.definition.actions.For_each.actions.FILE_ASSET_OCCUPANCY_BUDGET.actions.TecomAnalyticsXLSX2CSV_5.inputs.function.id' is invalid. Expect fully qualified resource Id that start with '/subscriptions/{subscriptionId}' or '/providers/{resourceProviderNamespace}/'.
It seems to me the parameter is not being evaluated and literally used instead.
Regards,
LS

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: fa5d67cf-2fd8-ab96-8b65-f41a9d7565b8
Version Independent ID: 8f74dc66-8c2c-2ec4-cb4d-cea94ca9acb0
Content: Create, edit, or extend logic app JSON workflow definitions - Azure Logic Apps
Content Source: articles/logic-apps/logic-apps-author-definitions.md
Service: logic-apps
GitHub Login: @ecfan
Microsoft Alias: estfan

@djpirra Thanks for the question! We're investigating this and will get back to you shortly.

@djpirra As Jeff mentions in this SO post, the resource ID of the function can only be set at deploy-time and not runtime.
You could instead populate the id property as follows:
"function": {
    "id": "[resourceId('Microsoft.Web/sites/functions', parameters('functionAppName'), variables('functionName'))]"
}

Please check this quickstart template on GitHub for the full working example.

@djpirra Just following up here to check if the above response helped.

@djpirra At this point, it appears that there are no edits or updates needed to the document in question. Hence closing this issue for now. If there are any further questions regarding the documentation, please tag me in your reply and we will be happy to continue the conversation.


PTVS discovers fewer tests in VS2017 compared to VS2015

https://github.com/Microsoft/knack
When I create a VS2015 solution for this repo, it discovers 165 tests, the same as when I discover tests from the command line.  However, when I create a VS2017 solution file, it only discovers 119 tests. As a result, I find myself constantly having to switch back to VS2015 just to debug tests. This has been a problem in the CLI repo since the release of VS2017.

I'm getting 164 tests discovered. I'm on the latest PTVS from master, which is slightly ahead of dogfood build 15.8 Preview 2.
Here's how I've setup the project, with a Python 3.6 venv:

Did you setup your project similar to mine?

Pretty much exactly like yours

I updated to the latest version of VS and PTVS and it looks like all is well. Love the new changes!


support COLLATE clause to any of the arguments

Is your feature request related to a problem? Please describe.
support COLLATE clause to any of the arguments, e.g. literal const, column, expression, etc.
The content of this issue is not the collation in create table.
Describe the solution you'd like
If we don't have plans to support collation yet, maybe we just need to simply handle collate in the frontend binder instead of throwing an error.
https://github.com/risingwavelabs/risingwave/blob/7db34d516356d11ed2562e798cbb28f66108f715/src/frontend/src/binder/expr/mod.rs#L196-L200
Describe alternatives you've considered
No response
Additional context
https://www.postgresql.org/docs/16/sql-expressions.html#SQL-SYNTAX-COLLATE-EXPRS
Our user is using drmio, and he complained that risingwave can’t support collation.
dremio is a data lakehouse platform, the sql query it generates uses a lot of collate clause.

maybe we just need to simply handle collate in the frontend binder instead of throwing an error

It seems not that easy. 😕 The collation affects the execution behavior like sorting and upper/lower.

Well, this is really complicated... So we should be able to accept COLLATE "C" and COLLATE "POSIX" now? (AFAIK, the two are exactly the same but written in different ways)
Dremio has not been observed using other collations, so we should be able to fix it by doing this first. 🙈

Oh I didn't see this issue and created another one. @jetjinser would you like to take it?

transfer to #13171.


Fix sharing snippets

The sharing snippets for FB and G+ were not getting passed the supported browser check (the bot check in the user agent gem is just a regex match for "bot" :dizzy_face:) and the terms check, so I extracted that out and added some special cases for Facebook and g+. It's easy to add in more exemptions if we need to, for whatever reason.
This seems like a big change, but I've just moved the user agent validation out in to a separate class as it's a separate responsibility from the app controller, and to make it testable.

:dancer: :dancer: :dancer: :dancer:


WIP Mnt fixup after mds changes + API changes

There are still some tests failing because I broke mds in ways that it's test suite does not catch, but dataportal does.

Also still need to re-work the dataportal internals to use the (still being finished) simplified mds api (get_Y_given_X for Y in documents and X in [uid, oid, runstart_uid, event_descritpor_uid])

Annoyingly, this PR is net-positive.....for now.

Maybe I missed this conversation, but when was it decided that all of the RunStart keys should not be copied into the header?
As of v0.1.0:
In [3]: print(hdr)

Header
======
beamline_config : None
beamline_id     : csx
group           :
owner           :
project         :
run_start_uid   : 758bc8b2-b5f2-4cf5-a841-4490ccdd8c62
sample          :
scan_id         : 5
start_datetime  : 2015-09-09 16:46:18.113764
start_time      :<PHONE_NUMBER>.113764

  EventDescriptor
  ---------------
  +-----------------+--------+----------+-------+--------+
  | data keys       | dtype  | external | shape | source |
  +-----------------+--------+----------+-------+--------+
  | Tsam            | number |   None   |   []  | PV:pv3 |
  | linear_motor    | number |   None   |   []  | PV:pv1 |
  | scalar_detector | number |   None   |   []  | PV:pv2 |
  +-----------------+--------+----------+-------+--------+
  run_start       : 758bc8b2-b5f2-4cf5-a841-4490ccdd8c62
  time            :<PHONE_NUMBER>.1183035
  time_as_datetime: 2015-09-09 16:46:18.118303
  uid             : 9133ff37-cd48-4091-b7d2-2d66477b5c5f

As of commit 76e6ae6 in this PR
In [3]: hdr = db[-1]

In [4]: print(hdr)

header
======
exit_reason     : None
exit_status     : None
run_stop        : None
sample          :
scan_id         : 5
start_time      :<PHONE_NUMBER>.113764
stop_time       : None
time            :<PHONE_NUMBER>.113764
uid             : 758bc8b2-b5f2-4cf5-a841-4490ccdd8c62

  EventDescriptor
  ---------------
  +-----------------+--------+-------+--------+
  | data keys       | dtype  | shape | source |
  +-----------------+--------+-------+--------+
  | Tsam            | number |   []  | PV:pv3 |
  | linear_motor    | number |   []  | PV:pv1 |
  | scalar_detector | number |   []  | PV:pv2 |
  +-----------------+--------+-------+--------+
  run_start       : 758bc8b2-b5f2-4cf5-a841-4490ccdd8c62
  time            :<PHONE_NUMBER>.1183035
  uid             : 9133ff37-cd48-4091-b7d2-2d66477b5c5f

  RunStart
  --------
  beamline_id     : csx
  group           :
  owner           :
  project         :
  sample          :
  scan_id         : 5
  time            :<PHONE_NUMBER>.113764
  uid             : 758bc8b2-b5f2-4cf5-a841-4490ccdd8c62

@ericdill Yeah that doesn't look right. I take all of this as yet more indication that Header is not quite the right thing. I think we have overthought this.

@tacaswell can you rebase this on master? Dan merged in some/many of these commits. It would be great to see what the diff is now


Uint256 hints integration tests

This is not ready to merge right now.
The cairo integration tests are detected because I have modified the cairozero_test.go file.
This PR is aims to do the tasks mentioned here : #413
Current Progress :

[ ]  Uint256Add
[ ]  Uint256AddLow
[x]  Split64
[ ]  Uint256Sqrt
[ ]  Uint256SignedNN
[ ]  Uint256UnsignedDivRem
[ ]  Uint256MulDivMod

@Sh0g0-1758  why is Split64 in Uint256? It works on a field element in the range [0, 2^192), splitting in in its low 64-bit and high 128-bit parts

hey @Sh0g0-1758  how is it going with this?

Ran into a roadblock yesterday. Resolved it and will hopefully have this ready by EOD.

Closing this as this was not forked from finalize segments.


Can I start or stop a specify program by pyLSV2?

I can read program status by con.get_program_status().
Can I stop/start a specify program ?

not as far as I know. maybe via dnc but not with only lsv2

@pangafu after reviewing some documents the only thing I could find was something called drip feeding but this still requires the user to press the start button on the control. This is probably a safety feature.

@drunsinn
Thansk a lot~


Migrated to library Nuxt design

To merge into the next branch.
Initial skeleton of the next version of Prez UI using an NPM library approach with Nuxt. We will develop this version on the next branch until it is ready for release as Prez UI v4, using a branching structure like next/jamie/*.

@hjohns there is no public/ or src/ directory in the base folder anymore. The top level directories are the 4 projects previously discussed. Refer to https://github.com/jamiefeiss/prez-ui/tree/main for the directory structure. The file browser on the left when viewing PR changes on GitHub might be misleading.


Update fs2-reactive-streams to 3.1.2

Updates co.fs2:fs2-reactive-streams from 2.5.9 to 3.1.2.
GitHub Release Notes - Changelog - Version Diff
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "co.fs2", artifactId = "fs2-reactive-streams" } ]

labels: library-update, semver-major

Superseded by #536.


Nodejs Project Management

To me Typescript seems to be a better choice for writing code where the code base is huge and requires more consistency.
But the examples and experiences i find around the internet are more on client side , may be coz of the Angular2.0 decision to use AtScript/Typescript.
Though Typescript adds a lot of power to Javascript , there is something that keeps on pissing me off is how to manage codebase with duplicate files.  According to my understanding i can have 2 ways to do it.
1st Way
Use a build tool like gulp/grunt , watch for changes and compile Typescript to Javascript in the same folder. which may look something like this.
├── models/
│ ├── User.ts
│ ├── User.js
│ ├── Likes.ts
│ ├── Likes.js

2nd Way
Another way will be the same but of instead of outputting Javascript into the same folder , i can clone the entire workspace.
├── typescript
│ ├── models/
│ │ ├── User.ts
│ │ ├── Likes.ts
├── javascript
│ ├── models/
│ │ ├── User.js
│ │ ├── Likes.js

Where both are not the best solution for me , is there any other way to manage codebase for nodejs. For client it is simple as we have only one output file.

I must say that this also pisses me off. I recommend the TypeScript team implement something like Browserify's bundling. It outputs everything into one file and you can both run it on the server and client.

@thetutlage you are looking to bundle all your outputs in one module, e.g. models.js (include both user.ts and likes.ts)?
if so, @tinganho has filed a similar issue (#2743) for bundling.

@mhegazy Does it really make sense to bundle server code into one file ?

then i did not understand your proposal/request. can you elaborate?

Sure , it is easy on client side to make easy of Typescript as i can easily bundle my entire code into 1 file.
But on server it is a problem as i will have a project structure which is not manageable according to me. So i was keen to know is there any best practice one can follow when using Typescript on Server ( with Node js ).
So in short the question is only related to code base management.

I do not think i have "best practices" to share per se. we use --out for the compiler code base, and emit only one file for our node package (see https://github.com/Microsoft/TypeScript/blob/master/bin/typescript.js).
In general I think bundling is similar on the server/client (in other words CommonJS/AMD). The same issue persists, you want as a developer to have a many files that represent your component, but want to hide this from your users by distributing a single file representing your component. I think if we end up doing bundling it will work for both AMD and CommonJS.

Okay cool , it makes sense . I was just wondering may be there is a 3rd way.

@thetutlage I am doing the following, using gulp building.

all files go into logical places, irrespective of type js / ts.
gulp takes a stream of js, and outputs it further down the pipe.
js stream is merge2 (npm it) with gulp-typescript-ed pipe, which takes ts and compiles then to js. Result is a pipe with js in it.
n) pipe may go into concat, browserify, just copy into dist folder, all depending on target(s).
Skip (2) if all your sources are in ts.
Set your IDE not to emit js files. Let gulp do it in a manner controlled by your script. Set names for your destination folders to be logical (dist-node, dist-browser, etc.), since "javascript" says nothing about intention (is it a source, should reader pay attention to it?).

Looks like there is no more information needed for this issue. closing.


Romain/fix readme

changed some names, clarified the readme for mongodb

Codecov Report

:exclamation: No coverage uploaded for pull request base (master@22eca03). Click here to learn what that means.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master     #435   +/-   ##
=========================================
  Coverage          ?   89.33%
=========================================
  Files             ?       30
  Lines             ?     3470
  Branches          ?        0
=========================================
  Hits              ?     3100
  Misses            ?      370
  Partials          ?        0

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 22eca03...8cb0f39. Read the comment docs.


Fine Grained PassWord Policy

To view, edit and Add FGPP
FGPP are is based on msDS-PasswordSettings Class

Added by @omiossec ! Thank you!


Support for Rigid, Similarity, Translation Models

It would be straightforward to port some of the AffineModel framework to a base HomographyTransform class and extend that to allow Rigid, Similarity, and Translation transforms to be represented in the transform module.  While to my knowledge EM_aligner does not implement these models, it could be helpful for trakem2-assisted workflows.

For the EM aligned reason I might advocate for allowing these on render python side, but writing them as AffineModels to render until such time as that support is more pervasive


Redirect Doesn't Allow Me To Go To Other Links

I am trying to make a login redirect using the following code:
Router.onBeforeAction(function () {
if (!Meteor.user() && !Meteor.loggingIn()) {
this.next();
} else {
// required by Iron to process the route handler
this.redirect('/map');
this.next();
}
});
But, when I login and it successfully redirects me to /maps, I can't click any of my link on my navbar. The link are working fine, but, when I click on them it just redirects me back to /map.
Help will be much appreciated!
Also, here is the github link:
https://github.com/Aggr0vatE/testbasichelp

Figured out the answer by using this code:
Router.route('/login',
{ name: 'login' });
var requireLogin = function(){
if (! Meteor.user() ) {
this.redirect('login');
} else {
this.next();
}
}
Router.onBeforeAction(requireLogin, {except: ['login']});


Swagger - How to use

Dears,
I am a basic user of swagger, and i don't know the procedure of using the swagger.
First: In Pistache i found the rest_description.cc to implement the swagger, but in the git repository there is no file "banker-api.json" to understanding the JSON structure and the file contents. and also octal uses the "swagger-ui-2.1.4/dist" inside the code, in the git page of swagger-ui there is no version 2.1.4 and current version of swagger is 3.18.3 (::now() = 11/01/2020).
Second: there is another library called swagger-codegen it seems there is an integration with Pistache or similar alternative implementation ( in examples i see the pistache server) now i am confusing
what is the different between these libraries and which one is usable to create swagger(the API help UI) for the web services ?
PS: sorry for about my less knowledge about the swagger, i am not the UI expert and mainly i just working in backed side, but now i should use the swagger as a manual page for the internal developed API's

Hi every body,
after some try i found these information that shows the how Pistache support swagger :
in example rest_description.cc we setup the server to serve the swagger-ui-dist and the path of the URL that should call from browser this path point to the swagger-ui-dist directory
you should download the swagger-ui-dist and put it into the directory that mentioned in "uiDirectory" in start function.
now the url you should called is the server address plus the uiPath route ( in example file is /doc)
for example <IP_ADDRESS>/doc
now the Pistache acts as the webserver to display swagger ui.
there is another parameter, the apiPath :
apiPath is represent the JSON that swagger can read and parse it in its UI structure, that shows the api functions, routes and any thing you mentioned in the code visually.
banker-api.json will be automatically generated during the pistache rest_descriptor starts.
you can see the JSON file in browser with your link (example http(s)://<IP_ADDRESS>/banker-api.json)
if this file pass to the swagger UI the entire swagger UI page will be generated.

the point about Swagger-Codegen :

Swagger Codegen can simplify your build process by generating server stubs and client SDKs for any API, defined with the OpenAPI (formerly known as Swagger) specification, so your team can focus better on your API’s implementation and adoption.

it mean this getting the Swagger JSON file and create server & client based on your preferred programming language it also supports the pistache (currently the codegen version is not supporting the last pistache version. you should add some namespace and change make files and remove some old flags of pistache to compatible with the last version)
it does not do the actions of rest_description.cc. it means nothing related to descriptor here.
Now i get whole story.. may this thread will usable for others does not familiar with swagger, and need to be use it in their code.


Use .mailmap when displaying committer/author names & emails

This PR should resolve #562. If there is no .mailmap file, authors/committers are displayed as normal.

Applied patch - works like a charm, thank you!


Rough edges with statefulness

In https://github.com/fchollet/keras/issues/1217#issuecomment-163072470 you said

However, one remaining gotcha with statefulness is that the RNN states won't be updated if you use predict, evaluate, etc. In other words, statefulness only works in training mode. The reason being that predict & co are implemented to be idempotent. Changing that would require crafting an exception specifically for RNN statefulness.

Also, there's a "ValueError: operands could not be broadcast together" exception when data is not divisible by batch size. Maybe this should be handled gracefully. This is particularly annoying when you set a validation_split parameter in fit(), because the final data size is variable.

I updated my previous message because the error is not caused by validation_split itself, but there's a side effect when split data isn't divisible by batch size.

this issue has mostly been closed by an addition of a nice error message when you try to fit on a set of samples that is not divisible by the batch size. it would be cool to truncate the data automatically and issue a warning, but that's a design decision.
regarding the validation_split causing this error, i've added the following to my code to help avoid this problem:
def quantize_validation_split(validation_split, sample_count, batch_size):
    batch_count = sample_count / batch_size
    return float(int(batch_count * validation_split)) / batch_count

then you can call model.fit(..., validation_split=fix_validation_split(0.05, len(X), batch_size)). but it would be cool if keras did this for you inside fit().


Z-index issues at dashboard presentation site

I'll give you more details about these issues
Mobile only - hardware and software description
Google Chrome: 65.0.3325.109(Versão oficial) 32 bits
Revision: f412f53af6f3abcd6165b6cf0859dad2e88ff87d-refs/branch-heads/3325@{#626}
SO: Android 6.0.0; LG-K430 Build/MRA58K
JavaScript: V8 <IP_ADDRESS>
User Agent: Mozilla/5.0 (Linux; Android 6.0; LG-K430 Build/MRA58K) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.109 Mobile Safari/537.36
Main menu is under section content

Main menu is under select element

Button overflow

Weird behave caused by 2nd issue (select element)

PS: english isn't my main language

I created a separate repo for the main site and moved this issue in there as it made more sense. :)
https://github.com/DesignRevision/designrevision.com/issues/1


Fix or document Quilt incompatibility

This mod does not work on Quilt with the standard QFAPI; it crashes when trying to enter a world or load the Create New World screen; see https://mclo.gs/3emFHB8.  Iʼm not sure I quite follow the reasons why, but someone on the Quilt server suggested that if you changed https://github.com/Commander07/data-loader/blob/master/src/main/java/dataloader/mixin/MinecraftServerMixin.java#L18 to inject at TAIL instead of HEAD it should work.  They also said itʼd be recommended to use the Resource Loader API instead of using that directly.
I have not yet tried to build and test this change myself, which is why Iʼm not making a PR.

This has been fixed in version 1.1.5.


Add backward compatibility test for __cuda_array_interface__

Followup of #2491. Resolves #2521.
cf. https://github.com/cupy/cupy/pull/2534#issuecomment-539341617.

Thank you for resending the PR.
Using a loop inside a test for changing configurations is not a good practice, because on failure it would be difficult to tell which configuration was failed, and also a failure in a single configurations would interfere other configurations.
Could you replace the loops with @testing.parameterize?
As this decorator can only applied to TestCase classes, you will have to refactor the tests so that the structure will be something like:
@testing.parameterize(*@testing.product({'version': [0, 1, 2]}))
class TestCudaArrayInterface(unittest.TestCase):
    def test_base(self): ... # use self.version
    def test_not_copied(self): ...
    def test_order(self): ...
    ...

@testing.parameterize(*@testing.product({'version': [1, 2]}))
class TestCudaArrayInterfaceMaskedArray(unittest.TestCase):
    def test_masked_array(self): ...

Done. PTAL. Thanks!

OK, done. PTAL. Thanks.

@niboshi, I found a bug in __cuda_array_interface__ 😞
>>> import cupy as cp
>>> a = cp.arange(3)
>>> b=a[0:1:-1]
>>> b.size
0
>>> b.__cuda_array_interface__
{'shape': (0,), 'typestr': '<i8', 'descr': [('', '<i8')], 'data':<PHONE_NUMBER>72608, False), 'version': 2, 'strides': None}

b is a 0-size array but it's pointer is not 0... Can I fix it in this PR, or should I create a new PR for it?

Can I fix it in this PR, or should I create a new PR for it?

Fixed in #2611.

@niboshi SkipTest added in 75ecc61 is removed (see https://github.com/cupy/cupy/pull/2536#discussion_r342387241). PTAL. Thanks.

Jenkins, test this please

Jenkins CI test (for commit a00787ca6509f7afe3038681926bc15300cd6be9, target branch master) succeeded!

Thank you so much!

Thanks, @niboshi!

@niboshi This pull-request is marked as to-be-backported, but the corresponding backport PR could not be found. Could you check?


Fix for custom element namespaces in element indexes mapper

[x] I updated the changelog
[x] I updated the composer.json version
[ ] I wrote tests
[x] I am proud of my code

@bvangennep I created this PR just a few seconds before my access was revoked. I should have forked it but I didn't even realise it then. Now I can't push anymore to this branch. Can you fix the tests or do you want me to fork and start over? It's an easy fix.

Fixed in a new PR


English (UK) is now Russian.

Rikka-Chan! English is now Russian!

If you mean values-uk, for language code, uk is Ukrainian, English (UK) is en_GB.

oh...sorry...


⚠ ClamAV has degraded performance

In d206119, ClamAV (https://spamassassin.apache.org/) experienced degraded performance:

HTTP code: 200
Response time: 71 ms

Resolved: ClamAV performance has improved in ed307f4.


fix: propagate organizationId when opening the search dialog

Closes #7199

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Thanks @jamesgkong to take a look at this PR.

I was able to replicate, @filfreire


Update README.md

"arrowLocation" attribute is missing, for me it was useful, I guess it will be for others as well, cheers.

Thanks!

with pleasure mr Shapiro love from Israel
On Wed, 22 Feb 2017 at 19:33 Ron Shapiro<EMAIL_ADDRESS>wrote:

Thanks!
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/venmo/tooltip-view/pull/13#issuecomment-281741828,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ALDwpgw9KDxDbJHm2IVqwNOoX47nF81Mks5rfHFOgaJpZM4MI7AC
.

בשמחה, אחי :smile:


Microsoft Teams Rooms Licensing Link

Microsoft Teams Rooms Licensing Link in the See Also section is still broken.  There had been previous feedback but it was closed.  So, either it wasn't fixed or it is broken again.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 05282aa8-ae14-117b-5885-7d851ee66b89
Version Independent ID: 85abe13c-d34c-f42c-e273-a69be8cd991e
Content: Deploy Microsoft Teams Rooms with Office 365
Content Source: Teams/room-systems/with-office-365.md
Service: msteams
GitHub Login: @LanaChin
Microsoft Alias: v-lanac

@greenchevy65  Thank you for submitting feedback and contributing to the docs. We are currently investigating this.

@LanaChin this looks like another rebranding impact should point to file currently at https://docs.microsoft.com/en-us/MicrosoftTeams/room-systems/skype-room-systems-v2 which presumably will need to be renamed.

@scanum - the link should point to https://docs.microsoft.com/en-us/MicrosoftTeams/room-systems/skype-room-systems-v2.

@LanaChin Thanks!

Merged updates by @j0rt3g4.


Publish 10:30 PST second attempt

Docs Build status updates of commit 405cb2a:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docset/mdop/appvclient/get-appvvirtualprocess.md
:white_check_mark:Succeeded

docset/mdop/mbam/enable-mbamwebapplication.md
:white_check_mark:Succeeded

docset/mdop/mbam/test-mbamwebapplication.md
:white_check_mark:Succeeded

docset/windows/multipoint/Get-WmsSession.md
:white_check_mark:Succeeded

docset/windows/multipoint/Set-WmsStation.md
:white_check_mark:Succeeded

docset/windows/multipoint/Set-WmsSystem.md
:white_check_mark:Succeeded

docset/windows/pkiclient/New-SelfSignedCertificate.md
:white_check_mark:Succeeded

docset/windows/remoteaccess/Get-BgpCustomRoute.md
:white_check_mark:Succeeded

docset/windows/remoteaccess/Get-BgpPeer.md
:white_check_mark:Succeeded

docset/windows/remoteaccess/Get-BgpRoutingPolicy.md
:white_check_mark:Succeeded

docset/windows/smisconfig/Unregister-SmisProvider.md
:white_check_mark:Succeeded

docset/winserver2012-ps/deduplication/Get-DedupVolume.md
:white_check_mark:Succeeded

docset/winserver2012r2-ps/msdtc/Get-DtcClusterDefault.md
:white_check_mark:Succeeded

docset/winserver2012r2-ps/msdtc/Get-DtcTransactionsTraceSession.md
:white_check_mark:Succeeded

docset/winserver2012r2-ps/msdtc/Set-DtcLog.md
:white_check_mark:Succeeded

docset/winserver2012r2-ps/msdtc/Set-DtcTransaction.md
:white_check_mark:Succeeded

docset/winserver2012r2-ps/msdtc/Start-Dtc.md
:white_check_mark:Succeeded

docset/winserver2012r2-ps/msdtc/Stop-Dtc.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

Docs Build status updates of commit 8ca4f88:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docset/mdop/appvclient/get-appvvirtualprocess.md
:white_check_mark:Succeeded

docset/mdop/mbam/enable-mbamwebapplication.md
:white_check_mark:Succeeded

docset/mdop/mbam/test-mbamwebapplication.md
:white_check_mark:Succeeded

docset/windows/dnsserver/Set-DnsServerDiagnostics.md
:white_check_mark:Succeeded

docset/windows/multipoint/Get-WmsSession.md
:white_check_mark:Succeeded

docset/windows/multipoint/Set-WmsStation.md
:white_check_mark:Succeeded

docset/windows/multipoint/Set-WmsSystem.md
:white_check_mark:Succeeded

docset/windows/pkiclient/New-SelfSignedCertificate.md
:white_check_mark:Succeeded

docset/windows/remoteaccess/Get-BgpCustomRoute.md
:white_check_mark:Succeeded

docset/windows/remoteaccess/Get-BgpPeer.md
:white_check_mark:Succeeded

docset/windows/remoteaccess/Get-BgpRoutingPolicy.md
:white_check_mark:Succeeded

docset/windows/smisconfig/Unregister-SmisProvider.md
:white_check_mark:Succeeded

docset/winserver2012-ps/deduplication/Get-DedupVolume.md
:white_check_mark:Succeeded

docset/winserver2012r2-ps/msdtc/Get-DtcClusterDefault.md
:white_check_mark:Succeeded

docset/winserver2012r2-ps/msdtc/Get-DtcTransactionsTraceSession.md
:white_check_mark:Succeeded

docset/winserver2012r2-ps/msdtc/Set-DtcLog.md
:white_check_mark:Succeeded

docset/winserver2012r2-ps/msdtc/Set-DtcTransaction.md
:white_check_mark:Succeeded

docset/winserver2012r2-ps/msdtc/Start-Dtc.md
:white_check_mark:Succeeded

docset/winserver2012r2-ps/msdtc/Stop-Dtc.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Unable to retrieve the project UUID

The Project UUID is important for the Terraform provider, however, it is not currently possible to retrieve this from the UI.

To clarify (I reported this via support): there doesn't seem to be a JSON view for projects, even though I have JSON views turned on in settings and can see them for most other kinds of objects. Or if there is, it's virtually impossible to find.
I believe this is also a recent regression because a few weeks ago when I was testing our integration library, I was able to retrieve this UUID no problem.

For folks stumbling on this issue, the workaround is to go to the project page, open devtools, refresh to refetch everything from the network, and filter for /api/v1/project. There you can see the JSON project object in the server response where you can retrieve the UUID.
If the response is blank, you probably selected the OPTIONS request instead of the GET request.

Yeah, we don't have JSON data view for Project, right? @kyle-ssg

Yes, JSON references were created for this reason. You can enable it in account settings.

From this, all pages will have context based raw JSON.
I can see however, the project list and project settings do not have a reference. I'll just get a PR up to include that.


Limit the 'operations block' to download phase of initial sync

Issues
Some users are reporting that they encounter the screen that says that the device is syncing (rectangular banner) at times when the device has been fully synced. It happens in the middle of an encounter and stops work from being completed.
Here are some details as described by the implementation team:
Tasks
Investigate the issue and describe the condition under which this screen would appear and why it would last more than just a momentary amount of time.

Via @anvmn
As agreeed, the solution will be limiting the 'operations block' to download phase of initial sync for the Health Center. Implemented at https://github.com/TIP-Global-Health/eheza-app/pull/843.


Check RF 5 Alpha 1 Compatibility

Coverage remained the same at 100.0% when pulling 77d3d7f09d2502800d11a56a89796ed9e37b41e1 on Check_RF5_Alpha1 into 6e66f09c8a29ecf695fec388b59ff43b2c0c06f1 on master.


Release v0.8.26

@muga If you're fine with the spec of MavenPluginSource (incl. classifier and a consensus about plugin's artifact names), I'll be releasing v0.8.26 soon. What do you think?
I'll be merging #685, #702 and #703 as well.

@dmikurube I'm fine. Please release 0.8.26 👍

Thanks! Releasing...


Fix broken package on Windows

My latest pull request that got merged into master was an adaptation to make this package to work with minizip 4.0.7 that has the wrong minizip.dll.
So undo that now that the latest minizip package has restored the former libminizip.dll. Sorry for having jumped into a topic I didn't master...

@conda-forge-admin, please rerender

Hi! This is the friendly automated conda-forge-linting service.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe/meta.yaml) and found it was in an excellent condition.

Hi! This is the friendly automated conda-forge-webservice.
I tried to rerender for you, but it looks like there was nothing to do.
This message was generated by GitHub Actions workflow run https://github.com/conda-forge/conda-forge-webservices/actions/runs/12295640620. Examine the logs at this URL for more detail.

Hi! This is the friendly conda-forge automerge bot!
I considered the following status checks when analyzing this PR:

linter: passed
azure: failed
rerender called: passed

Thus the PR was not passing and not merged.

@akrherz The Win64 build failure is intringuing.  So the build ran against the latest minizip build (4.0.7-h9fa1bad_2), as expected, and looking in the log one sees 	link /debug /dll /out:freexl.dll  /implib:freexl_i.lib freexl.obj freexl_xlsx.obj freexl_ods.obj  %PREFIX%\Library\lib\iconv.lib  %PREFIX%\Library\lib\libexpat.lib  %PREFIX%\Library\lib\libminizip.lib
as expected too.
But later one sees the following which is the likely cause of the failure:
WARNING (freexl,Library/bin/freexl.dll): $RPATH/minizip.dll not found in packages, sysroot(s) nor the missing_dso_whitelist.
.. is this binary repackaging?

So I'd say that the latest minizip update was broken. The files were renamed to libminizip.lib/dll but presumably the renamed libminizip.lib points to a minizip.dll (I believe the name of the dll is embedded in the .lib at the time it is built, and not directly related to the name of the .lib) ...
At that point, we should perhaps? bite the bullet and accept the new minizip.lib/.dll naming and rebuild, rather than working around...

At that point, we should perhaps? bite the bullet and accept the new minizip.lib/.dll naming and rebuild, rather than working around...

or if we still want to produce libminizip.lib/.dll, then we need to patch minizip CMakeLists.txt so it natively produces a libminizip library.
Persumably something along
if (WIN32)
   set_target_properties(${MINIZIP_TARGET} PROPERTIES OUTPUT_NAME "libminizip")
endif()

at line https://github.com/zlib-ng/minizip-ng/blob/cf5404bb714ee11ca2fdc3168d9770a11ec8b576/CMakeLists.txt#L677

patch minizip CMakeLists.txt

Thank you.  Do you think this will work?
--- minizip-ng-4.0.7.orig/CMakeLists.txt	2024-06-15 11:21:44.000000000 -0500
+++ minizip-ng-4.0.7/CMakeLists.txt	2024-12-12 06:32:12.979016530 -0600
@@ -677,7 +677,7 @@

 list(APPEND MINIZIP_INC ${CMAKE_CURRENT_SOURCE_DIR})

-set(MINIZIP_TARGET minizip${MZ_LIB_SUFFIX})
+set(MINIZIP_TARGET libminizip${MZ_LIB_SUFFIX})
 set(MINIZIP_PC ${CMAKE_CURRENT_BINARY_DIR}/${MINIZIP_TARGET}.pc)
 configure_file(minizip.pc.cmakein ${MINIZIP_PC} @ONLY)

Thank you. Do you think this will work?

maybe, but that might have side effects. The set_target_properties() approach I suggested above is likely the one that will minimize side effects


Add comment to ecoystem check re. ALL rules

Honestly I'm not sure if I agree with this logic though. What if my new rule is not detected in this small set of repositories?

I don't know if I agree with the logic either even though it's my comment, though I'm also not 100% sure I want to enable ALL for all projects. Hmm...


Add JUnit tests for Azure Storage implementation

Until an easy to use local emulator is available we are closing this as won't fix


Adding elasticsearch to docker setup

There will need to be a new Docker container defined to install, run, and manage an ElasticSearch instance to perform content indexing and search queries.

I'd think the stock dockerfile/elasticsearch image should suffice... no?
https://registry.hub.docker.com/u/dockerfile/elasticsearch/
Would need to add something like this to the fig.yml file after the rabbitmq declaration.
elasticsearch:
image: dockerfile/elasticsearch
ports:
- "9200:9200"
- "9300:9300"


Utilization review is possible in steampipe or not

Hi,
Is it possible to know utilization reviews on steampipe, to get a summary or report on which servers/resources are truly being used and which are sitting idle?

Welcome to Steampipe @Niba-nazar, great to see you using the DigitalOcean mod 👍 .
At present we do have a few controls that check if the resources are available or idle. You could lean onto the block_storage_volume_inactive_and_unused control which enlists the volumes which are unattached.
We do have good dashboards in https://github.com/turbot/steampipe-mod-digitalocean-insights/tree/main/dashboards which provide an in-depth overview of digitalocean resources.
@Niba-nazar are you looking out for some specific resources? Currently the plugin supports 22 tables, we will be more than happy to add controls/queries for the resources you are looking out for 👍 .

Hi @misraved,  is there any possibility to check or bring a feature for identifying idle resources like - a running droplet which is not being used and can be flagged as idle depending upon the measurements of its CPU/RAM consumption etc..

Welcome to Steampipe @rinshadka 👍 . I think we need to add new tables to the digital ocean plugin to query CPU/RAM consumption.
From an initial scan, I came across this API which can provide valuable insights on droplet metrics.
Could you please open an issue in https://github.com/turbot/steampipe-plugin-digitalocean/issues to add new tables for retrieving droplet metrics?

hi,
Thank you @misraved for the reply.
Is it possible to get  network utilization reports through SQL

Relevant slack thread - https://steampipe.slack.com/archives/C01UECB59A7/p1669292206131539


✨ Account freezing and CLI tool

Resolves https://github.com/stumpapp/stump/issues/168
PR mostly around account locking/unlocking, but I also did a bit elsewhere:

add the ability to lock/unlock a user account

account becomes locked automatically after 10 failed logins in a row (this will likely be made configurable)
server owners can lock/unlock from the UI

add a new CLI built into the Stump binary

when the docs get deployed after merge, you will find them at https://stumpapp.dev/guides/cli

replace axum-sessions with tower-sessions per deprecation notice
migrate off of a memory store for sessions and created a custom PrismaSessionStore

a follow-up will come with a bit better error handling

update rust version for repo
made some of the tables look a little better on mobile

Example usage


ERD: do not show table partitions on diagram

We should add a preference option for partitions view enable/disable.
Probably we shouldn't disable this option by default.

verified

Hi! any updates on this ?


Maiamcc/pass tty

I guess you didn’t mean to do this — looks like you’re forking tcell into a private copy.
- Garrett
On Oct 3, 2018, 10:56 AM -0700, Maia McCormick<EMAIL_ADDRESS>wrote:

Closed #240.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.

Correct, mis-clicked a button, ignore!


Question: Delay reporting from ADO

When running a GH Action for Azure Pipelines the status succeeded in 8 seconds.  However, on ADO, it actually continued running for another 16 minutes.  Is there a way to delay the reporting back to GH Actions until the true status is returned from ADO?
Thanks,

I am also wondering the same thing. The status check on PRs in Github are based on workflow actions, but the workflow (which only triggers an Azure DevOps pipeline) is always Valid as soon as it successfully launched the Azure DevOps pipeline, even if the ADO pipeline fails or finishes minutes later as you mentioned.  Still haven't found a way to get the status.

In the case of waiting for AzDO to finish before the status is returned to Github, would you be paying for 2 agents at that point?

Yes this is designed to fire asynchronously. If you sat waiting for the ADO pipeline to finish you would be paying for 2x the compute. The recommend approach here would be to have the ADO pipeline open a new status check at the beginning of the pipeline and then mark it as pass/fail at the end.  From GitHub this would be ${{ github.sha }} for a branch or ${{ github.event.pull_request.head.sha }} for a PR.
GITHUB_REPO: The full owner/repo_name of the GitHub repo. This would equate to ${{ github.repository }}


Flutter: How can I verify that the string entered by the user in the chat is a link?

I'm making a chat app with Flutter. Recently encountered a problem, the question is how can I verify that the string entered by the user in the chat is a link? I referred Dart / Flutter - Validating a string for URL, but when I type "google.com" it shows me false. When I type "www.google.com" it shows me false. When I type "https://www.google.com/" it shows me true. How can I fix this problem? I will be grateful when some users help me with this problem.

Please use one of the resources at https://flutter.dev/community#community-grid for help with your own code.

@stuartmorgan, https://flutter.dev/community#community-grid didn't help me.

@stuartmorgan, can you open my issue?

I found a solution! I use validators to solve this problem. Here's an example from someone who had the same problem as me:
import 'package:validators/sanitizers.dart';
import 'package:validators/validators.dart';

main() {
  print(toString(1));
  print(trim('  hello '));
  print(toDouble('1.23'));

  print(isDate('1988-01-01'));
  print(isNumeric('123'));
  print(isAlphanumeric('abc123'));
  print('isUrl: ${isURL('example', requireTld: false)}');
}


Add json files for Grafana Dashboards

Hi,
Could you please also add the json files of the dashboards?
I missed the Hosted Node Metrics and Pods and Containers Metrics and want to import it manually.
Could you please help with this?
Thank you very much

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 571a1a4a-600d-a5e9-d4bc-3b4efb7bf59f
Version Independent ID: 8649b0e6-450a-5273-892c-b3998584c326
Content: Monitor cluster with Grafana Dashboard - SQL Server Big Data Clusters
Content Source: docs/big-data-cluster/cluster-monitor-grafana.md
Product: sql
Technology: big-data-cluster
GitHub Login: @cloudmelon
Microsoft Alias: melqin

@jingli0118 -- thank you for your questions.  This repo is solely for documentation issues, and this issue seems more product-related.  Please consider these resources:

SQL Server on Microsoft Q&A
DBA Stack Exchange
Stack Overflow

You are welcome to add your suggestion here:

SQL Server Feedback

cc @cloudmelon for awareness
I am closing this issue now.  You are welcome to @ mention me for any followup.  We hope to hear from you again.


Failed to parse golang plugin config while running in Github Actions

Currently, I am working on adding actions e2e tests for envoy-go-ldap-auth. I am using the official image for compilation and execution. While the tests pass successfully in my local environment, I encounter parsing errors when running them in GitHub Actions.
How should I go about troubleshooting the issue?

environment :
local : mac m1 pro
github actions: ubuntu:latest
Since both the compilation and execution are using the official image, it is unlikely that the environment is the issue.

Following files you can find in here.
config.go
package main

import (
	xds "github.com/cncf/xds/go/xds/type/v3"
	"github.com/envoyproxy/envoy/contrib/golang/filters/http/source/go/pkg/api"
	"github.com/envoyproxy/envoy/contrib/golang/filters/http/source/go/pkg/http"
	"google.golang.org/protobuf/types/known/anypb"
)

func init() {
	http.RegisterHttpFilterConfigParser(&parser{})
	http.RegisterHttpFilterConfigFactory("envoy-go-ldap-auth", configFactory)
}

type config struct {
	host      string
	port      uint64
	baseDN    string
	attribute string
	bindDN    string
	password  string
	filter    string
	timeout   int32
}

type parser struct {
}

func (p *parser) Parse(any *anypb.Any) (interface{}, error) {
	configStruct := &xds.TypedStruct{}
	if err := any.UnmarshalTo(configStruct); err != nil {
		return nil, err
	}

	v := configStruct.Value
	conf := &config{}
	m := v.AsMap()
	if host, ok := m["host"].(string); ok {
		conf.host = host
	}
	if port, ok := m["port"].(float64); ok {
		conf.port = uint64(port)
	}
	if baseDN, ok := m["base_dn"].(string); ok {
		conf.baseDN = baseDN
	}
	if attribute, ok := m["attribute"].(string); ok {
		conf.attribute = attribute
	}
	if bindDN, ok := m["bind_dn"].(string); ok {
		conf.bindDN = bindDN
	}
	if password, ok := m["bind_password"].(string); ok {
		conf.password = password
	}
	if cFilter, ok := m["filter"].(string); ok {
		conf.filter = cFilter
	}

	if timeout, ok := m["timeout"].(float64); ok {
		conf.timeout = int32(timeout)
	}
	if conf.timeout == 0 {
		conf.timeout = 60
	}
	return conf, nil
}

func (p *parser) Merge(parent interface{}, child interface{}) interface{} {
	panic("TODO")
}

func configFactory(c interface{}) api.StreamFilterFactory {
	conf, ok := c.(*config)
	if !ok {
		panic("unexpected config type, should not happen")
	}
	return func(callbacks api.FilterCallbackHandler) api.StreamFilter {
		return &filter{
			callbacks: callbacks,
			config:    conf,
		}
	}
}

Makefile:

.PHONY: build test-bind-mode

build:
	docker run --rm -v $(PWD):/go/src/go-filter -w /go/src/go-filter \
		-e GOPROXY=https://goproxy.cn \
		golang:1.19 \
		go build -v -o libgolang.so -buildmode=c-shared -buildvcs=false .

test-bind-mode:
	docker rm -f envoy-ldap-test
	docker run --rm -v $(PWD)/example/envoy.yaml:/etc/envoy/envoy.yaml \
		-v $(PWD)/libgolang.so:/etc/envoy/libgolang.so \
		-e GODEBUG=cgocheck=0 \
		-p 10000:10000 \
		--name envoy-ldap-test \
		envoyproxy/envoy:contrib-dev \
		envoy -c /etc/envoy/envoy.yaml &
	sleep 5
	go test test/e2e_bind_test.go

	docker rm -f envoy-ldap-test

envoy.yaml
static_resources:

  listeners:
    - name: listener_0
      address:
        socket_address:
          address: <IP_ADDRESS>
          port_value: 10000
      filter_chains:
        - filters:
            - name: envoy.filters.network.http_connection_manager
              typed_config:
                "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
                stat_prefix: ingress_http
                access_log:
                  - name: envoy.access_loggers.stdout
                    typed_config:
                      "@type": type.googleapis.com/envoy.extensions.access_loggers.stream.v3.StdoutAccessLog
                http_filters:
                  - name: envoy.filters.http.golang
                    typed_config:
                      "@type": type.googleapis.com/envoy.extensions.filters.http.golang.v3alpha.Config
                      library_id: example
                      library_path: /etc/envoy/libgolang.so
                      plugin_name: envoy-go-ldap-auth
                      plugin_config:
                        "@type": type.googleapis.com/xds.type.v3.TypedStruct
                        value:
                          # required
                          host: localhost
                          port: 3893
                          base_dn: dc=glauth,dc=com
                          attribute: cn
                          # optional
                          # be used in search mode
                          bind_dn: # cn=admin,dc=example,dc=com
                          bind_password: # mypassword
                          # if the filter is set, the filter application will run in search mode.
                          filter: # (&(objectClass=inetOrgPerson)(gidNumber=500)(uid=%s))
                          timeout: 60 # unit is second.

                  - name: envoy.filters.http.router
                    typed_config:
                      "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
                route_config:
                  name: local_route
                  virtual_hosts:
                    - name: local_service
                      domains: ["*"]
                      routes:
                        - match:
                            prefix: "/"
                          route:
                            host_rewrite_literal: mosn.io
                            cluster: service_mosn_io

  clusters:
    - name: service_mosn_io
      type: LOGICAL_DNS
      # Comment out the following line to test on v6 networks
      dns_lookup_family: V4_ONLY
      load_assignment:
        cluster_name: service_mosn_io
        endpoints:
          - lb_endpoints:
              - endpoint:
                  address:
                    socket_address:
                      address: mosn.io
                      port_value: 443
      transport_socket:
        name: envoy.transport_sockets.tls
        typed_config:
          "@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext
          sni: mosn.io

workflow actions yaml
name: build ci

on:
  push:
    branches: [ "test-ci" ]
  pull_request:
    branches: [ "test-ci" ]

jobs:
  test-bind:
    runs-on: ubuntu-latest

    steps:
    - name: checkout code
      uses: actions/checkout@v3

    - name: Set up Go
      uses: actions/setup-go@v3
      with:
        go-version: 1.19

    - name: prepare glauth
      run: |
        curl -L -o glauth https://github.com/glauth/glauth/releases/download/v2.2.0-RC1/glauth-linux-amd64
        chmod +x glauth

    - name: build
      run: make build

    - name: run glauth
      run: |
        curl -L -o sample.cfg https://raw.githubusercontent.com/glauth/glauth/master/v2/sample-simple.cfg
        ./glauth -c sample.cfg &
        sleep 5

    - name: test-bind-mode
      run: make test-bind-mode

e2e_bind_test.go
package test

import (
	"net/http"
	"testing"
)

func TestE2EBind(t *testing.T) {
	req, err := http.NewRequest(http.MethodGet, "http://localhost:10000/", nil)

	resp1, err := http.DefaultClient.Do(req)
	if err != nil {
		t.Fatal(err)
	}
	defer resp1.Body.Close()
	if resp1.StatusCode != http.StatusUnauthorized {
		t.Fatalf("unexpected status code: %v", resp1.StatusCode)
	}

	req.SetBasicAuth("unknown", "dogood")
	resp2, err := http.DefaultClient.Do(req)
	defer resp2.Body.Close()
	if resp2.StatusCode != http.StatusUnauthorized {
		t.Fatalf("unexpected status code: %v", resp2.StatusCode)
	}

	req.SetBasicAuth("hackers", "unknown")
	resp3, err := http.DefaultClient.Do(req)
	defer resp3.Body.Close()
	if resp3.StatusCode != http.StatusUnauthorized {
		t.Fatalf("unexpected status code: %v", resp3.StatusCode)
	}

	req.SetBasicAuth("hackers", "dogood")
	resp4, err := http.DefaultClient.Do(req)
	defer resp4.Body.Close()
	if resp4.StatusCode != http.StatusOK {
		t.Fatalf("unexpected status code: %v", resp4.StatusCode)
	}

	t.Log("E2E test passed")

}

locally test result
go test test/e2e_bind_test.go
[2023-06-01T03:05:38.249Z] "GET / HTTP/1.1" 401 - 0 16 21 - "-" "Go-http-client/1.1" "9c5ad002-c055-4d81-8b43-8c36401ccba4" "localhost:10000" "-"
ok  	command-line-arguments	1.122s
docker rm -f envoy-ldap-test
envoy-ldap-test

but panic in github actions, details you can find in actions result:
Error: 5-31 19:16:55.885][19][error][http] [contrib/golang/filters/http/source/golang_filter.cc:1241] failed to parse golang plugin config: proto: mismatched message type: got "xds.type.v3.TypedStruct", want ""
panic: get config failed, plugin: envoy-go-ldap-auth, configId: 0

goroutine 17 [running, locked to thread]:
github.com/envoyproxy/envoy/contrib/golang/filters/http/source/go/pkg/http.getOrCreateHttpFilterFactory({0xc0000003f0, 0x12}, 0x0)
	/go/pkg/mod/github.com/envoyproxy/envoy@v1.26.1/contrib/golang/filters/http/source/go/pkg/http/filtermanager.go:44 +0x168
github.com/envoyproxy/envoy/contrib/golang/filters/http/source/go/pkg/http.createRequest(0x4e17f91b180)
	/go/pkg/mod/github.com/envoyproxy/envoy@v1.26.1/contrib/golang/filters/http/source/go/pkg/http/shim.go:94 +0xda
github.com/envoyproxy/envoy/contrib/golang/filters/http/source/go/pkg/http.envoyGoFilterOnHttpHeader(0x7f632d0e2dd8?, 0x1, 0x8, 0xa8)
	/go/pkg/mod/github.com/envoyproxy/envoy@v1.26.1/contrib/golang/filters/http/source/go/pkg/http/shim.go:110 +0x5e
[2023-05-31 19:16:55.887][19][critical][backtrace] [./source/server/backtrace.h:104] Caught Aborted, suspect faulting address 0x6500000001
[2023-05-31 19:16:55.887][19][critical][backtrace] [./source/server/backtrace.h:91] Backtrace (use tools/stack_decode.py to get line numbers):
[2023-05-31 19:16:55.887][19][critical][backtrace] [./source/server/backtrace.h:92] Envoy version: 7e6031d2a16210660b0[153](https://github.com/Makonike/envoy-go-ldap-auth/actions/runs/5136806180/jobs/9244019321#step:7:154)1b54bc2dc14b1b6e61/1.27.0-dev/Clean/RELEASE/BoringSSL
[2023-05-31 19:16:55.888][19][critical][backtrace] [./source/server/backtrace.h:96] #0: __restore_rt [0x7f6331c8b420]
[2023-05-31 19:16:55.888][19][critical][backtrace] [./source/server/backtrace.h:96] #1: runtime.fatalpanic [0x7f632d11280d]
[2023-05-31 19:16:55.889][19][critical][backtrace] [./source/server/backtrace.h:96] #2: runtime.gopanic [0x7f632d111eda]
[2023-05-31 19:16:55.890][19][critical][backtrace] [./source/server/backtrace.h:96] #3: github.com/envoyproxy/envoy/contrib/golang/filters/http/source/go/pkg/http.getOrCreateHttpFilterFactory [0x7f632d4f5168]
[2023-05-31 19:16:55.890][19][critical][backtrace] [./source/server/backtrace.h:96] #4: github.com/envoyproxy/envoy/contrib/golang/filters/http/source/go/pkg/http.createRequest [0x7f632d4fb87a]
[2023-05-31 19:16:55.891][19][critical][backtrace] [./source/server/backtrace.h:96] #5: github.com/envoyproxy/envoy/contrib/golang/filters/http/source/go/pkg/http.envoyGoFilterOnHttpHeader [0x7f632d4fba1e]
[2023-05-31 19:16:55.892][19][critical][backtrace] [./source/server/backtrace.h:96] #6: _cgoexp_abf5d86fd2bb_envoyGoFilterOnHttpHeader [0x7f632d4fc4d0]
[2023-05-31 19:16:55.892][19][critical][backtrace] [./source/server/backtrace.h:96] #7: runtime.cgocallbackg1 [0x7f632d0e07e2]
[2023-05-31 19:16:55.893][19][critical][backtrace] [./source/server/backtrace.h:96] #8: runtime.cgocallbackg [0x7f632d0e0469]
[2023-05-31 19:16:55.894][19][critical][backtrace] [./source/server/backtrace.h:96] #9: runtime.cgocallbackg.abi0 [0x7f632d144591]
[2023-05-31 19:16:55.894][19][critical][backtrace] [./source/server/backtrace.h:96] #10: runtime.cgocallback.abi0 [0x7f632d141ff3]
ActiveStream 0x4e17f8b1100, stream_id_:<PHONE_NUMBER>8638519693&filter_manager_:
  FilterManager 0x4e17f8b11a8, state_.has_1xx_headers_: 0
  filter_manager_callbacks_.requestHeaders():
    ':authority', 'localhost:10000'
    ':path', '/'
    ':method', 'GET'
    ':scheme', 'http'
    'user-agent', 'Go-http-client/1.1'
    'accept-encoding', 'gzip'
    'x-forwarded-proto', 'http'
    'x-request-id', '1f2cfe63-edce-49af-9056-eac15c4af7a7'
  filter_manager_callbacks_.requestTrailers():   null
  filter_manager_callbacks_.responseHeaders():   null
  filter_manager_callbacks_.responseTrailers():   null
  &streamInfo():
    StreamInfoImpl 0x4e17f8b12d8, protocol_: 1, response_code_: null, response_code_details_: null, attempt_count_: null, health_check_request_: 0, route_name_:     upstream_info_:     null
    OverridableRemoteConnectionInfoSetterStreamInfo 0x4e17f8b12d8, remoteAddress(): [172](https://github.com/Makonike/envoy-go-ldap-auth/actions/runs/5136806180/jobs/9244019321#step:7:173).17.0.1:53446, directRemoteAddress(): <IP_ADDRESS>:53446, localAddress(): <IP_ADDRESS>:10000
Http1::ConnectionImpl 0x4e17f2f7708, dispatching_: 1, dispatching_slice_already_drained_: 0, reset_stream_called_: 0, handling_upgrade_: 0, deferred_end_stream_headers_: 1, processing_trailers_: 0, buffered_body_.length(): 0, header_parsing_state_: Done, current_header_field_: , current_header_value_:
active_request_:
, request_url_: null, response_encoder_.local_end_stream_: 0
absl::get<RequestHeaderMapPtr>(headers_or_trailers_): null
current_dispatching_buffer_ front_slice length: 96 contents: "GET / HTTP/1.1\r\nHost: localhost:10000\r\nUser-Agent: Go-http-client/1.1\r\nAccept-Encoding: gzip\r\n\r\n"
ConnectionImpl 0x4e17f81b380, connecting_: 0, bind_error_: 0, state(): Open, read_buffer_limit_: 1048576
socket_:
  ListenSocketImpl 0x4e17f85cd80, transport_protocol_: raw_buffer

By changing the version of the image used for execution to contrib-v1.26.1, the issue was resolved.


Bytes should be annotated @FunctionalInterface

It contains only one abstract method.
And WrappedBytes class is not really needed. Wherever there is:
new WrappedBytes(...)

in the code base, one can replace it with:
() -> ...

Bytes are no longer a functional interface since #59.
We still need a tool that will create a new Bytes object from a raw byte array.
Currently, this is pattern is being used:
/// bytes[] arr = ...
Bytes wrapped =
    new AbstractBytes() {
        @Override
        public byte[] asArray() {
            return arr;
        }
    };

This is too long and leads to code duplication.
Furthermore, it breaks the Bytes#asArray() contract - fresh array should be returned on every invocation of asArray().
Lambda is also not a solution since such objects don't respect the Bytes' contract (equals() and hashCode()).
This should work, provided that the caller doesn't change the contents of arr:
/// bytes[] arr = ...
Bytes wrapped =
    new AbstractBytes() {
        @Override
        public byte[] asArray() {
            return arr.clone();
        }
    };

Perhaps, add it as a static method to the Bytes interface, like so:
static Bytes wrapped(final byte... bytes) {
    return
        new AbstractBytes() {
            @Override
            public byte[] asArray() {
                return bytes.clone();
            }
        };
}

This could then be used in a much shorter way:
/// bytes[] arr = ...
Bytes wrapped = Bytes.wrapped(arr);


Adding Amadeus as an adopter

Adding Amadeus as another adopter of the external secrets operator

Thank you @dreadful-dragon ! 🥳

/approve

/merge


KITTI dataset preparation

Branch
master branch  https://mmdetection3d.readthedocs.io/en/latest/
📚 The doc issue
I am trying to prepare the KITTI point cloud data using the documentation but when I run the command "python tools/create_data.py kitti --root-path ./data/kitti --out-dir ./data/kitti --extra-tag kitti" I get the error "ValueError: file not exist: training/image_2/000000.png"
The documentation isn't clear on why is it taking the png file from training/image_2 when I want it to use the point cloud data from training/velodyne and how do I make those changes?
Suggest a potential alternative/fix
No response

Since the image info is used to generate pkl info file and it's not optional. I suggest you to download the full Kitti datasets. It could be complex to make changes for not using png file during dataset converter.


systemd: add systemd-importd

Motivation for this change
Adding systemd-importd to the build, so that machinectls import-.*
may actually do anything.  Currently they fail with
Failed to transfer image: The name org.freedesktop.import1 was not provided by any .service files

as systemd-importd is not built.
This potentially will fix #70348

Testing

spawn a nixos vm based on this branch with gnupg and curl in the system environment and working sudo
run this (as normal user with working sudo) :

sudo gpg --no-default-keyring --keyring=/etc/systemd/import-pubring.gpg --fingerprint
tfile=$(mktemp -u /tmp/masterkey.nspawn.org.XXXXXXXXXXX)
curl https://nspawn.org/storage/masterkey.pgp -o "$tfile"
sudo gpg --no-default-keyring --keyring=/etc/systemd/import-pubring.gpg --import "$tfile"
machinectl pull-tar https://nspawn.org/storage/ubuntu/bionic/tar/image.tar.xz
machinectl start image
machinectl shell image

You will get a lot of selinux warning (which is perfectly fine if your vm has no se linux). In the end, the image should be verified by checksum and the signature proven by the imported key.
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[x] Determined the impact on package closure size (by running nix path-info -S before and after)
[x] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

Notify maintainers
cc @

I'll try to have a look at this in the next days. I vaguely remember reasons for not doing what you did there with udev. I can't remember right now. Hopefully that gets back to me within the next days.

pin @andir

What's the effect on closure size? It doesn't seem desirable to me to make systemd dependent on X.org (via libxkbdcommon).

What's the effect on closure size?

How would I find out?
Regarding libxkbdcommon: I have no clue why I added it, but it seems to be not necessary. I removed the unneeded deps and rebased on master. I will test 7d20b67 it in a vm and report back in a few hours.

How would I find out?

nix path-info -S (or -rS to see the paths in the closure).

Ah, with the path to the nix store. I always tried nix path-info -S systemd. The closure size becomes 147.0M, so round about +35M.
While testing, I found out that there is an issue with direct calls to gpg and tar in the import code, though that should be fixed in the next push.

Hm, I'd really prefer to avoid a hard dependency on X11 or gpg in the base system.

That seems reasonable. Do we have any other systemd-* component which is seperated from the systemd base expression?

Is it feasible to have those additional binaries in another output without increasing the closure of the out output?

Is it feasible to have those additional binaries in another output without increasing the closure of the out output?

I'm afraid that it might become difficult marrying both parts together. I will look into it though.

another systemd package that carries all the extra bloat that most/many users do not care about.

You do realize that the current state is shipping not working bloat (at least machinectl) in the normal package? Of course having finer control over what to build and whatnot is good, but the user gets bloat+regressions right now (of which at least the regressions shall go IMO). Maybe a general decision should be made on how to deal with systemd, IMHO the current package ships a lot of things which may be considered bloat since they are not at all needed for an init.

Please merge, I need this.

I don't think NixOS has an obligation to enable every systemd feature. Just because upstream feels that systemd needs to be docker replacement doesn't mean we have to impose that bloat on our users.

What is it that makes being dependent on gpg bad?

 (This mostly seems to be because of gpg-agent / pinentry, a program that doesn't work for me half of the time...)
Of course this is all transitive, so any app in Nixpkgs that has a (transitive) dependency on systemd will now depend on those things. (Though usually it's a dependency on systemd.lib so maybe it's not too bad.)

isn't the gnupg build here build without guiSupport, so the x11 dependency would go away?
also, can't we hide this feature behind an enable flag in the systemd derivation and people could enable it via an option?

Just because upstream feels that systemd needs to be docker replacement doesn't mean we have to impose that bloat on our users.

I can not stress this enough: nixos already ships machinectl, nspawn & machined by default. You already ship this bloat, but with a regression, as you do no ship the single component which is missing to make machined/machinectl any usable, because there is no official way of importing images! If you wish to make an argument about this then stop shipping broken bloat!

Let's have a look at the dependencies of gpg:

Let's have a look at the dependencies of systemd from this pr:
/nix/store/08al8hwj0hvqahb0yibslsmsvj50b1qz-acl-2.2.53                  	     109920
/nix/store/0m9rsg8dqsr7pq0i42qr6in8lfrqqyvn-libcap-2.27-lib             	      26936
/nix/store/0ni6d49nci8vpzxr7672qjwr6f5c5kc4-util-linux-2.33.2           	    1658096
/nix/store/0waqa7f6l0lzxk815ir633gaq0mas445-libidn2-2.2.0               	     214360
/nix/store/1a037dsgz0615j1hf9nshmpykc7qj39l-libtasn1-4.14               	      89824
/nix/store/22mmqqgdhzrs7l9vv324knbsz4zbwycp-libksba-1.3.5               	     262496
/nix/store/234i6czbva9dzwv3y4w0z9rr5p72m4g6-npth-1.6                    	      49792
/nix/store/2p33f8vmb094grg691f8w50gzpp7x884-libnftnl-1.1.4              	     280128
/nix/store/38jvvx3z7cz8k58v535r4sbswgp07yaw-cracklib-2.9.7              	    9971448
/nix/store/440vkrx6mf7v4j4z88vjcp6pavk05lig-bzip2-<IP_ADDRESS>.1-bin         	      70016
/nix/store/4ad65dga6w342wh9brlxdxwca0l31n33-audit-2.8.5                 	     311504
/nix/store/5dn03xfw3wimm78nfpg67p1zj1025sbc-bzip2-<IP_ADDRESS>.1             	      80912
/nix/store/62ar9xmrlcnlgmwgfi77xz6bq1180vhi-zlib-1.2.11                 	     132608
/nix/store/63jwkphcgl7yzcx168yns0540f2c4msm-keyutils-1.6-lib            	      42016
/nix/store/6qggmn4di5rhylh9jlah6rbcz6vwhb2x-libevent-2.1.11             	     896840
/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27                  	   28147008
/nix/store/7hmm5sawx4mbb9jkczjgrn0gm96681qy-libtool-2.4.6-lib           	      58080
/nix/store/8kaqqz7x3fgyy84wn2zpbpkmdzvrm6x6-p11-kit-<IP_ADDRESS>           	    3643704
/nix/store/8wr3wcj9qldb2ak8ccxvyf4dm9zzh7cx-libseccomp-2.4.1-lib        	     305720
/nix/store/9pfanxsm4wpygrx13r9332lsmhbqnmmj-gnupg-2.2.17                	   11221672
/nix/store/a66s8hzy6w34dlh6ahgfdwaiglpcny65-gnutar-1.32                 	    2858776
/nix/store/agc9nn5r0jm399lsfm0rx3m1gv40frsr-libunistring-0.9.10         	    1634416
/nix/store/b3pr9iv2v3j1y0gvdhy86pq5s9lqgzg7-openldap-2.4.48             	    4203992
/nix/store/b6fp8g085zyhq02vivdzdirb4mgp5ggi-openssl-1.1.1d              	    4158648
/nix/store/bswbcw4y9mr4ngj6r7gsv3drqlcnpldf-libkrb5-1.17                	    2389336
/nix/store/cc6ilhxss244dy7lqs6573bn8xp4wpq9-systemd-243-lib             	    1710104
/nix/store/d9mj4lwxcibgdm5aaa66r8i9svnb6bpm-linux-pam-1.3.1             	    2004888
/nix/store/g0d5ykr6jxbl4h7rhsz3mi5hp7gz4p2y-kbd-2.0.4                   	    2618360
/nix/store/g6hcw252yn4shagc7wg40biy3hmmpn09-libmnl-1.0.4                	      43704
/nix/store/gasyimps4d9lwl1pc44mhzgjb20dm7i6-libnetfilter_conntrack-1.0.7	     205256
/nix/store/hljwhpqjhpy5327zfmzjjh8xyly3hiy2-glibc-2.27-bin              	    2704872
/nix/store/i945gbylyzlz5gcdsj81vfbaz9yji37w-lz4-1.9.1                   	     496608
/nix/store/isg5d6rh4zdva16hxs7qg6k1s96v39mx-sqlite-3.28.0               	    1245824
/nix/store/j2p1qjbajqfb95ba3dqgjbsknipknikk-gmp-6.1.2                   	     698784
/nix/store/ji9yrngymyigsqzcqaavng7v4dws32q7-nghttp2-1.39.2-lib          	     193176
/nix/store/jssgml4p8vzs7ic5cqy7hh48m57swdb9-dns-root-data-2019-01-11    	       4480
/nix/store/jsx6f89kxy4apq8w9y434rf5mjlsawzi-nettle-3.4.1                	     513144
/nix/store/kglyaag9larw9hc1pc7iybzj1yzyy5x2-libassuan-2.5.3             	     100472
/nix/store/kn9w6k46qmzrv3hgzs1csrrs3dcad7h6-attr-2.4.48                 	      89136
/nix/store/kvia4rwy9y4wis4v2kb9y758gj071p5v-ncurses-6.1-20190112        	    3488416
/nix/store/kx56d14p0qm2awn0qrqrv4rb8xdhvhpi-coreutils-8.31              	    1798432
/nix/store/l6h4ya0wzb4b8mr0y58k2gh2nhfql4sn-bash-4.4-p23                	    1321096
/nix/store/l9g11dsdj97d5q0747bqjacrakyrf8dk-db-5.3.28                   	    4119360
/nix/store/lng218nwk66q5l3c3pq4826ggj4rvyfr-libmicrohttpd-0.9.66        	     167744
/nix/store/m52arbrnwm1km3yagr9iamsp2sqbmmip-libssh2-1.9.0               	     290464
/nix/store/mig9a44n4swlhwyamy8hhghnh4ifxk4j-unbound-1.9.3-lib           	     902928
/nix/store/mvs0vqiqp6186wpgs7z18agw0gs6l6ji-pcre2-10.33                 	    1672000
/nix/store/najvrr9mppdrgjh529l79w8px36wn58z-gcc-8.3.0-lib               	    5719760
/nix/store/pdpgm7h0mcbc9xidn0x1xyzcbqdzn2ad-libgcrypt-1.8.5             	    1261064
/nix/store/pk6h6pzmwaclxb9rpzvh5rk346mq8nyl-iptables-1.8.3              	    2936360
/nix/store/q1b45q29p5vw5bvq7p8prq94sgxnqyx3-systemd-243                 	   24236288
/nix/store/q8796x08wb1cmzqg5b6mx6k1k0kcxp8i-libgpg-error-1.36           	     718080
/nix/store/q9vxib6xc66i3w9pqyddbdlhqc4rshc3-kexec-tools-2.0.20          	     243424
/nix/store/qira6x24xinnzg3ksgz1znj0k0i04v6i-gnutls-3.6.9                	    2838224
/nix/store/r4i08s94s97jiwx5pwnw2qzgdwi404vn-libapparmor-2.13.1          	     445640
/nix/store/r5s3w32ahjzdlzsfrhybc3l2qcpi6yb2-libpcap-1.9.0               	    1028920
/nix/store/sgganmpm2bbnqmgz35hrgp4d0cvml6ir-gzip-1.10                   	     150120
/nix/store/wx84sq1p3ji0va77hchhfr1d6jd1b24h-cyrus-sasl-2.1.27           	     622368
/nix/store/x0yc4mysvi3p0k46wiipd4v7n6g4kwb4-libnfnetlink-1.0.1          	      57400
/nix/store/xnhkyg25aw87ivf7adbnfivfg779a5aw-util-linux-2.33.2-bin       	    5676928
/nix/store/yk8lp0x7sml42gaf9gfkr6cwx99glz7b-xz-5.2.4                    	     341056
/nix/store/ynyacrpj8876kqv7xmbhak1w2gmp4rk4-curl-7.65.3                 	     641560
/nix/store/z0bpm56favz7yzas3j9023lqisbr4q5s-kmod-26                     	     329368
/nix/store/z39757pg1gp5lgkxcn0yv6nv4lgmpnad-libffi-3.2.1                	      51368
/nix/store/z7y1hqcw0h6x5pm9piilxpv68v0s6n31-shadow-4.6                  	    3794936
/nix/store/zpav4lfjikaz21qkyq3hnaxsa9a7kqzy-readline-6.3p08             	     396016
/nix/store/zrd8dbxrq8shpfy2xh68l02qqyd4vy97-db-4.8.30                   	    3494896

Here is a diff between the systemd actually running in my systemd and the one in my branch:
diff systemd-normal systemd-importd
8a9,10
> curl-7.65.3                 	 626.5K
> cyrus-sasl-2.1.27           	 607.8K
9a12
> db-5.3.28                   	   3.9M
14a18,19
> gnupg-2.2.17                	  10.7M
> gnutar-1.32                 	   2.7M
19a25
> keyutils-1.6-lib            	  41.0K
21a28
> libassuan-2.5.3             	  98.1K
27a35,36
> libkrb5-1.17                	   2.3M
> libksba-1.3.5               	 256.3K
34a44
> libssh2-1.9.0               	 283.7K
35a46
> libtool-2.4.6-lib           	  56.7K
38a50
> ncurses-6.1-20190112        	   3.3M
39a52,55
> nghttp2-1.39.2-lib          	 188.6K
> npth-1.6                    	  48.6K
> openldap-2.4.48             	   4.0M
> openssl-1.1.1d              	   4.0M
41a58
> readline-6.3p08             	 386.7K
43c60,61
< systemd-243                 	  22.8M
---
> sqlite-3.28.0               	   1.2M
> systemd-243                 	  23.1M
45c63
< unbound-1.9.4-lib           	 881.8K
---
> unbound-1.9.3-lib           	 881.8K

As gpg is only used to verify signatures, I can look into making the gpg expression used even more minimal, maybe I can get rid of openldap too. Would that make you less opposed to this pr?

isn't the gnupg build here build without guiSupport, so the x11 dependency would go away?

Yes!

also, can't we hide this feature behind an enable flag in the systemd derivation and people could enable it via an option?

That might be something to do in the future, but I see two problems here:

the current systemd ships a broken component. It ain't clever IMHO to ship something partially broken by default, and give an opt-in to fix it.
also it might cause some headache to have two systemd packages. If it is a switch in the expression, people who alter it will have two different versions of systemd at all time, since many packages depend on the systemd with the switch set to false. On the other hand, if the switch is global there will be two versions of all packages which depend on systemd.

I can not stress this enough: nixos already ships machinectl, nspawn & machined by default.

Because they are useful even without systemd-importd. E.g. nixos-container uses them. Also, the bloat doesn't come from those programs but from dependencies like gnupg.

isn't the gnupg build here build without guiSupport, so the x11 dependency would go away?

Using guiSupport = false is actually likely to lead to an increase in system closure size, because many users will end up with 2 builds of gnupg in their closure.
It would be better if pinentry was architected like systemd-ask-password, which doesn't require programs to have a direct dependency on some GUI components to show a password prompt.

Just because upstream feels that systemd needs to be docker replacement doesn't mean we have to impose that bloat on our users.

gave me another impression than

Because they are useful even without systemd-importd. E.g. nixos-container uses them. Also, the bloat doesn't come from those programs but from dependencies like gnupg.

Thank you for explaining that.

Using guiSupport = false is actually likely to lead to an increase in system closure size, because many users will end up with 2 builds of gnupg in their closure.

Only for those who use gpg, and then it's about +10Mbyte. But doing that decreases the deps of systemd by a lot (to the point that systemd from this pr has less deps in total than normal gnupg alone). I gott that you have both an issue with depending on any form of gpg and a personal dislike for how it is made. I've asked multiple questions, but I was not able to answer them based from your feedback. So: what does it take to have you not opposed to making machinectl working? Is the current state stripped enough from bloat, or is there some dep left which hurts a sleek base system?

@edolstra it's possible to set ncurses as pinentry for GPG. No GUI needed.

@edolstra it's possible to set ncurses as pinentry for GPG. No GUI needed.

That's not the point: pinentry still pulls GTK+ into the closure. Or you disable at compile time, but then GUI users suffer.

Only for those who use gpg, and then it's about +10Mbyte.

Well bloat happens 10 MB at a time. But I don't really object to gpg in itself, the issue is everything else it drags in.

Well bloat happens 10 MB at a time. But I don't really object to gpg in itself, the issue is everything else it drags in.

Roughly estimated +22MB in total compared to current nixos-stable systemd-243 (I just summed up the size from the diff in https://github.com/NixOS/nixpkgs/pull/70352#issuecomment-541010887 with the regarding sign)?

@edolstra work on different pinentry flavours is happening in https://github.com/NixOS/nixpkgs/pull/49270.

Reiterating:

What does it take to have you not opposed to making machinectl working?
Is the current state (+22MB) stripped enough from bloat, or is there some dep left which hurts a sleek base system?
If these +22MB are still too much, what is your strategy of choice to solve both machinectl issue for the users who want it to be solved and comply with your target of a not additionally bloated base system?

@wucke13 with the pinentry PR merged (to staging), gpg doesn't pull in GTK or qt on the minimal installer image anymore.
You could rebase on top of that, and see how this affects closure sizes of systemd (and the minimal installation iso image).

@wucke13 You're right, it didn't pull in pinentry. Sorry for the confusion.
I looked into the feature and your PR once more.
The version of gnupg we would build for systemd has the following runtime closure (I strike-though'ed all lines that were already part of systemd's runtime closure before this PR)

~29M	/nix/store/hnp72ygssavd526rp7qk3yjylyc2klcw-glibc-2.27~
~748K	/nix/store/1p4gc2l25ndx6pp6bpjmz0bjckwd73is-libgpg-error-1.36~

~already currently pulled in via systemd's resolved~

~1.3M	/nix/store/0pp7jy1kn9gpdbhnl5i205qg7hz16i15-libgcrypt-1.8.5~
~140K	/nix/store/3zmbbkvxkgsp6z0qhnpdaaz7kkksq3m7-zlib-1.2.11~
1.2M	/nix/store/4i67jlg1kpz3i71h39yyng5q2hyblzca-sqlite-3.28.0

pulled in via ${gnupg}/bin/gpg

~1.4M	/nix/store/v2x1ks46fq3yn8c94qfv38wrvmr5xn5r-bash-4.4-p23~
60K	/nix/store/6jpa1h86py30i085awdhbjb6gni56rdd-npth-1.6

pulled in via ${gnupg}/{bin/{dirmngr,gpg-agent},libexec/scdaemon}

112K	/nix/store/hqfvc8y4bspv49fykdd6lirf6wwb6j9r-libassuan-2.5.3

pulled in by many tools in ${gnupg}/{bin,libexec}

268K	/nix/store/ihavkf3gwc5rmmi9dc6chypw5p363swg-libksba-1.3.5

pulled in via ${gnupg}/{bin/{dirmngr,gpgsm,kbxutil},libexec/scdaemon}

11M	/nix/store/kkqhf5593r5vdnlzaz9ijasrdsbh0lxm-gnupg-2.2.17

systemd basically only executes gpg/gpg2 in --verify mode, in a temporary gpghome, with --keyring pointing to {USER,VENDOR}_KEYRING_PATH.
We don't need any of the other binaries in ${gnupg}/bin (given ${gnupg}/bin/gpg (hopefully) doesn't shell out to them by itself).
This probably includes dirmngr gpg-agent or the smartcard functionality, so we might be able to get rid of libksba and npth.
We also don't need any of the sqlite features so we should build it with --disable-sqlite, saving another 1.2MiB of dependencies.
On top of that, we probably don't need to ship manpages and locales in $gnupg/share, shaving off another 5.1MiB, so in the end, we might end up only with bin/gpg, bin/gpg2 at about 1MiB, which IMHO, is an "acceptable bloat", given we can then verify these signatures.

@flokli I can confirm. You don't need smartcard features or even a gpg-agent for the gpg support in systemd. The only purpose of this gpg keyring for systemd is too verifying gpg signed tarballs and raw gpt images. This should happen without user interaction so no smartcard support needed.

So then, shall I add a gnuppg-minimal package commit to this PR with all the thinks you mentioned stripped away? @flokli

We don't have network access in the container, so just running above script during the test won't work.
We could use fetchurl and a known-hash to fetch images upfront, but it doesn't look like the image on that path is stable, but regularily updated, right?
Using mkosi would also be hard - because the bootstrapping tools it calls need network access. I guess we should just fill a bunch of random numbers like the systemd testsuite does, and sign it with a dummy key which we also configure put at systemd's import-pubring.gpg. We can't start this bunch of random numbers, but we can at least check the import and verification phase worked.
In addition to the image.tar.xz, nspawn downloads SHA256SUMS and SHA256SUMS.gpg. I'm not sure if that's also the case if we import locally.
If it isn't, we could still cheat by enabling an nginx, creating the "image" there, and point machinectl pull-tar to localhost.
It might be a good idea to put these tests in a new nixos/tests/systemd-nspawn.nix - I think @arianvp also also has some nspawn-specific tests in the pipeline too.

We could use fetchurl and a known-hash to fetch images upfront, but it doesn't look like the image on that path is stable, but regularily updated, right?

Not so regularly, but there is no way to pinpoint. But the image is only about 25 MByte in size. Maybe having all of it on web archive will work?

We can't start this bunch of random numbers, but we can at least check the import and verification phase worked.

That sounds really clever. I don't know how to do it though :laughing:
This is the tail of output by current state of the PR:
SHA256 checksum of https://nspawn.org/storage/ubuntu/bionic/tar/image.tar.xz is valid.
gpg: Signature made Thu Aug  1 16:39:51 2019 UTC
gpg:                using RSA key 575DE88794A45D84456D8897A232A512E7D0BA83
gpg: Good signature from "Christian Rebischke (nspawn.org masterkey<EMAIL_ADDRESS>[unknown]
gpg: WARNING: Using untrusted key!
Signature verification succeeded.
Created new local image 'ubuntu'.
Operation completed successfully.
Exiting.
==== AUTHENTICATING FOR org.freedesktop.systemd1.manage-units ====
Authentication is required to start<EMAIL_ADDRESS>Multiple identities can be used for authentication:
 1.  System administrator (root)
 2.  user
Choose identity to authenticate as (1-2): 2
Password:
==== AUTHENTICATION COMPLETE ====
==== AUTHENTICATING FOR org.freedesktop.machine1.shell ====
Authentication is required to acquire a shell in a local container.
Multiple identities can be used for authentication:
 1.  System administrator (root)
 2.  user
Choose identity to authenticate as (1-2): 2
Password:
==== AUTHENTICATION COMPLETE ====
Connected to machine ubuntu. Press ^] three times within 1s to exit session.
root@bionic:~#

Can you rebase this on top of staging, change the destination branch to staging, and amend the 2nd commit message? Apart from that, LGTM.

What shall be ammended with which commit message? Do you want me to squash those two commits?

Not squashing is fine, the first is good on its own too.
"systemd: etc" is what's not really saying.
It could be something like:
systemd: add importd

This will fix the `machinectl import-.*` commands which currently fail with ...

Ah I see! Done.

@GrahamcOfBorg build systemd

Can you remove the gnupg-minimal = callPackage ../tools/security/gnupg/minimal.nix { }; addition / removal in between the two commits?

I added a systemd-nspawn test which downloads an nspawn container via http from a local server, and ensures sha256sum verification and gpg signature verification work. PTAL.

@GrahamcOfBorg test systemd-nspawn

Ran tests manually on both aarch64-linux and x86_64-linux.

Added this test to the list in #72828 (it still needs to trickle from staging to master until it can be fixed there).

 If I am reading meson.build correctly, want_importd requires zlib, but it is not in buildInputs.

 If I am reading meson.build correctly, want_importd requires zlib, but it is not in buildInputs.

I guess things which are not explicitly deactivated are enabled on demand!

The thing is in that PR you set -Dimportd=true, which activates this if and then, if zlib is not found, [this if][https://github.com/systemd/systemd/blob/v243/meson.build#L1245) should trigger and configure should fail with an error. And HAVE_ZLIB is only ever set to true if zlib was actually found.

Ok, it ends up being in the dependencies closure somehow, probably as a propagated input of some other package, huh.

Hi, this caused a regression on our machines.
58fb23f72ad
exchange.> reloading the following units: dbus.service, firewall.service

6317f5b81ce
exchange.> stopping the following units: dbus.service, dbus.socket, nscd.service
exchange.> Failed to create bus connection: Connection refused

I think this is the same issue as https://github.com/NixOS/nixpkgs/issues/74626 and not immediately caused by this PR.


Improve responses

Currently the responses are wrapped in a RPCResponse so an error can be returned.
I think we can avoid this by letting the user define its response types like this:
message MyMethodResponse {
  oneof nrpcresp {
    nrpc.Error error = 1;
    string message = 2
  }
}

When this pattern is recognized by the code generator, the generated handler interface would return (string, error) instead of a MyMethodResponse. Also, the handler would catch panics.
With this approach we don't need a double encoding anymore, and the protocol is more explicit. The trade-off is a little more code in the .proto files, but I think it is worth it.
Thoughts?

Let me see if I got this right. Given these two messages:
message HelloReply {
  string message = 1;
}

message HelloNewReply {
  oneof nrpcresp {
    nrpc.Error error = 1;
    string message = 2;
  }
}

The generated code would be:
// interface
SayHello(ctx context.Context, req HelloRequest) (resp HelloReply, err error)
SayHelloNew(ctx context.Context, req HelloRequest) (resp string, err error)

// client
func (c *GreeterClient) SayHello(req HelloRequest) (resp HelloReply, err error) {...}
func (c *GreeterClient) SayHelloNew(req HelloRequest) (resp string, err error) {...}

For SayHelloNew, the generated handler and client will not use the RPCResponse envelope, but directly use a HelloNewReply.
Is that about right?

This is exactly what I meant.
The nrpc.Error, which I did not detail, could remain a string at first, but I think it could carry a little more information, like if the error is a client ("normal" error returned by the handler) or a server error (an intercepted panic), and an optional debug field for a stacktrace when a debug flag is activated.
Something along these lines:
message Error {
    enum Kind {
        CLIENT = 0;
        SERVER = 1;
    }
    Kind kind = 1;
    string message = 2;
    string debug = 3;
}

In the go mapping, we could make the Error struct implement the 'error' interface, so it the handler could return it directly if it needs more control on the actual error content.

We could decide that a function that have no "error-ready" response is a function that may never return an error. If it fails server-side, it simply never replies, leading to a timeout on the client side.
Don't know if it makes a lot of sense though (I don't have a use-case for it)

IMO the Error returned by the client would identify the cases:

the service returned an error
the service panicked
the infrastructure failed to send the request (safe to retry)
the infrastructure failed (not safe to retry)

Sending the service's stacktrace to the client to facilitate debugging seems like a hack. Ideally, the stacktrace (and other logs generated while servicing the request) would get tagged with a tracing id sent in the client request.

I am unsure how cases 3 and 4 would be identified (I guess it is implementation related).
Good idea to send a tracing id.

3 would be a failure of nats.Request() or nats.Publish() with an error like "unable to reach any nats servers". In this case, the user can retry the operation safely (say an operation like payment).
4 would be a timeout of nats.Request() or nats.Publish(). It won't be possible to know if the request was processed by the service or not in this case.
I was hoping to build tracing on top of nRPC, integrating with something like jaeger.

Ok so cases 3 & 4 are purely on the client side. Meaning the Error message returned by the server would only identify cases 1 & 2.
Integrating opentracing is a good idea, but should probably be a separated issue.

To get a taste of the reply pattern, I implemented in xbus. It works pretty well and allow code factorization for error handling (which is useful in my case because everything is hand-written).
I will start working on this issue now. I intend to let the pattern configurable with options, we will see were it goes.

Implemented by #13


Add Custom Theme?

I'm using ui-select with an ng-options (not ng-repeat). I need to add static text before the unordered list. It seems like a custom theme would be the best way. Is it possible to add a custom theme? If so, is there any documentation on how to do this? I searched but couldn't find anything.

I added the custom themes to a run block inside of the app.config file. Then I set the uiSelectConfig.theme within the appconfig. This seems to have worked.