[CONSOLE] "Smart" setting of decimal places is not that smart

Really not liking the "Smart" setting in the current state, it changed during merge of GDA and now it shows the "~" in multiple places where it should not have, e.g. if there should be exactly 1400 tokens distributed then it will still show that sign , or if the value would be 0 it would still show it as an estimate
Another point is that it is rounding way too aggressive IMO , I would expect "smart" setting to leave a nice trail of decimals depending on the value of the transaction like in the dashboard e.g. 123456789.987654321 would show up in console as "123456789" , and 0.543212345 would end up being something like 0.543 , currently it would aggressively round it up to ~1 , which is a big difference in tokens like ETH and BTC

Another point is that it is rounding way too aggressive
Agreed; the user experience isn't that great when small fractions are being aggregated together. Browsing the streams for this project vault, several users are streaming in "~5/month" streams, but they show up as "~0/day". With more precision that would be  "~0.1667/day".

I'd recommend instead of adding more decimal places, instead start to shift up from "day" to "week" to "month" to "quarter" to "year", until you hit a number greater than one. This already seems to be the case in the "Map" view, as the labels on the map for that example address do show those streams as "4.93/Mo".


web3 global variable too delayed initialization for AngularJS

[ ] I've asked for help in the Truffle Gitter before filing this issue.

Issue
The web3 global variable is initialized too late, on page load event, after the angularjs application is already initialized and after the first rendering,  so it's not available inside a controller code.
This behaviour started at commit 9003a2ab70040da06e269254e9fdadd228f025c5 with truffle-default-builder module version 1.0.2, before this release the web3 was already initialized at controller code execution.
Steps to Reproduce
Use this project: https://github.com/mpolci/truffle-web3-lazy-initialization
run testrpc, then:
truffle migrate
truffle serve

then open http://localhost:8080
Expected Behavior
the web page should report the following messages:
inside controller initialization
window.web3 variable initialized: true
currenteProvider initialized: true

during rendering
window.web3 variable initialized: true
currenteProvider initialized: true

Actual Results
inside controller initialization
window.web3 variable initialized: false
currenteProvider initialized: false

during rendering
window.web3 variable initialized: true
currenteProvider initialized: true

Environment

Operating System: ubuntu 16.04
Truffle version: 2.0.8
Ethereum client: testrpc
node version: 6.5.0
npm version: 3.10.7

A solution is to postpone angular bootstrap

Thanks for figuring this out!
On Tue, Sep 6, 2016 at 3:55 PM, Marco Polci<EMAIL_ADDRESS>wrote:

Closed #260 https://github.com/ConsenSys/truffle/issues/260.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/ConsenSys/truffle/issues/260#event-780131093, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAFp1WSiW2GXfpJHd7qD7czfJGOkK3opks5qne99gaJpZM4J2HEp
.

FYI; I had this as well and solved it in the angular controller by postponing everything that needed web3 (like loading accounts) to after document ready:
app.controller('indexController', function($scope, web3Service) {

var init = () => {
    web3Service.getAccounts().then(function (accs) {
        $scope.accounts = accs;
    }, function(err) {
        console.log(err);
    });

};

angular.element(document).ready(function () {
    init();
});


Fix Travis build

Build requirements have changed significantly since last year - need to fix Travis such that:

It builds at all
It builds without error!

Fixed in https://github.com/inventree/InvenTree/pull/118


Allow multiple dynamic project generates profiles with workflow configurable

it's quite typical for Atlantis users to mange repo in below architecture:
  project-1/
  project-2/
staging: -> need a separate workflow, deploy to different aws account
  project-1/
  project-2/```

Currently in digger the only way is to manually clarify all the projects as generate projects all share the same default workflow config.

The most straight forward way is to enable multiple generate_preojects

generate_projects:

includes: xxx
excludes: xxx
includes: xxx
excludes: xxx

I suggest digger can enable us to clarify projects in any sub folders under the repo. In this case the main config file doesn't need to care about the project structure at all.

prod/
digger.yml -> config differ
project-1/
project-2/
staging/
digger.yml -> config digger
project-1/
project-2/
And make digger binary to discover the config files, and execute accordingly.
Much appreciate!

Hi there! Thanks for filing the issue ..
Another idea I also have to keep everything toplevel is to enable workflow under each inc/udes/excludes pair as follows:
generate_projects:
    - includes: xxx
       excludes: xxx
       workflow: staging
    - includes: xxx
       excludes: xxx
       workflow: prod

workflows:
  staging:
    envs:
      state:
      - name: AWS_ACCESS_KEY_ID
        value_from: STAGING_ACCESS_KEY_ID
      - name: AWS_SECRET_ACCESS_KEY
        value_from: STAGING_SECRET_ACCESS_KEY
      commands:
      - name: AWS_ACCESS_KEY_ID
        value_from: STAGING_ACCESS_KEY_ID
      - name: AWS_SECRET_ACCESS_KEY
        value_from: STAGING_SECRET_ACCESS_KEY
  prod:
    envs:
      state:
      - name: AWS_ACCESS_KEY_ID
        value_from: PROD_ACCESS_KEY_ID
      - name: AWS_SECRET_ACCESS_KEY
        value_from: PROD_SECRET_ACCESS_KEY
      commands:
      - name: AWS_ACCESS_KEY_ID
        value_from: PROD_ACCESS_KEY_ID
      - name: AWS_SECRET_ACCESS_KEY
        value_from: PROD_SECRET_ACCESS_KEY

btw there are fields like terragtunt: bool seems can't assign to this discovery

cool thanks!

Will the new github action be released together?

hey @zongsizhang , the new action is realeased in v0.2.0 you can try to use this version directly!

Hey @zongsizhang good speaking today :) By the way did you get a chance to test out this new feature ?


Implement server for comment

I have completed the server tutorial and written the frontend and backend for making comments in my portfolio. The form requires users to enter a name and a comment. It saves them in a java hashmap of string (name) and arraylist of strings (comments).
I am still working on displaying the previously entered comments to the web page as I encountered some difficulties parsing a Java Hashmap in JSON format with Javascript. However, I am making progress, and I can confirm that the data structure adds and stores the comments as intended.
Although I have not finished the comment section completely, I thought it would be better to make multiple small Pull Requests, so I am submitting one now. Thanks!
Some future functionalities I am still working on include: error checking user input and displays user-friendly suggestions/warnings about them; also storing the current timestamp to the hashmap (maybe a HashMap<String, pair<ArrayList, Datetime>>? I can see the JSON getting harder to parse); sorting the comments by timestamp; displaying the comments in index.html.

Added Jess as a reviewer

Please get an LGTM from Jess before merging. Thanks,


Updated index.js

now supports react versions after 15.5

Hey @mayank-patel Can you please merge this? It fixes some errors in latest react versions


Update pt-BR.json

just check if GitHub didn't remove the last lines, sometimes it bugs out

thanks for the improvements


Add more charts to place page

First batch adding the following topics:

Literacy

Count_Person_Literate
Count_Person_Illiterate

Social Category

Count_Person_ScheduledCaste
Count_PersonScheduledTribe

Households

Count_Household_Rural
Count_Household_Urban

Poverty

Count_Person_Rural_BelowPovertyLevelInThePast12Months
Count_Person_Urban_BelowPovertyLevelInThePast12Months

Homelessness

Count_Person_Houseless_Urban
Count_Person_Houseless_Rural

Employment

Count_Person_NonWorker
Count_Person_Worker

Thanks for adding these! A screenshot of how it looks might be nice...

Thanks for review!


Fix for headers not being sent.

Due to a typo, the request headers set up in EventSource.prototype.xhrHeaders are never being sent. I noticed this because I was not seeing the Accept: text/event-stream header on my server when using the polyfill.

Thank you very much for your support.
We are merging this, and will prepare a new release.

Thank you!


Dynare sgu (and rbc) initial condition

For SGU:

In julia we do not have any of the initial conditions matching the pseudotrue.
looks like in https://github.com/HighDimensionalEconLab/HMCExamples.jl/blob/main/scripts/dynare_replication/SGUdata.m#L7  it creates the varables and saves it to https://github.com/HighDimensionalEconLab/HMCExamples.jl/blob/main/scripts/dynare_replication/SGUdata.m#L9
Then it calls https://github.com/HighDimensionalEconLab/HMCExamples.jl/blob/main/scripts/dynare_replication/SU03ext.mod#L131  but does the mode_file end up setting the initial condition when mode_compute = 0?
If so, then it looks to me like it is starting the sampling at its pseudotrue?

@donskerclass   We should discuss this issue when you get a chance.  Need to know what to do for the SGU initial conditions on dynare (and verifying that there is an issue with the setup of the rbc 2nd order initial condition)

Update:  we think that RBC is correct now.  The 2nd order should be correctly picking these up in the mod file.
So the proposed solution is to take the furrent sG03rext_mod.mat file which has identical values to the DGP https://github.com/HighDimensionalEconLab/HMCExamples.jl/blob/main/scripts/dynare_replication/SGUdata.m#L7
ANd then perturb them.  Just tell us the equivalent for what to put to perturb that,
parameter_names = [{'alpha'};{'gamma'};{'psi'};{'beta_draw'};{'rho'};{'rho_u'};{'rho_v'}];
xparam1 = [0.4;1.0;0.000742;4.0;0.4;0.4;0.4];

FWIW, the HMC on julia has: psueotrues of
alpha=0.32
gamma=2.0
psi=0.000742
beta=0.9615
rho=0.42
rho_u=0.2
rho_v=0.4

and we started the initial condition to be
0.328877958223689
2.3399861113683453
0.0005602652317476214
4.094970724844387 (this is beta_draw, so beta is 0.96066120489)
0.3877637128809934
0.2240415428766295
0.5348464581738216

I know these are not directly comparable, but that should give you a sense on the amount we are deviating to make things roughly comparable.

I can verify in the current code that Dynare is starting its SGU chain exactly at the values [0.4;1.0;0.000742;4.0;0.4;0.4;0.4]. I don't know that this is actually a problem at all, but if you wanted to make things directly comparable, the easiest option is to set the values in generate_sgu_data.m to exactly the Julia values
[0.328877958223689; 2.3399861113683453;0.0005602652317476214; 4.094970724844387; 0.3877637128809934; 0.2240415428766295; 0.5348464581738216]
If you want me to do this I can update it immediately and close the issue.


Botocore dependency breaking Home Assistant on initial launch

As mentioned in #1, there's an apparent issue with mandate's dependencies that seems to be causing an annoying error, requiring a restart or two either after initial installation of the component, or after updating Home Assistant. The error also blocks initial configuration, resulting in a "This integration does not support configuration via the UI" error. An example of the error is below:
2022-09-12 14:16:25.482 ERROR (MainThread) [homeassistant.setup] Setup failed for custom integration MagiQtouch: Unable to import component: cannot import name 'resolve_checksum_context' from 'botocore.client' (/usr/local/lib/python3.10/site-packages/botocore/client.py)

Yep these dependencies have been the no.1 issue I've run into while developing / installing the original version of this package. Some things required an older version of boto, others required a newer one, with conflicts all around.
I suspect the easiest path forward may be to get a working set of the required packages manually installed locally, then just copy them into this package in source so they don't need to be installed dynamically. There's plenty of other packages around that do this, even pip

This has hopefully been resolved in 069e56ba63770642ad8af449e1ed4a2ccc542cc8


feat: Implement SubnetFromState

Hi Multy Team,
Please check the changes and let me know if anything to be changed.
I assigned
	out.AvailabilityZone = r.Args.AvailabilityZone
As far as I know, never subnet's az will be able to modify.
Closes #216

Hey Shan! Thanks for the great contribution! I just added 1 comment but everything else looks great
You're right that the availability zone never changes so we don't need to parse it


feat: Remove redundant helm charts and add console as a dependency

Ⅰ. Describe what this PR did

Remove redundant helm charts.
Rename the current higress chart to higress-core
Add a new higress chart and add higress-core and higress-console as its dependencies.

Ⅱ. Does this pull request fix one issue?
Ⅲ. Why don't you add test cases (unit test/integration test)?
Ⅳ. Describe how to verify it
Ⅴ. Special notes for reviews

Codecov Report

Merging #223 (f786d1a) into main (8c95fd9) will increase coverage by 0.31%.
The diff coverage is n/a.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #223      +/-   ##
==========================================
+ Coverage   43.09%   43.41%   +0.31%
==========================================
  Files          31       31
  Lines        5038     5038
==========================================
+ Hits         2171     2187      +16
+ Misses       2699     2683      -16
  Partials      168      168

Impacted Files
Coverage Δ

pkg/bootstrap/server.go
61.45% <0.00%> (+6.10%)
:arrow_up:

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


fix: rephrase no-recent-bots box

Description
Replaces the no-longer-current "open the product tour" text on the home screen when there are no projects loaded with a rephrased link to starting a new bot.
Task Item
closes #8337
Screenshots

Coverage remained the same at 55.771% when pulling 74898f301d38e981167d04d589d57d9fb140331b on beyackle/changeIntroText into 4dbfaa0abe8a1d738d230aed37a8daa1cf7e6c20 on main.


Traffic Monitor: Use Local Time for events

There might be 2 issues here.

MST being used to display time in "Status"
Events shows on local server timezone (instead of client local). For example I had timezone on some Traffic Monitor of EST and some at GMT. The Events would show up at different times.

In both occasion, I believe it should be client local time.

@smalenfant: In 1.4.0 we moved events into their own log, event.log, separating application logging from state change logging. This new log is similar to our other formats and the timestamp is in seconds since the epoch. This was done via #900.
We have two other timestamps in Traffic Monitor:

Timestamp in the Wicket web UI under the event log tab
The timestamp in traffic_monitor.log

Both of these are likely using the default locale, which in your case, differs from area to area. We standardize on UTC across the board, so we don't have this issue.
You can correct the timestamp used in traffic_monitor.log yourself via /opt/traffic_monitor/conf/log4j.properties, but a code change would be required to allow UTC to be shown in the UI.
My concern is that some users might expect the time to be expressed in their locale, while others may want UTC. It seems that using a local timezone on the OS and UTC in the application logs would produce an inconsistent experience for command line users. When an operations user types "date" they usually expect log lines to correspond with their commands. This same argument could apply, but maybe to a lesser extent, to the web UI.
I'm sort of leaning toward making the timezone displayed in the web UI  configurable (local vs UTC) but might leave log4 as is to ensure we're consistent with the OS.
Thoughts?

@elsloo I'm using UTC across the board as well. Having it consistent is nice thing to have. When we ingest logs into a platform like Splunk, it normalize to local time for us (depending on the timezone of the client). We ended up with different timestamps because our servers were incorrectly setup. Logs should be UTC. Confusion can be easily resolved by making sure the logs have the Timezone information in them. Which ever format you prefer. ISO 8601?
Having the web UI configuration to show local vs UTC is what I was hoping for which is done on the client side anyway.

We can add the timezone to the application log and web UI. As far as what timezone to use in the web UI, given the direction of things I don't think it's worth it to build a client side tool to change the timezone. I was thinking it would be quick and easy to add a server side configuration parameter that is a boolean that controls whether to show time in localtime or UTC.
 That seems like overkill for this, especially if we're talking about changing the software/UI for 2.0.

Local or UTC would be my only choices.


Add password generator as builtin.password FQCN function

Hi! I think this fix for password generator should help
https://github.com/aalaesar/install_nextcloud/pull/136 and https://github.com/aalaesar/install_nextcloud/issues/137

@wiktor2200 the error persists after this change.

I see, maybe sth else has changed. Anyway if this generators will remain in the code there would be a need to apply these changes anyway, because it's using FQCN ansible.builtin.password function and without it will become deprecated sooner or later.

@wiktor2200 I am adding the FQCN it in my PR, but the length is not necessary. We can use the default.

Cool! :)


Reintroduce support for submissions

Resolves this Todo
https://github.com/laingsimon/courage_scores/blob/4904bd0a7a9604642c3d8bea95d0a38bf674d780/CourageScores/Services/Command/UpdateScoresCommand.cs#L241

Related to #123


Sign AmongUsCapture.exe to prevent Smartscreen warnings

Signing the exe will help prevent Windows Defender Smartscreen warnings and increase trust in the capture program.

We would do this, but this is hugely expensive. We would need an Extended validation code signing certificate, and those go for about $300 a year - $700 (from digicert). I don't think this open source software has that kind of money laying around.

We would do this, but this is hugely expensive. We would need an Extended validation code signing certificate, and those go for about $300 a year - $700 (from digicert). I don't think this open source software has that kind of money laying around.

We are open for donations though lol

We are open for donations though lol

Ah I see. No worries - thanks for creating an amazing bot and capture program!

Ah I see. No worries - thanks for creating an amazing bot and capture program!


Update transaction documentation

This PR adds documentation for transaction support in ODM.
TODO:

[x] Document baseline transaction features
[x] Document lifecycle event behaviour in transactions
[ ] Add examples for running queries within a transaction
[ ] Add examples for using the DocumentManager::transactional helper
[ ] Add examples for database logic in lifecycle event handlers

@jmikola Requesting your review for a first look at the wording. The documentation will most likely be expanded, but I wanted to get a first opinion on what I've added so far.


bond::is_writer does not work for derived types if they don't implement WriteStructBegin

Repro (I copied check_method and is_writer from bond's headers): https://godbolt.org/g/2F76N9
The way is_writer works - it tries to assign member function pointer from T::WriteStructBegin. When derived class relies on base's WriteStructBegin, this fails because it tries to assign base's pointer to member function to derived one.
I'm also guessing is_writer is not the only problem.
Copy of repro above:
#include <type_traits>
#include <boost/type_traits.hpp>

struct Metadata
{};

struct Base
{
    void WriteStructBegin(const Metadata&, bool) {}
};

struct Derived : Base {};

template <typename T, T> struct
check_method
    : std::true_type {};

template <typename T, typename Enable = void> struct
is_writer
    : std::false_type {};

template <typename T> struct
is_writer<T, typename boost::enable_if<check_method<void (T::*)(const Metadata&, bool), &T::WriteStructBegin>>::type>
     : std::true_type {};

int main()
{
    static_assert(is_writer<Base>::value, "!Base is writer");
    static_assert(is_writer<Derived>::value, "!Derived is writer"); // Fails
}

Is there a deeper problem that this is causing? We need to know that to make sure that the fix addresses the core problem too.
Are you, for example, hitting the static assert in the Serialization transform?
As a work around, try adding an explicit specialization of is_writer for your derived type:
template <> struct
is_writer<Derived> : std::true_type {};

Yes, this was exaclty the issue - I hit that assert. And yes, I did figure out this workaround, or I can just implement the method in derived class too. I just filed the issue that future users won't have to suffer :)

Thanks for confirming it was the static assert. We'll make sure the fix actually fixes that case.


Change reverse.each

Change on reverse_each because reverse_each loops in reverse order (no
intermediate array created).

@TimurTarasenko Thanks for fix


Bump documentation and REST API version to 13.0

Requirements

Filling out the template is required. Any pull request that does not include enough information to be reviewed in a timely manner may be closed at the maintainers' discretion;
Review the contributing guidelines for this repository; remember in particular:

do not modify code without testing for regression
provide simple unit tests to test the changes
the PR must not fail unit testing
if you're adding/updating documentation of an endpoint, make sure you add/update the necessary parameters to the (template) configuration files in the ensembl-rest_private repo

Description
Bumps the version and updates the wiki submodule
Use case
Bumps the version in lib/EnsEMBL/REST.pm and updates the wiki submodule
Benefits
We're ready for release/101
Possible Drawbacks
Testing
Not a functional change. Submodule commit matches.
Changelog
N/A

Coverage remained the same at 44.409% when pulling b84b4a2ac3f4b25b536d48963478f0dba0c83d70 on e101_version_bump into 0a2adc38a3166bb7f4d8c18521853659674b4afd on master.


Refactor _img_sdl2.pyx load_from_surface (cleanup)

Closes #5269. This supersedes PR #5274, squashed to avoid commit spam + some minor cleanup (release gil during conversion, change order of some tests, added comments)
Summary of changes:

Fix loading of SDL_PIXELFORMAT_BGR24 surfaces (BMP2, BMP3 images)
Fix loading of (some) PNG formats with binary alpha
Fix byte order for big-endian arch (untested)
Some other surface formats were incorrectly handled, but not used by SDL_Image

Known issues:

Some PNG48 images with binary transparency are still not loading correctly
8/16-bit grayscale PNGs are not supported. This seems like an issue with SDL_Image, it returns unexpected SDL_PIXELFORMAT_RGB565 instead of expected grayscale palette.

Details:

Log of PR against test images: link (100% pass)
Log of original implementation with same images: link (AssertionError: _img_sdl2: 100 of 1832 tests failed, these are all BMP2/BMP3 images. Plus unreported failures, see below)

Some of the tested _BINARY (binary alpha) png files loaded here are reported with 'pass' but they don't "really" pass. The test doesn't mandate the returned pixel format, it just verifies that the pixels are consistent with the returned format. So these are  correctly converted to RGB, but we lost alpha
Hence, the PR test lists all _BINARY images as 'rgba', while 116 of them are incorrectly listed as "pass" with "rgb" above.

Tested formats (ImageMagick):

Opaque: TIFF, BMP2, BMP3, PNG, GIF87
Binary: GIF, PNG8, PNG24, PNG48, BMP
Full Alpha: PNG32, PNG64, TGA

Notes:

All formats above were tested with fully opaque pixels
"Binary" and "Full Alpha" were tested with binary transparency
"Full Alpha" were tested with the same images as for opaque, but semitransparent.
TIFF is only tested as opaque, because the test doesn't support premultiplied alpha
JPG not tested because pixel values are unpredictable (lossy compression)

I didn't check the full code of the PR, but is there anything against a merge?

Testing on windows/macos/android/ios remains to be done, although I am fairly confident it will work as they are all little-endian in practice (right?). Big-endian would be nice to test of course, but how? PowerPC? Can a bi-endian ARM be used, if so how? Does anyone have a working system? Is it possible to run kivy in big-endian qemu?
One issue does need clarification; the default conversion to RGB24 if no alpha channel is detected. That is technically best in terms of memory performance, but not obviously the best overall. SDL2 will pitch-align it to 4 bytes, which needs an OpenGL extension for unpacking (or we'll do it in software). But it'll likely be unpacked as 32bpp in the end, so the question is, would it be better to default to RGBA conversion, even if the image has no alpha information? I don't know...
All in all, I consider it a low-risk merge. As noted in previous issues, there are a several problems with the original implementation (the shift test, non-standard SDL_PixelFormat, no colorkey detection, dead code paths from the weird not, ...), and the ImageLoader test case (although far from perfect) speaks in favor of this implementation..
Unless 1.10.1 is planned for release shortly, merging is probably a good way to get it test it more widely?

@bionoid any updates on this? is this ready to merger ?

Note to myself: try this together with #5785. I think the core dump happened in the SDL2 image provider when doing the screenshot test.


refactor rendering & handle cursor properly

Introduce a "dirty" state for the view. A view is "dirty" if it has
been updated but not rendered yet. Two things can be dirty: the line
cache, and the cursor. Making this distinction allows us to avoid
re-drawing the whole screen just to update the cursor position.

Move the rendering from screen.rs to view.rs and line.rs. Due to the
previous change, after drawing the view, we need to mark it as "clean".
That means we need to mutate the view, which would change force us to
pass a mutable reference of the view to the screen, for just updating
the state. Instead, it seems cleaner to pass stdout from the screen to
the view, and let it render itself and update its state accordingly.

Handle the cursor properly for ascii characters. Before, we were
printing "fake" cursors because we were not able to find where the
cursor should be set when a line contained tabs (\t). The problem is
that tabs have inconsistent length. We now handle this correctly.
However we assume tabs are 4 spaces large. So when using xi-tui, one
must run tab -4.

These are relatively big changes, I'd really appreciate if you could test and review this @ajm188 :)

Sure thing! I'll try to find some time to review that this week.

Thanks! I noticed that the cursor is not positioned corretly when line wrapping occurs. I'll try to see if there is an easy fix for that.

Thanks a lot for the review :)
I'm going to merge this, and address whatever remains in #36

sg2m!!


Callback telemetry QOS

closes #109

    pxHubClientOptions->pucModelID = NULL;

is this redundant if we are doing explicit initizlization of every filed ? ANd compiler we are checking our unit test always complaint about some varables are not init?

Refers to: source/azure_iot_hub_client.c:575 in de75a9b. [](commit_id = de75a9b02716e6e6e817b185c83e689f1c877b1b, deletion_comment = False)

I do not think we should generate identifier if QoS is zero.

Refers to: source/azure_iot_hub_client.c:858 in de75a9b. [](commit_id = de75a9b02716e6e6e817b185c83e689f1c877b1b, deletion_comment = False)


Issue-331 Fix redfish validator issues in Release_21.01 branch

This PR is for issue #335

by mistake deleted branch, so pr also got closed, reopening

by mistake deleted branch, so pr also got closed, reopening


SBOM: remove version and format fields

Remove version field from SBOM object as the SBOM document can self-report this information
Remove format field from SBOM object as this can be specified by the file extension and HTTP headers of the SBOM url

Please also make a PR to the examples repo that follows this.


[Feature REQ] Font support

It would be great if you in the future added support for font rendering with support for both ttf files loaded at run time and ttf files included as .h /.bin file(s).

ttf loaded at runtime ? bwa.
Maybe have a look at smea's geoshader thing for this

There's this https://github.com/smealum/ftpony/blob/d499b59deeadf0f8235061ec0001aaecc9ebacb8/source/text.c#L13

or you'know this https://github.com/xerpi/sftdlib


Feature/ssm agent add on

Issue #, if available: #93
Description of changes:
Installs SSM Agent on worker nodes using DaemonSet.
Before deployment:
aws ssm start-session --target i-06bbfe4fdeb6839d3

An error occurred (TargetNotConnected) when calling the StartSession operation: <instance-id> is not connected.

After deployment:
aws ssm start-session --target i-06bbfe4fdeb6839d3

Starting session with SessionId: kuapoorv-Isengard-0622c845c3ffc12e1
sh-4.2$

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Found a CrashLoopBackOff issue. I am preparing a fix for it.


NixOS testing framework does not log stdout

Issue description
I noticed that the NixOS testing framework does not log stdout from commands in testScript. It only logs stderr. What is the reason for this? Seems to me like logging stdout too is a good idea.

It would have made it obvious that the installer tests were sometimes hanging due to "parted" asking for user input (ref. 0ff4bb5f875afe8eb208bbabb3f52dc29539d2bc)
It might eliminate all the manual redirection currently being done: git grep ">&2" nixos/tests (I presume this is simply a workaround due to how things work today)

Thoughts?

This is intentional, because stdout can be used by the caller (e.g. my $output = $machine->succeed("...")). So it shouldn't be redirected to a log.

Would it be possible to use a "tee"? (So stdout would be logged but still available to the caller.)
Or to put it differently, why is stderr logged? The caller might want to capture stderr too?

Because stderr is for logging :-) That's just the interface here: if you want to process the output of a command, you send it to stdout, otherwise you send it to stderr.

I find that interface unintuitive. I wish there was a separate command to extract stdout/stderr from a command and into perl, and that succeed() and fail() simply logged both stdout/stderr. Oh well :-)


Annotated exception handler methods

Hi, would be a nice addition to support @ExceptionHandler methods and @ControllerAdvice. Those could be injected into the exception resolver chain. This feature is available here (at least to some extend) https://github.com/graphql-java-kickstart/graphql-spring-boot

The underlying mechanism is still that of GraphQL Java, so when a controller raises an exception, it has to propagate at first. Then GraphQL Java calls the registered DataFetcherExceptionHandler and we can check which controller raised the exception, look for @ExceptionResolver methods, go through @ControllerAdvice, etc.
What we have currently, as explained in the docs, is that you can register one or more DataFetcherExceptionResolver beans which apply globally like a ControllerAdvice. For example:
@Bean
public DataFetcherExceptionResolver exceptionResolver() {
	return DataFetcherExceptionResolverAdapter.from((ex, env) -> {
		if (ex instanceof SomeException) {
			return GraphqlErrorBuilder.newError(env).message("...").errorType(...).build();
		}
		else if (ex instanceof AnotherException) {
			return GraphqlErrorBuilder.newError(env).message("...").errorType(...).build();
		}
		else {
			return null;
		}
	});
}

I'm wondering whether this is sufficient for GraphQL? For once the inputs are simpler and for output there is no need to write to a response, which reduces the amount of benefit from such methods. At the same time there is extra overhead in GraphQL where there may be multiple exceptions per request for different parts of the data graph and the exception resolution steps are repeated multiple times. The other aspect could be the need for localized handling. It would be useful to hear more feedback on all this.

@rstoyanchev I think it's just nicer to have separate @ExceptionResolver methods instead of a single method with a bunch of if (ex instanceof ....
Some exceptions will be in response for the client to see, some will be internal errors that the client should not see but we still want to recover from or at least log.

It's good way than using if (ex instanceof ....
     @ExceptionHandler(         EntityNotFoundException::class,         NoSuchElementException::class,         EmptyResultDataAccessException::class,         IndexOutOfBoundsException::class,         KotlinNullPointerException::class     )     fun notFoundException(e: Exception): GraphQLError {         return GraphqlErrorBuilder.newError().message(e.message).build()     }

It looks like there is an issue with applying annotated exception handling to non-controller DataFetchers as described in https://github.com/spring-projects/spring-boot/issues/34526#issuecomment-1464145095.


Apparmor simplification

This was tested by comparing the profile of an unprivileged container with a simplified version where the mount options alternations were removed:
# get a generated profile
lxc launch images:ubuntu/20.04 u1
cp /var/snap/lxd/common/lxd/security/apparmor/profiles/lxd-u1 lxd-u1.orig

cp lxd-u1.orig lxd-u1.simplified
vim lxd-u1.simplified # remove mount options alternations from this branch (essentially apply this diff: https://termbin.com/v1vd)

# DFA comparison
apparmor_parser -Q --dump=dfa-states -K lxd-u1.orig 2> lxd-u1.orig.dfa
apparmor_parser -Q --dump=dfa-states -K lxd-u1.simplified 2> lxd-u1.simplified.dfa
cmp lxd-u1.orig.dfa lxd-u1.simplified.dfa

# full compilation comparison
apparmor_parser -K lxd-u1.orig -o lxd-u1.orig.compiled
apparmor_parser -K lxd-u1.simplified -o lxd-u1.simplified.compiled
cmp lxd-u1.orig.compiled lxd-u1.simplified.compiled

Those tests were done once in each of:

20.04 VM with Apparmor 2.13.3-7ubuntu5.1

root@v1:~# ll lxd-u1.orig.dfa lxd-u1.simplified.dfa
-rw-r--r-- 1 root root 168556 Oct  5 03:29 lxd-u1.orig.dfa
-rw-r--r-- 1 root root 168556 Oct  5 03:29 lxd-u1.simplified.dfa
root@v1:~# ll lxd-u1.orig.compiled lxd-u1.simplified.compiled
-rw-r--r-- 1 root root 41793 Oct  5 03:29 lxd-u1.orig.compiled
-rw-r--r-- 1 root root 41793 Oct  5 03:30 lxd-u1.simplified.compiled

18.04 VM with Apparmor 2.12-4ubuntu5.1

root@v2:~# ll lxd-u1.orig.dfa lxd-u1.simplified.dfa
-rw-r--r-- 1 root root 168556 Oct  5 03:29 lxd-u1.orig.dfa
-rw-r--r-- 1 root root 168556 Oct  5 03:29 lxd-u1.simplified.dfa
root@v2:~# ll lxd-u1.orig.compiled lxd-u1.simplified.compiled
-rw-r--r-- 1 root root 41785 Oct  5 03:30 lxd-u1.orig.compiled
-rw-r--r-- 1 root root 41785 Oct  5 03:30 lxd-u1.simplified.compiled

P.S.: Special thanks to sarnold for the help with Apparmor.

This pull request didn't trigger Jenkins as its author isn't in the whitelist.
An organization member must perform one of the following:

To have this branch tested by Jenkins, use the "ok to test" command.
To have a one time test done, use the "test this please" command.

Those commands are simple Github comments of the format: "jenkins: COMMAND"

jenkins: test this please

@simondeziel I like the use of the apparmor tools to compare the actual compiled policy.
@stgraber I wonder if we should add something to the test suite to ensure that the apparmor policy for a test container is being applied as expected based on a static policy in the test suite, that would capture any regressions in the basic apparmor policy.

Not sure how easy that would be. It couldn't be a compiled policy as that would change based on compiler version.
So it would have to be an expanded text version of the policy that we would then need to mangle to match the conditional bits of the Go code (paths mostly).
It feels to me like one of those tests that effectively just checks if another file changed as I don't see what could alter our generated profiles other than a change to lxd/apparmor and we already know to pay very close attention to any PR which touches that package.

So are you effectively removing some specific rules that are already covered by more general rules?

It's basically removing rules with alternated order of mount options on the basis that those 2 rules are identical in meaning:
mount options(ro,noatime) ...
mount options(noatime,ro) ...

jenkins: test this please


Remove empty space between sent messages and input field

https://www.dropbox.com/s/wv1hx2ixnxy08x0/Screenshot from 2023-07-20 20-31-49.png?dl=1
Looks like there is no reason to look at empty space instead of seeing more messages from the history.

This is a duplicate of #16397.


xabort instruction terminates the control flow in certain conditions

While looking at https://github.com/mandiant/capa/issues/2406, I noticed a very strange issue that the number of basic blocks of a function can be reported different in certain conditions (yet unknown).
To start with, due to https://github.com/mandiant/capa/issues/2516, the capa is getting the list of basic blocks of each function twice instead of once. And for function 0x5494e0, when it pulls the list of basic blocks, binja reports 13 basic blocks for the first time, but only one basic block in the second time. If I open the binary in binja GUI, i also see 13 functions. It is yet unclear why we would report only one functions in the second time.
But the basic block is indeed unusual, it contains an xabort instruction. And as can be seen in the below screenshot, for some reason we split the code at 0x5494e7 (which is immediately after the xabort instruciotn) into a new basic block. So maybe xabort somehow terminates the basic block? And in the second time we query the list of basic blocks, maybe certain analysis info is not yet ready, so the function does not contain other basic blocks.

2f7f5fb5de175e770d7eae87666f9831.elf_.zip

get_instruction_info returns an UnresolvedBranch for xabort instruction, which might be relevant to the issue

I somehow remember this also affects the debugger -- I see that there is only one basic block in the graph view, but re-analyzing the function gives me back all of the expected blocks.
I am suspecting that in certain conditions, the analysis update removes the CFG info but does not recalculate it, leading to an incomplete CFG

This can be reproduced by the following script:
from binaryninja import *

bv = load('/Users/xusheng/Downloads/2f7f5fb5de175e770d7eae87666f9831.elf_.bndb')
func = bv.get_function_at(0x5494e0)
print(func.mlil)
print(len(func.basic_blocks))

for func in bv.functions:
    if func.start == 0x5494e0:
        continue
    try:
        print(func.mlil)
    except:
        pass

func = bv.get_function_at(0x5494e0)
print(func.mlil)
print(len(func.basic_blocks))

The last line print(len(func.basic_blocks)) will print 1 instead of 13.
I have to say this is the most mysterious bug that I have seen. The second print(func.mlil) at line 17 is important -- if we remove it, it will print 13 (correct) instead of just 1.
This bug is probably a combination of cache eviction and analysis update, which get the function into a weird state, and the indirect branch info needed by the basic block analysis is missing. The indirect branch info is used by the analysis to continue the control flow at 0x5494e7

Well, it seems the operation of requesting the MLIL causes the indirect_branches to be cleared and not populated after that:
from binaryninja import *

bv = load('/Users/xusheng/Downloads/2f7f5fb5de175e770d7eae87666f9831.elf_.bndb')
func = bv.get_function_at(0x5494e0)
print(func.indirect_branches)
print(func.mlil)
print(func.indirect_branches)

This prints
[<branch x86_64:0x5494e4 -> x86_64:0x5494e7>, <branch x86_64:0x54952e -> x86_64:0x549531>]
<MediumLevelILFunction: x86_64@0x5494e0>
[]

2f7f5fb5de175e770d7eae87666f9831.elf_.zip


GUACAMOLE-1765: Use maven-frontend-plugin instead of jasmine-frontend-plugin to fix docker build.

https://github.com/apache/guacamole-client/pull/826, but targeted at staging/1.5.1 instead of master.
This change:

Switches from jasmine-frontend-plugin to maven-frontend-plugin to get the build working
Switches from chrome to firefox for unit tests
Disables tests unless explicitly requested so the build can be run in environments without firefox installed.

Future improvements that would be good (but are out of scope for fixing the build):

Have npm run the build and tests
Create a full-fledged NPM package for guacamole-common-js that could be published.

Well, the github action isn't in 1.5.1 but I can verify that the docker build works successfully locally:
➜  guacamole-client git:(GUACAMOLE-1765-fix-broken-docker-build) docker build .
[+] Building 216.3s (19/19) FINISHED
 => [internal] load build definition from Dockerfile                                                                                0.0s
 => => transferring dockerfile: 2.72kB                                                                                0.0s
 => [internal] load .dockerignore                                                                                0.0s
 => => transferring context: 112B                                                                                0.0s
 => [internal] load metadata for docker.io/library/tomcat:8.5-jdk8                                                                                0.5s
 => [internal] load metadata for docker.io/library/maven:3-jdk-8                                                                                0.5s
 => [auth] library/maven:pull token for registry-1.docker.io                                                                                0.0s
 => [auth] library/tomcat:pull token for registry-1.docker.io                                                                                0.0s
 => [builder 1/5] FROM docker.io/library/maven:3-jdk-8@sha256:ff18d86faefa15d1445d0fa4874408cc96dec068eb3487a0fc6d07f359a24607                                                     0.0s
 => [stage-1 1/6] FROM docker.io/library/tomcat:8.5-jdk8@sha256:c7fbc01187348b7994efd36b03e1592e3757814bb33ac5ee3df5e0054df688e6                                                   0.0s
 => CACHED [stage-1 2/6] RUN apt-get update -qq     && apt-get install -y xmlstarlet     && rm -rf /var/lib/apt/lists/*                                                            0.0s
 => CACHED [stage-1 3/6] WORKDIR /opt/guacamole                                                                                0.0s
 => [internal] load build context                                                                                0.4s
 => => transferring context: 691.83kB                                                                                0.4s
 => CACHED [builder 2/5] RUN apt-get update && apt-get install -y firefox-esr                                                                                0.0s
 => CACHED [builder 3/5] COPY guacamole-docker/bin/ /opt/guacamole/bin/                                                                                0.0s
 => CACHED [builder 4/5] COPY . /tmp/guacamole-docker-BUILD                                                                                0.0s
 => [builder 5/5] RUN /opt/guacamole/bin/build-guacamole.sh "/tmp/guacamole-docker-BUILD" /opt/guacamole                                                                         211.7s
 => [stage-1 4/6] COPY --from=builder /opt/guacamole/ .                                                                                0.5s
 => [stage-1 5/6] RUN groupadd --gid 1001 guacamole                                                                                0.6s
 => [stage-1 6/6] RUN useradd --system --create-home --shell /usr/sbin/nologin --uid 1001 --gid 1001 guacamole                                                                     0.8s
 => exporting to image                                                                                0.9s
 => => exporting layers                                                                                0.9s
 => => writing image sha256:3e6f5dc87f1a9d3dced526584953dce6b3c64f2aba6e474227011143a8f75d27                                                                                0.0s

All comments from https://github.com/apache/guacamole-client/pull/826 should be addressed now.


Grid shift transformations

For Czech Republic, Switzerland, France and New Zealand are available grid files which would be cool to list under transformations.
These do not have records in official EPSG database.
Czech: http://grass.fsv.cvut.cz/gwiki/S-JTSK-Grid
Swiss CHENyx06: http://www.swisstopo.admin.ch/internet/swisstopo/en/home/products/software/products/chenyx06.html
New Zealand: nzgd2kgrid0005.gsb: http://help.maptiler.org/coordinates/australasia/new-zealand
France, ntf_r93: ..
Canada, ...
UK, ostn02: http://www.ordnancesurvey.co.uk/business-and-government/help-and-support/navigation-technology/os-net/ostn02-ntv2-format.html
more at http://trac.osgeo.org/proj/

Some of the GSB binaries linked from:
http://www.eye4software.com/hydromagic/documentation/ntv2/

https://svn.osgeo.org/metacrs/csmap/trunk/CsMapDev/Dictionaries/


Add catalog-info.yaml

So we can use a website in our Backstage demos.

Just realised there already is a metadata file. 😭


Don't use Application.gev_env

I need to run multiple Trot instances, and global configuration stamps on those dreams. Manually building a supervision tree works as a quick hack, but I still have to contend with pre_routing and post_routing being global.
Instead, configuration should be pulled via options passed into init/1.

So there are two places that Application.get_env/3 is used, and I think its worth making a distinction between them. One is in the router, which it sounds like is what you're having trouble working around. I'm totally fine with replacing that with init/1 options.
The second place is in the supervisor and used for things like what port to listen on. I think that makes sense to have as env settings, but I'd be interested in hearing about your use case if you're asking to configure those settings via passed args as well.

With regards to the latter, I need to run both an unsecured and secured (http and https) endpoints. Separating them into distinct applications (under an umbrella) just doesn't make sense (most of the code is shared, maintaining two separate applications is needlessly complex, and so on).
The thing is, you can have both. You just have to move the Application.get_env specific code into Trot.start/2, wherein you translate the environment settings into options that are passed to Trot.Supervisor.init/1 (via Trot.Supervisor.start_link.)
I can submit a pull request that does the aforementioned (and by nature, it will not break backwards compatibility).


feature(connections): disable ConnectionInfo table editing when client was connected

The following is the previous version:

We expect that the form cannot be modified when connected, Now we cannot modify the form while connected.

We expect that the form cannot be modified when connected, Now we cannot modify the form while connected.

Disabled input seems a bit ugly. Maybe you can set the input background color to transparent, and see some. Then look at the effect and try not to destroy the original style, our purpose is just to disable it.

before

after


[FEATURE] adding ACTIVE state to sidebar links, when clicked, or when url correspinding to it

[FEATURE] adding ACTIVE state to sidebar links, when clicked, or when url corresponding to it
=> It is more "user-friendly" I think...
[FIX]    rename "Display table centering" to "Centering display table", "Flexbox centering" to "Centering flexbox", "Transform centering" to "Centering transform"
=> when I browse all tips, I think is easier to find by the "what" (in my modification, it's centering).
For me, it was easier to know that I have 3 kinds of technic in order to "center something".
I hope it's help.
It's my 1st pull request, so please be kind 🙏


Feature: merging keras and tf.keras frameworks

Hi @Callidior
I am going to adapt my segmentation-models lib for tf.keras, so I also need efficientnet adapted for tf.keras.
I saw your implementation of Efficientnet for keras-applications. Very nice work, but it is too long to wait until keras will merge it and make a new release.
It would be nice if you make a PR with same code to this repo, or allow to copy it.

Hi @qubvel
I am glad that you find my contribution to keras-applications useful and I would be happy to help you. I am, however, not exactly sure how I can help you, since you already seem to have quite a nice implementation of EfficientNet. Could you provide me with more details?

@Callidior My model is not suited both for keras and tf.keras, I have two different branches for that. But your model written in keras applications style to work with both. If you want you can just paste your model code to https://github.com/qubvel/efficientnet/blob/feature/merging-keras-tf.keras/efficientnet/model.py (feature/merging-keras-tf.keras branch) and make a PR. I will do the rest. Or I can copy it by myself, if you are not against it, and mention you as author.

Okay, I created PR #42 with an exact copy of my code. Hope it helps!

Everything works ok for keras, but tf.keras throw an arrow in Dropout layer https://travis-ci.com/qubvel/efficientnet/jobs/218365138
@Callidior do you have any thoughts?

In my opinion, this is a bug in tf.keras. I compared the implementation of the Dropout layer with that of keras and found that tf.keras does not replace None-valued dimensions with the actual batch size. keras does this in keras.layers.Dropout._get_noise_shape:
def _get_noise_shape(self, inputs):
    if self.noise_shape is None:
        return self.noise_shape

    symbolic_shape = K.shape(inputs)
    noise_shape = [symbolic_shape[axis] if shape is None else shape
                   for axis, shape in enumerate(self.noise_shape)]
    return tuple(noise_shape)

tf.keras just leaves the dimension of the batch axis as None, which leads to the error.
A clean fix would be to derive a custom dropout layer compatible with both frameworks, which implements the correct behaviour:
class Dropout(layers.Dropout):

    def _get_noise_shape(self, inputs):

        if self.noise_shape is None:
            return self.noise_shape

        symbolic_shape = backend.shape(inputs)
        noise_shape = [symbolic_shape[axis] if shape is None else shape
                       for axis, shape in enumerate(self.noise_shape)]
        return tuple(noise_shape)

But then, you need to include this layer in the custom_objects dictionary for de-serializing saved models.
A more hacky fix, which I didn't completely test, would be to replace noise_shape=(None, 1, 1, 1) in the model construction code with noise_shape=(backend.shape(x)[0], 1, 1, 1), but I don't like this solution and suspect that it might lead to different errors at some point.

Thanks for clarification, I dont like that I have to shadow new Dropout class definition, because at first time layers module is None and I can not use inheritance from layers.Layer. But may be its a price for compatibility)
Are you going to make a bug report to tensorflow?
I also have included swish to global custom objects to make model deserializable. May be you also need to do it in your keras-applications PR?

I don't like any of these workarounds either and have file a bug report at https://github.com/tensorflow/tensorflow/issues/30946, but who knows how long it will take them to fix it.
Maybe the second "hack" I suggested might be a quick solution for you in the meantime.

Hack with noise_shape=(backend.shape(x)[0], 1, 1, 1) does not work, because object is not serializable, while defining a custom Dropout layer works fine. Thanks a lot :+1:

Hm, I somewhat expected that. Thanks for checking. :+1:
But besides the custom Dropout layer, I don't see another way at the moment, unfortunately.

It was noted in the tensorflow issue that this bug has recently been fixed in https://github.com/tensorflow/tensorflow/commit/ab2644e7178312fc9c89f90d7c8e61f73f7c1f64. But that commit does not seem to have made it into a release yet.

Thanks, will wait for release to update)
For now the work is done, thanks again for your help!


2.0 - don't add IDE version string to sandboxContainer

What happened?
Migrating to the gradle plugin v2, how can I set the full path of the sandbox container?
With plugin v1, if I set intellij { sandboxDir.set("foo/") }, starting the sandboxed IDE generates folders like /foo/config, /foo/system, etc.
With plugin v2, if I set intellijPlatform { sandboxContainer = File("foo/") }, starting the sandboxed IDE generates folders like /foo/IC-2024.1.1/config, /foo/IC-2024.1.1/config, etc.
Is there a way to remove the sandboxed IDE version string from the sandbox container's path? It looks like a regression.
Relevant log output or stack trace
No response
Steps to reproduce
Migrate to plugin v2 and run gradlew runIde.
Gradle IntelliJ Plugin version
2.0.0-beta3
Gradle version
8.7
Operating System
Windows
Link to build, i.e. failing GitHub Action job
No response

What is the reason for that?
Versioned directories within the sandbox container directory enable introducing custom tasks, like running a custom IDE with the plugin loaded with a different IntelliJ Platform.

I want to easily share the same config across multiple IDE minor versions (e.g. 2024.1.1, then 2024.1.2). Now, each time a new minor IDE version is released, I have to copy/past my config. This is boring.

What about
tasks {
    prepareSandbox {
        sandboxDirectory = intellijPlatform.sandboxContainer.dir("fixedName")
    }
}

This is perfect, I was not aware of that. Thanks!!

Hum wait, when running runIde, it generates folders in fixedName, and in the regular build/generated/idea-sandbox/2024.1.1, and the sandboxed IDE picks config from this regular folder, not from mu custom one.

An alternative would be to let the user override the idea.config.path property. But, if I do that via systemProperties("idea.config.path" to "/foo/config"), the config is stored in /foo/2024.1.1/config. Even there, the IDE version string is inserted inside the config path.

Thank you for highlighting the above issue! Apparently, tasks didn't inherit from the sandbox producer (prepareSandbox), but that is now fixed.
It'll be available this week with 2.0.0-beta4.
BTW, if you don't want to refer to the sandboxContainer (build/idea-sandbox by default), you can do as follows:
tasks {
    prepareSandbox {
        sandboxDirectory = layout.projectDirectory.dir(".idea-sandbox2/stable")
    }
}

Is it included in beta4? It changed nothing for me.

The sandbox management was revisited again in beta8. With the current beta9, it works as described above.


Use Accept header to guess the content type

The content-type response header should be guessed from both the file type (guessed from extension and/or content) AND from the Accept request header.
Uses cases:

files without extension
unknown files extension
text files that could be use in different ways: either displayed or processed using javascript or downloaded.

Hi! I've never heard a standard for this before. Can you point me to a standard that specifies what the steps would be, how to resolve multiple types in the Accept header, etc.? If there is not a standard, I'm not sure if it really belongs in this module, especially since you should be able to easily implement this by yourself using the headers event (https://github.com/pillarjs/send#events):
send(req, req.url)
.on('headers', guessContentType)
.pipe(res)

function guessContentType(res, path, stat) {
  var req = res.req
  // perform your logic here and eventually...
  res.setHeader('Content-Type', guessedType)
}

The Accept Header is a HTTP 1.1 standard header, one of the header used for content negociation.
Here the official spec https://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.1
Here a more readable doc: https://developer.mozilla.org/en-US/docs/Web/HTTP/Content_negotiation
This header is used by the browser or any agent to indicate what content is acceptable by the client
The spec indicate that the server may use it for content negociation and specify in a quite clear way how this header should be interpreted by the server.

content negotiation is the mechanism that is used, when facing the ability to serve several equivalent contents for a given URI, to provide the best suited one to the final user

the served content means response body AND headers. The most signifant one is Content-Type
Practically, the Accept header is set in all major browser to indicate to the server how will be used the resource (i.e. what content type is acceptable) and the Content-Type header is used by all major browsers to determine how to use the served content.
In most cases, the content-type header does not prevent the browser the use it in the way it want (for example an img tag seems to be always displayed as an image, ignoring the content-type header).
But when the browser does not know how to use the response, this creates unwanted behaviors. This could be the case when:

typing an url in the address bar
loading a resource in an iframe
preloading resources using javascript

For example, files without extension (or with an unhandled extension like .php) served by send are currently triggering a download.
Regarding send, the Accept header should be at least used:

to guess the content-type when the content type is not guessable from the file extension
to determine the content-type to use when there are several content type guessed from the file extension

Hi! Sorry if I was not clear. What I was asking for a standard for is regarding

to guess the content-type when the content type is not guessable from the file extension

Can you be more specific on the exact algorithm that we should use to do this? Is there a standard out there that defines this algorithm? The listed standard you provided does not, as far as I can tell. It only outlines an idea. For example, the Accept header contains many different types by most web browsers. How should this module select which one to use? What if the accept header is just Accept: */*? We cannot send back Content-Type: */* because that is not a valid type. I'm just asking for the algorithm to use to make this determination.

to determine the content-type to use when there are several content type guessed from the file extension

Currently, we don't have any way to do this, because internally, the module we use only gives a 1-to-1 mapping of file extension to type. If you have any ideas around how to accomplish this, that would be welcome!
A big concern that comes to my mind with this feature is security. For example, for users who allow uploads and use this module to serve them up. As an example, a site only allows image files, so accepts only uploads with .jpg or .jpeg extensions. Then, that image is served up by this module. Well, a malicious user uploads HTML with JavaScript with a file extension of .jpg. With the current code, it will contain Content-Type: image/jpeg. From the sounds of this change, if that malicious user know a web browser sends the header Accept: text/html, application/javascript, text/xml, */*;q=0.6, they would reference that .jpg file and this module would instead send back Content-Type: text/html, since that looks like what the browser wants. Now, malicious script is running on that domain, which is a persistent XSS attack scenario. What would we need to do to help mitigate against this with this feature?

to guess the content-type when the content type is not guessable from the file extension

I just have enquired a bit deeper. There are no standard for this but Apache HTTPD server implements it quite simply, with a one to one mime types map.
The main differences with the actual implementation of send are:

There is no default content type (the header is not set if not guessed. (Before 2.2.7 the default content type was text/plain in order to not trigger download)
The guessed content type is passed through their content negociation algorithm which may result in a 406 Error if the guessed content type is not acceptable (i.e. Accept header does not contains */* and the guessed content type does not match any of the acceptable types)

In a nutshell:

The issue of wierd behavior (triggering download or so) should be preferably fixed by not setting a default content-type header
The Accept header could be handled to improve send but in a filtering way (I don't see any added value to this ...)

Can you work on putting a pull request together?

Before 2.2.7 the default content type was text/plain in order to not trigger download

Are you sure you are not confusing this module with something else? There has never been a version 2.2.7 of this module; we have not even reached 1.0.0

Are you sure you are not confusing this module with something else? There has never been a version 2.2.7 of this module; we have not even reached 1.0.0

Sorry if I wasn't clear, I was talking about apache. Default mime type of apache httpd was text/plain in 2.2.x. In current release, there is no default content type, In this case the client have to determine itself how to use this undetermined content.

Can you work on putting a pull request together?

Yes, I'll have a look at the code and make a pull request.
But the discussion may not be over ...
Not setting the content-type header may be a breaking changes for some people, in case they asume it is always set and do tests on it, without testing its existence (I mean in the server side, in a later response filter, like gzip encoding for example).
What do you think about?
What is the send module policy about possibly breaking changes? is a version bump + a warning note enough or should we implement it in several steps

adding a defaultType setting which is recommended to be set to null but has a default to text/plain (or the the old default of send which seems to be application/octet-stream). We may also print a warning to console if the setting is not set.
Then switch later the default to null (and remove the recommendation & warnings)

I've just made a pull request: https://github.com/pillarjs/send/pull/97

Hi @felicienfrancois,

Not setting the content-type header may be a breaking changes for some people, in case they asume it is always set and do tests on it, without testing its existence (I mean in the server side, in a later response filter, like gzip encoding for example).
What do you think about?

Well, I'm not going to accept that into this module, I'm sorry. You may want to consider forking if you really want that behavior. A response with an entity body should always have a Content-Type header, at the very least to prevent the very persistent XXS attack I highlighted above, which I don't think you addressed/responded to.

What is the send module policy about possibly breaking changes? is a version bump + a warning note enough or should we implement it in several steps

Well, the policy right now is to implement it in a backwards-compatible way, if at all possible (perhaps that means the functionality have to be opt-in). Then use deprecation that we have to get people to change their code in such a way that they can get rid of the deprecation without changing the behavior of their existing code & make upgrading to the next major a non-issue.
As far as timing for another major, that is dependent upon the timeline of the Express project, as this project will not bump to 1.0 until Express bumps it's major version. Any pull request that targets the next major of this module should be made against the https://github.com/pillarjs/send/tree/1.x branch.

I've just made a pull request: #97

I see nothing in that pull request regarding the use of the Accept header, so as far as I can tell, that is not even related to the discussion here regarding the use of the Accept header?

I see nothing in that pull request regarding the use of the Accept header, so as far as I can tell, that is not even related to the discussion here regarding the use of the Accept header?

Yes your right, as I pointed out, this header is mostly used for not static contents or multiple static contents served on the same URI, which is not the case in send.
In the case of pure one URI = one file static servers, it may be used to check if the served content is acceptable and send a 406 error if not.
This could theorically be useful to prevent some XSS attacks (for example if the Accept header was equal to image/jpeg, image/*, it would mean it only accept images and serving a .html file should result a 406 error instead).
But all modern browsers append */* to the Accept header for all kind of requests, so the served content is always acceptable.
=> Implementing such a filter in send could be done but has no added value
So this issue can be closed.
I will open a new issue regarding the default Content-Type header


fix remove useless substr of string like type cast

Proposed changes
Issue Number: close #xxx

Further comments
If this is a relatively large or complex change, kick off the discussion at<EMAIL_ADDRESS>by explaining why you chose the solution you did and what alternatives you considered, etc...

run buildall


fixed #549

this fixes #549

This PR removes code that is not related to the issue being referenced...


Feature/Kubernetes: Allow overriding randomly generated namespaces

Hi there,
it would be great to override Drone creating randomly generated namespaces, thus running all builds in the same namespace as drone agents (DRONE_KUBERNETES_NAMESPACE) or using a custom namespace (DRONE_KUBERNETES_BUILD_STAGE_NAMESPACE ?) to address this.
What do you think?

:wave: hey there, we have an open issue dedicated to this topic at https://github.com/drone/drone-runtime/issues/53

Thanks for the prompt reply!


Error trying to install express

npm ERR! registry error parsing json
npm ERR! SyntaxError: Unexpected token ILLEGAL
npm ERR! Internal routing error
npm ERR!
npm ERR! Sorry, we cannot connect to the intended server.
npm ERR!
npm ERR! We have just been notified of this problem. We will correct it as soon as possible.
npm ERR!
npm ERR! Feel free to contact us if you have any questions<EMAIL_ADDRESS>npm ERR!
npm ERR!     at Object.parse (native)
npm ERR!     at IncomingMessage. (/usr/local/lib/node_modules/npm/lib/utils/npm-registry-client/request.js:157:25)
npm ERR!     at IncomingMessage.emit (events.js:81:20)
npm ERR!     at HTTPParser.onMessageComplete (http.js:133:23)
npm ERR!     at Socket.ondata (http.js:1226:22)
npm ERR!     at Socket._onReadable (net.js:683:27)
npm ERR!     at IOWatcher.onReadable (net.js:177:10)
npm ERR! Report this entire log at:
npm ERR!     http://github.com/isaacs/npm/issues
npm ERR! or email it to:
npm ERR<EMAIL_ADDRESS>npm ERR!
npm ERR! System Linux 2.6.24-23-xen
npm ERR! command "node" "/usr/local/bin/npm" "install" "express"

npm install express --registry http://registry.npmjs.org/
I found this in Stackoverflow


Add rate limit note; suggest caching/etags

Why:
Context is missing around why PAT is set when fetching GH public keys for verficiationrecommend authentication or caching for secret scanning partners validating the GH pub keys.

Closes [issue link]
What's being changed:
docs for secret scanning

Check off the following:

[ ] I have reviewed my changes in staging. (look for the deploy-to-heroku link in your pull request, then click View deployment)
[ ] For content changes, I have reviewed the localization checklist
[ ] For content changes, I have reviewed the Content style guide for GitHub Docs.

Thanks for opening a PR with this addition @jamisonhyatt and thanks for the technical review @greysteil ⚡ I'll get this triaged for review!


How to forward a host port to a local container ip?

I'm following the Calico as a Docker network plugin example with the install instructions for DigitalOcean.
Container-to-conatiner communication works just fine with --ipip --nat-outgoing enabled. However i can't figure out how to forward a host port to a local container. i've added this according to the faq:
iptables -A PREROUTING -t nat -i eth0 -p tcp --dport 80 -j DNAT  --to <IP_ADDRESS>:80

# iptables -t nat -L -n -v
Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
    0     0 felix-PREROUTING  all  --  *      *       <IP_ADDRESS>/0            <IP_ADDRESS>/0
  479 33282 DOCKER     all  --  *      *       <IP_ADDRESS>/0            <IP_ADDRESS>/0            ADDRTYPE match dst-type LOCAL
    0     0 DNAT       tcp  --  eth0   *       <IP_ADDRESS>/0            <IP_ADDRESS>/0            tcp dpt:80 to:<IP_ADDRESS>:80

But this doesn't work:
# curl <HOSTIP>
curl: (7) Failed to connect to  <HOSTIP> port 80: Connection refused

# curl -I <IP_ADDRESS>
HTTP/1.1 200 OK
Date: Mon, 11 Jan 2016 09:35:09 GMT
Server: Apache/2.4.18 (Ubuntu)
Connection: close
Content-Type: text/html; charset=UTF-8

Any help pointing me in the right direction is appreciated.

Hi @ThomasLohner.
You'll also need to ensure your profile is updated to allow this traffic.  It's a little trickier with the Docker network plugin example because the security profile is created "under-the-covers".  However it is fairly easy to locate the correct profile to update. The Advanced Policy for Calico as a Docker network plugin guide describes how to do this.
The Advance Network Policy guide contains a worked-example for the actual changes that need to be applied to the security profile.
Let me know how you get on.
Rob

Wohoo, thanks, i've got it almost.
profile is set up
# calicoctl profile show
+------------------------------------------------------------------+
|                               Name                               |
+------------------------------------------------------------------+
| 93235a8c34cda72744320efb491b7b1336d6c5993a1108aae6103e0a8dd94512 |
+------------------------------------------------------------------+

# calicoctl profile 93235a8c34cda72744320efb491b7b1336d6c5993a1108aae6103e0a8dd94512 rule show
Inbound rules:
   1 allow tcp to ports 80
   2 allow icmp type 8
Outbound rules:
   1 allow

nat rule is still in place:
# iptables -t nat -L -n -v
Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
    0     0 felix-PREROUTING  all  --  *      *       <IP_ADDRESS>/0            <IP_ADDRESS>/0
   18   917 DOCKER     all  --  *      *       <IP_ADDRESS>/0            <IP_ADDRESS>/0            ADDRTYPE match dst-type LOCAL
    1    60 DNAT       tcp  --  eth0   *       <IP_ADDRESS>/0            <IP_ADDRESS>/0            tcp dpt:80 to:<IP_ADDRESS>:80

I've double checked that <HOSTIP> is on eth0:
# ip a
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 04:01:98:cd:bd:01 brd ff:ff:ff:ff:ff:ff
    inet <IP_ADDRESS>/16 brd <IP_ADDRESS> scope link eth0
       valid_lft forever preferred_lft forever
    inet <HOSTIP>/20 brd <xxx> scope global eth0
       valid_lft forever preferred_lft forever
    inet <IP_ADDRESS>/16 brd <IP_ADDRESS> scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::601:98ff:fecd:bd01/64 scope link
       valid_lft forever preferred_lft forever

now i can curl <HOSTIP> from outside
node3 # curl -I <HOSTIP-OF-Node-2>
HTTP/1.1 200 OK
Date: Mon, 11 Jan 2016 19:24:33 GMT
Server: Apache/2.4.18 (Ubuntu)
Connection: close
Content-Type: text/html; charset=UTF-8

but i still can't curl the ip from the same host:
node2 # curl <HOSTIP>
curl: (7) Failed to connect to <HOSTIP> port 80: Connection refused

Anyways, this is only cosmetics, i can do what i wanted to achieve. Thanks.

@ThomasLohner The DNAT rule does not take effect on the same host.
When you curl node2's ip on node2, Linux think this is a local connection, and will bypass the iptables rules. If you want to access it, you must use container ip directly.

@ThomasLohner I just found another solution.
You can add a DNAT rule in NAT table OUTPUT chain like this:
iptables -t nat -I OUTPUT -p tcp -o lo --dport 80 -j DNAT --to-destination <IP_ADDRESS>:80

@frostynova Adding both nat rules works like a charm! Thanks for your help.
@robbrockbank Should i add this to the docs or just close this issue?

@ThomasLohner We always welcome updates to our documentation.  If you'd like to beef up the docs and raise a PR that would be awesome!

Hi @ThomasLohner,
Just going through our issues.  Did you happen to make a docs update for this?  If so I can review and merge.
Thanks!


Write buffer image to google cloud storage

I using image buffer to write to google cloud storage but just get unknown file.
const gcs = new Storage({
    keyFilename: `./keyfiles/${projectId}.json`
})
let bucket = gcs.bucket('demo')
const fileName = `static-map/${trainingId}`
const file = bucket.file(fileName)
const stream = file.createWriteStream({ metadata: { contentType: 'image/jpg' } })
stream.on('error', (err) => {
    console.log('error', err)
})
stream.on('finish', async () => {
    await file.makePublic()
    const link = getPublicUrl(fileName)
    console.log('link', link)
})
const buffer = await map.image.buffer('image/jpg', { quality: 75 })
stream.end(buffer)

File uploaded to google cloud storage but can't read.
Could you please give an example?

Oh, downloaded it, works


Unify the CR description in Kyma dashoard

Description
Changes proposed in this pull request:

Unify ApplicationConnector CR's description and the link to the Application Connector documentation to be consistent with all CR descriptions in Kyma dashboard

Related issue(s)
https://github.com/kyma-project/kyma/issues/18321

There is an issue with but it's not in any way related to this change, therefore I'm merging it kyma-integration-k3d-agent-tests.yml


Add settings for all user-configurable settings

Right now, the only user-configurable settings are

joining an org for producer accounts,
Display barcode in search results
Add an edit link in search results

My understanding is that org management has been moved to MongoDB. Therefore, it's not really a task for Keycloak.
The barcode and edit link could be seen as site-specific/client-specific/app-specific setting, which isn't necessarily a task for Keycloak? Right now, we're not getting rid of user.sto entirely (I'd like to not touch HTTP session handling for the first iteration), so we could just keep these settings as-is.

Hi @hangy . I think my main concern here is one of usability - it is not very nice at the moment to have one page for the above settings and then a different page (inside Keycloak) for things like changing your name and password.

It's a bit of a philosophical question, IMHO. The way I see it, lots of big sites (think Microsoft or Google) have central settings and maybe something like RBAC in an "account website" such as Keycloak, but if you want to configure your Gmail/Outlook mail settings, you use a separate UI in that application. Likewise, the "display barcode in search results" settings is specific to ProductOpener and irrelevant to the mobile apps, or Robotoff/Hunger Games, for example.
As the settings that are specific to ProductOpener are very few now, they might just be toggles on the search results page, for example

Hi @hangy . I'm happy if we want to move the barcode and edit link settings to somewhere where it is easier to toggle them, but I think having a specific settings page that just shows these two settings and then another button to get to the Keycloak page is pretty messy.
I guess the other way to do it would be to have two options when you click on your name:

Account Parameters (the current option which would now go straight to Keycloak)
Preferences (new option which would just show product specific settings

What do you think?

Yes, something like that was what I had in mind. Admins might have to deal with the complicated "parameters" page for bot/moderator/… for a while, but we really can't do everything in one go


[webui] Need better error reporting when websockets do not work

Operating system
all
Kluctl Version
2.24.1
Kubernetes Version
all
kubectl Version
all
Bug description
When websockets fail to work (e.g. misconfigured NGINX or LBs), the Webui should properly report an error instead of silently doing nothing.
Steps to reproduce
No response
Relevant log output
No response

When using the kluctl webui behind an nginx ingress, add the following annotation to the ingress entity like so:
metadata:
  annotations:
    nginx.org/websocket-services: "kluctl-webui"


Add Markdown capability globally

Allow to use Markdown notation on LNbits.
Make the description (SITE_DESCRIPTION) capable of being made in Markdown.
Showdown library added to the vendor packages.
Extensions can then use the global LNbits.utils.convertMarkdown() method.

Closes #1912

Why cant we have a normal textfield that no-coders expect.
For a chat in Copilot or NostrMarket we need it anyway, isnt it ?


Move or copy files 0.0.2

New Pull Request Checklist
Please mark the points which you did / accept.

[x] I read the Step Development Guideline
[x] I have a test for my Step, which can be run with bitrise run test (in the step's repository)
[x] I did run bitrise run audit-this-step (in the step's repository - note, if you don't have this workflow in your bitrise.yml, you can copy it from the step template.)
[x] I read and accept the Abandoned Step policy

Hey @Dave005 👋
How did you create this PR? Asking because v0.0.1 of the step shouldn't be included if you just created a new PR.
In any case, can you please remove that? Only 0.0.2 should remain

Actually i was wondering the same, i will look into that :D

Thanks @Dave005 and let us know if we can help with anything! ;)

Or, accept my PRs in your step and create 0.0.3 😂

:D i will do that, but probably in the morning, need to get some rest :D

Yeah, I know that feeling :P

Closing it for now, but feel free to create a new one with the fixes @Dave005 and we'll review it ASAP! :)


Eval with method location

class User < ActiveRecord::Base
    bitmask :roles, as: %i(...)
end

# old
User.new.method(:roles).source_location # => ["(eval)", 2]

# new
User.new.method(:roles).source_location # => ["/your/project/path/.../bitmask_attributes-0fa2343aaca2/lib/bitmask_attributes/definition.rb", 61]

This repository is no longer maintained 😢


Initialise from a vector

Check #173

@ocots, this branch is not up to date with main, should I checkout a new branch from main directly ?

@ocots, this branch is not up to date with main, should I checkout a new branch from main directly ?
@jbcaillau not really a fan of creating branches from issues: there is not necessarily a 1:1 correspondance, and by the time work starts on these branches they are often out of date. I guess it can still be useful if you know you'll start right away and will need only one branch to solve the issue.

@PierreMartinon good practice but not compulsory, of course. not being 1:1 is not an issue (🙂) as several branches can be associated with an issue (if I am not mistaken).

we close this PR since it has been replaced by https://github.com/control-toolbox/CTBase.jl/pull/184


Exit tailing logs for failed builds

Previously builds used to hang for failed builds with any non "buildExecuteFailed" reasons.
For more details: Fixes #579
Proposed Changes

Exit for any build failures

/lgtm


UI should list the available projects

The project name can be reused (must be reused when deploying in a starter cluster due to quota constraints).
It would be nice if the OpenShift Project name input could list the available projects or allow creating a new one.

Mission control already supports deploying in an existing project given its name.
This is necessary for https://github.com/openshiftio/appdev-planning/issues/50

PR send: https://github.com/openshiftio/launchpad-frontend/pull/176


Recommend miniforge instead of anaconda/miniconda

In GitLab by @Huite on Oct 28, 2021, 11:52
Anaconda's EULA have changed, such that "commercial use" of the anaconda repo is no longer free. To quote from https://www.anaconda.com/terms-of-service:

To avoid confusion, “commercial activities” are any use of the Repository which is NOT:
use solely by an individual using for personal, non-business purposes,
use by a student or employee of an educational institution in connection with educational activities,
use by an employee or volunteer on behalf of a non-profit institution in connection with the provision of charitable services,
use by a 501(c)(3) non-profit research institution in connection with non-commercial activities, such as research to address societal needs and global challenges, public policy development, or the advancement of science for the general good, where such work will not result in revenue to the institution, or
use by entities in common control with each other with fewer than 200 employees in aggregate.

This is regarding the use of the repository. We're currently already relying on conda-forge exclusively, so I don't believe this should be an issue. But better to guarantee it by recommending miniforge: https://github.com/conda-forge/miniforge

In GitLab by @Huite on Dec 20, 2021, 15:42
a85dcccac44f9b16ea06bb9aaed01c9b75bdfcd1


Using Kubermatic-lab boilerplate

What this PR does / why we need it:
https://github.com/kubermatic-labs/boilerplate
Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #59
Special notes for your reviewer:
Documentation:

Does this PR introduce a user-facing change?:

NONE

/lgtm


Extract the code that runs ElasticSearch from the test assembly

I am writing a persistence plugin for Akka.net and it would be nice if something existed akin to Mongo2Go for MongoDB, allowing the running and teardown of either a node or a cluster.
I believe all the code is already there in the tests assembly. How feasible would it be to extract the relevant bits to a separate assembly/NuGet package?
I'm happy to do this, if it's something that's warranted.
Kind regards
Sean.

It should be very feasible and we have plans for doing so but none in the immediate future, if you fancy doing a PR for it to get things rolling that'd be awesome!
Feel free to open a work in progress PR as soon as you do! 👍
I think it will need to be two packages Elasticsearch.Managed (all the ElasticsearchNode and Observable stuff) and Elasticsearch.Net.XUnit with all the XUnit bits allowing you to create integration tests against Elasticsearch more easily.
We can focus on Elasticsearch.Managed first though.

Ok, forking as we speak!
Will the tests need to depend on both packages—I’m assuming so!
From: Martijn Laarman<EMAIL_ADDRESS>Sent: Monday, May 22, 2017 09:51
To: elastic/elasticsearch-net<EMAIL_ADDRESS>Cc: Sean Farrow<EMAIL_ADDRESS>Author<EMAIL_ADDRESS>Subject: Re: [elastic/elasticsearch-net] Extract the code that runs ElasticSearch from the test assembly (#2767)
It should be very feasible and we have plans for doing so but none in the immediate future, if you fancy doing a PR for it to get things rolling that'd be awesome!
Feel free to open a work in progress PR as soon as you do! 👍
I think it will need to be two packages Elasticsearch.Managed (all the ElasticsearchNode and Observable stuff) and Elasticsearch.Net.XUnit with all the XUnit bits allowing you to create integration tests against Elasticsearch more easily.
We can focus on Elasticsearch.Managed first though.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://github.com/elastic/elasticsearch-net/issues/2767#issuecomment-303039259, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ABY1fr9TwlwhKxbgXEEdRvx7topwgiP-ks5r8UyLgaJpZM4NfU4r.

Also, is there an issue for this and do I need to sign a CLA?
From: Martijn Laarman<EMAIL_ADDRESS>Sent: Monday, May 22, 2017 09:51
To: elastic/elasticsearch-net<EMAIL_ADDRESS>Cc: Sean Farrow<EMAIL_ADDRESS>Author<EMAIL_ADDRESS>Subject: Re: [elastic/elasticsearch-net] Extract the code that runs ElasticSearch from the test assembly (#2767)
It should be very feasible and we have plans for doing so but none in the immediate future, if you fancy doing a PR for it to get things rolling that'd be awesome!
Feel free to open a work in progress PR as soon as you do! 👍
I think it will need to be two packages Elasticsearch.Managed (all the ElasticsearchNode and Observable stuff) and Elasticsearch.Net.XUnit with all the XUnit bits allowing you to create integration tests against Elasticsearch more easily.
We can focus on Elasticsearch.Managed first though.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://github.com/elastic/elasticsearch-net/issues/2767#issuecomment-303039259, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ABY1fr9TwlwhKxbgXEEdRvx7topwgiP-ks5r8UyLgaJpZM4NfU4r.

Once you've started open a PR (even if its nowhere near done) I'll mark it WIP and you can ping us whenever you need some input or we'll provide unsolicited feedback on commits while scanning through :)
That can act as 'the' ticket.
Our CLA is up over at: https://www.elastic.co/contributor-agreement

What branch should we be basing this off 5.x or master/develop?
Also, where is the best place to start looking at the usage of the code that manages ES?
From: Martijn Laarman<EMAIL_ADDRESS>Sent: Monday, May 22, 2017 10:17
To: elastic/elasticsearch-net<EMAIL_ADDRESS>Cc: Sean Farrow<EMAIL_ADDRESS>Author<EMAIL_ADDRESS>Subject: Re: [elastic/elasticsearch-net] Extract the code that runs ElasticSearch from the test assembly (#2767)
Once you've started open a PR (even if its nowhere near done) I'll mark it WIP and you can ping us whenever you need some input or we'll provide unsolicited feedback on commits while scanning through :)
That can act as 'the' ticket.
Our CLA is up over at: https://www.elastic.co/contributor-agreement
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://github.com/elastic/elasticsearch-net/issues/2767#issuecomment-303045385, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ABY1ftgNaLE8y9M-PfBRuoTcgyfD54JEks5r8VJ6gaJpZM4NfU4r.

I would go for 5.x since it also still has 2.x specific edge case handling.

Ok, what’s the best place to start looking at the usage?
From: Martijn Laarman<EMAIL_ADDRESS>Sent: Monday, May 22, 2017 15:36
To: elastic/elasticsearch-net<EMAIL_ADDRESS>Cc: Sean Farrow<EMAIL_ADDRESS>Author<EMAIL_ADDRESS>Subject: Re: [elastic/elasticsearch-net] Extract the code that runs ElasticSearch from the test assembly (#2767)
I would go for 5.x since it also still has 2.x specific edge case handling.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://github.com/elastic/elasticsearch-net/issues/2767#issuecomment-303118788, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ABY1foHdmKLIcZXTeXpM73H3BKYbYrCYks5r8Z1rgaJpZM4NfU4r.

It all starts from ClusterBase which holds an ElasticsearchNode which in turn holds an ObservableProcess.

Ok, thanks, where do the integration tests live?
I’ll extract everything off cluster base in to a package then.
From: Martijn Laarman<EMAIL_ADDRESS>Sent: Monday, May 22, 2017 15:59
To: elastic/elasticsearch-net<EMAIL_ADDRESS>Cc: Sean Farrow<EMAIL_ADDRESS>Author<EMAIL_ADDRESS>Subject: Re: [elastic/elasticsearch-net] Extract the code that runs ElasticSearch from the test assembly (#2767)
It all starts from ClusterBase which holds an ElasticsearchNode which in turn holds an ObservableProcess.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://github.com/elastic/elasticsearch-net/issues/2767#issuecomment-303125764, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ABY1fkANKBOWbw_GzWkpc885lZq8TP8yks5r8aLDgaJpZM4NfU4r.

Integration tests live in the Tests project, with most of the components to extract in the Framework, specifically Framework/ManagedElasticsearch directories.
As @Mpdreamz said, there may be some unpicking to do, to decouple it from NEST's test infrastructure.

Ok, should be able to start this tomorrow, what version of vs are we using/wht project type?
Also, how do I add NuGet packages using packet to a project? I was looking at the paket.dependencies and XUnit doesn’t seem to be referenced there.
Finally should we upgrade our NuGet packages? I’ll raise another issue for this!
Kind regards
Sean.
From: Russ Cam<EMAIL_ADDRESS>Sent: Monday, May 22, 2017 22:31
To: elastic/elasticsearch-net<EMAIL_ADDRESS>Cc: Sean Farrow<EMAIL_ADDRESS>Author<EMAIL_ADDRESS>Subject: Re: [elastic/elasticsearch-net] Extract the code that runs ElasticSearch from the test assembly (#2767)
Integration tests live in the Tests projecthttps://github.com/elastic/elasticsearch-net/tree/5.x/src/Tests, with most of the components to extract in the Frameworkhttps://github.com/elastic/elasticsearch-net/tree/5.x/src/Tests/Framework, specifically Framework/ManagedElasticsearchhttps://github.com/elastic/elasticsearch-net/tree/5.x/src/Tests/Framework/ManagedElasticsearch directories.
As @Mpdreamzhttps://github.com/mpdreamz said, there may be some unpicking to do, to decouple it from NEST's test infrastructure.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://github.com/elastic/elasticsearch-net/issues/2767#issuecomment-303225581, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ABY1fgTUnOU-eeoSX0EjkavD7fHLPmuOks5r8f6SgaJpZM4NfU4r.

Ok, I'm able to look at this over the weekend, what sort of project should I create in vs2017?

Whilst looking at this, it seems that ElasticSearchVersion is tightly coupled to TestClient, see the RootUrl field as an example. How do we get round this?

Ok, I'm able to look at this over the weekend, what sort of project should I create in vs2017?

I'd recommend using the new csproj format with a C# library, that seems most fitting here.

Whilst looking at this, it seems that ElasticSearchVersion is tightly coupled to TestClient, see the RootUrl field as an example. How do we get round this?

I think the configuration of TestClient would need to be pulled out of it; ITestConfiguration is used to determine the version of Elasticsearch that should be started and some other test specific behaviours e.g. test against already running cluster, whether to reseed with test data. I think this should be instantiated separately and passed to TestClient

Sorry to hijack this issue, but I have a quick question about the same subject matter.
Currently everything test related starts with the Tests. namespace. I guess everything related to ManagedElasticsearch would need to be refactored from Tests.Framework.ManagedElasticsearch to Elasticsearch.Managed, right?

@nphmuller @SeanFarrow
I have started work on this myself last week to abstract several new nuget packages out of this. I hope to have this done by the end of the week.

@nphmullerhttps://github.com/nphmuller
Yes, that was the plan.
What new NuGet packages are you looking to create for my reference?
From: Martijn Laarman<EMAIL_ADDRESS>Sent: Tuesday, August 08, 2017 10:49
To: elastic/elasticsearch-net<EMAIL_ADDRESS>Cc: Sean Farrow<EMAIL_ADDRESS>Mention<EMAIL_ADDRESS>Subject: Re: [elastic/elasticsearch-net] Extract the code that runs ElasticSearch from the test assembly (#2767)
@nphmullerhttps://github.com/nphmuller @SeanFarrowhttps://github.com/seanfarrow
I have started work on this myself last week to abstract several new nuget packages out of this. I hope to have this done by the end of the week.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/elastic/elasticsearch-net/issues/2767#issuecomment-320907896, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ABY1fm9nNGYVqrYuM3ijIGK4B3CwOUazks5sWC8VgaJpZM4NfU4r.

@SeanFarrow
@Mpdreamz is already creating a couple of official NuGet packages. So currently I don't plan to create anything. :)

@mpdreamz What NuGet packages are you creating?
From: Nick<EMAIL_ADDRESS>Sent: Tuesday, August 08, 2017 15:31
To: elastic/elasticsearch-net<EMAIL_ADDRESS>Cc: Sean Farrow<EMAIL_ADDRESS>Mention<EMAIL_ADDRESS>Subject: Re: [elastic/elasticsearch-net] Extract the code that runs ElasticSearch from the test assembly (#2767)
@SeanFarrowhttps://github.com/seanfarrow
@Mpdreamzhttps://github.com/mpdreamz is already creating a couple of official NuGet packages. So currently I don't plan to create anything. :)
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/elastic/elasticsearch-net/issues/2767#issuecomment-320973866, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ABY1fqiZqOm2uQ8cyvawulXG1leeh0G4ks5sWHEugaJpZM4NfU4r.

@Mpdreamz Any update on the new NuGet packages?

Any updates on this?
I'm looking for a NuGet package that only contains the the binaries.

Here's what I got so far, pretty sure this is not the approach that is described above but it is what I basically need and I think it can help others.
The following is the nuspec file I created:
<?xml version="1.0"?>
<package xmlns="http://schemas.microsoft.com/packaging/2013/05/nuspec.xsd">
    <metadata>
        <id>Elasticsearch</id>
        <version>5.6.3</version>
        <authors>elasticsearch</authors>
        <owners>elasticsearch</owners>
        <licenseUrl>https://github.com/elastic/elasticsearch/blob/master/LICENSE.txt</licenseUrl>
        <projectUrl>https://github.com/elastic/elasticsearch/</projectUrl>
        <iconUrl>https://raw.githubusercontent.com/elastic/elasticsearch-net/master/build/nuget-icon.png</iconUrl>
        <requireLicenseAcceptance>true</requireLicenseAcceptance>
        <releaseNotes></releaseNotes>
        <description>Elasticsearch is a distributed, RESTful search and analytics engine capable of solving a growing number of use cases. As the heart of the Elastic Stack, it centrally stores your data so you can discover the expected and uncover the unexpected. </description>
        <copyright>© 2017. All Rights Reserved - Elasticsearch</copyright>
        <tags>elasticsearch,elastic,search,lucene</tags>
        <dependencies>
        </dependencies>
    </metadata>
    <files>
        <file src="Elasticsearch\Elasticsearch.targets" target="build\" />
        <file src="Elasticsearch\lib\*.*" target="content\Elasticsearch\lib\" />
        <file src="Elasticsearch\config\*.*" target="content\Elasticsearch\config\" />
        <file src="Elasticsearch\bin\*.*" target="content\Elasticsearch\bin\" />
        <file src="Elasticsearch\modules\*.*" target="content\Elasticsearch\modules\" />
        <file src="Elasticsearch\plugins\*.*" target="content\Elasticsearch\plugins\" />
    </files>
</package>

Then in order to link the relevant files to be copied to the output folder I added this Elasticsearch.targets file (It must have the same name as the package):
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
  <ItemGroup>
    <NativeLibs Include="$(MSBuildThisFileDirectory)..\content\**\*.*" />
    <None Include="@(NativeLibs)">
      <Link>%(RecursiveDir)%(FileName)%(Extension)</Link>
      <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
    </None>
  </ItemGroup>
</Project>

And that's it.
I have locally tested it to work.
Thoughts are welcome.
I can publish it under a different name with my user if you think this is not the right way to go.

NuGet is not for binaries IMO. You can get Elasticsearch from chocolatey for instance. Our managed elasticsearch is for running Elasticsearch from code. It will bootstrap aquiring es in ÷APPDATA÷

NuGet is not for binaries IMO

This is a strange opinion - most of what's in NuGet is binary files. Most of them are .Net managed DLLs and I agree that it will be strange to place java binaries there, but I can compare these Java binaries to C++ DLLs that are also stored in NuGet for libraries that need those native DLLs. My server and my development environment needs Elasticsearch and so to me it feels like the right place to put it.
My goal is to simplify the setup of development environment so adding a step to install elasticsearch is going to the opposite direction, this is for my use case anyway...

I agree with @mpdreamz, I don't think a Nuget package is the best way to distribute Elasticsearch, in much the same way you wouldn't use it to distribute SQL Server. Chocolatey is a better fit for this

@SeanFarrow I'm going to close this issue now as @Mpdreamz has decoupled the components that we use to spin up Elasticsearch within the NEST integrations tests into their separate projects:
https://github.com/elastic/elasticsearch-net-abstractions
These are available as packages on a MyGet feed currently:
https://www.myget.org/F/elasticsearch-net-abstractions-temp/api/v3/index.json
With plans to make them more official once they've had time to bake. You can see how they are wired up within the Tests projects in 6.x and master branches.
Feedback on them would be very welcome!