createReadStream() does not work with readline

Testing a module that uses readline with a unionfs createReadStream does not work.
Error encountered:
(node:17269) UnhandledPromiseRejectionWarning: TypeError: input.on is not a function
    at new Interface (readline.js:184:11)
    at Object.createInterface (readline.js:63:10)

Test file:
const fs = require('fs');
const readline = require('readline');
const mfs = require('memfs').fs;
const ufs = require('unionfs').ufs;
const util = require('util');

const contents = `this
is a
test
file
`;

mfs.mkdirSync('/tmp');

ufs.use(fs).use(mfs);

const onLine = (type) => (line) => console.log(type + ': ' + line);
const onEnd = (type) => () => console.log(type + ':end');

const systems = { fs, mfs, ufs };

Object.keys(systems).forEach(async (key) => {
  const tfs = systems[key];
  tfs.writeFileSync('/tmp/testfile', contents);
  const writeFile = util.promisify(tfs.writeFile);
  await writeFile('/tmp/testfile1', contents);

  const rl = readline.createInterface({
    input: tfs.createReadStream('/tmp/testfile')
  });

  rl.on('line', onLine(key));
  rl.on('end', onEnd(key));
});

Actual file under test:
https://gitlab.com/bytesnz/keeps-on-ticking/blob/6-add-function-testing/lib/file.js (https://gitlab.com/bytesnz/keeps-on-ticking/blob/6-add-function-testing/src/lib/file.ts)
Using files:
https://gitlab.com/bytesnz/keeps-on-ticking/blob/6-add-function-testing/lib/file.test.js (https://gitlab.com/bytesnz/keeps-on-ticking/blob/6-add-function-testing/src/lib/file.test.ts)
https://gitlab.com/bytesnz/keeps-on-ticking/blob/6-add-function-testing/test/lib/unionfs.js (https://gitlab.com/bytesnz/keeps-on-ticking/blob/6-add-function-testing/src/test/lib/unionfs.ts)

Would it be possible to just pass through the functions from the union-ed filesystems, eg go through the filesystems until one doesn't throw an error?


Don't vendor dependencies for plugins ?

We have different dependency declaration for different plugins which is a good thing to keep them self-contained and independent (allowing also for an independent release cycle).
However, verndoring the dependencies is really huge, especially for source where each source support for sources from eventing-contrib would have to vendor eventing-contrib individually although it only needs the client bindings for a single source out of it. (I wonder whether we could craft go mod that way to only pick that client dependency ?)
One suggestion for this particular dependency would be to get the client API with different means (e.g. git clone directly, by cherry-picking the client interface)
However, I would discuss in general whether for an aggregated repository vendoring makes sense at all or whether we should use go mod without a vendor/ dir and using the local go mod cache for downloading dependencies ?

@maximilien @daisy-ycguo @zhangtbj @zhanggbj What are your opinion in this ? I think this will help considerably in avoiding duplicated dependencies and download times as the shared cache below $GOROOT would be used.

Yes, I think if we keep the vendor in each plugin and if we have more and more plugins, the size of the repo will very large and even hard to pull the repo.
I also agree to remove the vendor in each plugin.

@rhuss
We do have some difficulties to pull vendor packages considering our network situation... but I support to use go mod without /vendor which it is going really huge both for the repo and each plugin, so 👍 for my side.


Add response attribute extraction

Supports extracting span attributes depending on the sttp response, for example to have HTTP status codes in traces.

@ybasket Thanks for the PR!
I reviewed it and realized we had inconsistency between the sttp and tapir modules wrt adding req/resp attributes to the traces, just because it was done by different people at different times.
My proposal is to change the code a bit:
val ctxHeaders = headersGetter.get(childCtx)
val req = request.headers(SttpHeaders.converter.to(ctxHeaders).headers: _*)

val isSampled = childSpan.context.traceFlags.sampled == SampleDecision.Include

// only extract uri attributes if the span is sampled as the host parsing is quite expensive
val reqUriAttributes = if (isSampled) SttpRequest.toAttributes(request) else Map.empty

for {
  _ <- childSpan.putAll(
    SttpHeaders.requestFields(Headers(req.headers), dropHeadersWhen) ++ reqUriAttributes: _*
  )
  resp <- lower(ctxBackend.send(req))
  _ <- childSpan.setStatus(SttpStatusMapping.statusToSpanStatus(resp.statusText, resp.code))
  _ <- childSpan.putAll(SttpHeaders.responseFields(Headers(resp.headers), dropHeadersWhen): _*)
} yield resp

What we'll have in this case:

ctx attributes go to both headers and span attributes
all req and resp headers go to span attributes, with filtering if needed
we use the same methods as in the tapir module

cc @janstenpickle for discussion

Oh, you needed some extra info apart from headers. Maybe we can combine both enhancements.

Oh, you needed some extra info apart from headers. Maybe we can combine both enhancements.

Just pushed a commit that combines both along your snippet. I only wonder whether the putAll(SttpHeaders.responseFields...) should also be wrapped in a whenA to have full lazy attribute handling. WDYT?

should also be wrapped in a whenA to have full lazy attribute handling

IMO, there's one case that is worth optimizing -- it's parsing the uri host.

but maybe still worth adding as an attribute by default

I would prefer default attributes. If so, do we still need an extra getter?

Fine! I just renamed a few vals to make them look consistent.


Feat rebase mecanism

closes #38

Users are now minting and holding shares of the total supply
In project.cairo, balance still returns the carbon_credits amount and not the shares thanks to the share_to_cc function.
erc1155 abi isn't embed anymore => reimplementation of all the functions in the project.cairo to keep compatibility.

/!\ Because internal balance is now using shares of supply, but the balance used by users are cc tokens, we need to use converters like cc_to_share and share_to_cc. There are rounding errors that needs to be solved, will be done in an other PR.

New rebase design has to be made to keep track of failed CC


Wave 5

Wave 5 documentation.

Hi Felipe!
Again, good job, I think we're almost there :tada:! I noticed some small things that still need fixing, it would be great if you could address these:

Somehow the dutch documentation files contain two duplicates of the text e.g. see here. I would suggest maybe just deleting the files and running the notebook again to make sure there is only one instance of each question.
The toc in wave5_questions_english_ordered.rst contains references to the dutch folder instead of the English folder.
In wave5_questions_english_topics.rst there seem to be some files missing? I am not sure why, usually these should be added automatically when you run create-variable-overview.ipynb in a configuration for the English files.
You can delete variable_table-waves-1-2-3-4-e.csv and add variable_table-waves-1-2-3-4-5-e.csv to the index file instead so wave 5 is included in the general overview table as well (you will need to adjust the columns specification when importing the .csv file in variables.rst. You will also need to adjust the text in that file to link to the single wave-5 variable overview table).
A summary of wave 5 should be added to README.md and the text in docs/source/wave-5/index.rst should be adjusted. Currently it's the same as for wave 4.
When building the documentation locally there are quite many errors still. Maybe you can have a look, I didn't assess them in detail but you might need to adjust some things in the codebook. Let me know if you have any questions about this!
(You can build the documentation locally by navigating to the docs folder and then typing make html in the command line. The documentation will then build and be saved to the folder docs/build/html.)
You might want to click through the documentation preview a bit to spot remaining mistakes in the files themselves. Many of the tables or bullet lists aren't displayed correctly. You will need to adjust this in the codebook table. Some of them probably just need spaces.

I hope these comments are helpful. Let me know if you need help with anything, we can also have a chat if you think it would be helpful to talk about the remaining things. :)


libdile_vt: add support for CreateEx and YUV422sp format

Ready for merging.

It looks like this at my WebOS5 TV.

It seems to be fixable in the future.

Hmmmm... Is this a regression, or was your TV using yuv422sp? Looks like broken stride.


Update year in License

Only made 1 change:

Update LICENSE to include latest year in the form of a year-range.

No other changes were made.

Thanks @fay-jai, I forgot about that!


Add support for 256 templates

There is currently no built-in* way to build themes based on so-called "256-bit templates".
*In the shot term you could always provide a path to the template like base16-builder -s gooey -t db/templates/mutt/dark-256.ejs.

@aloisdg, can you please explain to me the difference between normal and 256 templates?
What do you think about removing 256 templates from the db?
In the case of templates like mutt where there is only a 256 template, we can turn it into a  normal template so mutt/dark-256.ejs and mutt/light-256.ejs become  mutt/dark.ejs  and mutt/light.ejs respectively.
Update: I think 16-bit templates will look messed up in terminal windows with xterm-256color. If that is true, we need both.

There are 6 256-bit templates:

xresources
connectbot
console2
iterm2
mintty
mutt

All of these templates have 16-bit templates except for mutt.

What do you think we should do, @aloisdg?

@alexbooker A quick (and dirty solution) would be to move them as first template citizen. We could create a folder "mintty16" and one "mintty256". We can add a delimiter if we want too.


Example of WMTS ?

Sorry if this question makes no sense. I was wondering if there is an example of this package rendering WMTS data? I tried searching around but couldn't find any concrete examples.

As far as I can see, there is nothing within esri-leaflet specific to WMS or WMTS layers.  Since this library is just a react wrapper for esri-leaflet, there's nothing here for that either.
react-leaflet already supports WMS Tile Layers, as does native leaflet.  I recommend you try using the WMSTileLayer from react-leaflet, with your esri WMTS endpoint - should work just fine.
Closing as this is not an issue with this library.


Find Content doesn't search sQL

Find Content searches XSLT but not sQL queries

Seems to work in 7.4


Get last successful artifact instead of last (maybe unstable) build

refs. #10

@astj @kga @hagihala Please review this.

:eye:

@astj
Fixed https://github.com/aereal/capistrano-strategy-jenkins_artifact/pull/12#discussion_r93865455 !

@astj
https://github.com/aereal/capistrano-strategy-jenkins_artifact/pull/12#discussion_r93867962
It is renamed and fixed, so we'll merge this if CI passed.


Add file-code-2 icon

Just an idea: maybe if we add this, let's rename the current file-code to file-code-2 and add this one as file-code, so that the layouts are consistent (no suffix => contents inside, -2 suffix => contents to the side).

Just an idea: maybe if we add this, let's rename the current file-code to file-code-2 and add this one as file-code, so that the layouts are consistent (no suffix => contents inside, -2 suffix => contents to the side).

Yeah I noticed this inconsistency also… Would this require adding an alias in this particular case?

No, we can't add an alias for a name that will already taken.


Flow 0.54.0 fails on travis with EACCESS error

It seems that since you've released flow 0.54.0 travis fails on my build with an EACCESS error. Any idea why this is happening?
It works on my local machine, but on travis I get the following error: https://travis-ci.org/sulu/sulu/jobs/271668230#L522

:sadtrombone: – Ditto here.

Glad that it seems that it is not my error 🙈

Can people having this problem please report npm/yarn versions, os and version, etc.?
Install worked for me so there must be some pattern to who it doesn’t work for.

/cc @calebmer, there are several issues referencing flow 0.54 binaries not being executable. Any chance an explanation comes to mind?

@asolove It is working on my locale machine using Mac OSX 10.11.6 and NPM 5.3.0. However, it doesn't work on travis (Ubuntu 14.04.5 LTS, Node 8.4.0 and NPM 5.3 installed using NVM).

Seeing the same in docker (on travis). Base image debian:stretch-slim, node v8.1.4, npm v5.4.0, and flow v0.52.0.

Tried to downgrade to flow 0.53.1 and it didn't seem to help... Might this be a travis issue? Would be interesting to know if there is somebody having the same issue on a non-travis environment...

This is a problem with npm, not with Flow. See https://github.com/facebook/flow/issues/4733

Having looked into this, I agree with @vkurchatkin
If I download and look at the<EMAIL_ADDRESS>tarball (npm pack flow-bin@0.54.0), all the binaries are correctly marked as executable. The explanation on https://github.com/npm/npm/issues/18324#issuecomment-327081533 explains that they re-wrote how npm extracts packages and that it was accidentally stripping executable flags.


GetMetrics() for a particular app/version/signature

Current GetMetrics() (in the context of storage client) returns all metrics in the storage client. This may result in a lot of data and it can be simplified.
Change GetMetrics() so it uses the app name, version number, and signature to return a subset of the data.

Related to #1501


docs: revamp README

Preview link

Link to llamaindex docs
Make it shorter
Add status badges

Do we still want some kind of quick-start in the readme? (Feels like something that most readme's have)

Yea I like that idea -- just want to get users most quickly started from the readme 👍🏻


Added renLUNA, renFIL and renDGB info.json files

Added missing info.json files for three renAssets

Added


failed when uploading on production : recv() failed (104: Connection reset by peer) while reading upstream

hi,
I'm having problem, i can't upload file in production using nginx and passenger hosted on AWS. i keep getting error in browser console like so:
POST http://my-url/ net::ERR_INCOMPLETE_CHUNKED_ENCODING 200 (OK)
in my server log i got this
2018/09/27 16:30:04 [error] 2517#2517: *51 recv() failed (104: Connection reset by peer) while reading upstream, client: xxx.xx.xxx.xx, server: xx.xxx.xxx.xxx, request: "POST /my-url HTTP/1.1", upstream: "passenger:unix:/tmp/passenger.toEIX2t/agents.s/core:", host: "xx.xxx.xxx.xxx", referrer: "https://xx.xxx.xxx.xxx/my-url"

here is my nginx config
user www-data;
worker_processes  1;
events {
    worker_connections  1024;
}
http {
    include /etc/nginx/mime.types;
    default_type  application/octet-stream;

    sendfile        on;
    # tcp_nopush     on;
    # tcp_nodelay on;
    server_tokens off;

    keepalive_timeout 65;
    types_hash_max_size 2048;

    server_names_hash_bucket_size 64;

    access_log /var/log/nginx/access.log;
    error_log /var/log/nginx/error.log;

    gzip  on;
    gzip_disable "MSIE [1-6]\.";

    gzip_proxied any;
    gzip_http_version 1.0;
    gzip_min_length 500;
    gzip_types    text/plain text/xml text/css
                  text/comma-separated-values
                  text/javascript
                  application/x-javascript
                  application/atom+xml;

    include /etc/nginx/passenger.conf;
    include /etc/nginx/conf.d/*.conf;
    include /etc/nginx/sites-enabled/*;
}

server {
        listen 80 default_server;
        listen [::]:80 default_server;
        server_name _;
        return 301 https://$host$request_uri;
}
server{
        listen 443 default_server ssl;
        listen [::]:443 default_server ssl;
        ssl on;
        ssl_certificate    /etc/ssl/ca.pem;
        ssl_certificate_key    /etc/ssl/ca.key;
        ssl_dhparam /etc/ssl/dhparam.pem;
        server_name xxx.xxx.xxx.xxx;
        passenger_enabled on;
        passenger_sticky_sessions on;
        root /var/www/my_app/bundle/public;
        passenger_app_type node;
        passenger_startup_file main.js;
        passenger_env_var MONGO_URL mongodb://some_shard_urls
        passenger_env_var ROOT_URL https://xxx.xxx.xxx.xxx;
        passenger_env_var MONGO_OPLOG_URL: mongodb://some_shard_urls
        keepalive_timeout  1000;
        ssl_session_timeout 1d;
        ssl_session_cache shared:SSL:50m;
        ssl_session_tickets off;
        ssl_prefer_server_ciphers on;
        ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
        ssl_ciphers 'ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GC$
        ssl_stapling on;
        ssl_stapling_verify on;

        add_header Strict-Transport-Security "max-age=31536000;";
        access_log  /var/log/my_app/access.log;
        error_log   /var/log/my_app/error.log;

        proxy_send_timeout 60s;

        location / {
                proxy_set_header Connection "";
                proxy_http_version 1.1;
                proxy_redirect     off;
                client_max_body_size 100M;
                if ($uri != '/') {
                    expires 30d;
                }
                break;
        }
}

somehow, i found error in my passenger log file saying Not keep-aliving application session connection because application did not allow it here is the log:
[ D3 2018-09-27 16:53:44.2194 2500/Ta Ser/HttpChunkedBodyParser.h:183 ]: [Client 63] ChunkedBodyParser: parsing new chunk
[ D3 2018-09-27 16:53:44.2194 2500/Ta Ser/HttpChunkedBodyParser.h:123 ]: [Client 63] ChunkedBodyParser: chunk size determined: 982 bytes
[ D3 2018-09-27 16:53:44.2194 2500/Ta Ser/HttpChunkedBodyParser.h:162 ]: [Client 63] ChunkedBodyParser: parsing 982 of 982 bytesof remaining chunk data; 0 now remaining
[ D3 2018-09-27 16:53:44.2194 2500/Ta Ser/FileBufferedChannel.h:1416 ]: [FBC 0x7f71e801b670] Feeding 982 bytes
[ D3 2018-09-27 16:53:44.2194 2500/Ta Ser/FileBufferedChannel.h:486 ]: [FBC 0x7f71e801b670] pushBuffer() completed: nbuffers = 1, bytesBuffered = 982
[ D3 2018-09-27 16:53:44.2194 2500/Ta Ser/FileBufferedChannel.h:554 ]: [FBC 0x7f71e801b670] Reader: reading next
[ D3 2018-09-27 16:53:44.2194 2500/Ta Ser/FileBufferedChannel.h:586 ]: [FBC 0x7f71e801b670] Reader: found buffer, 982 bytes
[ D3 2018-09-27 16:53:44.2194 2500/Ta Ser/FileBufferedChannel.h:493 ]: [FBC 0x7f71e801b670] popBuffer() completed: nbuffers = 0,bytesBuffered = 0
[ D3 2018-09-27 16:53:44.2194 2500/Ta Ser/FileBufferedChannel.h:594 ]: [FBC 0x7f71e801b670] Reader: feeding buffer, 982 bytes
[ D3 2018-09-27 16:53:44.2194 2500/Ta Ser/FileBufferedChannel.h:554 ]: [FBC 0x7f71e801b670] Reader: reading next
[ D3 2018-09-27 16:53:44.2194 2500/Ta Ser/FileBufferedChannel.h:561 ]: [FBC 0x7f71e801b670] Reader: no more buffers. Transitioning to RS_INACTIVE
[ D3 2018-09-27 16:53:44.2194 2500/Ta Ser/FileBufferedChannel.h:539 ]: [FBC 0x7f71e801b670] Calling dataFlushedCallback
[ D3 2018-09-27 16:53:44.2194 2500/Ta age/Cor/Con/ForwardResponse.cpp:64 ]: [Client 2-63] Event: onAppSourceData
[ D3 2018-09-27 16:53:44.2194 2500/Ta age/Cor/Con/ForwardResponse.cpp:206 ]: [Client 2-63] Processing 7 bytes of application data: "\r\n0\r\n\r\n"
[ D3 2018-09-27 16:53:44.2195 2500/Ta Ser/HttpChunkedBodyParser.h:248 ]: [Client 63] ChunkedBodyParser: done parsing a chunk
[ D3 2018-09-27 16:53:44.2195 2500/Ta Ser/HttpChunkedBodyParser.h:183 ]: [Client 63] ChunkedBodyParser: parsing new chunk
[ D3 2018-09-27 16:53:44.2195 2500/Ta Ser/HttpChunkedBodyParser.h:123 ]: [Client 63] ChunkedBodyParser: chunk size determined: 0bytes
[ D3 2018-09-27 16:53:44.2195 2500/Ta Ser/HttpChunkedBodyParser.h:162 ]: [Client 63] ChunkedBodyParser: parsing 0 of 0 bytes of remaining chunk data; 0 now remaining
[ D3 2018-09-27 16:53:44.2195 2500/Ta Ser/HttpChunkedBodyParser.h:164 ]: [Client 63] ChunkedBodyParser: end chunk detected
[ D3 2018-09-27 16:53:44.2195 2500/Ta Ser/HttpChunkedBodyParser.h:267 ]: [Client 63] ChunkedBodyParser: end chunk reached
[ D2 2018-09-27 16:53:44.2195 2500/Ta age/Cor/Con/ForwardResponse.cpp:224 ]: [Client 2-63] End of application response body reached
[ D2 2018-09-27 16:53:44.2195 2500/Ta age/Cor/Con/ForwardResponse.cpp:1077 ]: [Client 2-63] Not keep-aliving application session connection because application did not allow it
[ D3 2018-09-27 16:53:44.2195 2500/Ta age/Cor/App/Socket.h:201 ]: Socket unix:/tmp/passenger.toEIX2t/apps.s/node.1si9u5: connection not checked back into connection pool. There are now 2 connections in total
[ D2 2018-09-27 16:53:44.2195 2500/Ta age/Cor/App/Gro/SessionManagement.cpp:150 ]: Session closed for process (pid=2519, group=/var/www/hmn/bundle (production))
[ D2 2018-09-27 16:53:44.2195 2500/Ta Ser/HttpServer.h:1211 ]: [Client 2-63] Ending request
[ D3 2018-09-27 16:53:44.2195 2500/Ta Ser/FileBufferedChannel.h:1460 ]: [FBC 0x7f71e8009b98] Deinitialize
[ D3 2018-09-27 16:53:44.2195 2500/Ta Ser/FileBufferedChannel.h:1416 ]: [FBC 0x7f71e801b670] Feeding 0 bytes
[ D3 2018-09-27 16:53:44.2195 2500/Ta Ser/FileBufferedChannel.h:486 ]: [FBC 0x7f71e801b670] pushBuffer() completed: nbuffers = 1, bytesBuffered = 0
[ D3 2018-09-27 16:53:44.2195 2500/Ta Ser/FileBufferedChannel.h:554 ]: [FBC 0x7f71e801b670] Reader: reading next
[ D3 2018-09-27 16:53:44.2195 2500/Ta Ser/FileBufferedChannel.h:566 ]: [FBC 0x7f71e801b670] Reader: EOF encountered. Feeding EOF
[ D3 2018-09-27 16:53:44.2195 2500/Ta Ser/FileBufferedChannel.h:582 ]: [FBC 0x7f71e801b670] Reader: EOF fed. Transitioning to RS_TERMINATED

Meteor 1.6.1
Meteor-Files 1.9.11
Ubuntu 16.04
nginx 1.14.0
passenger 5.3.5
AWS

we need some advice regarding this issue. Thanks in advance

Hello @ishadif ,
I'm very sorry for the late reply, let me know if you're still experiencing this issue, as we are using nginx at every server we're running and never faced issue like this.
If not, and you've solved issue on your own, please — share your solution with community

Closed due to silence at issue owner end.
Feel free to reopen it in case if the issue still persists on your end.


addressed concerns discussed during brownbag session

What this PR does / why we need it:

Add fields for enabling metrics and logs from application
made logging configurable
Which issue this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged): fixes #

Special notes for your reviewer:
Release note:

/azp

/azp run


Please provide values for helm installations of addons.

(This is used to request new product features, please visit https://discuss.istio.io for questions on using Istio)
I'd love it if the add ons here included the values.yaml for each add-on so that I could use helm to deploy these instead of configs that were dumped out of helm and applied directly. Clearly, these add-ons were produced by helm, so what values were passed TO helm to generate these in the first place? How were these samples created?
Describe alternatives you've considered
I don't see an alternative to having the 'real' original values.yaml for grafana, kiali, prometheus, etc. The documentation does not fully cover the installation of the add-on's using helm... only the sample yamls that helm created and stored in the istio project.
Affected product area (please put an X in all that apply)
[ ] Ambient
[X] Docs
[X] Installation
[ ] Networking
[ ] Performance and Scalability
[X] Extensions and Telemetry
[ ] Security
[ ] Test and Release
[ ] User Experience
[ ] Developer Infrastructure
Affected features (please put an X in all that apply)
[ ] Multi Cluster
[ ] Virtual Machine
[ ] Multi Control Plane
Additional context

You can see in manifests/addons.
Note these are samples. If you are looking for a production ready install, these aren't it - see the projects install docs which all have prod ready documentation.
For example, none have persistent storage.

thanks @howardjohn that's exactly what I was looking for.


[ISV-3366] Ignore deleted package lock objects

When searching for existing package lock ignore object with deleted flag set to true. The deleted flag was recently added to be able reuse package multiple times.

This should be merged after ISV-3365 is implemented.


excludes property should be renamed to exclude and support wildcards.

Would bring this API to be symmetrical to the other CLI exclude options for minification and compile suboptions.  Plus, wildcards make it a lot easier to specify things more concisely.

Probably want to add a deprecation warning for excludes but support it in the next version.


Add py.typed so that mypy knows that this package can be typed checked.

This commit contians the py.typed file needed, to satisfy the PEP-561
requirement, so that type checkers know that Yamale can be type checked.

This is a pull request for issue #114

Thanks for your contribution. However, I think there's a little confusion about what py.typed means. The existence of this file indicates that the proper annotations or stubs are available for the package. Yamale has neither, so just adding this file doesn't help with type checking.
To fully fix this, type annotations would need to be added to at least the public APIs....but ideally the whole codebase.

Closing due to lack of activity.

Closing due to lack of activity.


readthedocs badge shows failing

The status indicator shows failing build of readthedocs, but it seems passing when looking on readthedocs.io directly.

I noticed this behaviour before and I think it is related to your browser cache. Use private mode or delete the cache and it should be solved.

Thanks, I should have checked that myself first. my apologies.


docs(all): fix wrong github link

Issue
#27

Coverage remained the same at 16.397% when pulling 39b16660c430922f4f9054b681a0fbc95ce26189 on jongmoon:spinviewer-ghpage#21 into d8f3ef541546be20d451058ea6403e64f6b1f489 on naver:master.


Account storage is not updated in playground

Describe the bug
In Flow playground, changes to storage are not reflected in the accounts.
To Reproduce

Use tutorial 2
Deploy contract 1. It succeeds (Log: Deployed Contract To: 0x01)
. Account 0x01 shows:

eployed Contracts:
[
  {
    "contract": "HelloWorld",
    "height": 9
  }
]
Account Storage:
(Empty)

Now run the "Create Hello" transaction. It also succeeds ({"level":"debug","message":"Cadence log: \"Saved Hello Resource to account.\""}).
"Load hello" works too, so the asset was created properly.

However, the 0x01 account still displays as:
Deployed Contracts:
[
  {
    "contract": "HelloWorld",
    "height": 9
  }
]
Account Storage:
(Empty)

Expected behavior
The account storage should not be empty and reflect state as described in the tutorial text.
Desktop (please complete the following information):

OS: Ubuntu 22.04
Browser: Firefox 102.14.0esr
Playground info

Playground Version: 2.1.0
Cadence Version: 0.38.1
Emulator Version: 0.49.0
Language Server: 0.32.0

playground is under going changes to use flowkit in the backend. Account storage has not bee implemented at this time. It's on the todo list


[SOLVED] webcamd not starting

trident rc1:  webcamd not starting.
solution appears to be:
add  "directory /usr/local/etc/devd-openrc";
to the options section of /etc/devd.conf.
there is a webcamd.conf file all ready to go in that directory.  letting devd pick it up does the trick.
Note:  and the user still needs to add webcamd_enable="YES" to /etc/rc.conf

Automated for RC2 as part of the sys-init.sh setup script.
https://github.com/project-trident/trident-core/commit/c7b8919fa33e69bf126afc5f3c9f4e4a0ac865bf


Enhancement request: Docs: include returned fields

Hi,
It would be nice if the documentation would list which fields are returned.
Example: Memberships: http://ciscosparkapi.readthedocs.io/en/latest/user/api.html#memberships

Returns:
When iterated, the GeneratorContainer, yields
the memberships returned from the Cisco Spark query.

Nice would be the format that's used for the Parameters section , looking a bit like this

Returns:
When iterated, the GeneratorContainer, yields
the memberships returned from the Cisco Spark query.

id
roomId
personId
personEmail
isModerator
isMonitor
created

Or like this:

id, roomId, personId, personEmail, isModerator, isMonitor, created

Thank you DJF3, I actually have this item (updating the docstrings, which is where this information comes from) already logged as an issue #16 , and it is making its way up the backlog as I have been getting some items knocked out.  🙂
The returned SparkDataObjects (and their contents) are documented in the API docs.  I will work to clean these up and link the specifically returned data objects to the docs of the methods that return them.
I'm going to flag this as a duplicate, as this work will get done as part of issue #16 .


Added retry logic to KinesisPublisher.

Minor variable name cleanup elsewhere.

@gheine ^^

LGTM 👍
@mikegirkin mind reviewing this too?

Thanks, @gheine !
FWIW - I've updated our mobile project with this (locally built) version and it seems fine, I'm ready to merge/deploy.
Let me know when.

Go for it! Not sure if @mikegirkin has time to look right now...

Thanks, I've asked him, he's going to take a look tomorrow!

LGTM too!

Thanks, guys! Merging.

Is there a unit test ?

nope, sorry.

Then I have a question;
When I try it as below it does not send anything to kinesis, I am sure I am doing something wrong ...
    val aws_access_key_id = "ASIAJKABHFPJUBNFRGTA"
    val aws_secret_access_key = "RkYd0Lt0l5S2vpG33bzeA4D8lIuzW+ClFMyl9Fnb"
    val aws_session_token = "FQoDYXdzEJT//////////..."

    val records: List[String] = "apples" :: ("oranges" :: ("pears" :: Nil))

    val creds = new AWSCredentialsProvider {
        override def refresh(): Unit = ???

        override def getCredentials: AWSCredentials = new BasicSessionCredentials(aws_access_key_id, aws_secret_access_key, aws_session_token)
    }

    KinesisPublisher(10, 8, creds)
        .publishBatch("monitoring_test", records)(new KinesisRecordWriter[String]() {
            override def toKinesisRecord(r: String) = {
                println("new KinesisRecord(\"key\", r.getBytes)")
                new KinesisRecord("PartitionKey", r.getBytes)
            }
        })

The issue is not the credentials btw, I changed them when I pasted ...

Hmm, interesting, mind turning on debug logging (root logger, to get everything, including AWS calls) and waiting for the future it returns and printing the result?
I haven't tried custom credentials with this version (taking defaults from IAM instance profile) but just checked and we seem to pass it along, should be some error logged somewhere.
Or does that future never return?
 Might make a difference for a short-lived test process where you do a bit of work from main() and exit right away. The kinesis ops are async, so unless you de-reference those Futures it returns or shutdown() explicitly and wait for enqueued ops to drain - it may just exit before anything happens.

I'll try and let you know. Thanks

Cool, if that helps, here's how I deal with it in my project:
  private[this]
  val kinesisPublisher = KinesisPublisher()

  Runtime.getRuntime.addShutdownHook(new Thread() {
    override def run(): Unit = { kinesisPublisher.shutdown() } // attempt orderly shutdown
  })

/// other stuff ....

    Batcher[GenericRecord](kinesisStreamName, 100, 1 second) {
      kinesisPublisher.publishBatch(kinesisStreamName, _) onComplete {

        case Failure(e) =>
          error(s"Failed to publish kinesis batch: ${e.getMessage}", e) // just in case to see it in the logs, should never happen with current publisher implementation

        case Success(r) =>
          metrics.countKinesisBatchResult(r)
          if (r.failureRecordCount > 0) {
            error(s"Failed to publish batch to ${kinesisStreamName}: ${r}")
          } else {
            info(s"Published batch to ${kinesisStreamName} with the result: ${r}")
          }
      }
    }

Adding Shutdown helped. It is working, thank you. You have "Batcher[GenericRecord]" in your code, whats the purpose of it ? Thanks  again.

Ah, no prob. That Batcher is not part of the lib, it's used to batch up events as they come in, up to 1 sec to minimize number of kinesis API calls (to keep the costs down). That part is specific to my project, getting data one event at a time.

Great, thanks again. It's working like a charm ...


[Feature Request] Backward function for LayerNorm

Is your feature request related to a problem? Please describe.
We currently use moreh_layer_norm_backward, but it shows pretty bad performance.
Describe the solution you'd like
Fused operation to calculate input, gamma and beta.
Describe alternatives you've considered

moreh_layer_norm_backward - performance critical
composite op - performance critical

@davorchap  both forward and backward layernorm are pretty slow (jsut after matmuls and elemetnwise ops).
Need to find who can work on it.

@davorchap  both forward and backward layernorm are pretty slow (jsut after matmuls and elemetnwise ops).
Need to find who can work on it.

@bbradelTT and his team are taking on the fused OPs.

HI @bbradelTT  you can take a look at the moreh implementation as the baseline. But backward is pretty complex and very slow.
Id like to get an one op for forward version which can do both layernorm, rmsnorm or just norm and returning intermediate tensors required for backward if requested.

@dmakoviichuk-tt we'll have to hash out what your request would look like once we start looking at this issue, and maybe spawn off separate issues.

@bbradelTT  please let me know when you are going to star, I can provide more details.


The metadata output has gotten big for multilevel images in v5.6.0

Contributing guidelines

[X] I've read the contributing guidelines and wholeheartedly agree

I've found a bug, and:

[X] The documentation does not mention anything about my problem
[X] There are no open or closed issues that are related to my problem

Description
Hi :wave:
The recently released version 5.6.0 broke our CI because the format of the metadata GHA output changed. I understand that this is probably caused by dependencies of this action, but I was not sure where else to open the issue --- happy to be redirected elsewhere.
Concretely, one of our steps started to fail since we've been parsing the metadata output and we were passing it via an environment variable like this:
  - name: Set output variables
    run: |
       .github/workflows/extract-image-names.sh | tee -a "${GITHUB_OUTPUT}" | awk -F'=' '{print $2}' | jq
    env:
        BAKE_METADATA: ${{ steps.build-upload.outputs.metadata }}

Because of the size, we started to hit this error:
Error: An error occurred trying to start process '/usr/bin/bash' with working directory '/home/runner/work/aiidalab-docker-stack/aiidalab-docker-stack'. Argument list too long

We've been able to workaround this by saving the metadata output into a file first like this
  - name: Set output image names
    run: |
        cat << EOF > bake_metadata.json
        ${{ steps.build-upload.outputs.metadata }}
        EOF
        images=$(.github/workflows/extract-image-names.sh bake_metadata.json)
        echo "images=${images}" >> "${GITHUB_OUTPUT}"

Nevertheless, even with this workaround, it might be nice to try to prune the metadata output a bit to make it a bit more readable in the GHA logs. Specifically, there is now a section {"buildx.build.provenance": {"invocation": {...}} that is huge for multilevel builds.
Here's the output from the base image, nothing out of the ordinary
    "base": {
      "buildx.build.provenance": {
        "buildType": "https://mobyproject.org/buildkit@v1",
        "materials": [
          {
            "uri": "pkg:docker/jupyter/minimal-notebook@python-3.9.13?platform=linux%2Famd64",
            "digest": {
              "sha256": "f7022a83184c73b04fdb16d3f2f2b6b014c819ac086b87496f1c0641a0614f3a"
            }
          }
        ],
        "invocation": {
          "configSource": {
            "entryPoint": "Dockerfile"
          },
          "parameters": {
            "frontend": "dockerfile.v0",
            "args": {
              "build-arg:AIIDA_VERSION": "2.5.1",
              "build-arg:BASE": "jupyter/minimal-notebook:python-3.9.13"
            },
            "locals": [
              {
                "name": "context"
              },
              {
                "name": "dockerfile"
              }
            ]
          },
          "environment": {
            "platform": "linux/amd64"
          }
        }
      },
      "buildx.build.ref": "builder-df0f1d9d-0da6-4602-89bf-f0dd8e3e962d/builder-df0f1d9d-0da6-4602-89bf-f0dd8e3e962d0/2yfr5apv8yx146zzg287f66an",
      "containerimage.config.digest": "sha256:dc3491813cf2fa44b8ed0dc6a914b221a7d1668544193496a516de248224a29e",
      "containerimage.descriptor": {
        "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
        "digest": "sha256:a4ecb05d8859237d79ff229f84d1835f9d639aba923c3e8292318c42763b7267",
        "size": 6356,
        "platform": {
          "architecture": "amd64",
          "os": "linux"
        }
      },
      "containerimage.digest": "sha256:a4ecb05d8859237d79ff229f84d1835f9d639aba923c3e8292318c42763b7267",
      "image.name": "ghcr.io/aiidalab/base"
    },

Here's the second part from the image built on top of base image
    "base-with-services": {
      "buildx.build.provenance": {
        "buildType": "https://mobyproject.org/buildkit@v1",
        "materials": [
          {
            "uri": "pkg:docker/docker/dockerfile@1",
            "digest": {
              "sha256": "fe40cf4e92cd0c467be2cfc30657a680ae2398318afd50b0c80585784c604f28"
            }
          },
          {
            "uri": "pkg:docker/jupyter/minimal-notebook@python-3.9.13?digest=sha256:f7022a83184c73b04fdb16d3f2f2b6b014c819ac086b87496f1c0641a0614f3a&platform=linux%2Famd64",
            "digest": {
              "sha256": "f7022a83184c73b04fdb16d3f2f2b6b014c819ac086b87496f1c0641a0614f3a"
            }
          }
        ],
        "invocation": {
          "configSource": {
            "entryPoint": "Dockerfile"
          },
          "parameters": {
            "frontend": "gateway.v0",
            "args": {
              "build-arg:AIIDA_VERSION": "2.5.1",
              "build-arg:PGSQL_VERSION": "15",
              "build-arg:RMQ_VERSION": "3.9.13",
              "cmdline": "docker/dockerfile:1",
              "context:base": "input:0-base",
              "frontend.caps": "moby.buildkit.frontend.contexts+forward",
              "input-metadata:0-base": "{\"containerimage.config\":\"eyJhcmNoaXRlY3R1cmUiOiJhbWQ2NCIsIm9zIjoibGludXgiLCJyb290ZnMiOnsidHlwZSI6ImxheWVycyIsImRpZmZfaWRzIjpbInNoYTI1NjoxN2Y2MjNhZjAxZTI3N2M1ZmZlNjc3OWFmODE2NDkwN2RlMDJkOWFmN2EwZTE2MTY2MmZjNzM1ZGQ2NGYxMTdiIiwic2hhMjU2OjIxMjU2Y2ZiZjVjZDc4MjNjNjk3OTNiZmZlMmEyMmZjNWFiN2U2Mzg0MzZiNzY3MTFjMmIwZTNkYjgwNjExMTUiLCJzaGEyNTY6NWVhNjgxOTRjM2JmMmRjM2U0YzIxMDFiMDdlMWEwMGMyYmI4MmEyOGUyZDk0ZTYwNzNiNTFhOTNhZGE0NzE3YiIsInNoYTI1Njo1ZjcwYmYxOGEwODYwMDcwMTZlOTQ4YjA0YWVkM2I4MjEwM2EzNmJlYTQxNzU1YjZjZGRmYWYxMGFjZTNjNmVmIiwic2hhMjU2OjRhZmE4YzMzY2QyYzZkYTJhNmEyMzllMTcxZGJjMTgyNWVmMzIyYWM3ZmRjMWUwZWFmZGZiZjYyYzkwMTEzNDIiLCJzaGEyNTY6MGNhMzhlZjY2ZGZkZDI2ODRjMjhiMGE4MTQ2YTc2MGQ3YTdhYmJhNmQzYWZlZGRmMjk4NGM4MWFiZWNjZGJiNCIsInNoYTI1NjpjNzZlZTJkMDU0YjBkZWVlZDgzOTc0ZGE5M2Q0ZDBjMzg2NDJmY2JmMGNiYjQ2NTRhZjAzNjUzNzE0MWNmOWFhIiwic2hhMjU2OjRiOGE0YzlkNjM1OTNmMDNjMWI4ZjU1ZGQ4MDE5OGM2YTc3NWQ4ZTFkNjg0MGUzNzRmZTNlMTZlZjg0Y2IxM2IiLCJzaGEyNTY6NWY3MGJmMThhMDg2MDA3MDE2ZTk0OGIwNGFlZDNiODIxMDNhMzZiZWE0MTc1NWI2Y2RkZmFmMTBhY2UzYzZlZiIsInNoYTI1Njo3NWM2OGY2YjA4YjJhNGU2NjMxYTA1MDMwZDA0NTliMjA5ZjZmODdiYTU4ZmI1NWExNzliM2MxYzI1ZWFjYTQ2Iiwic2

(I elided the full output)
It would be great if the "input-metadata:0-base": "{\"containerimage.config\": section was not there.
Repository URL
https://github.com/aiidalab/aiidalab-docker-stack
Workflow run URL
https://github.com/aiidalab/aiidalab-docker-stack/actions/runs/10283635135/job/28465139810
YAML workflow
- name: Build and upload to ghcr.io
                id: build-upload
                uses<EMAIL_ADDRESS>                with:
                    push: true
                    # Using provenance to disable default attestation so it will build only desired images:
                    # https://github.com/orgs/community/discussions/45969
                    provenance: false
                    set: |
                        *.platform=${{ inputs.platforms }}
                        *.output=type=registry,push-by-digest=true,name-canonical=true
                        *.cache-to=type=gha,scope=${{ github.workflow }},mode=max
                        *.cache-from=type=gha,scope=${{ github.workflow }}
                    files: |
                        docker-bake.hcl
                        build.json
                        .github/workflows/env.hcl

              - name: Set output image names
                id: bake_metadata
                # bake-action metadata output has gotten too big, so we first write it
                # to a file. See https://github.com/aiidalab/aiidalab-docker-stack/issues/491

              - name: Set output variables
                run: |
                    .github/workflows/extract-image-names.sh | tee -a "${GITHUB_OUTPUT}" | awk -F'=' '{print $2}' | jq
                env:
                    BAKE_METADATA: ${{ steps.build-upload.outputs.metadata }}

Workflow logs
No response
BuildKit logs
No response
Additional info
No response

Thanks for reporting, what outputs are being created in Set output variables step?
Can you show what returns .github/workflows/extract-image-names.sh?
I think it would be better to rely on https://github.com/actions/github-script instead of shell for this use case.

I think something like this could to the trick (not tested):
  - name: Set output variables
    uses: actions/github-script@v7
    with:
      script: |
        for (const key in JSON.parse(process.env.BAKE_METADATA || '{}')) {
          if (data.hasOwnProperty(key)) {
            const entry = data[key];
            core.setOutput(`${key.toUpperCase().replace(/-/g, "_")}_IMAGE`, `${entry["image.name"]}@${entry["containerimage.digest"]}`);
          }
        }
    env:
        BAKE_METADATA: ${{ steps.build-upload.outputs.metadata }}

Closing this one for now but feel free to leave a comment


Code coverage for Dask tests is underestimated

What would you like to report?
The Dask tests are not fully reporting the coverage. For instance, check out the coverage lines for the @subflow decorator, which is reported as having no coverage. That's only because that block is launched via the cluster, which codecov doesn't see. Not sure how to more accurately mark that.

Closed in #1602.


generate indexes from multiple models

hi , i was wondering to know if there is any way not to use your migrations package and like before just create indices from models .
my concern is that i have like 20 models with 60 columns in each for example . writing migration for that would be so time consuming .
and my second question is that can we populate the indices manually ? because i want to store 3 models in 1 index .
thanks for your great work btw

Hi,
Writing migrations is the recommended way to create indices, but you can also create them manually, or use Elastic\Adapter\Indices\IndexManager::create()/Elastic\Adapter\Indices\IndexManager::createRaw() to somehow create indices from models on the fly.
You can override searchableAs model's method to define the target index. If a few models have the same searchableAs value then the data should be stored in the same index.


Comment

Link to JIRA ticket
Add a link to the JIRA ticket that the changes in this PR describe.
Describe this PR
What is the problem we're trying to solve
Add a summary of the problem and any additional context that helps explain what the issue we're trying to solve is.
What changes have we introduced
Add descriptions of what changes were made to address the problem and provide a solution.
Include any links to commits that introduce significant additions of code if they help explain the process of coming to the solution. e.g Addition of test database setup, addition of first test and production class, removal of buggy code, etc.
Checklist

[x] Added end-to-end (i.e. integration) tests where necessary e.g to test complete functionality of a newly added feature
[x] Added tests to cover all new production code
[x] Added comments to the README or updated relevant documentation (add link to documentation), where necessary.
[x] Code pipeline builds correctly
[x] Checked all code for possible refactoring

Follow up actions after merging PR
Are there any next steps that need to addressed after merging this PR? Add them here.

What was this PR about?  Was it merged without peer review?


docs: :memo: add docs

Purpose of PR

PR Checklist

[ ] I have read the relevant readme.md file(s)
[ ] Tests are added/updated/not required
[ ] Tests are passing
[ ] Storybook stories are added/updated/not required
[ ] Usage notes are added/updated/not required
[ ] Doesn't contain any sensitive information

Thx @YeSuX 👻


When used outside the main web project, the uSync and client files aren't wanted

If you host all your code in one web project, the current setup is fine.
If you have a larger solution with code separated into multiple Visual Studio projects (typically Razor Class libraries) that are brought together into one web project at build or deploy time, then those separated projects always have GOV.UK uSync files, SASS files and wwwroot files that they don't want.
This might be resolved by:

analysing the code that's referenced in this scenario, and finding a way to move it to the DLL-only ThePensionsRegulator.Umbraco project; or
adding an MSBuild switch that can prevent the unwanted files being copied to the project at build time.

We could create another package that just contains the core assemblies and no other output. We could then make a dependency from this package to the 'Core' package, just to get the assemblies. That would also have the desired effect.
I ran into a build issue because of the client files being copied multiple times. This has been documented here: https://stackoverflow.com/questions/69919664/publish-error-found-multiple-publish-output-files-with-the-same-relative-path

We already have a core library which is the ThePensionsRegulator.Umbraco package. It only has types in it that are purely about Umbraco though, nothing to do with GOV.UK or TPR. Nonetheless it does cover one of your scenarios, which I'll flag on the relevant PR.
For Razor class libraries, rather than create yet more packages we could look first at extending this approach which already tackled a similar issue of unwanted files in test projects.
You might want the SASS files, because they're variables and mixins intended to be used in the SASS code for your Razor class libraries, but it could be that you only get them if you have a 'Styles' folder.
I agree uSync files wouldn't be needed, but that could be excluded using the approach above.
wwwroot might be trickier, because there could be a wwwroot folder without you wanting the files from the package. We could potentially look for the <AddRazorSupportForMvc>true</AddRazorSupportForMvc> property of razor class libraries to determine whether to copy.


Display Guardian's health on the dashboard

One thing we've used while debugging our own federation is the Guardians tab on the dashboard. IMO it is a little confusing though that it only displays your peers health and not your own. Would it be possible to add your own status to this tab?

Also, it is a bit ambiguous what "Health" means in this context. I know its not possible to display every type of error that might go wrong with Fedimint, but would it be possible to add a bit of context as to what "Health" means? (i.e is it contributing to consensus?)
cc @tvolk131

Health here means if they've been contributing to recent sessions, think it's something like if contribution was in the last 10 sessions. Can add a tooltip or something explaining it, and then could add a "Healthy" green thing to the guardian's card at the top of the page so they feel better.

I was wondering about this, commenting for future reference, but maybe in the future, it could have more info in such a "guardian's status page" format, e.g https://www.atlassian.com/software/statuspage.
I think the same could be achieved with open-source alternatives, such as https://cachethq.io/


[FLINK-11732] [docs] Add a language switch to the sidebar for Flink documents

What is the purpose of the change
Add a language switch to the sidebar for Flink documents.
Brief change log

Add a language switch to the sidebar for Flink documents. The language switch links to the same page in the opposite language.

Add a "Want to contribute translation?" link at the bottom of every Chinese pages.

Verifying this change
N/A
Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): (no)
The public API, i.e., is any changed class annotated with @Public(Evolving): (no)
The serializers: (no)
The runtime per-record code paths (performance sensitive): (no)
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (no)
The S3 file system connector: (no)

Documentation

Does this pull request introduce a new feature? (no)
If yes, how is the feature documented? (not applicable)

Hi @xccui @fhueske , do you have time to have a look at this PR ?

@flinkbot approve-until architecture

Hi @sunjincheng121  I have added a same "contribute translation" link for English pages.

Thanks for add the link to the English page. @wuchong
+1 to merged.
Best,
Jincheng


Modify filter for spi_flash sample

Before that change the sample was filtered out by the Twister

SHAs in commit messages are wrong. It looks like you have cherry-picked commits from your PR, not from Zephyr main. Use the following ones: 2396423, 8fdb800, and 7b78393, respectively.

@anangl done

CI is green https://github.com/nrfconnect/sdk-nrf/pull/13392


run registry-v2 container failed

hi everyone,

generated tls certificate by myself

openssl req \
     -newkey rsa:2048 -nodes -keyout certs/domain.key \
     -x509 -days 365 -out certs/domain.crt

2.download source code then modify cmd/registry/config.yml, add code in "http" section as blow:
    tls:
        certificate: certs/domain.crt
        key: certs/domain.key

docker build -t secure_registry .
4.docker run -p 5000:5000 registry_local:latest
then error occur:

time="2016-03-14T04:58:28.822971793Z" level=fatal msg="open /search/registry/distribution-2.0.0/certs/domain.crt: no such file or directory" environment=development instance.id=ffa0e1a5-8b38-4cc5-a3bf-5247d9f8cd8e service=registry version="-extld=gcc"

i'm sure  "/search/registry/distribution-2.0.0/certs/domain.crt" really exist!
how can i fix it ?
Thanks!

Why don't you just try to follow the documentation? You don't need to build your own image.
https://github.com/docker/distribution/blob/master/docs/deploying.md
Nevertheless, from what you did here, it's quite likely your certificate is absent from the container you built.
Running this command:
docker run registry_local:latest ls /search/registry/distribution-2.0.0/certs/domain.crt will tell.

Just in case someone has the same problem and lands on this issue. In my case, this was because I had my volume and registry config set to:
/etc/letsencrypt/live/conradk.com:/certs
...
REGISTRY_HTTP_TLS_CERTIFICATE: /certs/fullchain.pem
REGISTRY_HTTP_TLS_KEY: /certs/privkey.pem

But the /etc/letsencrypt/live/xxx folders contain symlinks. Symlinks to files that may not be present in the container. This will work:
/etc/letsencrypt:/certs
...
REGISTRY_HTTP_TLS_CERTIFICATE: /certs/live/conradk.com/fullchain.pem
REGISTRY_HTTP_TLS_KEY: /certs/live/conradk.com/privkey.pem


Refactor dataplane

Consists of 2 commits:

Drop service reconciliation from nodeset controller
Services should be pre-req for the deployment and not
nodeset. Also, they're currently overwritten after they
are patched using oc apply -f which is not desirable.

Refactor nodeset/deployment CRDs

Moves ansibleee related fields to deployment
Moves services to deployment

Depends-On: https://github.com/openstack-k8s-operators/install_yamls/pull/579
Depends-On: https://github.com/openstack-k8s-operators/install_yamls/pull/580

recheck

I don't think refactoring is necessarily the way to go. Personally, I think we need to focus on stabilizing what we enabled in Q3. We have not had time or feedback to really see if the changes we made in Q3 were effective.

I don't agree as this is not a big change that would make things unstable. PR is already in good shape and once https://github.com/openstack-k8s-operators/install_yamls/pull/579 merges the kuttl job would be green.

I'd prefer to keep the NodeSet the declarative representation of the NodeSet, and the Deployment models the execution.

So you want to store things like what networks would be attached to aee pod and what verbose level ansible would be run in the nodeset properties?

It makes sense to me to keep that information stored on the NodeSet services. That way if an OpenStackDataPlaneDeployment is triggered for that NodeSet, the right set of services is run for each unique NodeSet, where the services may be different.

What set of services would be run should be deployment specific IMO. There is already the default set of services for a deployment as before.

With this proposed change, the set of services, mounts, networkAttachments are the same across all NodeSets for a Deployment. That really limits the flexibility of the Deployment. All nodes need to be the same, other than ansible vars.

We would not use anything other than ctlplane as the network attached to aee pod. aee pod can't talk to openstack services using other networks with the current design using mcvlan. It has to use the openshift internal network. Also, I mentioned earlier services to be run are deployment specific and can change from time to time. We can still have the option to provide services per nodeset in deployment spec.

As for the service reconciliation (EnsureServices), it seems you are expecting the user to do that going forward? That seems to be the case given the install_yamls PR (openstack-k8s-operators/install_yamls#579). I'm also not in agreement with that approach. I don't see why dataplane-operator can't create the default services automatically. If we want to enable editing the default services after they're created, we could still allow that by dropping the part that patches the full service Spec if the service is already created. We could also have the NodeSet "own" the services (setControllerReference), so that each service could be customized per NodeSet if desired.

It can't create those as it does not own them and still reconciles them from the image which is an anti-pattern and patching them externally has no impact. If nodeset owns the services with the full spec and then reconciles it, that would probably would ok. But I still say that what services has to be run should be deployment specific and users can run a subset of them when required.

Dropping the concept of a servicesOverride means it's on the user to remember the list of services for each NodeSet

I don't understand the issue here. The default services in a nodeset and deployment (proposed) are the same, if they use a custom set services in the last run, then deployment CR would have that info in both cases.

Moving these fields here to Deployment means they can't be unique per NodeSet.

These are mounts on the ansible pod right and what ansible expects? Why we want to tie them with nodeset? If we want to have different ones per nodeset we can still do that in deployment spec.

What "it" do you mean by "It can't"? The NodeSet controller is reconciling them today.

NodeSet controller does not own them neither watches them. If you patch the services it would reflect the modified CR in etcd. But when any nodeset CR is reconciled it would flip it back to the old definition in the dataplane-operator image. This is terrible design IMO.

There are dataplane-operator defaults (The hardcoded list of defaults).

NetworkAttachments are attached to aee and who owns the aee? I think it's the deployment controller[1] which also owns deployment.

And the change makes the Deployment more similar to the old DataPlane that we just refactored away.

No I don't agree. Deployment does not have nodeset definition. It just have everything that a deployment run would need to deploy services.

If you changed it so that all these fields were individual per NodeSet, then it is very similar to what we already have with NodeSet, except it's a list of NodeSets

Does not matter if a deployment has nodeset references, creating the deployment CR does not reconcile nodesets. So it's not the same .

What "it" do you mean by "It can't"? The NodeSet controller is reconciling them today.

NodeSet controller does not own them neither watches them. If you patch the services it would reflect the modified CR in etcd. But when any nodeset CR is reconciled it would flip it back to the old definition in the dataplane-operator image. This is terrible design IMO.

I'm not convinced that a use case of editing our default OpenStackDataPlaneService definitions is needed outside of a dev/test context, for which there are various other easier methods. We did not support or encourage editing our rpm packaged tripleo-heat-templates services either. You had to copy and maintain that copy, and refer to it as a different service name or mapping. I don't recall that causing all that much consternation.
The current design works the same way. If an edit is needed, instead of overwriting what we ship as a default resource, you can customize it using the supported composable services interfaces, and use a different name.

I'm not convinced that a use case of editing our default OpenStackDataPlaneService definitions is needed outside of a dev/test context

How do we differentiate between standard services vs custom services in the nodeset spec? Customers will definitely write and use custom services and for me everything is a service (read playbook) with single interface. We can publish definition of some standard ones, which customers can use and change as per their requirement.  I also saw the note on the install_yamls PR that nova and other services won't be using defaults we have for them.
The current way of using a different image for a so called default service is to patch the AEE default image (which will impact all default services) or build dataplane-operator with changed services https://github.com/openstack-k8s-operators/dataplane-operator/blob/main/Dockerfile#L64 and then patch dataplane-operator CSV.
The fact that some services would be reconciled by nodeset controller and some not and then custom ones can be created/patched outside is a confusing/bad user experience and can lead to number of customer case. I've seen several questions on that behavior on slack till now. Anyway, I don't really understand the concerns  here exactly, but I can't argue more, if we want to block this. I rest my case.

I'm not convinced that a use case of editing our default OpenStackDataPlaneService definitions is needed outside of a dev/test context

How do we differentiate between standard services vs custom services in the nodeset spec?

Other than by name, there is no differentiation in the NodeSet spec. The difference between a standard vs custom is:

RH ships the standard services in the dataplane-operator image (I'm assuming this is what you mean by standard)
The standard services are reconciled by the NodeSet controller
Custom services are reconciled by the user (oc apply)

Customers will definitely write and use custom services and for me everything is a service (read playbook) with single interface. We can publish definition of some standard ones, which customers can use and change as per their requirement.

I don't think the original intent is that customers would change the standard services directly on the same resource. They can however copy the standard service YAML, change the name, change the service spec in any way (including setting a different ansibleEE image), and then create it as a new service with the different name.

I also saw the note on the install_yamls PR that nova and other services won't be using defaults we have for them.

Yes, some will be, some won't be depending on the customizations.

The current way of using a different image for a so called default service is to patch the AEE default runner image (which will impact all default services) or build dataplane-operator with changed services https://github.com/openstack-k8s-operators/dataplane-operator/blob/main/Dockerfile#L64 and then patch openstack-operator CSV.

I think this is exclusive to dev/test context. There are also other ways, such as running dataplane-operator from git and editing under config/services. For prod, then the user creates a new service with a different name.

The fact that some services would be reconciled by nodeset controller and some not and then custom ones can be created/patched outside is a confusing/bad user experience and can lead to number of customer case. I've seen several questions on that behavior on slack till now. Anyway, I don't really understand the concerns here exactly, but I can't argue more, if we want to block this. I rest my case.

My main objection is to the CRD changes. I don't have strong objections to changes to how the services are reconciled. But, there are some base lines I feel are needed, such as the set of standard services that can be created automatically (as we know all the values).
Service reconciliation and the user experience workflow around it could be separate from refactoring the CRDs. We can leave CRDs as-is, and either drop the service reconciliation entirely so it's all manual, or fine tune it so that it does not undo user edits, or something else. I can't say there will be broad agreement on that (I have not heard from others), but they could be separate proposals if you feel strongly about it.

I can't say there will be broad agreement on that (I have not heard from others), but they could be separate proposals if you feel strongly about it.

I'm not sure there was a broad agreement on reconciling services in nodeset/role controller? I did point out my concerns in https://github.com/openstack-k8s-operators/dataplane-operator/pull/236 and the reasoning I  got was we've to fix the kuttl tests and similar stuff has been done elsewhere. The fact that the so called 'standard` services are not specific to nodeset (not owned by nodeset/nodeset controller) and live ever after till they're manually deleted, makes it a bad design.

My main objection is to the CRD changes.

For me, NodeSet is the definition of set of provisioned nodes that are ready for deployment or fully deployed. Deployment is the definition of what needs to be deployed and how. So all AEE related details should be in Deployment.If I understand correctly, you're saying  Deployment is just a trigger and nothing more than that and NodeSet contains all properties including what should be deployed in it and how, then the ServicesOverride in deployment spec does not really make sense. Any change in services to be deployed in a nodeset should be changed in the nodeset spec.

My main objection is to the CRD changes.

For me, NodeSet is the definition of set of provisioned nodes that are ready for deployment or fully deployed. Deployment is the definition of what needs to be deployed and how. So all AEE related details should be in Deployment.If I understand correctly, you're saying Deployment is just a trigger and nothing more than that and NodeSet contains all properties including what should be deployed in it and how, then the ServicesOverride in deployment spec does not really make sense. Any change in services to be deployed in a nodeset should be changed in the nodeset spec.

Deployment is the trigger with optional overrides which are offered for convenience and specific use cases (upgrades, etc).

Deployment is the trigger with optional overrides which are offered for convenience and specific use cases (upgrades, etc).

That's where I think the design is broken. We're saying it's just a trigger, but then the deployment resource owns the aee resources and also can override what would be deployed (services). Looking at the nodeset spec no one can tell what has been deployed on it at any point as the services could be overridden with more (including new ones) could have been deployed with service override.

I'm going to close this one, in the interest of closing stale PRs.


PasswordInput and changeset

PasswordInput should get it's value from from field/changeset, not local state.

I fixed this, fix incoming. Waiting for "v2", so this task can be closed.


WaterMarks still shows.

The meeting starts successfully but the options has no effects.
         SHOW_JITSI_WATERMARK: false,
          SHOW_WATERMARK_FOR_GUESTS: false,
          SHOW_CHROME_EXTENSION_BANNER: false

If I understand correctly you can only hide the logo if you self-host, if you use meet.jit.si then you are not allowed (and are literally not able) to hide the logo


Amazon Transcribe (Streaming Speech To Text)

Is your feature request related to a problem? Please describe.
I am frustrated with the amplify documentation located here
https://docs.amplify.aws/lib/predictions/transcribe/q/platform/ios/
It does not explain if streaming is available in swift 2.0
Describe the solution you'd like
clear documentation on how to use streaming with amplify in swif2.0  if the feature is available, if not what are the options
Describe alternatives you've considered
have not considered any alternative
Is the feature request related to any of the existing Amplify categories?
Predictions
Additional context
No response

Thanks for opening this @msidd
The documentation you linked is for Amplify Swift v2. The Predictions category was made available in 2.11.6, so you can use any version after that. We recommend using the most recent version (currently 2.15.4).
Does that answer your question?
Can you elaborate on what you'd like to see in the documentation?

as i understand, the documentation has an example for batch transcribing (url of the file) and does not have an example on to use https://docs.aws.amazon.com/transcribe/latest/APIReference/API_streaming_StartMedicalStreamTranscription.html API

*Note: Amplify doesn't support StartMedicalStreamTranscription as linked in your previous comment. The following response assumes this is StartStreamTranscription.

Yes, that's correct.
speechToText(url: URL) uses the StartStreamTranscription WebSocket based API under the hood. This provides the transcription back from Amazon Transcribe in chunks as they're processed, however Amplify doesn't currently support streaming the input from the client side. To do so would require adding a new API to Amplify, something like speechToText(stream: AsyncStream<Data>).
Can you provide details about your use case? That will help us with this feature request. Thanks!

The use case is that we need to do inline dictation but we cannot use Siri on the device due to HIPAA regulations. Therefore, latency is critical and we found that server side ASR is too slow. Instead direct ASR connection from device to ASR is best.
Further, when doing client-side transcription, the network may be flaky as the user moves around, so we need the SDK to give us feedback on what portion of the audio was transcribed, so we can resend / retry the parts that were not.


[DRAFT] C++ Export

Description
This is a draft, I suggest we keep the conversation in the associated issue:
https://github.com/DLR-RM/stable-baselines3/issues/836
Motivation and Context

[X] I have raised an issue to propose this change (required for new features and bug fixes)

https://github.com/DLR-RM/stable-baselines3/issues/836
Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[X] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] Documentation (update in the documentation)

Checklist:

[ ] I've read the CONTRIBUTION guide (required)
[ ] I have updated the changelog accordingly (required).
[ ] My change requires a change to the documentation.
[ ] I have updated the tests accordingly (required for a bug fix or a new feature).
[ ] I have updated the documentation accordingly.
[ ] I have reformatted the code using make format (required)
[ ] I have checked the codestyle using make check-codestyle and make lint (required)
[ ] I have ensured make pytest and make type both pass. (required)

For some application, I would also be interrested in having the values predictions available, so I am adding them as well, some questions:

Are the Q/V networks decent estimations of the true functions or not ? (answer may depend on algorithm involved)
ContinuousCritic has several q_networks

Why is that ? In TD3, looks like the networks are indeed all optimized to fit the value, but only the first one is used to improve the actor
Is it ok in production to only look at the value of the first network ?

Also, is there a consistent pattern across all algorithms to access values functions just like predict for the action ?

Another question/note:
If we use the normalize hyperparameter while training, I guess we will need to use the normalization during the inference, right? That would imply getting some data from the env wrapper and exporting it as well

I guess we will need to use the normalization during the inference, right?

yes, we already save the mean and std for observation in a separate file (vecnormalize.pkl) but should easy to export.

onsistent pattern across all algorithms to access values functions just like predict for the action ?

not really... Only between algorithms of the same family...
But I would not focus on that for now (value functions are not needed for inference, unless it is DQN)

@Gregwar I could successfully test it =)
I had to do some tweaks to use it with conda env:
export CMAKE_PREFIX_PATH="${HOME}/.local/lib/libtorch:/home/user/miniconda3"

where the conda prefix can be retrieved with ${CONDA_PREFIX:-"$(dirname $(which conda))/../"}
I also had to comment out the hardcoded set (Python_EXECUTABLE "/usr/bin/python3.8") (I don't think that's needed)

@Gregwar I could successfully test it =)
That is nice!

I had to do some tweaks to use it with conda env:
Yes you are right this should not be there

I think I will have to setup some (automated) tests to check for consistency between Python and C++ predictions. It will be hard to be sure to cover all the cases else, I am digging in the code of all the possible algorithms and I might miss some information (there are many possible options as well like using images as input that should get normalized, using SDE, the Wrapping "normalize" that is not handled at all currently etc.)

It will be hard to be sure to cover all the cases else,

Let's do a first working version that covers only some algorithms (let's say PPO, DQN and TD3) and only covers basic case (MLP, no images, no additional feature like normalization or SDE).
Once that's working and merge, we can work on adding additional features, I would start with normalization and then image support ;)

Hello,
Sorry for the lag, we are currently working on our humanoid robots for RoboCup, we integrate DRL algorithms in the robots for the first year.
We spent some time investigating and finally using OpenVino runtime because of our robots architecture (we use ONNX as intermediary representation OpenVino's model exporter).
In first place I thought of implementing pre/post processing in C++ but it is actually a better idea to use it in the PyTorch module that is being traced or exported. We can't provide a lot of runtime-specific implementation, so we could focus on libtorch as first intended and provide ONNX possibility for people that want to use something else.

@Gregwar could you give me access to your repo so I can push changes? (mainly merging master with this branch)

@araffin Is there any update on this PR? I would be interested in exporting SB3 models into C++ executables but I am not sure on how to approach this problem

I would be interested in exporting SB3 models into C++ executables but I am not sure on how to approach this problem

For inference, you can have a look at https://stable-baselines3.readthedocs.io/en/master/guide/export.html
and https://github.com/DLR-RM/stable-baselines3/issues/1349#issuecomment-1446161768


All v1 POST requests fail due to Twitter now will return empty content if request is with ssl.OP_NO_TICKET context

Summary
When making post request to Twitter, if SSL context has ssl.OP_NO_TICKET option (urllib3 default), the server will always return HTTP 200 with empty content.
Reproduction Steps

do any post request to Twitter on v1.1 end points. I test with simple_upload().

Minimal Reproducible Example
keys = ["key", "secret", "access_token", "access_token_secret"]
file = "path/to/file.jpg"
tweepy_auth = tweepy.OAuth1UserHandler(*keys)
tweepy_api = tweepy.API(tweepy_auth)
media = tweepy_api.simple_upload(file)

Expected Results
File upload success.
Actual Results
Exception has occurred: TweepyException
Failed to parse JSON payload: Expecting value: line 1 column 1 (char 0)
StopIteration: 0

During handling of the above exception, another exception occurred:

json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

  File "D:\!lastfm\lastfm-collage-generator\lastfm_collage_generator.py", line 134, in tweet
    media = tweepy_api.simple_upload(file)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\!lastfm\lastfm-collage-generator\lastfm_collage_generator.py", line 160, in main
    tweet(text, output)
  File "D:\!lastfm\lastfm-collage-generator\lastfm_collage_generator.py", line 163, in <module>
    main()
tweepy.errors.TweepyException: Failed to parse JSON payload: Expecting value: line 1 column 1 (char 0)
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Twitter API Access Plan
Free
Tweepy Version
4.14.0
Checklist

[X] I have searched for duplicate issues.
[X] If applicable, I have shown the entire traceback.
[X] If applicable, I have removed any visible credentials from my code and/or screenshots.

Additional Context
It takes me admittedly very long time to figure out why, since I cannot reproduce the bug with either curl or Postman.
Apparently, requests/urllib3 by default includes an SSL option called OP_NO_TICKET:
https://github.com/urllib3/urllib3/blob/5325c2b179945a9931662412b3bc9a428ab16af3/src/urllib3/util/ssl_.py#L314-L318
And this flag now causes Twitter server to always return empty content (together with code 200) since yesterday (or a few days ago).
You can test it simply with:
import requests

r = requests.post('https://upload.twitter.com/1.1/media/upload.json')
print(r.status_code)
print(r.content)

With curl, it returns error message normally:
>curl -X POST https://upload.twitter.com/1.1/media/upload.json
{"errors":[{"code":215,"message":"Bad Authentication data."}]}

To fix the issue, one has to manually modify the HTTPAdapter:
import requests
from requests.adapters import HTTPAdapter
from urllib3.util.ssl_ import create_urllib3_context
import ssl

class AdapterFix(HTTPAdapter):
    def __init__(self):
        self.ssl_context = create_urllib3_context()
        self.ssl_context.options &= ~ssl.OP_NO_TICKET
        super().__init__()

    def init_poolmanager(self, *args, **kwargs):
        kwargs["ssl_context"] = self.ssl_context
        super().init_poolmanager(*args, **kwargs)

    def proxy_manager_for(self, *args, **kwargs):
        kwargs["ssl_context"] = self.ssl_context
        return super().proxy_manager_for(*args, **kwargs)

adapter = AdapterFix()

s = requests.Session()
s.mount('https://', adapter)
s.mount('http://', adapter)

r = s.post("https://upload.twitter.com/1.1/media/upload.json")
print(r.status_code)
print(r.content)

(You can also just use ssl.create_default_context(), which does NOT have ssl.OP_NO_TICKET set.)

Similar, but on client.search_recent_tweets().
Started 04:00 UTC (06:00 GMT+02:00) from Google Cloud Function. The same code works from my local (a different network)
Debug logging returns empty content, but HTTP 200
Received API response: 200 OK
Headers: {'date': 'Fri, 04 Aug 2023 07:50:00 UTC', 'perf': '<redacted>', 'server': 'tsa_b', 'cache-control': 'no-cache, no-store, max-age=0', 'backoff-policy': 'backoff=1234;serial-duration=30000;serial-delay=500;no-retry=true', 'content-length': '0', 'x-transaction-id': '<redacted>', 'x-rate-limit-limit': '40000', 'x-rate-limit-reset': '<redacted>', 'x-rate-limit-remaining': '39993', 'strict-transport-security': 'max-age=<redacted>', 'x-response-time': '5', 'x-connection-hash': '<redacted>'}
Content: b''

Tweepy exception
    ), data_type=Tweet, user_auth=user_auth
  File "/layers/google.python.pip/pip/lib/python3.7/site-packages/tweepy/client.py", line 135, in _make_request
    response = response.json()
  File "/layers/google.python.pip/pip/lib/python3.7/site-packages/requests/models.py", line 975, in json
    raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)"```

Same problem with create_tweet (V2) : json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Same problem with create_tweet (V2) : json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

You can right, the issue actually affects v2 too.
I only tested with requests.post("https://api.twitter.com/2/tweets") which can return proper http 401 and proper content, but after another test with a valid data upload, it fails too. I modified my title to reflect this.

Thanks for the detailed bug report.
I'm no longer able to reproduce this.
This seems to have been an issue on Twitter's end that's resolved now?

I'm no longer able to reproduce this.

Same here... I was seeing the error but then 42 mins ago the same code started working again.

twitter fix this


Episode Idea: Gatekeeper and OPA

This episode will be an update to the existing OPA TGIK with a focus on gatekeeper.
Monday or Tuesday:

[x] Pick a topic and create/use an issue in the repo
[x] If applicable coordinate with an upstream project if they're interested in attending the show
[x] Create the graphic
[x] Schedule on Youtube
[x] Add scheduled episode to TGIK playlist on YT
[x] Create short link in rebrandly
[x] Tweet about it from your personal account

[x] Post tweets on internal #tgik slack channel and cc @castrojo and agrey
[x] Get confirmation from boschem that she's seen the tweet and is scheduling it

Before episode:

[x] Create HackMD page for live notes.  Start with README template in this repo
[x] Paste HackMD URL on internal #tgik channel and cc @castrojo or other assistant

[x] Assistant adds ~5-10 news items from around the web as possible "News of the Week"

[x] Get everything tested well before episode so you can debug
[x] Use internal slack channel to ensure that AV setup is good

After show (ideally right away but can wait until Monday):

Create/submit PR for this repo

[x] Episode directory and README from hackmd
[x] Any other files from the episode that might be useful
[x] Update playlist.md with episode
Feel free to self merge

[x] Edit YT description to point to episode directory here
[x] OBS/OS Upgrade if applicable (Never on the day of a show)

Resolved in https://github.com/vmware-tanzu/tgik/pull/290


Ftrack: Open Multiple Notes action to tasks.

Resolved with https://github.com/pypeclub/OpenPype/pull/1882


481 lang is deprecated remove references to it in the docs

also removed a few references to docker: in favor of jobtype_extra:


Clarify -b/-c/-p usage

Currently, there're three command line options whose behavior are confusing:

-b specify the build file
-c specify the settings file
-p specify the project directory

For example, in the documentation:

You can use the -b option to select another build file. If you use -b option then settings.gradle file is not used.

What's more, the implementation is also confusing/buggy.
We should clarify what's gonna happen if these options are applied (combinedly). Or, if they're useless, deprecate/remove them.

I think fixing them to match their current documentation would be a good first step, e.g. fixing #3100

@blindpirate The documentation has been changed recently for those option: https://docs.gradle.org/current/userguide/command_line_interface.html#_environment_options. Does this improve the situation or is additional work needed?

The goal of this ticket is to evaluate:

which of these options we want to keep at all
what their exact contract should be (e.g. the current behavior of -b is highly confusing and not mentioned in the new documentation)
create issues to change Gradle to be in line with those decisions


Add artifact to deploy apps via the Appsody Application Operator

Each Stack needs an app-deploy.yaml file that represents the minimal set of configuration needed to deploy the application image.
apiVersion: appsody.dev/v1alpha1
kind: AppsodyApplication
metadata:
  name: REPLACE-NAME
spec:
  applicationImage: REPLACE-IMAGE
  stack: java-microprofile

The field REPLACE-NAME should be modified to match the service's name (must be unique), and REPLACE-IMAGE should be modified to represent the full image path, such as registry/imageName:imageTag.

Here's a more complex example, demonstrating a lot of the other fields for completeness:
apiVersion: appsody.dev/v1alpha1
kind: AppsodyApplication
metadata:
  name: demo-appsodyapplication
spec:
  applicationImage: docker-registry.default.svc:5000/demo/appsody-app:1.0.0
  stack: java-microprofile
  env:
    - name: "dbUrl"
      value: "http://couchdb-svc-couchdb:5984"
    - name: dbUsername
      valueFrom:
        secretKeyRef:
          name: couchdb-couchdb
          key: adminUsername
    - name: dbPassword
      valueFrom:
        secretKeyRef:
          name: couchdb-couchdb
          key: adminPassword
  architecture:
    - amd64
  expose: true
  resourceConstraints:
    limits:
      memory: 512Mi
      cpu: 2
    requests:
      cpu: 0.5
  readinessProbe:
    failureThreshold: 12
    httpGet:
      path: /health
      port: 9080
    initialDelaySeconds: 5
    periodSeconds: 2
  service:
    port: 9080
    type: ClusterIP

This has been implemented for all stacks in https://github.com/appsody/stacks/pull/213


Disable Peer Exchange in PRUNE message

Description
As described here https://github.com/ethereum/consensus-specs/issues/3500, Peer Exchange isn't relevant to the Ethereum Network and PRUNE messages sent by Teku can have up to 9 KB and 210 peer IDs due to Peer Exchange.
The options to limit the number of peers sent and accepted in PRUNE messages have been fixed and improved in this PR https://github.com/libp2p/jvm-libp2p/pull/336.
We suggest 2 phases for the implementation:

A new jvm-libp2p version containing the mentioned PR is released. Teku is updated to use this version and set maxPeersSentInPruneMsg to zero. For compatibility reasons, maxPeersAcceptedInPruneMsg is set to a high value, potentially to the max Integer value.
After some time has passed and all Teku nodes are expected to have been updated, maxPeersAcceptedInPruneMsg is also set zero.

Steps to Reproduce (Bug)
Inspect PRUNE messages sent by Teku over time.
Expected behavior:
Phases:

Teku doesn't send peer IDs in PRUNE messages, but accept it from other peers.
Teku doesn't send peer IDs in PRUNE messages and doesn't accept it from other peers.

Actual behavior:
Teku sends peer IDs in PRUNE messages and accept it from other peers.
Frequency:
Often
Versions (Add all that apply)

Software version: Not sure, but probably the latest and previous ones.
Java version: [java -version]
OS Name & Version: [cat /etc/*release]
Docker Version: [docker version]
Cloud VM, type, size: [Amazon Web Services I3-large]

Thank you @diegomrsantos for raising this.:) We will look into releasing a new version of the libp2p and will take it from there.

@diegomrsantos just to confirm both phases have been implemented


Automated tests for UI workflows should test retrieval of data from Elasticsearch

Automated tests should validate that the workflow and retrieval of data create by the workflow using the same methods as the UI.

No longer applicable. Elasticsearch was removed during the front-end / back-end split in v2.0.4.


Avail's fixes

I forgot to make a new branch for every pull request, so I guess this will have to do


PLT-1384 Added handling for new preferences updated/deleted events (3.7)

Ticket Link
https://mattermost.atlassian.net/browse/PLT-1384
Test Information
This PR was tested on: iOS Simulator

Sorry, @enahum is right. Do we need this for 3.7?

Oh yeah, you're right. It wouldn't really hurt to have, but it's definitely not required for 3.7


Use text version of question pages as regression test artefacts

Description
Trello card: https://trello.com/c/P6tuDjcg
This introduces a text version of question pages and converts all existing regression test artefacts for question pages to the new text version.
Note that we lose a little "end-to-end" test coverage of the "previous answers" section on question pages. However, this was only ever incidental test coverage and it's not the right place to test such functionality. I suggest this is better addressed by making sure we have sufficient coverage before making changes in that area of the code.
Motivation
The idea is to make the regression tests less brittle for question pages, e.g. when @selfthinker is making accessibility improvements to the HTML.
External changes
The new text version of question pages should be accessible in the integration environment, e.g. https://www-origin.integration.publishing.service.gov.uk/calculate-statutory-sick-pay/y.txt
However, the new text version of question pages will not be visible in staging or production environments.

I'm starting to review this now.

I found an example question page artefact for each type of question to help me check the changes:

multiple_choice example
country_select example
date_question example
value_question example
money_question example
salary_question example
checkbox_question example
postcode_question example

Aside from the date_question options that I've already asked about, the only other question I have is whether we should display the options (week, month, year) for a salary_question? Maybe it doesn't make sense if the options are always shown?

I've chatted to @chrisroos about the issues he has raised about options for date_question & salary_question not appearing in the new text version of the question page artefacts.
The date_question artefacts are generated using Rails helper methods (i.e. select_day, select_month, select_year) within the app/views/smart_answers/inputs/_date_question.html.erb template. Rendering these options as part of the text version of the artefact would be awkward, because (a) there are up to 3 sets of options to display, and (b) the options to display are not readily available from the DateQuestionPresenter. Given the amount of work involved and the resultant additional complexity in the code, I'm not convinced rendering the options is worthwhile.
The salary_question artefacts are generated from a hard-coded array of strings in the app/views/smart_answers/inputs/_salary_question.html.erb template. We could move the hard-coded options into an options method on SalaryQuestionPresenter - that would mean they would show up in the text version of the artefact. However, there are only 3 instances of salary_question in published flows (all in pay-leave-for-parents) and I believe they are all redundant and in need of removal. Given this and the fact that they are hard-coded options (i.e. how risk of breaking), I'm not convinced rendering the options is worthwhile.
Stepping back slightly, both Chris & I are wondering how useful the question page regression test artefacts are. We might get enough value by changing the regression tests to just check that that each question page renders without an error i.e. stop checking that the content matches exactly.
Anyway, I plan to rebase this against master, resolving conflicts, and force-push in preparation for merging.

I've added an extra commit to update the docs which explain how to view the Govspeak version of pages.


Starting ElastAlert

Hi
How do I run elastalert, I did run python -m elastalert.elastalert --verbose --rule example_rules/example_frequency.yaml but when I 'Ctrl c' it stops. I would like it to run all the time. Am I missing something?

I did try python -m elastalert.elastalert --verbose --rule example_rules/example_frequency.yaml & and it seem to be running but I know this is not correct.
On the elastalert docs [Running ElastAlert | http://elastalert.readthedocs.org/en/latest/running_elastalert.html#running-elastalert] it say to run ElasticAlert as a daemon using [Supervisord | http://supervisord.org/]. This is where I am kinda stuck and experience comes in handy.
on the supervisord doc  [Installing a Distribution Package | http://supervisord.org/installing.html#installing-a-distribution-package]

Use the package management tools of your distribution to check availability, on CentOS you can run yum info supervisor

when yum info supervisor is run, I get
Loaded plugins: priorities, update-motd, upgrade-helper Error:
No matching Packages to list

After that I ran
pip install supervisor
which returns
Traceback (most recent call last):
  File "/usr/bin/pip", line 5, in <module>
    from pkg_resources import load_entry_point
  File "build/bdist.linux-x86_64/egg/pkg_resources/__init__.py", line 2912, in <module>

  File "build/bdist.linux-x86_64/egg/pkg_resources/__init__.py", line 2898, in _call_aside

  File "build/bdist.linux-x86_64/egg/pkg_resources/__init__.py", line 2925, in _initialize_master_working_set

  File "build/bdist.linux-x86_64/egg/pkg_resources/__init__.py", line 644, in _build_master

  File "build/bdist.linux-x86_64/egg/pkg_resources/__init__.py", line 657, in _build_from_requirements

  File "build/bdist.linux-x86_64/egg/pkg_resources/__init__.py", line 830, in resolve

pkg_resources.DistributionNotFound: The 'pip==6.1.1' distribution was not found and is required by the application

Also I did try
yum install supervisor
which returns
Loaded plugins: priorities, update-motd, upgrade-helper
amzn-main/latest                                                                                | 2.1 kB     00:00
amzn-updates/latest                                                                                | 2.3 kB     00:00
No package supervisor available.
Error: Nothing to do

Any idea what the errors are and how I can get this to run.

A really easy way might be to run it in screen or tmux.


gui-wm/river-9999: build failed, invalid version of wlroots

Hi,
emerge --ask =river-9999

Wants to install gui-libs/wlroots-0.16.2-r2 which is incompatible (and results in a compilation error) with the latest version of river.
The latest river requires wlroots version 0.17.1
This is fixed after executing
emerge --ask =wlroots-0.17.1

And compiling river manually.
emerge--info.txt

Thanks for the report, fixed the minimum version.


Add support for Uint8Array in addition to Buffer

It might just be a question of updating the TypeScript types. At least the Node.js fs APIs support both. I don't know about other drivers.

I am less familiar with Uint8Array. Can you share when you will use it and does it make sense to convert it to buffer inside s3 or gcs drivers (if they don't support it)?

Technically, Buffer is built on Uint8Array (class Buffer extends Uint8Array). It just adds some utility methods on top of it.
There are many libraries which return binary data as Uint8Array (often to be usable in the browser).
It may make sense to convert internally in the drivers (there is a very fast way to create a Buffer from Uint8Array, without copying the underlying data)

Added to flydrive. https://flydrive.dev/docs/disk_api#getarraybuffer

Why is it called getArrayBuffer if it doesn't return an ArrayBuffer?

Yes, sgtm


[feat] Sorting on the real record date - on createdAt / either updatedAt

we should add additional field and fill it after AI -it should be the default sort
Then we can show up the life timeline

https://www.solid-ui.com/docs/components/timeline

Simplet way to make it - regarding the UX would be to display the date tags always by the filters button:


sitemap.xml metadata

Does the collector add metadata (title, keywords, etc.) it find from the sitemap.xml itself or just metadata it finds inside the document itself?

Not sure I grasp your question.  sitemap.xml files normally do not have title, keyword, and related fields.   For every link it follows, it will index the target page with its metadata.
It will also add these fields to each documents from the sitemap:
collector.sitemap-lastmod
collector.sitemap-changefreq
collector.sitemap-priority

Does that answer?

I think so.  I'm assuming wont parse SEO elements in sitemap.xml like:
video:titleXero</video:title>
video:descriptionXero online accounting software.</video:description>
If not, then we will have an issue with indexing web pages with embedded video links.  The MP4 have metadata itself so those links will have the title and description data.  But the parent page is just a generic template that is passed a video id.
So let's say the page hosting the mp4 is http://mysite.com/playVideo.html and it has a link in the page called http://video.3rdparty.com/36136564246.mp4.  And the video has a title metadata element of "Xero".  I want the playVideo.html to come up in my search results, not the actual video.
So, is there a way to tell Norconex to add the metadata that came from a video link in it's page?

If the info is in your sitemap.xml, one way to do it is to consider the sitemap a regular page and you extract its links using a custom ILinkExtractor.   Your custom solution will extract links which can have a title and description attached, which will be associated with the target URLs.

Thanks.  We will look at extending the code.

If I implement a ILinkExtractor would I use Link.setTitle() and setText() to inject the info into the link?
Or do these properties get overwritten by the collector?

The answer is... none of the above! :-)
They are added as new fields to target documents that are processed, They are:
collector.referrer-reference
collector.referrer-link-tag
collector.referrer-link-text
collector.referrer-link-title

If you want those to take over some other fields you have, I suggest you use RenameTagger or CopyTagger in your Importer section as a post-parse handler.


TAVOSS-TR:Joplin

Version of the project
server-v2.2.11-beta
Tech Stack
Application [A]
Project details
Joplin is a free, open source note taking and to-do application, which can handle a large number of notes organised into notebooks. The notes are searchable, can be copied, tagged and modified either from the applications directly or from your own text editor.
Languages used
TypeScript, JavaScript
Domain
Technology
Industry
Education
Open Source Project type
Community led
Repo URL
https://github.com/laurent22/joplin
Webpage
https://discourse.joplinapp.org/
License
MIT
Other license
No response
Why we should track it.
Joplin is a free, open source note taking and to-do application, which can be customized for end-user consumption.

This project is accepted for tracking hence closing this issue with status as Tracked


Unable to dynamically load : C:/Users/dalvarez/Documents/maya/2020/plug-ins/maya2glTF.mll

Trying to use the exporter in a machine with windows 10 and getting this error in Maya 2019, Maya 2020.
// Error: file: C:/Program Files/Autodesk/Maya2020/scripts/others/pluginWin.mel line 317: Unable to dynamically load : C:/Users/dalvarez/Documents/maya/2020/plug-ins/maya2glTF.mll
No idea what can be causing these error.

Thanks for reporting. Is the latest rc3 release you are trying?
Could you make a screenshot or dump of your C:/Users/dalvarez/Documents/maya/2020/plug-ins folder, or just tell me if maya2glTF.mll is in that folder?

This is the problem with not installing the proper msvc redistributable.

Ah yes, that could be it.
Also, the link to the msvc redistributables in the README isn't up-to-date.
The latest ones can be found here
You most likely need to install these

Thank you I will try that and will let you know. I did try in another machine and I didn't have any problem.

It works! Thank you very much!

@fire Thank you!