Dynamic viewport height test

This is a test PR for switching thee CSS use of 100vh to 100dvh. The dvh is a dynamic value that adjusts to the size of the browser chrome on mobile as you scroll, the keyboard expands/collapses/etc. This is used to calculate the screen height in certain places, like the code editor and mobile nav. With just vh, it can happen that elements go beyond the visible screen window and are blocked by browser chrome.
Reason for testing is that I found scrolling in the mobile nav a bit funky, some content at the bottom was not reachable. The nav shares a class to restrict this with the code editor, which I updated here. But, this may cause other problems, so the final solution might be for the mobile nav to have its own CSS instead of using the shared one. It's just a line or two anyways.
But let's test and see.
💼Check the preview🧳

Does anyone know what these pixel values stand for? I think 70px is the top nav bar, but not sure about the others.

A lot of those values have to do with the monaco editor and its inability to easily be resized, the 40px is the language selection and the 56px is the bottom success bar

Ahh I see, i definitely accidentally included that class, it shouldn't be here


No description of options

Body
Ridiculous

Version
not sure
What browsers are you seeing the problem on?
Other
What OS are you seeing the problem on?
Other
Anything else?
What do these options do? there is no explanation that I can find anywhere; should be in the wiki; please consider explaining them; like what is 'the destination of a tracker'? and why is it disabled by default

Destination of a tracker is the website it leads to like if clicking on t.co/xyz takes you to google.com then google.com is the destination of the tracker. By default the extension shows a preview of the link you are about to be redirected to, if you don't want that then check the "Instantly take me to the destination..." option

Is apimon.de a proxy for untangling redirects?
And does blocking ip loggers if they can't be bypassed break the bypass altogether?

apimon.de is now discontinued (see #784)
Yes, blocking IP loggers blocks sites like https://grabify.link completely.

Thank you.


xctool stalls after "Loading settings for scheme 'MyScheme' ..." if scheme is not shared

Hi!
We've seen this a few times where a project is set up without a shared scheme and then tries to run xctool on Travis CI, where it just stalls right after "Loading settings for scheme 'MyScheme' ...":
$ xctool -workspace .xcworkspace MyWorkspace.xcworkspace -scheme MyScheme -sdk iphonesimulator7.0 build test
[Info] Loading settings for scheme 'MyScheme' ...

Marking the scheme as "shared" has fixed this for the people seeing this issue, I wonder if it's possible to have xctool print an error message instead of stalling?
Let me know if there's any other information I can help provide to help debug this.

I have made some changes to how we handle and search for schemes. It shouldn't be a case after commit 728150d6ce8bc90e2603c2b92eec79ac8fd15b08.


cudaminer build error on Ubuntu 16.04

I followed the building process for Ubuntu 14.04. OpenCL + CUDA (for NVIDIA cards), except that I installed "libjsonrpccpp-dev" instead of "libjson-rpc-cpp-dev" because it was renamed.
I ran
cmake -DBUNDLE=cudaminer ..
make -j8

to build cudaminer. At the end of make -j8, it shows
[100%] Linking CXX executable ethminer
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libjsonrpccpp-client.so: error: undefined reference to 'curl_global_init', version 'CURL_OPENSSL_3'
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libjsonrpccpp-client.so: error: undefined reference to 'curl_global_cleanup', version 'CURL_OPENSSL_3'
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libjsonrpccpp-client.so: error: undefined reference to 'curl_easy_setopt', version 'CURL_OPENSSL_3'
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libjsonrpccpp-client.so: error: undefined reference to 'curl_easy_init', version 'CURL_OPENSSL_3'
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libjsonrpccpp-client.so: error: undefined reference to 'curl_slist_append', version 'CURL_OPENSSL_3'
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libjsonrpccpp-client.so: error: undefined reference to 'curl_easy_getinfo', version 'CURL_OPENSSL_3'
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libjsonrpccpp-client.so: error: undefined reference to 'curl_slist_free_all', version 'CURL_OPENSSL_3'
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libjsonrpccpp-client.so: error: undefined reference to 'curl_easy_cleanup', version 'CURL_OPENSSL_3'
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libjsonrpccpp-client.so: error: undefined reference to 'curl_easy_perform', version 'CURL_OPENSSL_3'
collect2: error: ld returned 1 exit status
ethminer/CMakeFiles/ethminer.dir/build.make:117: recipe for target 'ethminer/ethminer' failed
make[2]: *** [ethminer/ethminer] Error 1
CMakeFiles/Makefile2:493: recipe for target 'ethminer/CMakeFiles/ethminer.dir/all' failed
make[1]: *** [ethminer/CMakeFiles/ethminer.dir/all] Error 2
Makefile:127: recipe for target 'all' failed
make: *** [all] Error 2

I google and found this page
http://stackoverflow.com/questions/16476196/undefined-reference-to-curl-global-init-curl-easy-init-and-other-functionc
but I have no idea where to add the -lcurl flag.
What should I do?

I've found the solution.
I read the log of cmake -DBUNDLE=cudaminer .. and found that the linked curl lib (and some other libs) was from Anaconda, which I installed before. I cancelled the PATH change made for Anaconda, ran the cmake and make again, and it's successfully built.


docs: add documentation around importing CRD as GVK

Description
The Kubernetes Fluent Client enables importing TypeScript types from CRDs. This PR contains the documentation explaining to the user how to achieve this with an example.
Related Issue
Fixes #148

Relates to #
Type of change

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[x] Other (security config, docs update, etc)

Checklist before merging

[x] Test, docs, adr added or updated as needed
[x] Contributor Guide Steps followed

You shouldn't have to re-register the custom objects; you should only have to import them for it to work, otherwise I that might be a bug in KFC.


can't install presentator on php 7.0

First of all thank you very much for this project.
I would really love to start testing and contributing - but I can't install it.
I sais that I need PHP 7.1 but I only have 7.0
Is it true? I saw in your requirements description, that 7.0 should work

Just tested it. Indeed, there is an error with phpunit/codeception in php7.0.
This weekend I will try to review all dependencies and their requirementents in details and eventually will upload a fix as v1.8.1 release.
Meanwhile, If you want to install the project locally, I recommend you to use the integrated Docker dev environment - https://github.com/ganigeorgiev/presentator/wiki/Develop-with-Docker.

@mokra v1.8.1 with regenerated composer.lock file is released. The automation tests and the installation process were tested with php5.6, php7.0 and php7.1.
If you still are unable to install/setup the project, please comment/reopen the issue.

thanks!
The yii file was also missing in the root folder.


View is pretty large on Ipad

Hey
I love your implementation of this and how simple it is to use!
Tried it for Ipad but it covered almost the entire screen which is a bit brute :D
So I added this setting which let's you manually set the width divider (standard was 1.6)
Probably not the prettiest way to add to it but it works and it's a very fast enhancement :)
In Settings struct
static var CLAWidthDivision = 1.6
In loadingActivity:
let width = sender.view.frame.width / CGFloat(Settings.CLAWidthDivision)
Before calling showMethod:
CozyLoadingActivity.Settings.CLAWidthDivision = 4.6 (about right for Ipad)
Thanks again!
PS don't use github that much so thought I'd add it here and you can mark it as enhancement or something :) Cheers

Hello, can you take a screen shot before and after you made this difference? I might add this as a feature that automatically works in ipads if needed. Just need to see your screen

Tested on iPad 2/ iOS 9.1 simulator

I'll create an update soon

https://github.com/goktugyil/CozyLoadingActivity/commit/517b8240039587cf9af7d70a1fe5fc9e5471a62a
https://github.com/goktugyil/CozyLoadingActivity/commit/fa2cdf23a91fc0dae407c596c62ee65033c84b6d
fixed


Adjust 16mb minimum compression size for<EMAIL_ADDRESS>

Some developers start to use this package within the expo. Unfortunately, Expo still uses an old version of React Native up to React Native 0.64.3.
Can you  remove this  condition  https://github.com/Shobbak/react-native-compressor/blob/dd5c106522461005acd89d6f679c0ef2f96fdd74/ios/Video/VideoCompressor.swift#L205
and old version users can benefit from that patch.

+1

@emmbyiringiro @xuanhieupd today I am busy with office work. hope I will add this tomorrow. thanks

Start from Expo SDK 42, the Expo team release a new service that makes the Expo app support a native module.
Expo community has no built-i media compression package. This package is the one that is a great candidate as It has an easy setup and API to work with.
I expect many Expo users to adopt this.
I wish I can help more unfortunately I don't have a background in native development.
You can read more about  Expo  native build here
https://docs.expo.dev/build/setup/

@emmbyiringiro @xuanhieupd can you check 0.5.8
yarn add<EMAIL_ADDRESS>

Thanks for the patch. I have another issue or I'm doing wrong something.
By default, the video size for compression is 640. When I record video with camera on android, the  camera record video with  width:480 and height:640 dimensions.
I was expecting this video to be compressed on to default of 640. Unfortunately, the app crash and I need to lower video compression to either 480 or 360.
I guess that the logic should check if the size is less than or equal so provide compression size.
I

@emmbyiringiro can you confirm, in 0.5.8 version minimumFileSizeForCompress is working or not?

@emmbyiringiro can you send me the video, so I can check that video on my side?

I investigate the issue and I found that all videos you want to compress must have a video size great than maxSize otherwise it will crash.
For example, if you have a video like this  https://sample-videos.com/video123/mp4/360/big_buck_bunny_360p_1mb.mp4 which has width 640 and height 360 and you set maxSize to 640 this will crash.
Even if video size 640 is equal maxSize. The device cameras sometimes record large video sizes which require be optimized.
In this situation, both video size and maxSize are equal but the app crashes.

@emmbyiringiro can you confirm, in 0.5.8 version minimumFileSizeForCompress is working or not? so i can write this version in docs

I installed but it crashed with Native module not found error.
I'm using Expo SDK 42  which uses React Native 0.63 .
You can read more about this Expo Native module integration
https://blog.expo.dev/introducing-custom-development-clients-5a2c79a9ddf8

@emmbyiringiro can you confirm, in 0.5.8 version minimumFileSizeForCompress is working or not? so i can write this version in docs

I installed but it crashed with Native module not found error.
I'm using Expo SDK 42 which uses React Native 0.63 .
You can read more about this Expo Native module integration https://blog.expo.dev/introducing-custom-development-clients-5a2c79a9ddf8

This issue happen on android

@emmbyiringiro can you check 0.5.9 version. I have added support for the expo. you can read docs
yarn add<EMAIL_ADDRESS>

I investigate the issue and I found that all videos you want to compress must have a video size great than maxSize otherwise it will crash.
For example, if you have a video like this https://sample-videos.com/video123/mp4/360/big_buck_bunny_360p_1mb.mp4 which has width 640 and height 360 and you set maxSize to 640 this will crash.
Even if video size 640 is equal maxSize. The device cameras sometimes record large video sizes which require be optimized.
In this situation, both video size and maxSize are equal but the app crashes.

i will investigate this later

@emmbyiringiro can you check 0.5.9 version. I have added support for the expo. you can read docs
yarn add<EMAIL_ADDRESS>
Add the Compressor plugin to your Expo config (app.json, app.config.json or app.config.js):
{
  "name": "my app",
  "plugins": ["react-native-compressor"]
}

Have tested this package with Expo 42. I tried and result in this error.
[TypeError: null is not an object (evaluating 'NativeVideoCompressor.compress')
It seems that EXPO SDK 42 does not register native module

@emmbyiringiro after the above steps, can you run these three commands and test them?
expo prebuild

eas build

yarn ios

@emmbyiringiro closing this because I have fixed it and released the 0.5.9 version. you can open another issue for the expo if you want


[AITT] Add publish_full

Add publish_full
Set default protocol as AITT_TCP

:memo: TAOS-CI Version: 1.5.20200925. Thank you for submitting PR #184. Please a submit 1commit/1PR (one commit per one PR) policy to get comments quickly from reviewers. Your PR must pass all verificiation processes of cibot before starting a review process from reviewers. If you are new member to join this project, please read manuals in documentation folder and wiki page. In order to monitor a progress status of your PR in more detail, visit http://ci.nnstreamer.ai/.

:octocat: cibot: @gichan-jang, A builder checker could not be completed because one of the checkers is not completed. In order to find out a reason, please go to http://ci.nnstreamer.ai/nnstreamer-edge/ci/repo-workers/pr-checker/184-202309151703310.9002730846405-2d1d10a4f0b22eadaa4081fb2a5b22d9e22f65c3/.


Update fits.hpp to add binary_table to read_extensions() function

Description

The read_extension() function supported only image and ascii_table extensions as parser for binary_table had not been developed at the time of writing of the function.
binary table option has been added in this PR.

Can you please remove the "if" in your comments no need of it.

Made the change... Thank u for ur review!

Many things have changed in recent months in the library and this PR is no more needed. Though thank you for taking the time to contribute into the library. We value your time, interest and contribution.


refactor(WIP): WIP on subscription refactor DO NOT MERGE

Attn @trxcllnt, who I know is working on similar refactors.
I think we should just put up a different branch and work against that. It will likely take multiple PRs to straighten out this refactor.

Actually, @trxcllnt, I've done just that... this is merged into a subscription-refactor branch here. Let's try to work off of that. Basically the game is now to get all tests green and add tests around issues that other people have seen.
cc @staltz, @kwonoj
This is probably our biggest priority at the moment.

Here is the branch: https://github.com/ReactiveX/RxJS/tree/subscription-refactor

@blesh we should start with #1065 merged in. I started my own branch from that PR on Friday and got fairly far, I think I might continue with that and see if I can solve it there, then merge that branch into this one.

@blesh this is finished in my branch.

@trxcllnt since your commits touched ConnectableObservable, I'd recommend pulling in the tests I wrote in PR #1042 into your branch and make sure the tests pass there.

@trxcllnt did you want to submit a PR?


ConfigMap data with array

fabric8 maven plugin goal resource fails to parse the yaml data with array of values
Description
When we have configmap with array of values for the data, the plugin resource goal fails with the following error:
[ERROR] Failed to execute goal io.fabric8:fabric8-maven-plugin:3.2.15:resource (default-cli) on project bug-hunter-service: Execution default-cli of goal io.fabric8:fabric8-maven-plugin:3.2.15:resource failed: Can not deserialize instance of java.lang.String out of START_ARRAY token
[ERROR] at [Source: N/A; line: -1, column: -1] (through reference chain: io.fabric8.kubernetes.api.model.ConfigMap["data"]->java.util.LinkedHashMap["elastic-search-indexes"])

Info

f-m-p version : 3.2.15
Maven version (mvn -v) :

Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5; 2015-11-10T22:11:47+05:30)
Maven home: /usr/local/homebrew/Cellar/maven/3.3.9/libexec
Java version: 1.8.0_92, vendor: Oracle Corporation
Java home: /Library/Java/JavaVirtualMachines/jdk1.8.0_92.jdk/Contents/Home/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "mac os x", version: "10.12.2", arch: "x86_64", family: "mac"

Kubernetes / OpenShift setup and version :

lient Version: version.Info{Major:"1", Minor:"4", GitVersion:"v1.4.6+e569a27", GitCommit:"e569a27d02001e343cb68086bc06d47804f62af6", GitTreeState:"not a git tree", BuildDate:"2016-11-12T09:26:56Z", GoVersion:"go1.7.3", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"5", GitVersion:"v1.5.1", GitCommit:"82450d03cb057bab0950214ef122b67c83fb11df", GitTreeState:"clean", BuildDate:"1970-01-01T00:00:00Z", GoVersion:"go1.7.1", Compiler:"gc", Platform:"linux/amd64"}

If it's a bug, how to reproduce :

Create a simple maven project and add f8-m-p to it
add a following yaml config map to the src/main/fabric8

   ---
  apiVersion: "v1"
  data:
    elastic-search-service-name: "elasticsearch"
    elastic-search-service-port: "9200"
    elastic-search-indexes:
       - "logstash-*"
   hunting-interval: "10"
  kind: "ConfigMap"
  metadata: {}
  labels: {}
  name: "some-service"

* run the maven like `mvn clean fabric8:resource`

_Please format the yaml as needed as copy paste might cause indentation issues

@rhuss  - can you please take a look at this in next cycle of yours :)

Configmap data field is only string->string so this is correct behaviour.
If you want to specify several values then you need to do so in a string & separate into multiple values in your application code.
Alternatively consider putting a whole config file as a string value of a key so your app code can parse accordingly.

Thanks @jimmidyson, i have already updated my application to use string and then some parsing ..  since it was yaml i thought it will support all YAML syntax and semantics.


Securing API endpoints L5

Hello Luca!
How can I secure my api endpoints in Laravel 5?
In Laravel 4, I do it:
Route::group(array('prefix' => 'api', 'before'=>'oauth'), function() {
    ...
});

But L5 i have several errors when try to check the scope. It seems that the filter don't work well
     Call to a member function hasScope() on null

I tried to add to the routeMiddleware array but didn't work.
Routes.php
     Route::group(array('prefix' => 'v1', 'middleware'=>'oauth'), function() {

Kernel.php

	/**
	 * The application's route middleware.
	 *
	 * @var array
	 */
	protected $routeMiddleware = [
		'auth' => 'App\Http\Middleware\Authenticate',
		'auth.basic' => 'Illuminate\Auth\Middleware\AuthenticateWithBasicAuth',
		'guest' => 'App\Http\Middleware\RedirectIfAuthenticated',
		'csrf' => 'App\Http\Middleware\VerifyCsrfToken',
        'oauth' => 'LucaDegasperi\OAuth2Server\Middleware\OAuthExceptionHandlerMiddleware'
	]

Hello, the package in L5 works just like it does in L4.  You add the "oauth" before filter to your routes (or routes groups) and on top of that add the OAuthExceptionHandlerMiddleware to your global middleware, not the route ones.

Solved!
I forgot to add to .htaccess
RewriteEngine on
RewriteCond %{HTTP:Authorization} ^(.*)
RewriteRule .* - [e=HTTP_AUTHORIZATION:%1]


Goldfish comment barely visible

Hey! awesome theme pack, I found my (almost) perfect theme, Goldfish!
I've been using the latest VSCode, with the latest rainglow pack version.
my issue is rather small, the comments text is too dark imho, and it's almost invisible on diff tool (screenshot provided).
this issue exists in several themes but not in Joker or others.
I assume a few clicks lighter would solve this.

I would be happy to submit a PR to fix this, but its your show and art - so your call.
Thanks! :)

Is there a list of properties that we can use to write our overrides?
The extension has a code block,
"workbench.colorCustomizations": {
    "statusBar.background": "#333",
    "statusBar.foreground": "#fff",
    "statusBar.noFolderBackground": "#333",
    "statusBar.noFolderForeground": "#fff"
}

But where can I find a complete list of these color customizations?


Fix build issues with bundle project templates

fixes #2044
Bonus: removes bundle project's need for the WinObjC.Language package

:shipit:


Issue #4300 is happening again.

See: https://github.com/MicrosoftDocs/msteams-docs/issues/4300
I am getting the following error in the dev manifest portal for Teams:
defaultGroupCapability.team | Value "" is not defined in the enum.
defaultGroupCapability.groupchat | Value "" is not defined in the enum.
defaultGroupCapability.meetings | Value "" is not defined in the enum.
on a new project that has been provisioned with TeamsFx on visual studio 17.5.3
last issue just says it was fixed but no indication on how.

changed these to tab to fix the issue:


Multi-Property Value Application

This issue was migrated from https://github.com/NIVeriStandAdd-Ons/SLSC-EDS-Plugin-Custom-Device/issues/5
I'd love to see the ability to change properties that share the same datatype simultaneously, i.e. select all the relevant switches using Shift and set the value appropriately to be reflected across all of them.

I think I implemented this feature in the latest version of the EDS. You can use "shift" and select several elements of the same type and change its value. Before it used to not allow that. Could you confirm that this is behaving as Lynn would expect, @rtzoeller, @csjall ?

Hey @Austrim and @Karl-G1 , I am tagging you so you can "manage" this request. I believe this was already fixed. Just wanted to clean up house before this gets incorporated into DevOps with no need.

Confirmed this feature was added in #15


DBZ-5328 field.exclude.list doesn't accept dashes (-) in database name

https://issues.redhat.com/browse/DBZ-5328

@harveyyue Applied, thanks. Very nice test coverage.


join group using code

The test fails because $user->teams is cached between the two calls (which does not happen in reality). A workaround would be $user->teams()->get() which forces a refetch of the relationship (this is taken care of by syncWithoutDetaching).


Material Design

Hello, I want to contribute in your project, I need something similar with material design, I saw you have a branch whit some changes, How can I help you? Do you have a roadmap? in fact, I saw that there is a fork that has a lot of progress on that too:
https://github.com/theRichu/angular-surveys
Regards.

Hi, I've merged branch with material design templates to master.
Material templates are in directory '/src/builder/templates/material/',  and demo using these templates is in '/demo-material'.
Files from 'dist/form-builder.min.js and 'dist/form-viewer.min.js don't contain any templates; you need to add them from file 'form-builder-material-tpls.min.js'
Online demo is here: http://mwasiluk.github.io/angular-surveys/material
Regarding the roadmap and help: https://github.com/mwasiluk/angular-surveys/issues/3
Regards.


Improve sources orga

Into src/app/medInria change files tree.
Create folder 'Area' to shelter all area.
Into Workspace area you can find different folder for each workspace

Is it possible to write a description on this PR? For future use, it's always useful to understand directly with the title or the description what a PR is doing :)

Done


Model conversion for 6-channel data

We have a model that uses 6-channel data. Is there any way to convert a model with quantization? The API seems to ask for a dataset.txt containing a list of image files, which is not an option for us. It would be great if we could pass data samples directly to the build() function as np arrays or something. Any possibilities there?

I've found a note in the manual that says we can supply npy files. Having some trouble with the format (NCHW vs. NHCW), but looks promising.

I've found a note in the manual that says we can supply npy files. Having some trouble with the format (NCHW vs. NHCW), but looks promising.

Have you solve this problem?

@colin-broderick Have you solve this problem?

No net yet I'm afraid, but it's been a lower priority than some other issues we're having, such as getting a model converted accurately without quantization.

Is there any new progress in quantization of the models with more than 4-channel inputs ?


chore(Datastore): Correct the timestamp type

This PR is to resolve PR comments in the PR https://github.com/googleapis/google-cloud-php/pull/5734

Closing this PR because we will instead merge: https://github.com/googleapis/google-cloud-php/pull/5801


Provide an interface to specify absolute cliff time as dates, and fix docs

Time locked accounts currently don't work as documented, because the docs assume cliff time is set relative to the transaction which sets timing, while in reality it seems to be interpreted as an absolute slot number (of the slot at which the cliff time is over).
We should either fix the docs or (my suggestion) make the interface in snarkyjs treat the cliff time as a relative slot number, because that will be most certainly easier to use.
EDIT: plan changed, see below

Instead of the above, we decided to keep the absolute time and provide an interface for converting times / dates into global slots

@mitschabaude I opened PR https://github.com/o1-labs/docs2/pull/355 for updates to the the https://docs.minaprotocol.com/zkapps/advanced-snarkyjs/time-locked-accounts page
I have to learn how to bring in that updated graphic.
Can you use this PR for your changes? Teach me if there is a better way to work this. Thank yoU!

Can you use this PR for your changes? Teach me if there is a better way to work this. Thank yoU!

Yes that works 👍🏻


Similarity estimation

semantic similarity estimation modules dedicated for RDF processing

feature extraction in utils
similarity estimation models (7): Batet, BraunBlanquet, Dice, Jaccard, MinHash (reusing Apache SPark MLlib MinHashLSH) Ochiai, Simpson, Tversky
minimal calls provided in examples: how to call with few lines of code the modules
experiment: SimilarityExperimentPipline for reproducable measuring processing times in hyperparameter grid
Metagraph creation by RDF data in utils

read in of files resolved of typesafe config files
some minor unit tests are available
tiny files for using these contributions like movie.nt and parameter files

I guess I just go ahead with the merge; I see rdf.nt which should eventually be renamed, I am not very fond of this 'mode' string (probably the mode should be reference to a lambda, a class instance or an enum and some util function may construct it from a string) and comments about example datasets/code which should not be part of the ml library itself but rather a separate example module (either within the layer or in sansa-examples). Making it more easy to put the code/data into the places where it belongs is one of the reasons for merging all layers.


Add block/state multiproof request/subscription feature (WIP)

These endpoints are crucial for light client support, would be great if this gets incorporated. cc @seunlanlege

Closing as stale, we can re-open if/when required.


Build a REST-like interface around the routing tool

In progress, a flask draft for a server api is in place


CopterHack-2022 - Джедаи 1581

Команда инженерной школы 1581 Заявка на коптерхак

Dear friend,
Thank you for your interest in the CopterHack 2022 contest. Your application has successfully passed the experts’ review and was accepted for the second project stage of the competition, which will be held until February 28, 2022. This stage will be followed by monthly updates, lectures, and guidance from industry experts.
What's next?

Join the Telegram group with participants of the second project stage “CopterHack 2022, participants chat”.
Every month, you're expected to prepare free-form reports on the progress of your project (update) in this draft pull request. In your report please indicate your progress, what tasks have been solved. You are encouraged to attach photos and videos that demonstrate the results of the work done within the month.
Send the link to the update to the participants chat in the Telegram group.

In response to the message, you will receive feedback from the competition jury.
There is no specific date for sending the report, so you can send the report between the beginning and the end of the month.
Monthly updates become mandatory from November 2021. But as soon as you are in the telegram group, you can already send updates and receive feedback from the jury.
We are interested in helping you to complete your project as much as possible by the final stage. We wish you success!
Always in touch! Your CopterHack team.

Dear Джедаи 1581 team,
We are closing pull request for your project as you didn’t get to the final round of the competition.
You are welcome to submit your article to the clover documentation as an clover platform-based project. In order to do this please create a new pull request or make a comment to the existing one.
We wish you success in implementing your project and invite you to the next events.


Preprocess for Segmentation

Thank you for providing the data! But I've found mismatch in different modalities. Did you register images of different modalities before segmentation?

And another quesition: during training process which label is used as the GT, adc_tumor_reader1, adc_tumor_reader2, or t2_tumor_reader1?

They were resampled using DICOM header information, but not registered using more advanced methods. I believe we used reader 1 as reference during training


HBASE-24199 Procedure related metrics is not consumed in the JMX metric

Modified getApi call in Metrics implementations to expose metrics
registered in new registry as well

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
0m 30s
Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s
No case conflicting files found.

+1 :green_heart:
hbaseanti
0m  0s
Patch does not have any anti-patterns.

+1 :green_heart:
@author
0m  0s
The patch does not contain any @author tags.

_ master Compile Tests _

+0 :ok:
mvndep
0m 34s
Maven dependency ordering for branch

+1 :green_heart:
mvninstall
3m 33s
master passed

+1 :green_heart:
checkstyle
1m 21s
master passed

+1 :green_heart:
spotbugs
2m 29s
master passed

-0 :warning:
patch
2m 14s
Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 13s
Maven dependency ordering for patch

+1 :green_heart:
mvninstall
3m 18s
the patch passed

-0 :warning:
checkstyle
1m  6s
hbase-server: The patch generated 4 new + 1 unchanged - 0 fixed = 5 total (was 1)

+1 :green_heart:
whitespace
0m  0s
The patch has no whitespace issues.

+1 :green_heart:
hadoopcheck
10m 59s
Patch does not cause any errors with Hadoop 3.1.2 3.2.1.

+1 :green_heart:
spotbugs
2m 47s
the patch passed

_ Other Tests _

+1 :green_heart:
asflicense
0m 25s
The patch does not generate ASF License warnings.

34m 50s

Subsystem
Report/Notes

Docker
Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1616/1/artifact/yetus-general-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/1616

Optional Tests
dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle

uname
Linux fc559a1204fd 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
master / 512d00e75d

checkstyle
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1616/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt

Max. process+thread count
94 (vs. ulimit of 12500)

modules
C: hbase-hadoop-compat hbase-server U: .

 Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 15s
Maven dependency ordering for patch

+1 :green_heart:
mvninstall
3m 49s
the patch passed

+1 :green_heart:
compile
1m 21s
the patch passed

+1 :green_heart:
javac
1m 21s
the patch passed

+1 :green_heart:
shadedjars
5m  5s
patch has no errors when building our shaded downstream artifacts.

-0 :warning:
javadoc
0m 18s
hbase-hadoop-compat in the patch failed.

-0 :warning:
javadoc
0m 39s
hbase-server in the patch failed.

_ Other Tests _

+1 :green_heart:
unit
0m 33s
hbase-hadoop-compat in the patch passed.

+1 :green_heart:
unit
121m 46s
hbase-server in the patch passed.

148m 47s

Subsystem
Report/Notes

Docker
Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1616/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/1616

Optional Tests
javac javadoc unit shadedjars compile

uname
Linux 9b7d1eef86e8 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
master / 512d00e75d

Default Java
2020-01-14

javadoc
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1616/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-hadoop-compat.txt

javadoc
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1616/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt

javadoc
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1616/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-hadoop-compat.txt

javadoc
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1616/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt

Test Results
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1616/1/testReport/

Max. process+thread count
4625 (vs. ulimit of 12500)

modules
C: hbase-hadoop-compat hbase-server U: .

Console output
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1616/1/console

versions
git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)

Powered by
Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
0m 32s
Docker mode activated.

-0 :warning:
yetus
0m  3s
Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck

_ Prechecks _

_ master Compile Tests _

+0 :ok:
mvndep
0m 33s
Maven dependency ordering for branch

+1 :green_heart:
mvninstall
3m 32s
master passed

+1 :green_heart:
compile
1m 13s
master passed

+1 :green_heart:
shadedjars
5m  9s
branch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
0m 56s
master passed

-0 :warning:
patch
6m 26s
Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 16s
Maven dependency ordering for patch

+1 :green_heart:
mvninstall
3m 34s
the patch passed

+1 :green_heart:
compile
1m 17s
the patch passed

+1 :green_heart:
javac
1m 17s
the patch passed

+1 :green_heart:
shadedjars
5m 10s
patch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
0m 56s
the patch passed

_ Other Tests _

+1 :green_heart:
unit
0m 34s
hbase-hadoop-compat in the patch passed.

+1 :green_heart:
unit
137m 28s
hbase-server in the patch passed.

163m 27s

Subsystem
Report/Notes

Docker
Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1616/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/1616

Optional Tests
javac javadoc unit shadedjars compile

uname
Linux 079f98ec98a1 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
master / 512d00e75d

Default Java
1.8.0_232

Test Results
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1616/1/testReport/

Max. process+thread count
3750 (vs. ulimit of 12500)

modules
C: hbase-hadoop-compat hbase-server U: .

Console output
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1616/1/console

versions
git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)

Powered by
Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
1m  7s
Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s
No case conflicting files found.

+1 :green_heart:
hbaseanti
0m  0s
Patch does not have any anti-patterns.

+1 :green_heart:
@author
0m  0s
The patch does not contain any @author tags.

_ master Compile Tests _

+0 :ok:
mvndep
0m 34s
Maven dependency ordering for branch

+1 :green_heart:
mvninstall
3m 51s
master passed

+1 :green_heart:
checkstyle
1m 25s
master passed

+1 :green_heart:
spotbugs
2m 35s
master passed

-0 :warning:
patch
2m 19s
Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 11s
Maven dependency ordering for patch

+1 :green_heart:
mvninstall
3m 37s
the patch passed

+1 :green_heart:
checkstyle
1m 23s
the patch passed

+1 :green_heart:
whitespace
0m  0s
The patch has no whitespace issues.

+1 :green_heart:
hadoopcheck
12m  1s
Patch does not cause any errors with Hadoop 3.1.2 3.2.1.

+1 :green_heart:
spotbugs
2m 56s
the patch passed

_ Other Tests _

+1 :green_heart:
asflicense
0m 23s
The patch does not generate ASF License warnings.

37m 32s

Subsystem
Report/Notes

Docker
Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1616/2/artifact/yetus-general-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/1616

Optional Tests
dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle

uname
Linux 2b2313bab883 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
master / 512d00e75d

Max. process+thread count
84 (vs. ulimit of 12500)

modules
C: hbase-hadoop-compat hbase-server U: .

 Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 14s
Maven dependency ordering for patch

+1 :green_heart:
mvninstall
4m  5s
the patch passed

+1 :green_heart:
compile
1m 26s
the patch passed

+1 :green_heart:
javac
1m 26s
the patch passed

+1 :green_heart:
shadedjars
5m 28s
patch has no errors when building our shaded downstream artifacts.

-0 :warning:
javadoc
0m 23s
hbase-hadoop-compat in the patch failed.

-0 :warning:
javadoc
0m 51s
hbase-server in the patch failed.

_ Other Tests _

+1 :green_heart:
unit
0m 40s
hbase-hadoop-compat in the patch passed.

+1 :green_heart:
unit
139m 17s
hbase-server in the patch passed.

167m 46s

Subsystem
Report/Notes

Docker
Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1616/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/1616

Optional Tests
javac javadoc unit shadedjars compile

uname
Linux db12b9afc3e9 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
master / 512d00e75d

Default Java
2020-01-14

javadoc
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1616/2/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-hadoop-compat.txt

javadoc
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1616/2/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt

javadoc
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1616/2/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-hadoop-compat.txt

javadoc
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1616/2/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt

Test Results
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1616/2/testReport/

Max. process+thread count
3916 (vs. ulimit of 12500)

modules
C: hbase-hadoop-compat hbase-server U: .

 Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 15s
Maven dependency ordering for patch

+1 :green_heart:
mvninstall
3m 30s
the patch passed

+1 :green_heart:
compile
1m 12s
the patch passed

+1 :green_heart:
javac
1m 12s
the patch passed

+1 :green_heart:
shadedjars
4m 56s
patch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
0m 54s
the patch passed

_ Other Tests _

+1 :green_heart:
unit
0m 31s
hbase-hadoop-compat in the patch passed.

+1 :green_heart:
unit
149m 20s
hbase-server in the patch passed.

174m 43s

Subsystem
Report/Notes

Docker
Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1616/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/1616

Optional Tests
javac javadoc unit shadedjars compile

uname
Linux ed2eb91cc743 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
master / 512d00e75d

Default Java
1.8.0_232

Test Results
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1616/2/testReport/

Max. process+thread count
3741 (vs. ulimit of 12500)

modules
C: hbase-hadoop-compat hbase-server U: .

Console output
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1616/2/console

versions
git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)

Powered by
Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.


feat(http): support webhook message delete, update

Support two new Webhook routes: message updating and message deleting.
If the ID of a message created by a webhook is kept, it can be used with Client::update_webhook_message to update a webhook's created message by ID. The route supports updating the content and embeds.
Additionally, Client::delete_webhook_message can take the message ID to delete it, but only if the message was created by the webhook.
Checklist:

[x] webhook message deletions
[x] webhook message updates
[ ] make route generic for other webhook-related applications

Closes #640.

Marked as ready for review, if anyone else has comments. No changes since the last 2 approvals. See #634 for why this was un-drafted


Crypto.AEAD.Main.fsti: cut dependency on FStar.Pointer

So that miTLS does not depend on FStar.Pointer or anything like that.
Also, later, we might switch the implementation to using FStar.Modifies, new modifies clauses which are not necessarily related to ArrayStruct.

Please see if things work out by leaving the let in the interface, marking it as inline_for_extraction noextract then you have my 👍 to commit. Thanks!

Hmm, I'm not sure I understand your comment. I actually want to say that loc is an abstract ghost type, which is why I need to go through an auxiliary loc' type that will not be extracted.

Ok, I understand now. But this only fixes clients of the library, right? When extracting secure_api, we will still have FStar.Pointer.Base in our dependency graph.
I have the feeling that the uses of fp were only local and tentative. Is it possible to remove the uses of fp as a temporary band-aid to restore extraction and the everest build?

Yes, this is only to fix miTLS for now; secure_api/aead/Crypto.AEAD.Main.fst still depends on FStar.Pointer.
I can't judge it by myself, but if you are confident about the fact that nobody is using fp so far, then yes, I can remove it from the interface altogether.

Unless we fix it for HACL* too, this won't buy us much since everest upgrades will keep failing.
I would suggest you go ahead and comment out fp for the moment. Then, we see what happens at the next everest upgrade.


Support for notification settings

Currently, the Notifications section of the application displays all notifications, regardless of the type of content.
Prominent apps (X [Twitter], Instagram, etc.) allow you to choose which types of notifications to show, and we would like the Bluesky app to allow you to choose the content of your notifications as well.
Example:
X (Twitter)

Instagram

Please implement the feature.

I truly miss the filter from only having notifications from people I follow only..
this is a huge help in saving battery life

@pfrazee Sorry for the direct tag, but with these issues over a year old can I please ask what's going on with notification settings? With the recent influx of users, it's effectively rendered the notifications portion of my app broken since I can't filter out follower notifications

Being able to simply toggle off/on notifications of likes, follows, followbacks, replies, reposts, or quote posts would be amazing.

This doesn't directly solve this issue as you would still receive the notifications, but FYI there's a draft PR open to add a Conversations tab to the Notifications screen:

#7044


Fix POD_NAME env

What type of PR is this?

What this PR does / why we need it:
Which issue(s) this PR fixes:
Fixes #
Special notes for your reviewer:
Does this PR introduce a user-facing change?:

Tested with @lec-bit
cc @YaoZengzeng @nlgwcy


RDKTV-31831 : adding FTA_USER_MODE_MONOLITH and FTA_PCI flags

adding FTA_USER_MODE_MONOLITH and FTA_PCI flags for supporting a separate TV Factory PCI model.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


[FEATURE REQ] Cosmos: Allow accessing plain byte array / JSON string when dealing with query results.

Is your feature request related to a problem? Please describe.
As far as I understand, currently the only way to handle the query responses is to use Jackson and JavaBeans. This is not ideal, when the application might have more advanced/faster deserialisation options. These could be allowed with no overhead, if the response would allow accessing the underlying response body as a string or byte array.
In our case, we are using SDK from Scala, where there are several alternative JSON deserialisation options.
Describe the solution you'd like
Allow direct access to the underlying response JSON string / byte array.
Describe alternatives you've considered
JavaBeans and POJOs - not ideal for Scala as they are inherently mutable.
Information Checklist
Kindly make sure that you have added all the following information above and checkoff the required fields otherwise we will treat the issuer as an incomplete report

[X] Description Added
[X] Expected solution specified

@mbhaskar  - please take a look at this issue.

@mbhaskar  - please take a look at this issue.

Any update on this? 😃 @mbhaskar @kushagraThapar

@mpartan  - what is the SDK that you are using ?

@kushagraThapar
I'm experiencing similar issues with azure-cosmos 4.10, I guess Cosmos is mentioned in the title but not in the description.

@mpartan - I see, @mbhaskar  - do we have this in our plan for exposing byte array results for query execution ?

This is in the backlog, but not planned at the moment. Will update the incident if this changes.

I guess this is still open and not planned? Just makes it a bit more hard to use the library which is really useful but if you don't want to use POJOs 🤔

It is possible to use ObjectNode (com.fasterxml.jackson.databind.node.ObjectNode) thought - wouldn't that address the request above?
cosmosAsyncContainer.queryItems(sqlSpec, options, classOf[ObjectNode])

That actually works like a charm for this use case. Thanks a lot @FabianMeiswinkel 😃


auto update does not always update...

When i did preprocessing/processing for a few studies, qiita showed for a long time the "Waiting on Preprocessing..." page. Maybe because i closed my mac/connected to a different wifi, it does not automatically update.
if i try to reload the page, i get "ERROR: CODE 405!"
then when i press back (on the browser), it showed me "not preprocessed". However, if i went to the "edit" study information and cancel, it now showed as preprocessed...

The specific example has been solved so closing, but if there are other cases where the "auto-update" doesn't work please open independent issues. Thanks!


Change Unique Names to Names

There are no instances of the word "unique" in any of the published Help or About pages on test3.
Looking at the Records section, the word "unique" appears to have been removed from the displays already.

Unless there are other places to look, then this issue is fixed.


Pass response into calculate blocks.

Brings such blocks into line with other types of blocks e.g. validate,
define_predicate, next_node.

Reduces duplication and makes code slightly terser.

I've only declared the block argument where it is actually needed i.e. there
are still many calculate blocks which do not have a response argument
declared.

This may seem like a small change, but I think it will make some other
refactoring easier.

Looks good to me :+1:

@tadast: Thanks. I've just merged it.


Linter not working

Describe the bug
I installed the linter to my VSC and followed the instructions, but is not 'marking' where the errors are happening in a **FREE source and I think it's because the JSON when written into the iSeries is not recognizing the {} like this:
This is when I open the linter on VSC

This is how it's seen in the pub400 source file

Environment (please complete the following information):

Extension version 0.3.0 of RPLE language tools
IBM i OS version V7R4M0

Could it be that I need to map some values for these chars?
thank you
Ramon Sandoval

Looks valid to me.
Did you enable it correctly in the VS Code settings?
Control+Comma -> RPGLE language tools -> enable the linter


Need to create Component (dynamic popover, modals, calendar, etc...)

Need to create Component (dynamic popover, modals, calendar, etc...)

@ismail280489 I am not sure it's the purpose of tailwind.

@ismail280489 Steve and Adam are working on a component library: https://twitter.com/steveschoger/status/1126938094312271872
But it will likely be published as a separate project as it's outside the scope of Tailwindcss.


Enhanced dcp error checking

Error checking for dcp has been enhanced to be more thorough and report specific error codes depending on the failure.
Currently, the new error codes are contained to dcp.c.
In the future, it is possible that we could also use error codes in the lower level libraries, but properly propagating these errors back to dcp.c is a more difficult undertaking which I will save for a later time, as the need arises.
I ran the DAOS functional tests (in progress) against these changes to verify basic functionality for:

DAOS -> DAOS
DAOS -> POSIX
POSIX -> DAOS
POSIX -> POSIX

New file: common/mfu_errors.h

Please review this for convention
Contains some error codes for dcp and DAOS

Within the range of 0-255

Can be expanded in the future as needed.

Modified: dcp.c

Use the new error codes in mfu_errors.h
Better error checking for DAOS

Check for required/dependent args
Early exit if dfs_mount fails

Print a final message after summary if there were any errors during copy

Closes #363

You may consider "squashing" these commits, since a few of the commits were work-in-progress.

@adammoody
Just a friendly reminder about this PR. (I'm sure you're probably busy)
Really, I would mostly like your input about common/mfu_errors.h in terms of convention. I.e. error ranges, naming conventions.
I chose to go with the 0-255 range because this is the most compatible range for reading error codes from a returned process.

Actually, this isn't quite ready to be merged.
I realize now that I need to rethink the way the errors are being returned, due to some limitations.
Instead of dcp.c returning the rc for different errors, it will be more compatible and flexible to return 0 if everything checks out, and 1 otherwise (how it was originally). But this time, the change will be that the logged messages will include the rc for different errors.
This is more similar to how it is done in DAOS, and I think now I see why this approach was taken.

Thanks @daltonbohning .  Got some time to take a look at this.  Looks good to me so far, but it sounds like you're wondering about something.  What are you looking at?

Thanks @daltonbohning . Got some time to take a look at this. Looks good to me so far, but it sounds like you're wondering about something. What are you looking at?

I ran into as issue with the way I was using the error codes. Initially, I went with a very simplified approach of introducing errors in the range of 0-255, since this is most compatible with getting the return code from the various ways a process can be called (command line, fork(), etc.)
But there were some issues with this approach:

Many MFU functions return negative values, 0-255 wouldn't strictly be a drop-in replacement.
Working with multi processes means that sometimes there just isn't easily a single return code for dcp.
0-255 isn't a very large range, so it's difficult to designate ranges to different modules.

My latest changes have refactored this, and I'll explain how in more detail once those changes are ready for PR.

My branch deviated a bit from my previous PR, I had many small commits, and git wasn't cooperating, so I'm just creating a new PR entirely.


Cannot use the action in a workflow

Describe the bug
Hi there, first of all thanks for the efforts in developing this library. We are trying to include this step in our workflow, but we are facing an issue related to the docker image. It cannot pull it:

Might the issue be related to the actual image defined in action.yaml ?
To Reproduce
Steps to reproduce the behavior:

follow either the readme example or the snippet as reported in the marketplace

Example configuration:
     - id: dbt_checks
        uses<EMAIL_ADDRESS>        env:
          [ ... ]
        with:
          args: run --files ${{ steps.file_changes.outputs.files }}

Version:
v1.0.0

Thank you for posting this. Will look on that

i also faced the same issue - also how do i use the hook syntax here?
can i do the following to run dbt-compile hook in my github action?
with:
         args: dbt-compile --model-prefix + --cmd-flags ++project-dir dbt_pipelines --

I'm having the same issue

Hi! I am also having this issue

Thank you all! This will be fixed in the new version.


[FEATURE REQUEST] Permanent output folder

Please add settings to save output folder as default. Now it is asked every time where to store files.
Possible solution:

(This is program called TrimTo)

With this update, we have added an option to save to a specified folder.

When this option is specified, the file is saved to the specified location permanently and the save dialog is not displayed.
If you still have any problems, please comment here.

Thanks, it is much better now.
One more thing: temporary wave files, they stay where the first mp3 was:

I think it would be better to use some temp folder for them or just delete them (as option) after WAVE to ATRAC encode.

It also says every time, that folder exists, why? I know it is there, because I chose it in settings as subfolder. But it doesn't say anything if converted at3 files are already there.

If input file's path has some cyrillic symbols, output files are named as current date.

We have made code fixes for a series of problems presented in v1.27 update.
Please comment again if you have problems.


Error connecting to the server when clicking on Recent entries

When clicking on the Recent entries in Toggl Button popup, this error shows up.

Operation system: macOS Sierra
Extension version: Chrome (Version 58.0.3029.110 (64-bit)) - TogglButton 1.1.6.

Expected behavior
Since Recent entries isn't clickable, I'd expect not to have any errors showing up when clicking on the field.
Actual behavior
As soon as the field is clicked, the "Error connecting to the server" shows up.
Steps to reproduce the behavior
Click on Recent entries in Toggl popup.
Screencast: https://drive.google.com/file/d/0B2woEdzhKYh9M3BPVExfbm9WRkk/view
Case: #470196

I can confirm this is indeed happening. Clicking the "recent entries" header produces the following error:


chore(deps): upgrade dependencies

Upgrades project dependencies. See details in workflow run.

Automatically created by projen via the "upgrade-k8s-25-main" workflow

⚪ Backport skipped
The pull request was not backported as there were no branches to backport to. If this is a mistake, please apply the desired version labels or run the backport tool manually.
Manual backport
To create the backport manually run:
backport --pr 2067

Questions ?
Please refer to the Backport tool documentation


RediSearch and RedisJson checks do not fire against redis:latest

When testing with make test_oss we get TypeError: 'NoneType' object is not subscriptable with Traceback in redis-py not in redis-om:
tests_sync/test_json_model.py:29: in <module>
    if not has_redis_json():
redis_om/checks.py:17: in has_redis_json
    command_exists = check_for_command(conn, "json.set")
redis_om/checks.py:9: in check_for_command
    cmd_info = conn.execute_command("command", "info", cmd)
../../.cache/pypoetry/virtualenvs/redis-om-2-PyqfHp-py3.8/lib/python3.8/site-packages/redis/client.py:1218: in execute_command
    return conn.retry.call_with_retry(
../../.cache/pypoetry/virtualenvs/redis-om-2-PyqfHp-py3.8/lib/python3.8/site-packages/redis/retry.py:45: in call_with_retry
    return do()
../../.cache/pypoetry/virtualenvs/redis-om-2-PyqfHp-py3.8/lib/python3.8/site-packages/redis/client.py:1219: in <lambda>
    lambda: self._send_command_parse_response(
../../.cache/pypoetry/virtualenvs/redis-om-2-PyqfHp-py3.8/lib/python3.8/site-packages/redis/client.py:1195: in _send_command_parse_response
    return self.parse_response(conn, command_name, **options)
../../.cache/pypoetry/virtualenvs/redis-om-2-PyqfHp-py3.8/lib/python3.8/site-packages/redis/client.py:1240: in parse_response
    return self.response_callbacks[command_name](response, **options)
../../.cache/pypoetry/virtualenvs/redis-om-2-PyqfHp-py3.8/lib/python3.8/site-packages/redis/client.py:553: in parse_command
    cmd_name = str_if_bytes(command[0])
    TypeError: 'NoneType' object is not subscriptable

You can replicate this against redis:latest with
import redis

redis.StrictRedis().execute_command("command", "info", "json.set")

There is a problem in redis-py with parsing the response from redis. (It's nill)
Maybe it's related to: https://github.com/redis/redis-py/blob/09a52dba48221353eafa8188d73ab97e8f4ccc49/redis/commands/core.py#L740-L743
    def command_info(self, **kwargs) -> None:
        raise NotImplementedError(
            "COMMAND INFO is intentionally not implemented in the client."
        )

So when I try to use redis-om against redis:latest:
from redis_om import HashModel, Migrator, get_redis_connection, Field

class TestModel(HashModel):
    test_field: int = Field(index=True)

    class Meta:
        database = get_redis_connection(port=6381)  # redis:latest

Migrator().run()
TestModel.find()

I get this:
Traceback (most recent call last):
  File "/home/lamar/PycharmProjects/redis-om-python-flask-skeleton-app/check.py", line 12, in <module>
    TestModel.find()
  File "/home/lamar/.env/redis-om-python-flask-skeleton-app/lib/python3.8/site-packages/redis_om/model/model.py", line 1168, in find
    return FindQuery(expressions=expressions, model=cls)
  File "/home/lamar/.env/redis-om-python-flask-skeleton-app/lib/python3.8/site-packages/redis_om/model/model.py", line 347, in __init__
    if not has_redisearch(model.db()):
  File "/home/lamar/.env/redis-om-python-flask-skeleton-app/lib/python3.8/site-packages/redis_om/checks.py", line 25, in has_redisearch
    if has_redis_json(conn):
  File "/home/lamar/.env/redis-om-python-flask-skeleton-app/lib/python3.8/site-packages/redis_om/checks.py", line 17, in has_redis_json
    command_exists = check_for_command(conn, "json.set")
...
  File "/home/lamar/.env/redis-om-python-flask-skeleton-app/lib/python3.8/site-packages/redis/client.py", line 530, in parse_command
    cmd_name = str_if_bytes(command[0])
TypeError: 'NoneType' object is not subscriptable

instead of https://github.com/redis/redis-om-python/blob/00ed537fcaf43d93ea26e0332d5cb2f1a4c1c4a1/aredis_om/model/model.py#L347-L352
So maybe we can test with
try:
    db = redis.StrictRedis()
    db.json().set("tmp_key", ".", {})
    db.delete("tmp_key")
except redis.ResponseError:
    raise RedisModelError("...")

here https://github.com/redis/redis-om-python/blob/00ed537fcaf43d93ea26e0332d5cb2f1a4c1c4a1/aredis_om/checks.py#L14 and something similar for RediSearch here https://github.com/redis/redis-om-python/blob/00ed537fcaf43d93ea26e0332d5cb2f1a4c1c4a1/aredis_om/checks.py#L22

And what if user uses async API:
Model would get async conn, then the check that should be sync(because in __init__) gets async conn...
Either we create async and sync Redis instance in metaclass. (which is bad?)
Or maybe check might go to the place where async and sync is a part of protocol instead of __init__ that is always sync
like execute() in FindQuery or save(), get() in JsonModel


🐛 Bug Report: Premature close error

👟 Reproduction steps
When downloading large file (couple of GB)  the download fails after few minutes.
The docker throws:
[Nest] 47  - 12/08/2023, 12:47:07 PM   ERROR [ExpressAdapter] Premature close
Error [ERR_STREAM_PREMATURE_CLOSE]: Premature close
    at ServerResponse.onclose (node:internal/streams/end-of-stream:159:30)
    at ServerResponse.emit (node:events:526:35)
    at emitCloseNT (node:_http_server:1023:10)
    at Socket.onServerResponseClose (node:_http_server:278:5)
    at Socket.emit (node:events:526:35)
    at TCP.<anonymous> (node:net:337:12)

Sometimes this happens several times before the file if finally downloaded.
👍 Expected behavior
Successful download regardless the file size.
👎 Actual Behavior
As described above.
🌐 Browser
Firefox, Chromium, Brave

That's strange I've never seen this error before.
A few questions to debug this further:
Can you try it with another file?
Does the error occur exactly at the same position (e.g at 2.3GB) ?

Ok. I've tested it a little bit more.

no, it doesn't happen at the same position (I am not hitting any limits on reverse proxy either)
it doesn't happen every time (it is rather an exception than a norm)
it does happen with different files (but big ones - few GB)
it happens with different external PCs

Thanks for your detailed research. It's difficult to debug this if I can't reproduce it and I can't really find any useful information on the internet. I've created a Docker image that uses the LTS version of Nodejs, this might fix the issue. Could you try to use the image stonith404/pingvin-share:development temporarily?

sure thing. Will do more tests with LTS

A bit more investigation:
The LTS throws same errors:
[Nest] 41  - 12/14/2023, 12:53:24 AM   ERROR [ExpressAdapter] Premature close
Error [ERR_STREAM_PREMATURE_CLOSE]: Premature close
    at ServerResponse.onclose (node:internal/streams/end-of-stream:159:30)
    at ServerResponse.emit (node:events:526:35)
    at emitCloseNT (node:_http_server:1023:10)
    at Socket.onServerResponseClose (node:_http_server:278:5)
    at Socket.emit (node:events:526:35)
    at TCP.<anonymous> (node:net:337:12)
[Nest] 41  - 12/14/2023, 4:24:15 AM   ERROR [ExpressAdapter] Premature close
Error [ERR_STREAM_PREMATURE_CLOSE]: Premature close
    at ServerResponse.onclose (node:internal/streams/end-of-stream:159:30)
    at ServerResponse.emit (node:events:526:35)
    at emitCloseNT (node:_http_server:1023:10)
    at Socket.onServerResponseClose (node:_http_server:278:5)
    at Socket.emit (node:events:526:35)
    at TCP.<anonymous> (node:net:337:12)

However, I have tested different browsers this time.

Firefox: Errors out with above. Sometimes after a few seconds, sometimes after a few minutes. Occasionally download is successful.
Librewolf: Same as Firefox. (No mysteries here - it is Firefox too.)
Chromium: No problems.
Brave: No problems.
All tests have been done from remote Linux machines running Linux Mint, Manjaro or Arch Linux.

Hm okay, so it only occurs with Firefox. I'll try to investigate the further next week.

Hello,
I'd like to add my 2c to this - I'm having the same problem, downloading an 8.8GB file from my own server. It goes well until it hits 1-1.2GB, then drops the download like a hot potato.
[Nest] 41  - 12/19/2023, 10:06:06 AM   ERROR [ExpressAdapter] Premature close
Error [ERR_STREAM_PREMATURE_CLOSE]: Premature close
at ServerResponse.onclose (node:internal/streams/end-of-stream:159:30)
at ServerResponse.emit (node:events:526:35)
at emitCloseNT (node:_http_server:1023:10)
at Socket.onServerResponseClose (node:_http_server:278:5)
at Socket.emit (node:events:526:35)
at TCP. (node:net:337:12)
Firefox on Linux Mint 21.2 and on Windows 10 22H2 both exhibit the same behaviour. I've not tried with any Chromium based browsers.

Strange, I couldn't find anything useful on the internet.
I'm not a Firefox user but did you experience this bug with other downloads from other websites?

I doubt it would be related to Firefox. It is rather NestJS issue. Other projects reports similar problems...
Could we log some more to be able to pinpoint the issue?
https://github.com/nestjs/nest/issues/12283

Hey!
I may have found something while fiddling around.
The site doesn't timeout or cancel the download on Firefox while on HTTP, although the issue remains via HTTPS...
sadly it doesn't mean Pingvin container logs are fault-free, the same fault remains al be it less often.
The next thing I tested was accessing the site via HTTPS and the App URL is set to host ip and port.
Unfortunately, it resulted in the same fault where the download was canceled.
Still kinda pointing back to NestJs as @plague-doctor said.
Anyway is there a workaround or known working version?

If I have some time, I'll create a bug report in the NestJS repo with a small reproduction. Besides that I'm not really sure what I can do.
Did the download of large files work before on Firefox?

Did the download of large files work before on Firefox?

yes, it did. I can't download files bigger than 100mb while before a 2 GB file wasn't an issue.

@sebasdt Can you remember approximately when downloading large files have stopped working?

The annoying part is I do not know at what version pingvin broke. (My backups don't go back that far and the last monthly december was just purged.)
The last known working state was back in 9 Dec, Although it could be later.

This probably doesn't help much, but I tested it out of interest. I uploaded a 5GB file, then tested the following scenarions:

Download with Chrome -> no issue
Download with Firefox after downloading the same file with Chrome -> no issue
Download with Firefox only after closing Chrome -> Premature close (however, the download continues in case it's over unsecure HTTP)
Download with Firefox and Chrome simultaniously with Firefox starting first -> Premature close and fail to download on Chrome even over HTTP.
Resure download on Chrome after cancelling the simultanious download on Firefox -> no further issues.

I don't know if this helps in any way and if it's expected behaviour for this particular bug since I don't know how nestjs works, but thought I'd still post my findings.

I've just tested older versions of pingvin-share, and I can tell that the issue starts happening with v0.21.1, while v0.21.0 seems to be working as expected.

Thanks @axiomen for your observations!
Could you guys test version v0.21.0 too? There aren't any db migrations since then so you only have to change the Docker image tag to v0.21.0 .

I've tried all versions from and including 0.21.0 through latest and they all exhibit the same behaviour, i.e. dropping the download at around 1GB data transferred.

That is strange. I have tested multiple large files, with single and multiple files per share and there was no issue on v0.21.0, while on v0.21.1 it happened immediatelly. I can re-test later today on another machine.

I've tried all versions from and including 0.21.0 through latest and they all exhibit the same behaviour, i.e. dropping the download at around 1GB data transferred.

I've only tried 0.21.0 and am still only able to download on http although it with some errors.
Man, what is annoying is that we all have a common problem but the symptoms are all kinda different.

Yeah.. I've poked at this for a bit to see if my environment was the culprit. I'm testing pingvin-share as a docker container on unRAID, and have it sitting behind an Nginx Proxy Manager reverse proxy which handles the TLS termination and passes the request through. NPM is also a docker container.
Thinking that was my issue, I spun up a Debian server and set up pingvin-share there, still routing it through NPM, and still the same problem. I then exposed the Debian host for pingvin-share directly on the internet - no dice.
I still saw the same errors in the logs and concluded that the environment is not the issue; something within the pingvin-share container must be wrong, but that's where my debugging skills end.
As I stated in my previous entry here, I've tried all versions from and including 0.21.0 through current, and they all do the same thing. I'll give 0.19.x a go and see if that has the same issues and report back.

Tested on my second machine on v0.21.0, and the error is back. I have no idea how on my first machine the error consistently happens on v0.21.1+, but on v0.21.0 doesn't even after testing multiple scenarios with 1GB+ files. Both are containers spun on docker. Honestly at a loss here.

Thank you guys for helping to debug this. It's frustrating as I can't really do much because it must be caused by the StreamableFile class of NestJS. And on the internet there is nothing that is related to this issue. I just upgraded all packages to the newest version, would you mind to test the stonith404/pingvin-share:development image?
If this doesn't work, I have to create an issue in the NestJS repository.

No Premature close error on my first machine, but I did get another error upon finishing the download.

On my second machine, Premature close error is still present even on the dev image.

On the development tag version, I was able to download one 3.5GB file and one 5.1GB file, but the zip file containing the two, with a total size of 8.8GB, failed after 1,138,822KB (1.13GB).
Retrying the first 3.5GB, it failed at 1.9GB:
[Nest] 41  - 01/11/2024, 2:35:23 PM   ERROR [StreamableFile] Premature close
Error [ERR_STREAM_PREMATURE_CLOSE]: Premature close
at ServerResponse.onclose (node:internal/streams/end-of-stream:159:30)
at ServerResponse.emit (node:events:526:35)
at emitCloseNT (node:_http_server:1023:10)
at Socket.onServerResponseClose (node:_http_server:278:5)
at Socket.emit (node:events:526:35)
at TCP. (node:net:337:12)
I've noticed a few other things as well, but I'll open separate tickets for those.

Okay thanks. I think the only option now is to report it to NestJS.

I did some more testing, and noticed that on Chrome the error happens if multiple simulatneous downloads from Pingvin Share are running. For instance, if I'm downloading three files from the same share (file1, file2, and zip of file1 and file2), I'll get Premature close error, but not all downloads are cut off (file1 and file2 stop downloading, while the zip continues to download).

So I debugged this issue a bit further and I found out that the backend logs Premature close error if the the browser cancels the download. My guess is that the issue occurs when the network is unstable and the client devices gets disconnected for a short time.
For example I'm using a VPN and I was able to reproduce this issue with the VPN but if I disable the VPN the download works. My VPN isn't really reliable because sometimes if I visit a website I get the error "Network has changed".
Could this be the issue or are you guys sure that your connection on the server and client are stable?

I tested it with 2 friends, both with fiber connection and near 5km radius. I'm using pingvin-share behind SWAG reverse-proxy. Download All button seems to throw Premature Close error more often on a share with multiple files, than downloading the files separately. Downloading single files have been giving variable success rate, sometimes it downloads it fully, sometimes it fails the file download with Premature close error.

@fosq Thanks for taking the time for helping to debug this. Would you mind to test if the issue also occurs when you download the file over your local network, without a proxy in front?

So, I just rewrote the download endpoint completely and replaced the StreamableFile with the download method of Express. I've tested the download with a VPN, over the local network, over the internet, and concurrent downloads. Would you mind testing the stonith404/pingvin-share:development image again? 🙏

Tested on two hosts. Unfortunately, still the same probelm.
Host #1: remote over HTTPS, three simultaneous downloads, all failed at a certain point.

Host #2: local network over HTTP, three (different) simultaneous downloads, all but one failed.

Tested on Firefox.

@axiomen Thanks for your research. I made some researches too and it's extremely strange. You're right that the issue still occurs because I only tested it with a chromium based browser. Downloading files in a chromium based browser concurrently works with no problems for me. Downloading one file at the same time in Firefox works for me too, but not multiple files at the same time. If I start the download in Chrome first and then start the download in Firefox, the Firefox download fails, but if I start the download in Firefox first the files get downloaded successfully in both browsers.

it seems that this is more of an Nginx issue

You said that you tested the download over the local network, was there Nginx in front too?

Sorry, I was in a rush, so I didn't elaborate on my comment. From my understanding, Docker image comes with bundled Nginx, correct? If not, disregard that.
This is what I've noticed the process usually goes like: Firefox shows the download is still in progress, even though the transfer shows no activity anymore (i.e. 0 MB/s) -> Network Monitor reports "Blocked" -> Pingvin share throws a "Request aborted" / "Premature close" error.

Okay, further investigation has led me to something interesting (again, might not be relted at all). There is a mismatch between reported file size on Pingvin Share interface vs the one that the browsers report (valid for both Chrome and Firefox).

Which has led me to this issue. Relevant parts:

I've looked at the response headers when the file's downloading and all is in order, until I get this NS_ERROR_NET_PARTIAL_TRANSFER message and it fails.

I have an issue with FireFox where it can't download generated private file PDFs, yet Chrome has no such issue.

The real issue is the mismatch between the file's real size on the server and the size Drupal stores in the file_managed table, filesize column.

I did check the content-length in the HTTP Headers when the transferred fails, which reports the correct size, so I'm not really sure where the mismatch happens.

@stonith404 I got it working, but feel like it's more of a workaround?
Mounted nginx.conf to the host and added "proxy_buffering off", so it looks like this:
location /api {
            proxy_buffering off;
            proxy_pass http://localhost:8080;
            proxy_set_header X-Forwarded-Host $host:$server_port;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $scheme;
        }

I'm not sure what exactly is going on, and from my understanding this is not the best practice, but it works now.

Docker image comes with bundled Nginx, correct?

It seems like it yes. Even the bare install without docker uses Nginx.
Maybe it's possible to temporarily replace Nginx with Caddy or Apache to see if Nginx is being a dufus?

I just replaced Nginx with Caddy in the latest development image. For me the error strangely still occurs if I disable proxy buffering or use Caddy. Would you mind to test the image again?
@axiomen Is the error completely gone if you disable proxy buffering?

I managed to do a little bit of testing over HTTPS (latest image, slightly adjusted nginx on my host), but on my phone with Firefox Android using mobile data. Ran one test with two concurrent downloads with no errors. I'm starting to think there's something up with the firewall on the network I've done all (or most of) my testings on so far. Once the downloads fail, I noticed I cannot access anything on my domain on this network for a while, however everything remains accessible through my phone data.

I did a quick test, and it's not only on Firefox under the aformentioned conditions. Chrome is also affected.
A rather strange thing I've noticed is my domain becomes unreachable for a minute or so after the downloads fail. Containers keep running, so does swag (nginx), and there are no error logs indicating something's wrong. For good measure, I tested file transfer over my Nextcloud instance (in case there's some domain hosting issue), and that worked fine.

Hi!
Well I just gave the latest dev build a look, It seems like I got a different error..
That might just be because im on a dev built.

Thanks @axiomen and @sebasdt. This issue is really difficult the fix, because I'm still not sure where the problem is. But I'll keep you up to date.

I looked a bit more through the swag logs, including fail2ban module, and also Network stats in Firefox, and this popped out for me.

As you can see, I'm getting quite a few 401 errors due to "Slow server response time" while downloading, which ultimately ends with fail2ban temporarily blocking the IP address, and failing the transfer.

@axiomen Thanks for participating in #401 let's discuss related things there, instead of here.
Did disabling the fail2ban jail really fix the issue? I'm not using fail2ban and the issue still occurs. I still have no clue where this issue occurs as the symptoms are different for every user. For example you said that the issue also occurs in Chrome but I made many tests and it never failed for me in Chrome.

@stonith404 I've run multiple tests, upload + download of two 4-5 GB files, and all tests passed. That's on the dev image with Caddy. As long as I disable nginx-unauthorized fail2ban jail OR log in while downloading, the dev image works as expected.

I did stumble on another issue with this image though, where editing a share and attempting to add a file results in an upload error and the share disappearing completely on the web interface. The share is still present on the disk, however. This is the error:
[Nest] 49  - 02/12/2024, 10:08:38 AM   ERROR [ExceptionsHandler] request entity too large
PayloadTooLargeError: request entity too large
    at readStream (/opt/app/backend/node_modules/raw-body/index.js:163:17)
    at getRawBody (/opt/app/backend/node_modules/raw-body/index.js:116:12)
    at read (/opt/app/backend/node_modules/body-parser/lib/read.js:79:3)
    at rawParser (/opt/app/backend/node_modules/body-parser/lib/types/raw.js:81:5)
    at Layer.handle [as handle_request] (/opt/app/backend/node_modules/express/lib/router/layer.js:95:5)
    at trim_prefix (/opt/app/backend/node_modules/express/lib/router/index.js:328:13)
    at /opt/app/backend/node_modules/express/lib/router/index.js:286:9
    at Function.process_params (/opt/app/backend/node_modules/express/lib/router/index.js:346:12)
    at next (/opt/app/backend/node_modules/express/lib/router/index.js:280:10)
    at expressInit (/opt/app/backend/node_modules/express/lib/middleware/init.js:40:5)

@axiomen Okay thanks for your feedback. I'll test it ASAP.

@axiomen I don't have fail2ban and the error still occurs when I'm signed in :/ Did you maybe change something else?
Regarding the PayloadTooLargeError, this is fortunately easy to fix and only caused on the dev image as I have increased the chunk size for testing purposes.

That's really bizarre. I haven't changed anything else. This is my nginx configuration file, if it helps in any way:
## Version 2023/05/31
# make sure that your pingvin-share container is named pingvin-share
# make sure that your dns has a cname set for pingvin-share

server {
    listen 443 ssl http2;
    listen [::]:443 ssl http2;

    server_name REDACTED.*;

    include /config/nginx/ssl.conf;

    client_max_body_size 0;

    location / {
        include /config/nginx/proxy.conf;
        include /config/nginx/resolver.conf;
        set $upstream_app REDACTED;
        set $upstream_port 3000;
        set $upstream_proto http;
        proxy_pass $upstream_proto://$upstream_app:$upstream_port;

    }

}

The bug #401 should be resolved in the most recent 'development' image. Would you mind to try if you still don't encounter the "premature error" anymore?

@stonith404 I'm getting 502 Bad Gateway with the latest dev image.

Deleted and redownload the image, it's working now. I'll test for the 401 error as soon as I get to my PC.

@stonith404 removed and pulled the newest dev image, tested a 24GB file transfer, and haven't seen any errors. Neither the 401, nor the Premature.

@stonith404 Thanks for this quick version
I've just tested it with an 8GB archive, several 2GB video files and no crashes.
But still a very high use of RAM and even the transfer finished downloading I keep this use of RAM.
I have to reboot my VM to get it back to normal.
This happens to me on upload and download.

I'm guessing whatever you run your VM on can't tell apart free vs cached memory, so it reports that most of your memory is taken, while in reality 84% is cached (as shown with the free command). The cached part is probably the file you're downloading cached directly in the memory.

This should be fixed in v0.22.2. Feel free to reopen the issue if you encounter the bug again.

I'm reopening this as @axiomen has reported that the issue still exists.

@stonith404 When implementing streaming of files with experienced users (getting a ranged file) from Minio using the S3 SDK client and obtaining a readable stream, I utilize 'StreamableFile' to pipe the client. Initially, it works fine. However, when I attempt to tassel the video, an error "ERROR [StreamableFile] Premature close" occurs. I've tested this multiple times, and the error consistently appears, indicating a potential interruption in the stream.

This bug is really strange. @axiomen has reported that it doesn't occur anymore so I'll close it again... If you notice the error again please comment here.

It's just like a herobrine you try to fix It/remove the bug and it just pops up again.😜

@axiomen  On a remote network, behind an Nginx reverse proxy

The weird thing is that it's always the first try that fails, and on the second attempt, it works just fine.

I am facing the same issue, and in my case, I was trying to stream pdf files. I tested with different file size and found the Premature close issue only for files whose size is above 5MB. I tested on my Windows machine along with Edge, Firefox and Chromium browsers. Followings are the test cases:
Test Case 01:
File type: PDF, File Size: 1MB
Result: Pass.
Test Case 02:
File type: PDF, File Size: 5MB
Result: Pass.
Test Case 03:
File type: PDF, File Size: 10MB
Result: Failed. ( ERROR [StreamableFile] Premature close)
Test Case 04:
File type: PDF, File SIze: 7MB
Result: Failed. ( ERROR [StreamableFile] Premature close)

Also having this issue. Just spent a couple hours thinking it must be the config of my reverse proxy in front of Pingvin, but now I'm not so sure.
Some details:

500MB junk file download fails 100% of the time with:

Firefox

500MB junk file download succeeds 100% of the time with:

Chrome
curl
wget

Internet connection is definitely a bit flaky, but nothing that's noticeable "during normal usage"
Cloudflare DNS with requests proxied through Cloudflare
NGINX reverse proxy sitting in front of Pingvin. Config:

server {
    listen 443 ssl http2;
    server_name [REDACTED];

    include /etc/nginx/conf.d/snippets/ssl.conf;

    set $upstream http://${PINGVIN_HOST}:${PINGVIN_PORT};

    client_max_body_size      0;
    proxy_buffering           off;
    proxy_request_buffering   off;
    sendfile                  off;

    location / {
        proxy_set_header     Host $host;
        proxy_set_header     X-Real-IP $remote_addr;
        proxy_set_header     X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header     X-Forwarded-Proto $scheme;
        proxy_pass           $upstream;
    }

    location /robots.txt {
        add_header Content-Type text/plain;
        return 200 "User-agent: *\nDisallow: /\n";
    }
}

The error is as follows:

pingvin  | {"level":"error","ts":1720589993.0274456,"logger":"http.handlers.reverse_proxy","msg":"aborting with incomplete response","upstream":"localhost:8080","duration":0.046721392,"request":{"remote_ip":"[REDACTED]","remote_port":"39394","client_ip":"[REDACTED]","proto":"HTTP/1.0","method":"GET","host":"[REDACTED]","uri":"/api/shares/IwOTM2O/files/6b2d8478-7eba-43ef-8cbb-08a3759c75a0","headers":{"X-Real-Ip":["[REDACTED]"],"Accept-Encoding":["gzip, br"],"Accept-Language":["en-US,en;q=0.5"],"Cdn-Loop":["cloudflare"],"Upgrade-Insecure-Requests":["1"],"X-Forwarded-For":["[REDACTED]"],"Sec-Fetch-Dest":["empty"],"Cf-Ray":["8a0e35c4cd811000-LAX"],"Sec-Fetch-Site":["same-origin"],"Cf-Connecting-Ip":["[REDACTED]"],"Sec-Fetch-Mode":["same-origin"],"Priority":["u=4"],"X-Forwarded-Host":["[REDACTED]:{server_port}"],"User-Agent":["Mozilla/5.0 (X11; Linux x86_64; rv:127.0) Gecko/20100101 Firefox/127.0"],"Referer":["[REDACTED]"],"Cf-Ipcountry":["US"],"Cookie":[],"X-Forwarded-Proto":["http"],"Accept":["text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8"],"Cf-Visitor":["{\"scheme\":\"https\"}"]}},"error":"writing: write tcp [REDACTED]->[REDACTED]: write: connection reset by peer"}
pingvin  | [Nest] 38  - 07/10/2024, 5:39:53 AM   ERROR [StreamableFile] Premature close
pingvin  | Error [ERR_STREAM_PREMATURE_CLOSE]: Premature close
pingvin  |     at ServerResponse.onclose (node:internal/streams/end-of-stream:159:30)
pingvin  |     at ServerResponse.emit (node:events:531:35)
pingvin  |     at emitCloseNT (node:_http_server:1020:10)
pingvin  |     at Socket.onServerResponseClose (node:_http_server:278:5)
pingvin  |     at Socket.emit (node:events:531:35)
pingvin  |     at TCP.<anonymous> (node:net:338:12)

Same issue here with v1.2.0.
Downloads stop before 1GB with Firefox 131.0.2, but work with Chromium 129.
Error log on server side:
[Nest] 48  - 10/16/2024, 5:18:19 PM   ERROR [StreamableFile] Premature close
Error [ERR_STREAM_PREMATURE_CLOSE]: Premature close
    at ServerResponse.onclose (node:internal/streams/end-of-stream:159:30)
    at ServerResponse.emit (node:events:531:35)
    at emitCloseNT (node:_http_server:1021:10)
    at Socket.onServerResponseClose (node:_http_server:277:5)
    at Socket.emit (node:events:531:35)
    at TCP.<anonymous> (node:net:339:12)

In my case, the bytes go through a Traefik reverse-proxy.
Everything on my local (and reliable) network

Hello, some time ago I had again the problem with the last version of Pingvin and thanks to ChatGPT I modified my proxy.conf file via Swag with what he suggested and it's been working perfectly ever since.
Désactiver le buffering proxy
proxy_buffering off;
proxy_request_buffering off;

# Désactiver la redirection proxy
proxy_redirect off;

# Timeout augmenté pour les fichiers volumineux
proxy_connect_timeout 600;
proxy_send_timeout 600;
proxy_read_timeout 600;
send_timeout 600;

@babsz02 you imply that it might come from the behavior of your frontal reverse-proxy?
In any case, I'd be happy to help debugging this issue if I can (I won't be able to help on the typescript code, unfortunately)

Indeed, it comes from my riverse proxy in my case Swag proxy.
Since I made the changes mentioned above, everything's working again.

@babsz02 If you scroll through the comments, you'll see that I reproduced the error on an instance without a reverse proxy, so I don't think it's that in every case.

@babsz02 if you scroll through the comments, you'll see the issue has been reproduced even on instances that aren't behind a reverse proxy, so I don't think that's really it in all cases.


ipsec: Add support for public key authentication

Feature Description
The current IPsec plugin implementation does not support public key authentication, which allows for a more secure mutual authentication than PSK while still not introducing the complexity of X509 certificates. The authentication can easily be set up by generating a bare RSA keypair chain on both machines, followed by exchanging the public keys between the two peers.
This commit introduces public key authentication functionality by adding a new authentication method to phase 1 configuration called "Mutual Public Key" and adding a menu entry "Public Key Authentication", which allows adding public keys (+ optional private keys). It was successfully tested against a Linux virtual machine running Strongswan 5 and the entered RSA keys are automatically verified for correctness.
Useful commands for generating a bare RSA keypair:
ipsec pki --gen --type rsa --outform pem --size 4096 > private.pem
ipsec pki --pub --outform pem --in private.pem > public.pem

Screenshots
https://zig.pw/b2a74ca916e0dc9a58a5b503759b53d8.png - IPsec Phase 1 Configuration
https://zig.pw/6832696dc1f616f1d1fbd5c47683fc56.png - IPsec Public Key Authentication Overview
https://zig.pw/1789578d2e7bd5b43f2349e6d9b2b413.png - IPsec Public Key Authentication Editing
References
The DN42 project features a Wiki page which explains the benefits of IPsec Public Key Authentication in further detail and also provides documentation for setting it up on various systems: https://dn42.net/howto/IPsec-with-PublicKeys

Based on feedback from @AdSchellevis and @fichtner, I have rewritten this pull request to make use of the new OPNsense MVC system and made several changes as I've been unhappy with a few parts of the previous implementation after putting further thought into them:

The implementation now refers to Key Pairs, as the previous term Public Key Authentication in the menu has been misleading. After all, the menu item isn't there to configure pubkey-auth but to manage RSA key pairs.
The implementation only supports RSA keys as of today, but can be easily extended to support further types (e.g. ed25519, if Strongswan should start supporting this)
The implementation uses the builtin OpenSSL functions and does not rely on any third-party code
The keys are being automatically stored as a sanitized representation during validation. So e.g. unnecessary newlines or spaces after the end of a key are being trimmed away.
The key size and fingerprint are being calculated and stored into the model subsystem during validation. This avoids recalculating these stats on every page load for every single key pair, which could waste many CPU cycles.

The legacy code makes use of the new MVC model to access the configuration data. To allow the new configuration page to reload the legacy service,  LegacySubsystemController has been added which allows querying the status of the legacy subsystem (dirty / clean) as well as applying the configuration.
Thank you for the advice so far and looking forward to your feedback!

I just had a quick view on the frontend code. Have not looked at the backend.

Thank you @AdSchellevis for your code review as well. I have implemented the requested changes - the new MVC model code no longer makes use of any legacy code. Reconfiguring IPsec using pluginctl through configd works fine on my system, so I think this was a great idea AND it got rid of the hacky error_reporting(...) workaround. :)
The only other change I've made myself, aside from rebasing to master, was ensuring the "subsystem is dirty" message does not flicker when an exception occurs by checking if there has been a successful message at all before making these DOM changes.

@ppmathis it looks good, thanks! I would like to give it a spin on my local machine before merging, at the moment I'm a bit too busy to do this, so I'll merge as soon as I have a bit of time available (maybe at the end of this week).

@ppmathis its merged, small change in the volt template to fix the background, further than that it looked nice. Thanks again!

Thanks to everyone who was involved with this PR, I am really happy with how the result turned out and the review process felt smooth and effective to me. :+1:


Azure temples from portal

In the Azure portal there's also an option to create and share templates.(in preview but has been for a long time)
When deploying with Azure CLI could you use these templates as a "remote template" deployment? And if so; how do you obtain the uri?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: f0aca150-4747-7670-f465-7439aecc40e5
Version Independent ID: f836844c-0089-1477-57af-43a76181e5d6
Content: Deploy resources with Azure CLI and template - Azure Resource Manager
Content Source: articles/azure-resource-manager/templates/deploy-cli.md
Service: azure-resource-manager
Sub-service: templates
GitHub Login: @mumian
Microsoft Alias: jgao

@bvduijn Thank you for your query, we will look into it and get back to you at the earliest.

@bvduijn - sorry, that is not possible. An alternative for this will be template specs.

Thank you @tfitzmac for helping on this thread.
@bvduijn Please review the information shared by @tfitzmac and let us know if you have any further queries.
Hope the above information helps. At this point it appears that there's no further edits or updates needed to the documentation. We will go ahead and close this issue, however if you have any further queries please tag us in your reply and will be happy to reopen the thread and continue the conversation. Thank you

@tfitzmac  Thanks for the info! I guess Template specs is an evolution of the current templates in the Azure portal(?)
@BharathNimmala-MSFT Issue can be closed indeed. I'll sign up for the preview of the template specs.

Yes, it is an evolution but also different. Template Specs are a much better way to store and retrieve templates in your subscription because they are implemented as resources. However, there is not a portal UI for writing templates. VS Code is the recommended way to write templates.


update pytube to allow downloading after new youtube updates

Youtube has updated its interface, and Pytybe doesn't work anymore, we need to updated it or figure a work around
https://stackoverflow.com/questions/7301110/why-does-return-list-sort-return-none-not-the-list

fixed with https://github.com/brilliant-ember/YouTube-Channel-Downloader/pull/23


Change styling in config-UI's trigger collection page.

Description

Links to how to use the trigger wiki in config-UI's trigger collection page is not clear.

The description above is not clear enough.

Proposed solution

Links to how to use the trigger should be highlighted

The description above should be updated to :
Create a http request to trigger data collect tasks, please replace your gitlab projectId and jira boardId in the request body. This can take up to 20 minutes for large projects. (Gitlab 10k+ commits or Jira 5k+ issues)
There are two types of plugins in our application, corresponding to the 2 lists in the following json.
The regular plugins collect and enrich data (the 1st list) while the domain layer plugins (the 2nd list) prepare the data for the graphs in Grafana dashboards.  Editing domain layer plugins is for advanced usage only. Detailed configuration guide: How to use the triggers page

Recommended font styling for the "Full documentation" link:

                    Full documentation here:

                      How to use the Triggers Page

'Jira' should also be written as 'JIRA'


Extract css for prod builds

css sourcemapping
css requested in parallel
css cached separate
faster runtime (less code and DOM operations)

Only bummer is obviously the console will throw an error regarding /app-resources/styles.css from index.html for development since it will be programmatically inserted, but is anyone opposed to using html-webpack-plugin to avoid that case?

Looks great, Chris! Will take a closer look on Monday morning. Have a great weekend and thanks so much for the CSS cleanup! :confetti_ball:

@RickCarlino, I'll implement html-webpack-plugin for the development environment in addition to this PR sometime tomorrow for review/testing :)

@MrChristofferson I don't recall if we still needed that one or not (I very well could have left a stray TODO in the codebase). I originally wanted to use it for some i18n stuff we were doing, but we don't actually need it anymore. If you think there's a good use case for it though, go right ahead.
I'm going to merge this one now.

Oh, whoops- just saw why you want to add html-webpack-plugin. Yeah, that makes sense.

Looks like pulling out that style.css is a quick fix for now.


Remove deprecated key

Hi,
long time no see haha, I guess we are not using proprietary: true anymore?

👋 heyo, yeah. Hope you're doing fine.
Thanks for the PR. Indeed, we don't use it anymore.
Found a couple more places, that I'm gonna clean up as well.

👋 heyo, yeah. Hope you're doing fine.

a bit busy in past few months, company grows, so my work load grows as well haha, hope you doing fine as well 😄

Found a couple more places, that I'm gonna clean up as well.

I see, I will clean up if I found them

Yeah, there's always plenty of work. 😆