Error publishing AV1 streams to RTSP server using latest ffmpeg and latest mediaMTX (2023-10-29) "clock rate not found"

Which version are you using?
v1.2.1
Which operating system are you using?

[X] Linux amd64 standard
[ ] Linux amd64 Docker
[ ] Linux arm64 standard
[ ] Linux arm64 Docker
[ ] Linux arm7 standard
[ ] Linux arm7 Docker
[ ] Linux arm6 standard
[ ] Linux arm6 Docker
[X] Windows amd64 standard
[ ] Windows amd64 Docker (WSL backend)
[ ] macOS amd64 standard
[ ] macOS amd64 Docker
[ ] Other (please describe)

Describe the issue
Description
Failure to publish av1 encoded stream to
Describe how to replicate the issue

Create sample files using latest ffmpeg by taking an h264 encoded mp4 file, and transcode it to vp9+h265+av1 using:
av1: ffmpeg -i h264.mp4 -c:v libsvtav1 -an -crf 35 -svtav1-params fast-decode=1 -g 50 av1.mp4

(h264, h265 and vp9 working)
h265: ffmpeg -i h264.mp4 -c:v libx265 -an -crf 35 -tag:v hvc1 h265.mp4
vp9: ffmpeg -i h264.mp4 -c:v vp9_qsv -crf 35 -g 50 av1.mp4

start the server (linux 1.2.1, or windows 1.2.1, this bug happens in both)

publish with: ffmpeg -stream_loop -1 -re -i av1.mp4 -c:v copy -an -f rtsp rtsp://<IP_ADDRESS>:8554/av1

(working lines: )
ffmpeg -stream_loop -1 -re -i vp9.mp4 -c:v copy -an -f rtsp rtsp://<IP_ADDRESS>:8554/vp9
ffmpeg -stream_loop -1 -re -i h265.mp4 -c:v copy -an -f rtsp rtsp://<IP_ADDRESS>:8554/h265

read with: ffplay -i -f rtsp rtsp://<IP_ADDRESS>:8554/av1

(working lines)
ffplay -i -f rtsp rtsp://<IP_ADDRESS>:8554/vp9
ffplay -i -f rtsp rtsp://<IP_ADDRESS>:8554/h265

H264 WORKS!
VP9 WORKS!
H265 WORKS!

AV1 makes error on mediaMTX while publishing (before connecting client):
Conn... opened
session created by...
conn closed: invalid SDP: media 1 is invalid: clock rate not found

ffmpeg breaks done because of it writing:
[out#0/rtsp @<PHONE_NUMBER>29b500] Could not write header (incorrect codec parameters ?): Server returned 400 Bad Request.
Error opening output file rtso://<IP_ADDRESS>:8554/av1
*I didn't create the SDP, the latest version of ffmpeg did, just as it should have done for the working H264\H265\VP9 files.
**ffplay directly on the file works
Did you attach the server logs?
mediaMTX logs
2023/11/09 11:26:07 INF [RTSP] [conn <IP_ADDRESS>:50052] opened
2023/11/09 11:26:07 DEB [RTSP] [conn <IP_ADDRESS>:50052] [c->s] OPTIONS rtsp://<IP_ADDRESS>:8554/av1 RTSP/1.0
CSeq: 1
User-Agent: Lavf60.17.100
2023/11/09 11:26:07 DEB [RTSP] [conn <IP_ADDRESS>:50052] [s->c] RTSP/1.0 200 OK
CSeq: 1
Public: DESCRIBE, ANNOUNCE, SETUP, PLAY, RECORD, PAUSE, GET_PARAMETER, TEARDOWN
Server: gortsplib
2023/11/09 11:26:07 DEB [RTSP] [conn <IP_ADDRESS>:50052] [c->s] ANNOUNCE rtsp://<IP_ADDRESS>:8554/av1 RTSP/1.0
CSeq: 2
Content-Length: 157
Content-Type: application/sdp
User-Agent: Lavf60.17.100
v=0
o=- 0 0 IN IP4 <IP_ADDRESS>
s=No Name
c=IN IP4 <IP_ADDRESS>
t=0 0
a=tool:libavformat 60.17.100
m=video 0 RTP/AVP 96
b=AS:4194
a=control:streamid=0
2023/11/09 11:26:07 INF [RTSP] [session 319bde60] created by <IP_ADDRESS>:50052
2023/11/09 11:26:07 DEB [RTSP] [conn <IP_ADDRESS>:50052] [s->c] RTSP/1.0 400 Bad Request
CSeq: 2
Server: gortsplib
2023/11/09 11:26:07 INF [RTSP] [conn <IP_ADDRESS>:50052] closed: invalid SDP: media 1 is invalid: clock rate not found
2023/11/09 11:26:07 INF [RTSP] [session 319bde60] destroyed: not in use
ffmpeg logs:
ffmpeg -stream_loop -1 -re -i ./videos/av1/Sintel_720_10s_5MB.mp4 -c:v copy -an -f rtsp rtsp://<IP_ADDRESS>:8554/av1
ffmpeg version 2023-10-29-git-2532e832d2-full_build-www.gyan.dev Copyright (c) 2000-2023 the FFmpeg developers
built with gcc 12.2.0 (Rev10, Built by MSYS2 project)
configuration: --enable-gpl --enable-version3 --enable-static --pkg-config=pkgconf --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-bzlib --enable-lzma --enable-libsnappy --enable-zlib --enable-librist --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-libbluray --enable-libcaca --enable-sdl2 --enable-libaribb24 --enable-libaribcaption --enable-libdav1d --enable-libdavs2 --enable-libuavs3d --enable-libzvbi --enable-librav1e --enable-libsvtav1 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxvid --enable-libaom --enable-libjxl --enable-libopenjpeg --enable-libvpx --enable-mediafoundation --enable-libass --enable-frei0r --enable-libfreetype --enable-libfribidi --enable-libharfbuzz --enable-liblensfun --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-dxva2 --enable-d3d11va --enable-libvpl --enable-libshaderc --enable-vulkan --enable-libplacebo --enable-opencl --enable-libcdio --enable-libgme --enable-libmodplug --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libshine --enable-libtheora --enable-libtwolame --enable-libvo-amrwbenc --enable-libcodec2 --enable-libilbc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-ladspa --enable-libbs2b --enable-libflite --enable-libmysofa --enable-librubberband --enable-libsoxr --enable-chromaprint
libavutil      58. 30.100 / 58. 30.100
libavcodec     60. 32.102 / 60. 32.102
libavformat    60. 17.100 / 60. 17.100
libavdevice    60.  4.100 / 60.  4.100
libavfilter     9. 13.100 /  9. 13.100
libswscale      7.  6.100 /  7.  6.100
libswresample   4. 13.100 /  4. 13.100
libpostproc    57.  4.100 / 57.  4.100
[libdav1d @ 000001ec81d511c0] libdav1d 1.3.0-4-gfd4ecc2
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from './videos/av1/Sintel_720_10s_5MB.mp4':
Metadata:
major_brand     : isom
minor_version   : 512
compatible_brands: isomav01iso2mp41
encoder         : Lavf58.64.100
Duration: 00:00:10.00, start: 0.000000, bitrate: 4196 kb/s
Chapters:
Chapter #0:0: start 0.000000, end 10.000000
Metadata:
title           : Chapter 01
Stream #0:00x1: Video: av1 (Main) (av01 / 0x31307661), yuv420p(pc, progressive), 1280x545, 4194 kb/s, SAR 1:1 DAR 256:109, 30 fps, 30 tbr, 15360 tbn (default)
Metadata:
handler_name    : VideoHandler
vendor_id       : [0][0][0][0]
Stream #0:10x2: Data: bin_data (text / 0x74786574), 0 kb/s
Metadata:
handler_name    : SubtitleHandler
[out#0/rtsp @ 000001ec81d55740] Could not write header (incorrect codec parameters ?): Server returned 400 Bad Request
Error opening output file rtsp://<IP_ADDRESS>:8554/av1.
Error opening output files: Server returned 400 Bad Request
Did you attach a network dump?
no

Please post server logs with logLevel: debug as clearly stated in the instructions.

Hi.
Just added the accurate logs, as long as whats ffmpeg is giving.
Worth mentioning I tried many AV1 (some encoded with SVT other AOM) files in many different containers (AVI\MP4\WEBM),
with somewhat different parameters (though more oriented to LIVE feeds, thas why I use -re in the command)
Some files I downloaded from 'https://test-videos.co.uk/sintel/mp4-av1' instead of encode with my ffmpeg.
All had the same result.
I'm tried using both the last ffmpeg windows and ffmpeg linux versions to stream the files to mediaMTX.
(Maybe worth saying, that wen I tried streaming to rtp in stead of RTSP, ffmpeg alerted that it doesnt support AV1.
But I'm guessing it's only the rtp output that doesnt support it, iy has no problem to encode, send to rtsp or play av1)
Thank you very much for taking the time to check it out.

The issue here is that FFmpeg doesn't support publishing AV1 tracks with RTSP yet. Take a look here:
https://github.com/FFmpeg/FFmpeg/blob/96d2a40b9ee0ddd1844b82c23a074804c316caa5/libavformat/sdp.c#L512
You can see support for VP8, VP9, H265, but there's no AV1 yet.
On the other hand, latest GStreamer is able to publish AV1.
Open an issue in the FFmpeg bug tracker or just implement the feature by yourself and send a patch to the FFmpeg mailing list.


Every doRpcCall times out

This Telegram API is confusing, already wasted 2 days. Every doRpcCall I make throws TimeoutException. If I make request doRpcCallNonAuth, it does not time out. Here is the code:
apiState = new MemoryApiState(sessionFile);
api = new TelegramApi(apiState, new AppInfo(107508, "devicemodel", "systemversion", "appversion", "EN"), new ApiCallback() {
    @Override
    public void onAuthCancelled(TelegramApi api) {

    }

    @Override
    public void onUpdatesInvalidated(TelegramApi api) {

    }

    @Override
    public void onUpdate(TLAbsUpdates updates) {

    }
});

try {
    TLConfig config = api.doRpcCallNonAuth(new TLRequestHelpGetConfig());
    apiState.updateSettings(config);
} catch (RpcException e) {
    e.printStackTrace();
} catch (TimeoutException e) {
    e.printStackTrace();
}

////// SIGN UP NEW USER

String phoneCodeHash = sendCode(phoneNumber.phoneNumber);
if(phoneCodeHash == null)
{
    System.out.println("ERROR SENDING SMS");
    throw new Exception("ERROR SENDING SMS");
}

// read phone code from sms
String phoneCode = null;
long beginTime = System.currentTimeMillis();
while(phoneCode == null)
{
    Thread.sleep(5000);
    phoneCode = smsApi.getMessage(phoneNumber.orderId);

    if(System.currentTimeMillis() - beginTime > 4 * 60 * 1000)
        throw new Exception("sms timeout");
}

// register user

TLRequestAuthSignUp signUp = new TLRequestAuthSignUp();
signUp.setFirstName(firstName);
signUp.setLastName(lastName);
signUp.setPhoneCode(phoneCode);
signUp.setPhoneCodeHash(phoneCodeHash);
signUp.setPhoneNumber(phoneNumber.phoneNumber);
TLAuthorization authorization = api.doRpcCallNonAuth(signUp, 10000, apiState.getPrimaryDc());
apiState.doAuth(authorization);

/// HERE IS THE FIRST REQUEST WITH DORPCCALL

TLRequestAccountUpdateUsername updateUsername = new TLRequestAccountUpdateUsername();
updateUsername.setUserName(username);
try {
    api.doRpcCall(updateUsername, 10000); // TIME OUT EXCEPTION
} catch (IOException e) {
    e.printStackTrace();
} catch (TimeoutException e) {
    e.printStackTrace();
}

I've added -Djava.security.egd=file:/dev/./urandom to VM options, but still does not work. I think something might be wrong with authentication, but I have no idea what

@Gintasz Yeah, I'm in the same boat. Any progress on your side ? Overall codebase is of impressively bad quality - zero documentation, zero comments, naming of interfaces and class with Borland Delphi style. I'm still digging through it and will report if I build minimal working client.

Did you use Deepthought sample?
I use that sample and modify it in a 6-7 project and its work without any timeout. yes, it's no comment if you download Javadoc you only find a name of the function in Javadoc and some other thing but work correctly.
I found some bug and I can resolve some of them.
I told everyone in this repository to start from Deepthought sample but I don't know why some body doesn't use it.


Add stop() function for cartridge roles.

It should clean up _G after the init() function.

Added in #145


[INLONG-2989] Fix the coding style problem of TubeMQ Go SDK

Title Name: [INLONG-XYZ][component] Title of the pull request
where XYZ should be replaced by the actual issue number.
Fixes #2989
Motivation
Explain here the context, and why you're making that change. What is the problem you're trying to solve.
Fix the coding style problem of TubeMQ Go SDK
Modifications
Describe the modifications you've done.
Fix the coding style problem of TubeMQ Go SDK
Verifying this change

[ ] Make sure that the change passes the CI checks.

(Please pick either of the following options)
This change is a trivial rework / code cleanup without any test coverage.
(or)
This change is already covered by existing tests, such as (please describe tests).
(or)
This change added tests and can be verified as follows:
(example:)

Added integration tests for end-to-end deployment with large payloads (10MB)
Extended integration test for recovery after broker failure

Documentation

Does this pull request introduce a new feature? (yes / no)
If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)
If a feature is not applicable for documentation, explain why?
If a feature is not documented yet in this PR, please create a followup issue for adding the documentation

/cc @charlely


Android/IOs support

is it being planned?

No plans at present — While the framework's reactivity could be used to build any kind of application (see the gtk example in the repo) it is primarily intended for web applications. Given how tightly iOS and Android interfaces are tied to Swift/ObjC and Kotlin/Java, it seems unlikely that any Rust framework will have particularly good support in the near future. Of course it's already possible using something like Capacitor to wrap a webview into a native app, and my understanding is that Tauri now has at least some support for mobile apps. But I don't plan on doing any work myself to use Leptos to drive native mobile app interfaces.


Fix scrolling to comments

Problem with componentDidMount not triggering scroll was caused by not rendering #comments handle if content was undefined.
Post became very bloated so I created PostContent component, that is container of StoryFull. PostContent is only rendered if content is loaded so scrollIntoView is triggered there.
It's triggered only once, so in some cases (long post with major part of images) comments might be scrolled down (and same thing happens in Condenser).
This should fix those issues:
https://trello.com/c/yV2fsD3S/159-bug-scroll-with-comment-link
https://trello.com/c/NqRVZUMd/196-click-on-comment-icon-from-feed-redirect-to-top-of-post

Seem to solve this issue too: https://trello.com/c/gjchs1fB/184-bug-scroll-when-voting-on-page-with-comment-hashtag


Using __esri namespace in more than one component

I've noticed that I am unable to use the __esri namespace in more than one component or service in my applicaiton.  For example, I have a MapComponent class which sets up my basemap, and and a LegendComponent class which sets up an interactive legend to turn on/off layers.  However, when I attempt to use access the __esri namespace in the LegendComponent class, I get this error:
ReferenceError: __esri is not defined
    at Module../src/app/legend/legend.component.ts (c:\...\src\app\legend\legend.component.ts:3:15)
    at __webpack_require__ (http://localhost:4200/runtime.js:79:30)
    at Module../src/app/app.module.ts (http://localhost:4200/main.js:137:105)
    at __webpack_require__ (http://localhost:4200/runtime.js:79:30)
    at Module../src/main.ts (c:\...\src\main.ts:1:1)
    at __webpack_require__ (http://localhost:4200/runtime.js:79:30)
    at Object.0 (http://localhost:4200/main.js:1009:18)
    at __webpack_require__ (http://localhost:4200/runtime.js:79:30)
    at checkDeferredModules (http://localhost:4200/runtime.js:46:23)
    at Array.webpackJsonpCallback [as push] (http://localhost:4200/runtime.js:33:19)

Removing the LegendComponent class from the application resolves the error.  How can I use the __esri namespace in places other than just a single component?

@mfcallahan can you provide a stackblitz that reproduces the issue? I've seen some weird stuff recently with Angular material and namespaces. Any chance you are using Angular material?
Here's a working non-material example: https://stackblitz.com/edit/angular-cli-esri-map2-test?file=src%2Fapp%2Fmap-text%2Fmap-text.component.ts

I had to add
   "types": ["arcgis-js-api"]
to my tsconfig, even though I'm using v7.2 of angular.

Oh, that's interesting, thanks for the update. We tested quite a bit with v7.2 but with so many interdependencies these days it's hard to capture all permutations. I'll close this for now.

Hi thanks for the follow up - I will post an example tomorrow.


docs: Updated doc-check to generate config + format errors

Closes https://github.com/ipfs/aegir/issues/1273
Related: https://github.com/libp2p/js-libp2p/pull/1505

Strange CI failure as all these tests pass on my local env 🤔

It seems that the underlying library i.e. https://github.com/bbc/typescript-docs-verifier does not properly compile snippets on Windows due to path resolution methods. I raised an issue. In the interim we could consider ignoring the tests on windows since this particular tool is less likely to run on that OS.

If Windows fails the doc-check, instead of skipping the the doc-check tests the command itself should end early on Windows and print a message - for example see how coverage is skipped at the moment, though it'd be better if it printed the relevant GH issue too, https://github.com/ipfs/aegir/pull/1195 in this case.
Otherwise we could ship this, then someone who happens to do their dev on windows will find aegir doc-check fails for hard to debug reasons.


Fix/improve demo documentation

Describe the behavior changes introduced in this PR
I was running the Pelorus demo and decided to update the documentation on that part (but I ended up doing more and this became a big PR :cry:).
The idea of this PR is to improve our documentation, focused on the Pelorus demo.
Linked Issues?
not closing ones.
Testing Instructions
Run the documentation locally, with
make dev-env
source .venv/bin/activate
mkdocs serve

and compare with our live documentation.

/test 4.10-e2e-openshift

I would change the order of philosophy and getting started, what you think?

@mpryc this will likely conflict w/ https://github.com/konveyor/pelorus/pull/723/files  which imho we should hurry up and merge.

Depends on: #723 #690

Applied some changes from https://github.com/konveyor/pelorus/pull/723

/lgtm


Support docker 1.13 for accessing nodeport from remote node

Since docker 1.13, docker set the default policy of FORWARD to drop package, this will cause a service nodeport can't be accessible from remote node, but it can be accessible from local node. But when using weave or flannel, this will work fine.
Expected Behavior
If user use docker 1.13 with K8S, the nodeport of any service should be accessible from remote node.
To fix this issue, we need to set iptable rule "iptables -P FORWARD ACCEPT" when booting node.
Whether calico can set such iptable rule when starting component of calico.
Current Behavior
When user create service with nodeport in k8s, the nodeport can be accessible from the node which is running the pod for this service, but when user access the nodeport from remote client, the nodeport will not be accessible.
Possible Solution
set iptable rule "iptables -P FORWARD ACCEPT" when starting component of calico.
https://github.com/kubernetes/kubernetes/issues/40182
Steps to Reproduce (for bugs)
1.Install k8s with docker 1.13
2.Create a k8s service in type of NodePort
3.Access the nodeport from any node which is not running pods
Context
When expose a k8s service with the nodeport, user can't access it from remote client, unless the ip is the node ip which is running the pod for service.
Your Environment

Calico 2.3
Orchestrator version :K8S
Operating System and version: centos 7.3
Docker version: 1.13 or 17.0x

@James-Dao Did you find an alternate solution when using docker 1.13 with Calico?


Change width of submenus

Fantastic plugin and many thanks for making it available!! This is more of a request than an issue.  Is it possible to make the width of the submenu's less than that of top parent menu?  For example I am using the menu with an Iconbar, but would like the icons to be in view and clickable when a user goes down into lower levels.  Is it possible to modify the width of the submenus without affecting the width of the top menu?  Many thanks for your help in advance.

Cool idea, I'll see if I can create an add-on for this behavior.

Check out the new add-on "iconPanels":

If you want a small part of the parent panel to always be visible (most commonly when you're using icons), include the "iconPanels" add-on .js and .css files and use the iconPanels options.

http://mmenu.frebsite.nl/documentation/addons/icon-panels.html
Actually using icons is not required... :)

Great update Fred! This will be very useful to a lot of people I suspect so thanks very much for taking the time to produce this add on.  I am currently working with your menu and have adapted it to suit certain requirements.  I will get a link up when its completed, I think it looks well.  May also have to pick your brains about the cloning option.

Hmm, there seems to be an issue when opening a submenu after having scrolled down.


Resolve SR-1028

What's in this pull request?
A fix for SR-1028, which is an Xcode 7.3 regression. This just cherry-picks the fix for SR-590 (b54976) and adds an additional test (6db942).
Resolved bug number: (SR-1028)

Before merging this pull request to apple/swift repository:

[ ] Test pull request on Swift continuous integration.

Triggering Swift CI
The swift-ci is triggered by writing a comment on this PR addressed to the GitHub user @swift-ci. Different tests will run depending on the specific comment that you use. The currently available comments are:
Smoke Testing
    Platform     | Comment
    ------------ | -------------
    All supported platforms     | @swift-ci Please smoke test
    OS X platform               | @swift-ci Please smoke test OS X platform
    Linux platform              | @swift-ci Please smoke test Linux platform

Validation Testing
    Platform     | Comment
    ------------ | -------------
    All supported platforms     | @swift-ci Please test
    OS X platform               | @swift-ci Please test OS X platform
    Linux platform              | @swift-ci Please test Linux platform

Note: Only members of the Apple organization can trigger swift-ci.

@swift-ci Please test


inspect modules to make their fully independent project

the most stable part of the core should be moved into the independent projects with their own release cycle etc.
it should minimize classpath and avoid all kind of conflicts when these concrete feature is not used in test project.
Candidates for movement in 7.4+ releases are:
carina-dataprovider
carina-appcenter
carina-aws-s3
carina-azure
carina-crypto
in this ticket let's try to move carina-dataprovider module only

carina-dataprovider used in IAbstractTest and AbstractTestListener. it has only net.sf.opencsv as unique dependency so let's keep it as sub-module. Changes in abstract classes and interfaces lead to complicated extra steps to integrate this utility into the test project.

inspected. carina-crypto is 1st one which is already available at: https://github.com/zebrunner/carina-crypto


Replace PHPhotoLibrary.requestAuthorization(_:) with PHPhotoLibrary.requestAuthorization(for:handler:)

As of iOS 14.0 requestAuthorization(_:) and authorizationStatus() are now deprecated they should be replace by requestAuthorization(for:handler:)and authorizationStatus(for:).

Check c3e9343b21c1e1b147eb41eb0f2c191ff394910d


Determine if an image falls under open access policy

Hi, is it possible in any way to automatically determine if an image in pusblished_images.csv falls under open access policy so that it can be used freely?

Is this issue related to "Imagery for an object should not be displayed unless the canShowImagery flag is 1"?  The statement appears in the data dictionary text, but not as a field in any table that I can find.   TIA

Yes, it is very much related but "canshowimagery" refers to an internal deprecated flag.  It should not be in the published version of the data dictionary.  We recently discussed this github issue and will be releasing new flags in the published images table to indicate which images fall under our open data program and therefore a permissive content license.


Internal/External Notifications

Ability for wallet to send notifications. No specific method or standard. Open to community feedback.


Message rotation on /var/log/messages

I don't see rotation of /var/log/messages happening in the GCE ref arch, and we have a very high debug level configured.  First, do we have this already and I missed it?  Second, if not, do we have it in OpenShift Ansible?  Third, if not, we should probably add it to the ref-arch, since this is a very likely cause of node failure on GCE.
@pschiffe

I added https://github.com/openshift/origin-gce/commit/4b792d4d8e0c22a9967baa7b3461a8f973580437 for this - it's resulted in several nodes dying painful deaths.

@smarterclayton @pschiffe any thoughts on lowering this debug level? If thats not an option we can add in the the rotate but this arch currently has the highest debug level

I'm going to close this as we have not seen issues with other providers and debug 2


IE8 - mega menu is blank

Maybe fix https://github.com/cfpb/cfgov-refresh/issues/963 at the same time.

Well then...


Link to original files and not to exec root

Fixes #40

Please see discussion on #40 on why the current code is by design.


Packaging into a gem

Hello,
Recursive Open Struct is great.
I just packaged it into a gem using the latest version of jeweler. I dusted a bit some side files (without altering the code, version or summary/etc.)
I also fixed a unit test which was failing.
I published the gem with jeweler, so it's now available through RubyGems. I'll gladly give you the ownership if you want.
Best regards and thanks for giving this code to the community.
Offirmo.

Hi Offirmo,
Apologies for taking so long to get back to you, but I am glad you like it. Also, thank you for turning it into a gem and writing specs for it!
If you want to continue being the steward for the gem, feel free to do so. If you would like me to take over though, I am happy to handle it.
Thank you,
William Orvis


Add a conda recipe

Such as:
https://github.com/pydata/pandas/tree/master/conda.recipe

I created [conda packages for Ibis and its dependencies] (https://anaconda.org/koverholt) for linux-64 and osx-64. You can conda install Ibis using:
conda install -c koverholt ibis-framework

I have the conda recipe files for ibis-framework and its dependencies if you would like me to open a PR and commit them somewhere.
You also might be interested in creating an account on [Anaconda.org] (http://www.anaconda.org) (e.g., named cloudera or ibis) so that you can maintain the conda packages and have users install from that channel using something like:
conda install -c cloudera ibis-framework

Let me know if I can help with any of those efforts.

Be happy to accept a PR — we'd probably need to set up a corporate CLA with Continuum that enumerates individual contributors

Nudging you about this, in case we can get it into the 0.4 release. Note that we're going to modify the  requirements (and remove snakebite, which is not used at all), so I may bug you to update the recipe once we do that

Definitely, I can do that. Regarding the CLA, I should have something over to you by tomorrow.

Update: Added list of contributors to the CLA on our side. Still waiting on the CLA to be signed off. Will get it to you as soon as that happens. Let me know if you need any updates in the mean time.

I just sent a copy of the CLA to your email. Please let me know if you need any additional information. Thanks!


Suggestion: Make an option to not to force the output window open on test run

When terminal is not in the same place as the output window (e.g. on the right), it is really annoying to have the output window popping up from the bottom and the terminal from the right. I really don't care about the output window as it only shows the command.
It makes sense to have an option that disables that behaviour:
this.channel.show(true);

Hi, I removed the show completely, as you say it's useless since you wanna just see the terminal output or problems. Released update now


SmarkScreen link for testing Network Protection doesn't get blocked

[Enter feedback here]
I went through the steps to enabled Network Protection and the link (https://smartscreentestratings2.net/) used to confirm Network Protections is functioning correctly is not blocked. This link is blocked with SmartScreen but not network proection

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 2c453008-4ccc-186c-b6da-64536d62c89c
Version Independent ID: a00180f1-4f69-956f-cd92-77b7bf21fc16
Content: Evaluate network protection
Content Source: microsoft-365/security/defender-endpoint/evaluate-network-protection.md
Product: m365-security
Technology: mde
GitHub Login: @Dansimp
Microsoft Alias: dansimp

@yogkumgit , i tried on google chrome 32bit browser, i  received the following.

@over2under3 Thank you for your feedback. I was able to replicate the issue aswell. But this is not by design and needs to be reported to Microsoft support via the need help section in the admin center so that the engineer can raise a product bug and fix this in the backend.
Hope this helps!
Thanks
Sri

Did you implement this using GPO or Intune? I used GPO and still can't get this to work using Edge.

@msbemba Thank you very much for the contribution and sharing this explanation. @over2under3 Hope this comment is helpful for you. Considering the nature of the issue, as advised, please open a service ticket in your tenant and follow with them for the resolution. We proceed to close it as here we are limited to documentation fixes and improvements only.
We will be very happy if you can share the outcome or resolution with us if you see documentation update is required. We will certainly update this documentation with that valuable information. Thanks again, for taking out some time to open the issue. Appreciate and encourage you to do the same in future also.


[me.lucko.luckperms.lib.hikari.pool.PoolBase] luckperms-hikari - Failed to validate connection me.lucko.luckperms.lib.mariadb.Connection@7e20ff86 ((conn=5955) Connection.setNetworkTimeout cannot be called on a closed connection). Possibly consider using a shorter maxLifetime value.

Description
[me.lucko.luckperms.lib.hikari.pool.PoolBase] luckperms-hikari - Failed to validate connection me.lucko.luckperms.lib.mariadb.Connection@7e20ff86 ((conn=5955) Connection.setNetworkTimeout cannot be called on a closed connection). Possibly consider using a shorter maxLifetime value.
Mariadb config: https://pastebin.com/QwTZjwDP
Luckperms config: https://pastebin.com/02Tkf6YF
Reproduction Steps
1
Expected Behaviour
1
Server Details
purpur 1.20.6-2211-3d25693 (MC: 1.20.6)*
LuckPerms Version
v5.4.128
Logs and Configs
1
Extra Details
No response

same with the velocity


Deprecate 'l1' and 'l2' in favor of 'absolute_error' and 'squared_error' (#82)

Summary
Closes #82
Details and comments
The TrainableModel class supports L1 and L2 loss functions and are chosen using the argument values 'l1' and 'l2'. Since this can be confusing, this PR deprecates these values by replacing occurances of 'l1' with 'absolute_error' and 'l2' with 'squared_error'. The PR also updates unit tests and documentation.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

@woodsp-ibm Yes I see now, thank you so much for the helful comments!

CI still failing with some words it does not like instantiation and instantiations. Since the docstrings become our main API reference documentation we do the spell check to help keep up the quality. For words its built-in dictionary does not have, we have custom dict in the root here https://github.com/Qiskit/qiskit-machine-learning/blob/main/.pylintdict You will see they are in alphabetic order and lowercase. If you can add the above words to that file - yes, both the singular and plural forms are needed to be added there. That should get things through CI here. Thx

I changed the code to use the deprecate_values decorator.

My pleasure! Thank you all so much all for helping me out with my first issue. Please feel free to let me know if there are any other beginner/intermediate tickets you want taken care of!

@mattwright99 Thanks a lot! If you fill comfortable you can take a look at #62 and #112. They are more interesting for sure, but in the same time more difficult for a beginner.


Simplify tox.ini file

Description
This PR simplifies the tox.ini file in the following way:

Remove redundant pyXY environments. These environments are created by tox automatically.
Remove specific environments for fast mode. This config is passed in via the INMANTA_EXTRA_PYTEST_ARGS environment variable.

Part of inmanta/irt#917
Self Check:

[x] Attached issue to pull request
[x] Changelog entry
[ ] Type annotations are present
[ ] Code is clear and sufficiently documented
[ ] No (preventable) type errors (check using make mypy or make mypy-diff)
[ ] Sufficient test cases (reproduces the bug/tests the requested feature)
[x] Correct, in line with design
[ ] End user documentation is included or an issue is created for end-user documentation

Reviewer Checklist:

[ ] Sufficient test cases (reproduces the bug/tests the requested feature)
[ ] Code is clear and sufficiently documented
[ ] Correct, in line with design

Processing this pull request

Merged into branches iso4 in fd4a6f1fca06f79f73473c8efe007cf8ec2dd673


Getting an error while launching PyMAPDL using latest version 0.68.1 and Python v3.12.0.

🤓 Before submitting the issue

[X] I have visited the Troubleshooting section.
[X] I have searched among the existing issues.
[X] I am using a Python virtual environment.
[X] I am using the latest version of PyMAPDL and its dependencies (fully updated virtual environment). You can update them using pip install --upgrade --upgrade-strategy eager ansys-mapdl-core in your activated virtual environment.

🔍 Description of the bug
Getting an error while launching PyMAPDL using latest version 0.68.1 and Python v3.12.0. Please refer attached image to see the error message and pip list.

🕵 Steps To Reproduce
The log will automatically be formatted as Python code! No need to type backticks.

💻 Which Operating System are you using?
Windows
🐍 Which Python version are you using?
3.12
📝 PyMAPDL Report
Show the Report!

# PASTE HERE THE OUTPUT OF `python -c "from ansys.mapdl import core as pymapdl; print(pymapdl.Report())"` here

📝 Installed packages
Show the installed packages!

# PASTE HERE THE OUTPUT OF `python -m pip freeze` here

📝 Logger output file
Show the logger output file.

# PASTE HERE THE CONTENT OF THE LOGGER OUTPUT FILE.

@clatapie : Please find attached report for the further investigations.

@clatapie Please find attached report using PyMAPDL version 0.68.1,

@clatapie Based on my further investigations, it appears that, initial errors occurs if we launch mapdl using default option, like,
mapdl = launch_mapdl() ---> end up with the errors as reported initially.
However, we able to launch mapdl successfully, if we specify the version, e.g.
mapdl = launch_mapdl(version = 24.1) or mapdl = launch_mapdl(version = 241)

Hi @vnamdeo
I think there is a problem with either your envvars, or which one of your versions.
Please follow the steps given in https://mapdl.docs.pyansys.com/version/stable/user_guide/troubleshoot.html#debug-in-pymapdl and post here the log file (mylog.log). That file will tell you which command is trying to execute, and then we can try that in a terminal.

Hi @germa89 , Followed steps as mentioned in the Troubleshooting help doc, but, it couldn't save log file. However, please refer to the attached image for the further debug.

When we add version = 24.1, we able to launch PyMAPDL with no errors, please refer to the attached image for reference.

@vnamdeo
Can you try in a terminal:
"C:\Program Files\ANSYS Inc\v231\ansys\bin\winx64\ansys231.exe" -j file -np 2 -b __tmp__.inp -o __tmp__.out -port 50052 -grpc

and put the output here?

@germa89 Is the default MAPDL version is v231 on launch_mapdl( ) ?
If we execute above line in a terminal, the output is,
"The system cannot find the path specified. "

I realise you are trying to launch MAPDL 231, but you do not have it installed.
Maybe you had it installed and uninstalled it. But the config.txt still has it.
You can check this file by doing in python:
>>> from ansys.tools.path.path import CONFIG_FILE
>>> print(CONFIG_FILE)
/Users/user/Library/Application Support/ansys_tools_path/config.txt

to change this, the best is either delete that file, or:
>>> from ansys.tools.path import change_default_mapdl_path, get_mapdl_path
>>> change_default_mapdl_path('C:\Program Files\ANSYS Inc\v231\ansys\bin\winx64\ansys231.exe')
>>> get_mapdl_path()
C:\Program Files\ANSYS Inc\v231\ansys\bin\winx64\ansys231.exe

@germa89 In the first place, why launch_mapdl() usage Ansys v231 to launch. What is the default version is ?

whatever is written in the CONFIG_FILE.
This file is created the first time you run PyMAPDL EVER in a PC.

@germa89 Earlier, I am not getting such errors and using PyMAPDL from long time. Recently, I migrated to newer PyMAPDL version and new Python interpreter since then I am observing such issues. Ideally, we should set some default version "latest mapdl version" on version option of launch_mapdl () function to invoke rather grab it from the CONFIG_FILE, which is not ideal approach as you have observed it. Thanks !

You are right. I acknowledge there are some corners to polish. PyMAPDL should probably re-detect MAPDL if the file cannot be found.
I will look into that. I'm keeping this issue open for future fix.
In the meantime, please delete CONFIG.txt file, it should work OK after that.


Disassembly memory references should be treated as opaque strings

Currently VS Code assumes the memoryReference in the context of disassembly is a decimal address, but this is a constraint not dictated by DAP. See discussion: https://github.com/microsoft/vscode-discussions/discussions/712

What are some verification steps for this issue?
I've attached a pic of what I'm seeing in case that verifies the issue immediately.

this actually dupes https://github.com/microsoft/vscode/issues/129655


Downloaded file has underscores prefixed ans suffixed

Hey,
Thanks for the great lib.
The content-disposition package adds quotes around the filename, causing chrome to replace the quotes with underscores, so:

Is actually downloaded as:

If I manually set the header, without the quotes:

File is download as expected:

I know this is an issue for the content-disposition package, but this is also an issue with the current usage of res.download.

Your screenshot shows the file download is not coming from Express, but from your front end framework. That is why the URL displayed starts with blob: and not http: .
I just tested with latest Chrome and there were no underscores even though it was quoted file name.
I can reopen if you can provide the code I can run and steps to reproduce the error so I have a reproduction I can investigate.

@dougwilson thanks for the superfast response. I guess it is because of our company's proxy, thanks for pointing it out, sorry for the hussle.


Annotations watcher

This now offers the --annotations <provider> flag, this is used to watch for BGP annotations being applied and will watch for:
<provider>/src-ip
<provider>/node-asn
<provider>/peer-ip
<provider>/peer-asn
Once all these have been populated the watch ends and starts the BGP server

A few questions:

Are the annotations configurable?
Are the docs updated to reflect this option?
Will the whole "packet" option go away, as this can just use those?

A few questions:

Are the annotations configurable?
Are the docs updated to reflect this option?
Will the whole "packet" option go away, as this can just use those?

As we discussed in issue #153 I understand the feature and I know why it is coded in this way (with the possibility to override the prefix). I can't buy that flexibility. It looks a bit confusing for somebody that does not have the same context we have (and the opportunity to ping Dan on a daily bases).
But code is code and it implemented a needed feature!

If there is a better alternative i'd be happy to implement it? I just need some standard for getting BGP configuration from annotations?

As we discussed in issue #153 I understand the feature and I know why it is coded in this way (with the possibility to override the prefix). I can't buy that flexibility. It looks a bit confusing for somebody that does not have the same context we have (and the opportunity to ping Dan on a daily bases).
But code is code and it implemented a needed feature!

If there is a better alternative i'd be happy to implement it? I just need some standard for getting BGP configuration from annotations?

If there is a better alternative i'd be happy to implement it? I just need some standard for getting BGP configuration from annotations?

I think you should not require a provider as a prefix, you can make your own one. It can be kube-vip or bgp-configure. It will make kube-vip easier but it will probably require CCM to adapt to your standard (but I think it is something CCM already has to do because only Packet CCM will work out of the box).
If I am wrong all the CCM already set BGP in the right way what I am saying does not have any sense!

If there is a better alternative i'd be happy to implement it? I just need some standard for getting BGP configuration from annotations?

I think you should not require a provider as a prefix, you can make your own one. It can be kube-vip or bgp-configure. It will make kube-vip easier but it will probably require CCM to adapt to your standard (but I think it is something CCM already has to do because only Packet CCM will work out of the box).
If I am wrong all the CCM already set BGP in the right way what I am saying does not have any sense!

If there is a better alternative i'd be happy to implement it? I just need some standard for getting BGP configuration from annotations?

I think you should not require a provider as a prefix, you can make your own one. It can be kube-vip or bgp-configure. It will make kube-vip easier but it will probably require CCM to adapt to your standard (but I think it is something CCM already has to do because only Packet CCM will work out of the box).

+1 on a kube-vip specific the prefix here...  I would say that something like:
bgp.kube-vip.io/<attribute> would probably be a good convention to follow for bgp-related annotations.

If I am wrong and all the CCM implementations already set BGP in the right way what I am saying does not have any sense!

I'm not aware of any other CCM implementations that are interacting with any BGP related config outside of ours.
That said, to ease migrations it might make sense to have some backwards compat support on the kube-vip side at least temporarily.

If there is a better alternative i'd be happy to implement it? I just need some standard for getting BGP configuration from annotations?

I think you should not require a provider as a prefix, you can make your own one. It can be kube-vip or bgp-configure. It will make kube-vip easier but it will probably require CCM to adapt to your standard (but I think it is something CCM already has to do because only Packet CCM will work out of the box).

+1 on a kube-vip specific the prefix here...  I would say that something like:
bgp.kube-vip.io/<attribute> would probably be a good convention to follow for bgp-related annotations.

If I am wrong and all the CCM implementations already set BGP in the right way what I am saying does not have any sense!

I'm not aware of any other CCM implementations that are interacting with any BGP related config outside of ours.
That said, to ease migrations it might make sense to have some backwards compat support on the kube-vip side at least temporarily.


Remove all uses of report.lastActorEmail

Details

As part of the personalDetails email to accountID migration, I removed all uses of lastActorAccountEmail in the App.
Fixed Issues

https://github.com/Expensify/App/issues/19007
Tests

Make sure you are able to create tasks and reports as normal and without error.
Make sure the LHN loads report details as normal
Make sure the search page loads report details as normal

[x] Verify that no errors appear in the JS console

Offline tests

Same as regular tests. Chats/tasks should be created optimistically
QA Steps

Same as tests.

[ ] Verify that no errors appear in the JS console

PR Author Checklist

[x] I linked the correct issue in the ### Fixed Issues section above
[x] I wrote clear testing steps that cover the changes made in this PR

[x] I added steps for local testing in the Tests section
[x] I added steps for the expected offline behavior in the Offline steps section
[x] I added steps for Staging and/or Production testing in the QA steps section
[x] I added steps to cover failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[x] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)
[x] I tested this PR with a High Traffic account against the staging or production API to ensure there are no regressions (e.g. long loading states that impact usability).

[x] I included screenshots or videos for tests on all platforms
[x] I ran the tests on all platforms & verified they passed on:

[x] Android / native
[x] Android / Chrome
[x] iOS / native
[x] iOS / Safari
[x] MacOS / Chrome / Safari
[x] MacOS / Desktop

[x] I verified there are no console errors (if there's a console error not related to the PR, report it or open an issue for it to be fixed)
[x] I followed proper code patterns (see Reviewing the code)

[x] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick)
[x] I verified that the left part of a conditional rendering a React component is a boolean and NOT a string, e.g. myBool && <MyComponent />.
[x] I verified that comments were added to code that is not self explanatory
[x] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[x] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method

[x] If any non-english text was added/modified, I verified the translation was requested/reviewed in #expensify-open-source and it was approved by an internal Expensify engineer. Link to Slack message:

[x] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[x] I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is approved by marketing by adding the Waiting for Copy label for a copy review on the original GH to get the correct copy.
[x] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[x] I verified the JSDocs style guidelines (in STYLE.md) were followed

[x] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[x] I followed the guidelines as stated in the Review Guidelines
[x] I tested other components that can be impacted by my changes (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar are working as expected)
[x] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[x] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[x] I verified that if a function's arguments changed that all usages have also been updated correctly
[x] If a new component is created I verified that:

[x] A similar component doesn't exist in the codebase
[x] All props are defined accurately and each prop has a /** comment above it */
[x] The file is named correctly
[x] The component has a clear name that is non-ambiguous and the purpose of the component can be inferred from the name alone
[x] The only data being stored in the state is data necessary for rendering and nothing else
[x] For Class Components, any internal methods passed to components event handlers are bound to this properly so there are no scoping issues (i.e. for onClick={this.submit} the method this.submit should be bound to this in the constructor)
[x] Any internal methods bound to this are necessary to be bound (i.e. avoid this.submit = this.submit.bind(this); if this.submit is never passed to a component event handler like onClick)
[x] All JSX used for rendering exists in the render method
[x] The component has the minimum amount of code necessary for its purpose, and it is broken down into smaller components in order to separate concerns and functions

[x] If any new file was added I verified that:

[x] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[x] If a new CSS style is added I verified that:

[x] A similar style doesn't already exist
[x] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(themeColors.componentBG))

[x] If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
[x] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[x] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[x] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[x] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.
[x] I have checked off every checkbox in the PR author checklist, including those that don't apply to this PR.

Screenshots/Videos

Web

Mobile Web - Chrome

Mobile Web - Safari

Desktop

iOS

Android

Code is looking good, but yeah definitely need to test thoroughly :D

iOS seems to be acting very weirdly. Is this a known issue?

There is a known issue with assigning tasks to someone you've never chatted with before, yes. That's being handled separately.

I just tried iOS safari with a user who I already have chatted with and it seems to work fine.

Same thing with iOS:

I'm having an issue with Android chrome where it won't load the app. Won't load new.expensify.com either which is weird.

Merging since @neil-marcellini and @allroundexperts already approved! 👍

:hand: This PR was not deployed to staging yet because QA is ongoing. It will be automatically deployed to staging after the next production release.

🚀 Deployed to staging by https://github.com/Beamanator in version: 1.3.39-0 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅

🚀 Deployed to production by https://github.com/francoisl in version: 1.3.39-11 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅


creating multiple folder with UUID when mounting a drive

I am glad found this. but every time when mounting a drive it creating separate folder like UUIDx , [where x = 1,2,3,...] . Anyone know how to get rid of this. And this happens  after fresh reboot otherwise it mounting same folder.

Hello, mounting is accomplished using udiskctl2: https://github.com/SL-RU/ranger_udisk_menu/blob/master/menu.py#L208
Sadly it has no options to set mount path. Why multiple directories are bothering you?

not really. I am kind of person who keep files, scripts etc, clean and delete unnecessary files. After I found  your ' ranger_udisks_menu ' I am using it frequently because when comparing it to my little script, it has more functionality. But it keep increasing those folders which feels me  you know, and I though it was bug a or something, with the same thought, I showed up above issue.
I really appreciate your work of this and thank you

Hello, mounting is accomplished using udiskctl2: https://github.com/SL-RU/ranger_udisk_menu/blob/master/menu.py#L208
Sadly it has no options to set mount path. Why multiple directories are bothering you?

not really. I am kind of person who keep files, scripts etc, clean and delete unnecessary files. After I found your ' ranger_udisks_menu ' I am using it frequently because when comparing it to my little script, it has more functionality. But it keep increasing those folders which feels me you know, and I though it was bug a or something, with the same thought, I showed up above issue.
and once  again thank you this plugin


Iterators: Lower iterators to LLVM+func

This PR replaces the lowering from iterators to the placeholder C++ runtime with a proper lowering through LLVM. The lowering is implemented with a custom walker, which provides the state type of the upstream iterators to the lowering of each iterator op such that these state types can be inlined.
At least for the moment, this PR also uses llvm.structs for the values that are passed from one iterator to the next with Next in order to avoid the need for further lowering passes from tuple. I plan to revise this decision in one of the very few next PRs.

@nicolasvasilache: I believe I have addressed all of your comments, here in the review and what we discussed offline. The code has changed significantly -- I recommend that you briefly skim the commit history since your first review but then do a review of the full PR from scratch.

landing this and iterating on followups in-tree


How to run test for L2SM?

Are there any existing benchmarks for L2SM to test the read/write performance or YCSB workloads?
Does L2SM need any extra configuration parameters than LevelDB if I want to test L2SM with my own code?
How to adjust the read/write ratio for different type workloads? The benchmark code in db_bench is confusing.

Another question: Why here needs to multiply 120x for read and 220x for write when calculate the throughput ?

db_bench.cc/LoadThenRead()
https://github.com/ericaloha/L2SM/blob/master/db/db_bench.cc#L2850-L2852

    fprintf(debug,"Read Thrpt:%f\n",(r*120)/(float)readTotal);
    fprintf(debug,"Write:%ld, Latcy:%f msc/op \n",writeTotal,writeTotal/(float)w);
    fprintf(debug,"Write Throughput: %f MB/msc \n",(w*220)/(float)writeTotal);

Thanks for your question. "120" may be related to the size (bytes) when calculating throughput, you can change it based on your demand.
you can write your own benchmark function based on LoadThenRead with different read/write ratios, etc.


We should get an email.

I needed to put an email in the gpg key we're using to sign maven central artifacts. I used the dynatrace opensource one, but that feels weird. We should get an email address for governance things.

@oleg-nenashev, do you know if the CNCF has any recommendations for this? We're managing the domain in Netlify, so setting up email should be easy.

CNCF does not provide email services to Sandbox projects. We would have to set UP and maintien things on our own

I do not think so, AFAIK groups.io is only for incubating projects. You can ask in the CNCF Slack

I have registered<EMAIL_ADDRESS>w/ the free plan. We can migrate it to CNCF when the time comes. Ping me on slack w/ your email for an invite.


[CT-854] Cache authentication (token) for subsequent connections

Describe the bug
With Snowflake, logging in is slow (as it should be, given password hashing etc), so it should be avoided.
Every query that is executed causes a new login, as the connection is marked as closed. This should not happen, the connection should be marked as open so you're not forced to login again.
Steps To Reproduce

Have multiple queries, or set the threads to a low limit
Issue the queries
See multiple logins, you can verify by:

select *
from table(information_schema.login_history_by_user())
order by event_timestamp desc;

Expected behavior
A single login happens, maybe by thread?
System information
Happens on MacOS/Linux, pretty sure it's not a OS issue..
Python 3.10.4, dbt 1.1.1

Thanks @joshuataylor!
I left a fuller response over in https://github.com/dbt-labs/dbt-core/issues/5489#issuecomment-1188908741
I'm going to close this issue, for now, in favor of centralizing the conversation over there

Reopening as there might be a dbt-snowflake only way to go here, or at least a thread worth pulling on:

@joshuataylor
As another alternative, if we could solve this on the dbt-snowflake level in the interim this would be a big speed win. We could add the token when logged in to the connection, then add it to connection. This would involve updating the connection contract though, maybe adding a metadata or other key that connectors could use?

@jtcohen6
I don't have a good sense of whether there are any potential security risks with taking that approach. If it works, though, and substantially speeds up the process of opening new connections, that our current dbt-core approach (treating connections as a commodity) might hold muster for the foreseeable future.

the token is only valid for a certain amount of time, I think ~4 hours by default

This is a legitimate concern. We'd need some way to detect that the token is invalid — probably by trying to open a connection with it, catching the error, and falling back to re-authenticate.

However this throws a RecursionError as we're trying to access the handle via the open. I think if we could alter the credentials with a token, that would work, but I'm unsure if that's threadsafe?

Probably not thread-safe. The only object not reused / shared between threads is the connection itself. Could we use the Connection Manager's self.lock? (see here) This seems to be our pattern when getting / checking / clearing connections.

I've got a PR here which should solve the issue safely, as it won't release it: https://github.com/dbt-labs/dbt-snowflake/pull/203
This doesn't change open/close logic so you shouldn't see any problems around that.

I ran into the same problem when I install dbt via brew. I uninstall dbt via brew then pip install from scratch , the problem is gone.


MLT xml adapter

Link the Issue(s) this Pull Request is related to.
This PR adds an MLT xml adapter to the contrib adapters and closes #267 if/when merged
Summarize your change.
The MLT adapter is a write-only adapter that creates simplified mlt files aimed at usage with the melt command line player/editor.
Melt can both render a video file based on the mlt file and play it directly.
Features:

Multiple tracks
Nested tracks
Audio/video
LinearTimeWarp
FreezeFrame
ImageSequenceReference (though melt itself seems to be a bit picky on supported file formats, png works)
Dissolve
Fade In/Out (from/to black)

You may pass either a Timeline, Track or Clip directly to the adapter to produce an mlt file.
Arguments for the profile of the video may be passed as adapter keyword arguments when calling otio.adapters.write_to_string(). Please refer to the available profile data here
Limitations:

Audio handling/control is a bit limited. Audio clips that have the same source in video track will be ignored as MLT will include audio from the video track by default.
Effects directly on Track and Stack is currently not implemented. I have to investigate if it's even supported in MLT.

Reference associated tests.
A unittest file is added and attempts to cover all features.

Closing PR
The adapter is released on pypi as discussed in TSC meetings.


Consider setting trailing_slash option to False

Original discussion here: https://github.com/multinet-app/multinetjs/pull/33#issuecomment-805106618

@AlmightyYakob let me know what you think.


Merge test with main

Sorry, I was trying to merge into my main branch which I forked from LLaVA, but I accidentally created a merge request to the main LLaVA.


Fix a typo in Timex.shift/2 doc

Summary of changes
Fix a typo in Timex.shift/2 doc.
I guess the one of 'may' is extra.
Checklist

[N/A] New functions have typespecs, changed functions were updated
[N/A] Same for documentation, including moduledocs
[N/A] Tests were added or updated to cover changes
[N/A] Commits were squashed into a single coherent commit
[N/A] Notes added to CHANGELOG file which describe changes at a high-level

Coverage increased (+0.07%) to 72.188% when pulling 3f5417300d7bc165adb82cae6e7e1c78e96f5772 on TORIFUKUKaiou:fix-a-typo-in-timex-shift-2 into a9c97e2dd9cb5bed286d5a1d688f3eea8e73e6ea on bitwalker:master.


Fix building Github Pages with CI

Github pages are now hosted here: https://dlr-sc.github.io/memilio/index.html. Probably needs some beautification at some point. This PR just replicates the Gitlab CI and fixes the currently broken Github CI.
Merge Request - GuideLine Checklist
Guideline to check code before resolve WIP and approval, respectively.
As many checkboxes as possible should be ticked.
Checks by code author:

[x] There is at least one issue associated with the pull request.
[ ] The branch follows the naming conventions as defined in the git workflow.
[ ] New code adheres with the coding guidelines
[ ] Tests for new functionality has been added
[ ] A local test was succesful
[ ] There is appropriate documentation of your work. (use doxygen style comments)
[ ] If new third party software is used, did you pay attention to its license? Please remember to add it to the wiki after successful merging.
[ ] If new mathematical methods or epidemiological terms are used, has the glossary been updated ? Did you provide further documentation ?
is present or referenced. Please provide your references.
[ ] The following questions are addressed in the documentation*:  Developers (what did you do?, how can it be maintained?), For users (how to use your work?), For admins (how to install and configure your work?)
For documentation: Please write or update the Readme in the current working directory!

Checks by code reviewer(s):

[ ] Is the code clean of development artifacts e.g., unnecessary comments, prints, ...
[ ] The ticket goals for each associated issue are reached or problems are clearly addressed (i.e., a new issue was introduced).
[ ] There are appropriate unit tests and they pass.
[ ] The git history is clean and linearized for the merge request.
[ ] Coverage report for new code is acceptable.

Fixes #9

Codecov Report

Merging #10 (577f917) into main (f10a30d) will decrease coverage by 0.01%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             main      #10      +/-   ##
==========================================
- Coverage   62.42%   62.41%   -0.02%
==========================================
  Files          88       88
  Lines        5885     5885
  Branches     2063     2063
==========================================
- Hits         3674     3673       -1
  Misses        536      536
- Partials     1675     1676       +1

Impacted Files
Coverage Δ

cpp/epidemiology/abm/person.cpp
82.00% <0.00%> (-2.00%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f10a30d...577f917. Read the comment docs.

Note to reviewer: In the main branch, the Github Action yml still uses the pull_request_target trigger, that we don't actually need. This means that the Workflow file of the target branch (main) will be triggered with every pull request. Obviously this job will fail, because the whole point of this PR is to fix the pipeline.
I removed the trigger here, and the PR can safely be merged if the workflows triggered by push and pull_request succeed, even if the one triggered by pull_request_target fails.


This demo site can not be visited.

All accounts are wrong (500 code).

Request: https://fearless.zce.me/api/auth/token
Response: {"error": {"code": "500", "message": "A server error has occurred"}}

@CoXier Thanks for your feedback


connections/models config changes

moves the connection config and model definition expected by adapters into an isolated property in config/connections.js[config] and config/models/model.js[definition] to prevent any conflicts between restify-microservice config and adapter config

i merged some of your commits via git cherry-pick and modified the outstanding issues to resolve the issues in question. ive updated the docs to reflect the changes


Revise timeseries_anomalies.py

Hi, I'm impressed with your code.
Looking at the code, I found something strange, so I fixed it.
I wonder if the value of best_cost needs to be changed.
Thank you for reading.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.nrbaek seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.

Hi @BAN2ARU, thank you for catching this!
best_cost should be updated as the cost decreases.

Hi @BAN2ARU, would it be possible for you to sign the CLA?

@sarahmish I have signed the CLA. I clicked the Contributor License Agreement clicked and agreed. When I click the link again, I see “You have agreed to the CLA for sintel-dev/Orion“.


[Question] Navigation bar

Hi,
Is it possible to also add option to disable rendering navi bar (_renderNaviBar)
Otherwise I need to update it style to be compliant with app style or vs.
Thank you for component!

@barchinsky I will add an option to control it show/hide. Thanks for using this library and it will be done in one day.

@barchinsky I have published v1.1.9 and it support navigation bar show or hide. You can see README or source code to get how to use it.

@gaoxiaosong Thank you a lot for fix and for nice lib!

Thanks for using this library again~~~


Weirdness with categories in mf2 output

https://granary.io/url?input=html&output=mf2-json&url=https%3A%2F%2Faaronparecki.com%2F2018%2F03%2F14%2F3%2F
turns html with mf2 like

…
spotify

into
      "type": [
        "h-entry"
      ],
      "properties": {
        "category": [
          {
            "type": [
              "u-category"
            ],
            "properties": {
              "name": [
                "spotify"
              ]
            }
          }
        ],

which looks like it's trying to build some kind of nested object for the category?

interesting! yeah that should probably be a plain string and not a nested object, like how http://pin13.net/mf2/?url=https%3A%2F%2Faaronparecki.com%2F2018%2F03%2F14%2F3%2F does it.
given the prevalence of mf2 parsers, html -> mf2 json is a rare use case for granary, so I'm not surprised it has warts like this... but still, easily fixable. thanks for filing!


Performance Ideas for Attribute Routing

We should improve the performance of attribute routing. We did some work in 1.0.0 to implement a scalable algorithm, but we didn't really do any optimization work on the code.
For perspective, the routing part of MVC (routing + action selection) can be up to 30-40% of time spent in various techempower benchmarks. This is an area where we can probably improve much more as we haven't done much beside implement a nifty algorithm.

The algorithm we use today is three stage.
1 First, we use a trie to match the literal text portions of the template with the URL path's structure. Each node matches a single segment of the URL path. We only process literal text, so api/products/{id}, we'd match api and products, and for the {id} parameter node, we'd verify that there's some non-empty text there. We walk the trie and collect all valid matches in order.
Note that we allocate in this stage to produce a list of matches. We do this up front to avoid mixing the compute heavy part of the code with async calls (see step 3). Note that this requires us to walk the entire tree, or at least down all the valid paths.
2 Next, we process the matches in order, this does a 'full' match of the path and will do things like populate route values. We can fail at this point due to a constraint, or a complex segment (which we treat like a parameter in step 1).
3 Lastly we do an async call to get a route handler. Since we need to call the next IRouter this is async, though it's generally not. We do this step in a loop with step 2, and short circuit on the first success.

Here's one take: https://gist.github.com/rynowak/a41dc558230e610d71584e8673a8849c
Benchmark.net says this is about 2x faster for the cases I tested. I haven't done any real profiling work on this code yet, I just changed the data structures. I also didn't implement support for catch-all, so caveat emptor and such.
There are two high-level difference between the current implementation and my prototype.
First, mine is dense.  Rather than a literal tree of nodes to represent the trie, I'm using an array-based representation. This is intentionally compact, cache and GC friendly. It could potentially be made more compact as well. My main goal here is to avoid pointer chasing. The current node and its siblings are always laid out consecutively in the entry array because we build it using a BFS.
Secondly, my implementation doesn't handle IRouter, it only handles IRouteHandler. IRouteHandler is a more minimal contract that fits how attribute routing is used in practice, but it's not async. MVC's handler only requires IRouteHandler. The reason for this is that if you're not trying to avoid making a lot of async calls, a lot of the complexity just melts away.
Thirdly (yeah it's a bonus) my implementation is doing a linear search on the strings. The current shipping implementation is doing a dictionary lookup and I think that it's non-optimal given the size of N and the relative slowness of OrdinalIgnoreCase hashing. Plus we have to substring 👎.  My prototype doesn't include it, but the design anticipates using a jump table based on the first letter or length of the string. Given that a jump table is possible, the jump table could be non-existent (linear search), a simple index based on the first letter into a sorted list (skiplist-like) or a dictionary that tells you exactly what index to jump to, and we could use all of these in one implementation depending on the branchiness of the node.

/cc @davidfowl @benadams for weird science

This test is now checked in here: https://github.com/aspnet/Routing/tree/dev/test/Microsoft.AspNetCore.Routing.Performance (without the new experimental implementation)

Just discussed this with @rynowak. The big question is how we want to deal with async support:

As Ryan notes above, the existing TreeRouter works with IRouter routes, where the process of matching them against an incoming context is (in general) asynchronous. Having to make an async call to each candidate route in series isn't good for perf.
Fortunately, MVC's attribute routing always creates routes that implement IRouteHandler, which means they can be matched against contexts synchronously. So we could make the router do all its matching purely synchronously - it would work.

But then there's back-compatibility. TreeRouter is a public API - we can't really drop its async support. And if it retains any async support, it has to thread async-ness throughout its operation, which significantly limits the perf gains - even if in practice everything gets resolved synchronously.
Therefore if we want a new, more lightweight alternative, it probably has to be a new separate API, with the old TreeRouter left as a vestige for back-compat. Unfortunately that means not only duplicating the code, but also ~150 unit tests (BTW the existing ones rely on async support).
Plan
We think it's probably too late to make such a dramatic change to routing before 2.0.0 goes out. We have had regressions in routing in the past caused by optimisations, and don't want that again. Also, it's likely that in 2.1.0, routing will somehow be moved into the middleware layer, where its API surface would be changing anyway. So that might be a better time for this.
Partial implementation
I just took @rynowak's quick experimental implementation and built a more readable version that's closer to what we might ship: https://github.com/aspnet/Routing/commit/203c65d005fa535c75689fd0f9510fe98d821b9c This still lacks a lot of functionality (like the experimental implementation does, such as handling complex route segments or catch-alls).
Stats
Using Ryan's benchmark,

The existing shipping TreeRouter takes approx 1000ns per iteration
The changes in the partial implementation (https://github.com/aspnet/Routing/commit/203c65d005fa535c75689fd0f9510fe98d821b9c) reduce this to approx 450ns per iteration (i.e., performs as well as Ryan's experimental implementation, because logically it's just about the same)
If I add back async support by threading Tasks through everywhere, it goes up to about 670ns per iteration. It also adds quite a bit to GC pressure because of all the Task instances, which might be optimisable by inlining some of the matching methods into each other so fewer Task instances get created.

Thanks, moving this to 2.1.0


Make flaky tests retry.

Closes #3214.

Merging when CI completes.

@lukesneeringer Good to delete this branch?

Yep. Sorry.


[chore] Regenerate offsets

I noticed that the offsets in main weren't the latest make offsets output.

lgtm, I just noticed this too


corrects costs when constants in objective for mosek(_cli)

Also includes unit tests

@pgkirsch I suggested a simpler approach in #305. If you like it, you can merge it (it will get merged into this PR). Then, ask jenkins to retest and if everything passes, you can merge to master.
I think #296 needs to remain open since this fix is not fully satisfying, but we can change the milestone to 0.3.0.

retest this please

Jenkins built successfully. Merging to master, and moving to 0.3.0.


[FORMATS-146] Add variant quality field.

Resolves #146.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/bdg-formats-prb/167/
Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/bdg-formats-prb/168/
Test PASSed.

Then about releases, this will be required for ADAM 0.23.0, so we cut a 0.11.2 for bdg-formats with only this PR in it?

Addressed review comments.

Then about releases, this will be required for ADAM 0.23.0, so we cut a 0.11.2 for bdg-formats with only this PR in it?

Yup!

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/bdg-formats-prb/169/
Test PASSed.

Thank you, @fnothaft!


bug: particle renderer bug

The state of the particles affects each other
Recurring demo : editor id 309000049

The current particle system will be deprecated


Docker version vulnerable to cve-2023-29406

Summary
The current docker version 20.10.15 is vulnerable to a few issues, like:
https://nvd.nist.gov/vuln/detail/cve-2023-29406
https://nvd.nist.gov/vuln/detail/cve-2023-39325
And same for containerd: 1.7.2 is also affected by a few issues
https://nvd.nist.gov/vuln/detail/cve-2023-47108
Is there any plan on updating docker and containerd versions for this image?

This was addressed in #231.

Thanks for the ticket, we have bumped to docker 25.0.3 for AL2023
Also keep in mind that the docker versions in NIST CVE notices don't always 100% map to the docker version in Amazon Linux. If you look at CVE 2023-39325 in ALAS, for example, and then click through to the Amazon Linux 2 advisories, you can see that this CVE is fixed in 20.10.15 in Amazon Linux: https://alas.aws.amazon.com/cve/html/CVE-2023-39325.html
As soon as Amazon Linux 2 releases docker 25.0.3 we will be bumping our AL2 AMIs.

As soon as Amazon Linux 2 releases docker 25.0.3 we will be bumping our AL2 AMIs.

I think Docker 25.0.3 is now in the AL2 repos, could we get an AMI bump? 🙏

I think Docker 25.0.3 is now in the AL2 repos, could we get an AMI bump? 🙏

Thank you for the quick response! https://github.com/aws/amazon-ecs-ami/pull/269
I think this issue can be closed.


Refactor Uni.awaitSuspending

Follow the existing extension modules from https://github.com/Kotlin/kotlinx.coroutines/tree/master/reactive, add Uni.awaitSingle() and Uni.awaitSingleOrNull instead, the naming is more clear and easy migrated from other ReactiveStreams/JDK9 Flow impls.

lets follow up on this discussion in https://github.com/smallrye/smallrye-mutiny/issues/1325


feat: Support loading config options from Info.plist

Goal
Read config options from Info.plist values, supporting workflows where options are specified directly or added to script build phases.
Design

BugsnagConfiguration adds a new utility constructor, loadConfig, which reads the infoDictionary values under bugsnag within the main bundle.
Options are then passed to the config builder which constructs a configuration if the options are all valid. Invalid configurations are discarded to avoid cases where endpoints or other critical options are mistyped and reports sent to an incorrect server.

This has been implemented in #582.  Closing.


Custom attributes replacement

Enhancements to HTML-to-JSON Converter
This pull request introduces several enhancements to our HTML-to-JSON converter for the ChaiBuilder project:
Custom Attribute Replacement:
Implemented support for generic attribute replacement using the data-chai-replace-* prefix. This allows for dynamic substitution of HTML attributes with specified values, providing greater flexibility and customization.
Enhanced the attribute processing logic to better handle custom attributes and replace specified attributes based on the data-chai-replace-* prefix.
Example: <div data-chai-replace-_name="CustomBlockName">Content</div> will replace the _name attribute with "CustomBlockName".

Closing this PR. this feature is added to import logic using $name attribute


Participation Rating Presentation Change (Frontend)

:warning: Please install the  to ensure uploads and comments are reliably processed by Codecov.
Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 63.64%. Comparing base (50c64d6) to head (6b6bbaa).

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@            Coverage Diff            @@
##             master     #151   +/-   ##
=========================================
  Coverage     63.64%   63.64%
  Complexity      625      625
=========================================
  Files            68       68
  Lines          2074     2074
  Branches         85       85
=========================================
  Hits           1320     1320
  Misses          720      720
  Partials         34       34

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


chore(gradle): update bundled tomcat 8 version to latest

Checklist

[x] the individual contributor license agreement is signed
[x] commit message follows commit guidelines

Description of change

Updates version to latest V8.0 series

@ChristianMurphy -- did you do a comparison, and visually confirm that the files in etc/tomcat -- which are overlaid in the build process -- match up with the versions of the same files in the updated Tomcat dependency?
I don't think it's ever safe to skip that step.

did you do a comparison, and visually confirm that the files in etc/tomcat -- which are overlaid in the build process

I did

match up with the versions of the same files in the updated Tomcat dependency?

They do

I don't think it's ever safe to skip that step.

Feel free to add this to the uPortal docs.

uPortal works with Tomcat 8.5. Why not upgrade to 8.5.23?

I'm good with that @bjagg.

closing in favor of #64


(serveStatic plugin) Add an option to not append req.path() to the file path

Hi,
I was kinda annoyed that the restify static plugin was behaving differently than the one in express.
Let me explain :
server.get(/^\/test\/.*$/, restify.serveStatic({
  directory: path.resolve(__dirname, '..', 'docs'),
  default: 'index.html'
}));

Here I expect that, when I make a request to /test/document.html, the plugin tries to serve me the content of ./docs/document.html - and not ./docs/test/document.html ... kind of like a symbolic link.
So, the change I made, is to add an optional appendRequestPath option. The change is backward-compatible (it's set to false by default).
Additionally I fixed the default file test, which wasn't testing the right thing ... (it was making a request to /dir/defaultfile.ext rather than /dir/)

I don't know why the tests are failing, it was working fine locally.
It doesn't seem related to my changes.

Thanks for the PR! Just a minor nit, your comment says that appendRequestPath is set to false by default, but the code indicates it's set to true when not defined. Setting it to true by default seems like the right thing based on the logic you wrote, just wanted to verify that was your intent.
Also, do you mind either adding another issue for documenting this new feature or updating the docs?

Yes, it should have been true in my comment.
Sure I can update the docs. But I'm not sure where I should send the PR exactly ? In the node-restify project on the 5.x-docs branch ?

The 5.x-docs branch is perfect. Thanks so much!

Thanks again for the PR.


Integrate New WebUI with dream.py

This is the first draft of integrating the new WebUI with dream.py
Major Changes

Launching dream.py with --web now launches the new WebUI.

NOTE
This is just a first draft. At the moment, I just ported over the entire thing as it is with some changes to let dream.py handle the loading of the models and etc.
Everything should work like it has been if you launched backend/server.py
There's a ton of code cleanup that can be done. I will work on it through the week and have this finalized by then.

@psychedelicious Do a thorough test of the new server_integrated.py please to make sure everything is working as intended. Feel free to make updates to it. Once we know it's all set in place, I'll remove the server.py file and rename the integrated version to server.py.

Almost everything works fine, thanks for helping improve things. Only problem is images not being served from flask.
I have made a couple very small changes in my branch of your branch:

Manually implemented PR#826 (was breaking progress images)
"Fix" for images 04304b26fce488247559b56e8cc9a68a63e5194e

Without the "fix" I made, flask was trying to serve paths like outputs/img-samples/img-samples/12345.png. The change I made fixes the issue for me, but only for the default outdir arg. I can't wrap my head around making it work.
As I went through, there are some obvious refactoring candidates, suspect you've also found others:

Wrapping each event handler with try: except:, emitting an 'error' with caught exceptions (see issue #817)
Clean up everything related to socketio sending progress/status updates
The handlers for runESRGAN and runGFPGAN share like 95% of the same code

Manually implemented PR#826 (was breaking progress images)

Implemented this fix.

Without the "fix" I made, flask was trying to serve paths like outputs/img-samples/img-samples/12345.png. The change I made fixes the issue for me, but only for the default outdir arg. I can't wrap my head around making it work.

This has not been an issue for me. In fact that fix you suggest is what is breaking it. Otherwise the files are being served from the output folder on the origin host which is what it should be doing.

Wrapping each event handler with try: except:, emitting an 'error' with caught exceptions (see issue Web server backend crashes if a file in local storage is missing #817)

While adding exceptions will work, the ideal way to solve this is to add the local storage entries as soon as the page loads if they don't exist. So calling them does not break the app.
But I'll go through a cleaning pass soon.

The handlers for runESRGAN and runGFPGAN share like 95% of the same code

Yep. Also find ways to minimize other code duplications if any. I'm also thinking of code splitting maybe because the file is pretty long.

@psychedelicious I am also going to work on a reskin while I am working on this. I noticed a few things that need fixing. Hope you can do them while I work on the styling.

When I press the cancel button, I expect the progress bar at the top to reset. Instead it just stops where it is.
Pressing the cancel button should stop the status text at stop to say 'Canceled' or something. But it stops at Generating.

Please fix these two. It's important to give the user a visual feedback when the cancellation happens.
I'll keep beeping you on this thread as I find more stuff. Hope you don't mind. :)

Can fix those no problem.
I'll respond in more detail a bit later, but for now, if you are reskinning, please check out the Chakra theming system. It's quite nice. I'm using extendTheme() here already to customize individual component styles.
You can add a global property, as described in the docs, use CSS selectors as keys with CSS rules as values and it gets compiled into the CSS. I have been hacking away at the node-based UI and styling with this, you can get a better example here. Dark mode is super easy with this.
If we have some awesome color scheme, it can be defined and override the color and other theme tokens automagically when selected. I.e. you use blue.500 and all the other theme tokens throughout theme.ts, then define a colorscheme that changes which actual color those tokens resolve to. So user selects the Cyberpunk theme from a dropdown and everything changes.
Sorry if you already know all this!

I'll respond in more detail a bit later, but for now, if you are reskinning, please check out the Chakra theming system. It's quite nice. I'm using extendTheme() here already to customize individual component styles.
You can add a global property, as described in the docs, use CSS selectors as keys with CSS rules as values and it gets compiled into the CSS. I have been hacking away at the node-based UI and styling with this, you can get a better example here. Dark mode is super easy with this.
If we have some awesome color scheme, it can be defined and override the color and other theme tokens automagically when selected. I.e. you use blue.500 and all the other theme tokens throughout theme.ts, then define a colorscheme that changes which actual color those tokens resolve to. So user selects the Cyberpunk theme from a dropdown and everything changes.
Sorry if you already know all this!

I've done some research and decided against using the extendTheme functionality to theme further. There are limitations to it beyond the API they provide. There are some cases where I would like a full override and that is not possible. Infact because Chakra uses Emotion to inject CSS after hydration, using CSS classes or any other framework on top of it becomes difficult.
So I implemented as a cache system with Emotion to prepend the Chakra classes instead of appending them. This now allows us to theme on top of the Chakra UI.

I'm doing a theming pass using SCSS. This styling will be purely visual as I intend to retain the functionality that ChakraUI components provide.
There are a few advantages of doing this.

Anyone wanting to style or theme does not have to learn Chakra UI for this. They can just use CSS.
While theming becomes easier, we can still continue to use Chakra to build further UI elements.
There will be a dedicated color scheme files. All anyone ever needs to do is alter color values or create a new file with the same values to generate a whole new theme. This is a lot easier than getting them to mess with the ChakraUI components which could be widespread across the entire app.

I'll give you a preview of how I'm doing it once I have something substantial to show. At the moment, I'm setting up the barebones to get rolling.

There is a LOT to do. You'll see when I'm done.

Thanks for giving it consideration. I tried to keep styling out of the project as I went, but there are some layout-related style props on most of the components. I'm not sure if stripping them so they are bare Chakra UI components, or leaving them and building on top is better for what you are doing.
Anyways, I look forward to the upcoming overhaul!

Thanks for giving it consideration. I tried to keep styling out of the project as I went, but there are some layout-related style props on most of the components. I'm not sure if stripping them so they are bare Chakra UI components, or leaving them and building on top is better for what you are doing.
Anyways, I look forward to the upcoming overhaul!

I'll take care of that as I go. I am removing styling exclusive components and retaining the functional components. This way we have a simpler JSX too.

@blessedcoolant I am running a setup with two GPUs, therefore running two instances. I was looking for a way to send prompts as a curl request to the web server. Do you know if that is supported by your work? I guess that would also make testing easier.

Styling update. Just the prompt bar done for now. But I like how its shaping up. The work will continue through the week.
https://user-images.githubusercontent.com/54517381/192588100-e8bd97ff-668f-4675-a92e-5078f282a849.mp4

Some more progress on the UI. Did some initial drafts for the image viewer and the gallery. Lots more to do. The options panel will have a major overhaul.
https://user-images.githubusercontent.com/54517381/192617239-8bafbdce-e978-44fd-8e8c-289260706fb9.mp4

I dig the purple :)
@plucked Please raise an issue with your request

@lstein This is good to go. I'm leaving @psychedelicious original server.py in the files for the time being. If there's any emergency porting bugs that need to be fixed, it'll be there for reference.  There are some niche issues that we know of which we will tend to in the coming days.
But for now this should be good to go.
The first integration of the new WebUI in to the main script. Big thanks and kudos to everyone who contributed to this massive undertaking.
In the coming days or week, I will make a push for a redesign of the new WebUI styling (screenshots that you can see above) and then refine it further based on user feedback.
Let's go.

@blessedcoolant. @psychedelicious;
I don't have time to thrash this right now, so I'm trusting that the two of you are fully confident in this.

I am very excited to check this out.

I may have deleted the import line from backend.invoke_ai_web_server when I resolved a conflict with dream.py. I've readded it and am testing. Will directly commit a patch if it checks out.

Windows 11, multiple browsers
python scripts/dream.py --web
and then browsing to the indicated  http://<IP_ADDRESS>:9090 nets me a blank white page.
Console says
Failed to load module script: Expected a JavaScript module script but the server responded with a MIME type of "text/plain". Strict MIME type checking is enforced for module scripts per HTML spec.

I may have deleted the import line from backend.invoke_ai_web_server when I resolved a conflict with dream.py. I've readded it and am testing. Will directly commit a patch if it checks out.
..yep, that was me. Fortunately I caught and fixed it in minutes. The new interface is working great. The one thing I noticed is that when I first loaded, I got a bunch of broken image icons because I had cleared out my output directory earlier and none of the cached local storage images were valid. The issue was fixed by reloading the page in the browser.
Here's the message that printed out on the command line:
>> Page None of images requested (page size 50 offset None)
Exception in thread Thread-9:
Traceback (most recent call last):
  File "/u/lstein/.conda/envs/ldm/lib/python3.9/threading.py", line 980, in _bootstrap_inner
    self.run()
  File "/u/lstein/.conda/envs/ldm/lib/python3.9/threading.py", line 917, in run
    self._target(*self._args, **self._kwargs)
  File "/u/lstein/.conda/envs/ldm/lib/python3.9/site-packages/socketio/server.py", line 730, in _handle_event_internal
    r = server._trigger_event(data[0], namespace, sid, *data[1:])
  File "/u/lstein/.conda/envs/ldm/lib/python3.9/site-packages/socketio/server.py", line 755, in _trigger_event
    return self.handlers[namespace][event](*args)
  File "/u/lstein/.conda/envs/ldm/lib/python3.9/site-packages/flask_socketio/__init__.py", line 282, in _handler
    return self._handle_event(handler, message, namespace, sid,
  File "/u/lstein/.conda/envs/ldm/lib/python3.9/site-packages/flask_socketio/__init__.py", line 826, in _handle_event
    ret = handler(*args)
  File "/u/lstein/projects/SD/InvokeAI/backend/invoke_ai_web_server.py", line 169, in handle_request_images
    chunk_size * (page - 1) + offset,
TypeError: unsupported operand type(s) for -: 'NoneType' and 'int'

Yep. This is a known issue. We'll fix this up in one of the coming bug fix patches.
@psychedelicious Maybe the solution to this is to add a check where we check if the file exists on disk. If it doesn't then we just remove it from the image gallery and push that.

Windows 11, multiple browsers
python scripts/dream.py --web
and then browsing to the indicated http://<IP_ADDRESS>:9090 nets me a blank white page.
Console says
Failed to load module script: Expected a JavaScript module script but the server responded with a MIME type of "text/plain". Strict MIME type checking is enforced for module scripts per HTML spec.

Can you clear the older stuff and give this a fresh pull and try again?

Can you clear the older stuff and give this a fresh pull and try again?

Same same. @psychedelicious and I ran into this once before, but I can't remember the resolution. Something about MIME types on Windows.

Can you clear the older stuff and give this a fresh pull and try again?

Same same. @psychedelicious and I ran into this once before, but I can't remember the resolution. Something about MIME types on Windows.

I'm on Windows too. Not facing this error. But let me try to look it up. Maybe @psychedelicious remembers the fix.

Pulling down to put it through the ringer. Will report back.


Enable JMX for Cassandra using standard container

@echauchot I found some time to check why tests are failing in my environment for https://github.com/apache/flink-connector-cassandra/pull/3. You're on a Linux machine, right?  It turns out it's docker networking (I suppose). The lack of host networking support on Macs prevents the JMX url from working.
Also, reading a bit on the topic (http://thegridman.com/coherence/oracle-coherence-on-docker/#jmx), JMX doesn't work reliably in different environments with mapped ports. There's two options:

use Cassandras standard JMX port with a fixed port binding (but risking that this port might be used otherwise already)
or dynamically pick a free port and configure JMX for Cassandra from scratch (without auth) to use that port (again with a fixed port binding).

I just pushed the code here for reference, feel free to use it or discard ...

@mosche thanks for your contribution and valuable improvements !

@echauchot I found some time to check why tests are failing in my environment for apache#3. You're on a Linux machine, right? It turns out it's docker networking (I suppose). The lack of host networking support on Macs prevents the JMX url from working.

Nice catch! I have no Mac to test, as I'm indeed on Linux. Better to be OS agnostic. Thanks !

Also, reading a bit on the topic (http://thegridman.com/coherence/oracle-coherence-on-docker/#jmx), JMX doesn't work reliably in different environments with mapped ports. There's two options:

use Cassandras standard JMX port with a fixed port binding (but risking that this port might be used otherwise already)
or dynamically pick a free port and configure JMX for Cassandra from scratch (without auth) to use that port (again with a fixed port binding).

Free port option LGTM

I just pushed the code here for reference, feel free to use it or discard ...

I'll test your code and use it if this works on Linux as well. I'll also keep your commit in the history for credits

@mosche I just tested locally on my linux machine, it work fine. Thanks for the improvement: no more tailored docker image, more OS agnostic tests and quicker tests (no more start/stop).


Redirect legacy search and browse actions

Fixes #3799.

Redirect /pypi/?:action=browse&c=<classifier_id> to /search/?c=<classifier_name>
Redirect /pypi?:action=search&term=foobar&submit=search to /search/?q=foobar

(Also it's possible they came from somewhere where that message isn't really actionable, e.g. Google: https://www.google.com/search?q=site%3Apypi.python.org+inurl%3Asearch+inurl%3Aaction)

Spiders should pick that up eventually though, right? 🕷

Yeah, especially b/c it's a 301, but still, a message like that isn't really appropriate for those users.