Only a single cpu core is utilized when join-ing on tokio runtime

Only a single CPU core is utilized when join-ing futures on tokio runtime. The issue does not arise when using async_std.
some stats:
# async_std:naive
Total duration: 2.3061 s
Rate:<PHONE_NUMBER> reqs/s

# async_std:futures_concurrency_join
Total duration: 2.8707 s
Rate:<PHONE_NUMBER> reqs/s

# tokio:naive
Total duration: 2.4885 s
Rate:<PHONE_NUMBER> reqs/s

# tokio:futures_concurrency_join
Terminated after a few minutes. Utilization here is limited to a single CPU core.

tokio test cases:
use std::time::{Duration, Instant};
use tokio::{spawn, time::sleep};

#[derive(Debug, Clone)]
struct Workload {
    task_duration: Duration,
    n_tasks: usize,
    start: Instant,
}

#[tokio::main]
async fn main() {
    let workload = Workload {
        task_duration: Duration::from_millis(100),
        n_tasks: 1000000,
        start: Instant::now(),
    };

    // naive(workload.clone()).await;
    futures_concurrency_join(workload.clone()).await;

    let total_duration = (Instant::now() - workload.start).as_secs_f64();
    let rate = workload.n_tasks as f64 / total_duration;
    println!("Total duration: {:.4} s", total_duration);
    println!("Rate: {:.4} reqs/s", rate);
}

async fn naive(workload: Workload) {
    let tasks = (0..workload.n_tasks)
        .map(|_| {
            spawn({
                async move {
                    sleep(workload.task_duration.clone()).await;
                }
            })
        })
        .collect::<Vec<_>>();

    for task in tasks {
        task.await.ok();
    }
}

async fn futures_concurrency_join(workload: Workload) {
    use futures_concurrency::prelude::*;

    let tasks = (0..workload.n_tasks)
        .map(|_| async move {
            sleep(workload.task_duration.clone()).await;
        })
        .collect::<Vec<_>>();

    tasks.join().await;
}

async_std test cases:
use std::time::{Duration, Instant};
use async_std::task::{spawn, sleep};

#[derive(Debug, Clone)]
struct Workload {
    task_duration: Duration,
    n_tasks: usize,
    start: Instant,
}

#[async_std::main]
async fn main() {
    let workload = Workload {
        task_duration: Duration::from_millis(100),
        n_tasks: 1000000,
        start: Instant::now(),
    };

    // naive(workload.clone()).await;
    futures_concurrency_join(workload.clone()).await;

    let total_duration = (Instant::now() - workload.start).as_secs_f64();
    let rate = workload.n_tasks as f64 / total_duration;
    println!("Total duration: {:.4} s", total_duration);
    println!("Rate: {:.4} reqs/s", rate);
}

async fn naive(workload: Workload) {
    let tasks = (0..workload.n_tasks)
        .map(|_| {
            spawn({
                async move {
                    sleep(workload.task_duration.clone()).await;
                }
            })
        })
        .collect::<Vec<_>>();

    for task in tasks {
        task.await
    }
}

async fn futures_concurrency_join(workload: Workload) {
    use futures_concurrency::prelude::*;

    let tasks = (0..workload.n_tasks)
        .map(|_| async move {
            sleep(workload.task_duration.clone()).await;
        })
        .collect::<Vec<_>>();

    tasks.join().await;
}

async-std v1.12.0
futures-concurrency v7.6.0
tokio v1.37.0

macOS Sonoma 14.4.1

Thanks for reporting this @manifest. I can reproduce this locally, and we should look into the root cause of this further.
You mentioned Tokio only uses a single core with this; how did you measure that? I've been able to see the slowdown, but I didn't yet look at core utilization.
I'd like to test the scaling with FuturesUnordered to see whether this has something to do with futures-concurrency specifically, or whether it's something to do with Tokio and intermediate wakers.

You mentioned Tokio only uses a single core with this; how did you measure that? I've been able to see the slowdown, but I didn't yet look at core utilization.

I've just checked the CPU usage in htop. The process with the latest test case using Tokio runtime utilized up to 100% (unitilization of a single cpu core). My comment isn't entirely accurate because different CPU cores were involved, but it seemed as sequential execution.

There is another interesting case resulting in a stalled execution. At least on my machine, I can constantly reproduce it when n_tasks is greater than 4.
#[derive(Debug, Clone)]
struct Workload {
    task_duration: Duration,
    n_tasks: usize,
    start: Instant,
}

#[tokio::main]
async fn main() {
    let workload = Workload {
        task_duration: Duration::from_millis(100),
        n_tasks: 10,
        start: Instant::now(),
    };

    futures_concurrency_co_stream(workload.clone()).await;

    let total_duration = (Instant::now() - workload.start).as_secs_f64();
    let rate = workload.n_tasks as f64 / total_duration;
    println!("Total duration: {:.4} s", total_duration);
    println!("Rate: {:.4} tasks/s", rate);
}

async fn futures_concurrency_co_stream(workload: Workload) {
    use futures_concurrency::prelude::*;

    (0..workload.n_tasks)
        .collect::<Vec<_>>()
        .into_co_stream()
        .for_each(|_| async {
            sleep(workload.task_duration.clone()).await;
        })
        .await;
}

@manifest The last test case you posted almost certainly will hit https://github.com/yoshuawuyts/futures-concurrency/issues/182 - tokio::sleep + ConcurrentStream currently don't work as expected. That still needs fixing.

I've created two more test cases using FuturesUnordered and futures_util::future::join_all:
async fn future_util_join_all(workload: Workload) {
    let tasks = (0..workload.n_tasks)
        .map(|_| async move {
            sleep(workload.task_duration.clone()).await;
        })
        .collect::<Vec<_>>();

    join_all(tasks).await;
}

async fn futures_unordered_join(workload: Workload) {
    let tasks = (0..workload.n_tasks)
        .map(|_| async move {
            sleep(workload.task_duration.clone()).await;
        })
        .collect::<Vec<_>>();

    let mut group = FuturesUnordered::from_iter(tasks.into_iter());

    while let Some(_) = group.next().await {}
}

The results for n_tasks = 10_000 is as follows:
tokio naive
Total duration: 0.1288 s
Rate: 77654.0458 reqs/s

futures_concurrency_join
Total duration: 0.6355 s
Rate: 15736.2799 reqs/s

futures_join_all
Total duration: 0.7635 s
Rate: 13097.1088 reqs/s

futures_unordered
Total duration: 0.8850 s
Rate: 11299.5670 reqs/s

Comparatively the futures_concurrency impl seems to perform better than the alternative concurrency impls, but all of them seem to do significantly worse than using tokio's for..in loop over handles. My hypothesis right now is that there are two things happening here which we should separate:

Tokio isn't particularly happy with handles being placed inside of concurrency operators. That's something to figure out on the Tokio side.
If we go from 10_000 to 100_000 the futures_concurrency::Join impl isn't too happy about it, and seems to hit a pathological case. That's likely something on our side, and we should figure out why that happens.

Interestingly, the excessive awaiting points seems to be hurting performance. For me that is counter-intuitive, because I was thinking about them as zero-cost abstractions. There is almost 30% degradation in performance between the following test cases. I use n_tasks = 1_000_000.
futures_unordered_join
Total duration: 0.2884 s
Rate: 3467896.8878 tasks/s

futures_unordered_join_excessive_await_points
Total duration: 0.4054 s
Rate: 2466827.3394 tasks/s

async fn futures_unordered_join(workload: Workload) {
    use futures::stream::{FuturesUnordered, StreamExt};

    let tasks = (0..workload.n_tasks)
        .map(|_| {
            sleep(workload.task_duration.clone())
        })
        .collect::<Vec<_>>();

    let mut group = FuturesUnordered::from_iter(tasks.into_iter());
    while let Some(_) = group.next().await {}
}

async fn futures_unordered_join_excessive_await_points(workload: Workload) {
    use futures::stream::{FuturesUnordered, StreamExt};

    let tasks = (0..workload.n_tasks)
        .map(|_| async {
            sleep(workload.task_duration.clone()).await
        })
        .collect::<Vec<_>>();

    let mut group = FuturesUnordered::from_iter(tasks.into_iter());
    while let Some(_) = group.next().await {}
}

I've compared naive vs futures_unordered_join on n_tasks = 1_000_000 just for the sake of curiosity. futures_unordered_join performs more then 3 times faster on my machine.
naive
Total duration: 0.8474 s
Rate: 1180038.8759 tasks/s

futures_unordered_join
Total duration: 0.2884 s
Rate: 3467896.8878 tasks/s

async fn naive(workload: Workload) {
    let tasks = (0..workload.n_tasks)
        .map(|_| spawn({
            sleep(workload.task_duration)
        }))
        .collect::<Vec<_>>();

    for task in tasks {
        task.await.ok();
    }
}

Here is stats for n_tasks = 10_000 on my machine. All implementations behave on par on my machine with naive and futures_concurrency_join slightly behind.
naive
Total duration: 0.1079 s
Rate: 92649.3189 tasks/s

futures_unordered_join
Total duration: 0.1049 s
Rate: 95305.6797 tasks/s

futures_unordered_join_excessive_await_points
Total duration: 0.1048 s
Rate: 95426.9807 tasks/s

futures_concurrency_join
Total duration: 0.1198 s
Rate: 83464.3257 tasks/s

My previous analysis wasn't right; I spent some time on a plane debugging this today, and I realized a number of things:

I indeed wasn't measuring things right - that means it's not a Tokio issue and it's squarely on us.
What we're seeing is quadratic scaling, that is almost certainly because our iterations are fairly expensive.
The reason why futures-concurrency uses a single core is because it's only concurrent, not parallel. The examples should use task.spawn to also be parallel.
When using task.spawn, FuturesUnordered actually ends up performing slightly better than the naive tokio impl.

I think what needs to happen is change the internal slab impl to one with better iteration speeds as well as referential stability. That would not only solve the quadratic scaling, it would also address the issues in #182.


How do I override the view

I want to override the view used to create notifications.  I used to be able to do that with a template, but that doesn't work any more.
I don't want to mess with a ton of php code, just override the view is all.

Hey @murrant 👋, have you by any chance started a pull request or some initial code on this?

If you look at the flasher.php config file in that PR, I have default set to template.librenms.  I need to update that :)

Looks like I'll just need to downgrade flasher, thanks.


[Bug]: brick/geo Package not installed as part of package dependencies

What happened?
I installed laravel-postgis via composer, however, while I was working on using this package within a test application, I noticed it could not find Brick\Geo\Point for use in a Model cast. It seems as though the brick/geo package wasn't installed as one of the dependencies during the composer require process.
It might help if it were installed by default along with the laravel-postgis package or if perhaps the README could be updated to state that brick/geo would need to be installed separately if the functionality it provides is needed.
How to reproduce the bug

Start a fresh Laravel project
Install laravel-postgis via composer.
Try to setup a model cast based on example in README.md

Package Version
0.4.0
PHP Version
8.3.14
Laravel Version
11.34.2
Which operating systems does with happen with?
macOS
Notes
No response

The brick/geo integration is part of the next version. I meant to release it a few days ago, but life happened. You can already use it if you install the dev branch, but I still need to test the model cast.
I’ll try to get a new version released today and will keep this ticket open till then!

I've just released v0.5.0, which includes the brick/geo integration.

Ok great. Thanks @boris-glumpler


cmd/objbench: check list error

close #3306
CI fixed by  #3310

Codecov Report
Patch coverage: 78.94% and no project coverage change.

Comparison is base (f82e250) 55.42% compared to head (8e24f19) 55.42%.

:exclamation: Current head 8e24f19 differs from pull request most recent head 6e66ec2. Consider uploading reports for the commit 6e66ec2 to get more accurate results

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #3308   +/-   ##
=======================================
  Coverage   55.42%   55.42%
=======================================
  Files         142      143    +1
  Lines       33920    33920
=======================================
  Hits        18799    18799
  Misses      13121    13121
  Partials     2000     2000

Impacted Files
Coverage Δ

cmd/objbench.go
56.69% <40.00%> (-0.27%)
:arrow_down:

pkg/object/utils.go
92.85% <92.85%> (ø)

... and 5 files with indirect coverage changes
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


Update schema to 0.6.2, enable py3k builds

In part for the python 3 pledge, and to enable apps which depend on this to make their own py3k builds.

Not sure what's going on with the build, but it's probably not related to your changes as PR #583 seems to have the same problem. Still investigating.

This is the script that causes the error. It looks like for some reason Travis can't find basic Python conda packages.

I think it might be either first or last line of this block which is causing the try: block to fail. On my local box, conda_build.config does not exist and .short_build_prefix is not a property. A bit trickier to figure out if that's whats failing on the docker image.

Hooray!

In this case it should be fine, but in the future, version strings should be quoted so there's no risk of them being parsed as floats

In [11]: yaml.load('version: 0.6.2')
Out[11]: {'version': '0.6.2'}

In [12]: yaml.load('version: 0.6')
Out[12]: {'version': 0.6}


[Low Priority] ERR_HTTP_HEADERS_SENT warning

Response object of NextJS shows this error when we're setting multiple "set-cookie" headers, which is required by the logic of session handling. I'm still not clear how to fix this issue. I played with different suggestions from here: https://stackoverflow.com/questions/52122272/err-http-headers-sent-cannot-set-headers-after-they-are-sent-to-the-client, but it wasn't helpful in our case. So probably it's something nextjs specific.
Ref: https://moovweb.atlassian.net/browse/PTR-14

@egormesyats Please try upgrading to the latest version of XDN and retesting. I was seeing the same issue for the salesforce connector, but after upgrading, I am not seeing this error anymore.


Neither click event nor touchstart event seem to be working on Android chrome

I used the sample here: https://hiukim.github.io/mind-ar-js-doc/examples/interative - simplifying it to be just an image slide show and just the email and web icon links underneath.  It works great using my laptop browser, but opening it on my Android phone chrome browser, the target image is recognized and I have it showing the first slide automatically, but tapping the arrows or the icons underneath don't do anything.  The sample code just uses the click event.  I added the touchstart event as well to see if that would help, but no luck.  Anyone else run into this?

I realized it worked if I got closer.  Not sure how to get it working when out a bit, but it is working so I will close this.


In some cases, writeToc can be called before the last toc entry is ac…

…tually processed, fixes #334

Coverage increased (+0.0002%) to 95.625% when pulling 4afe1908fb5f8bffb4853ebde0daa07a618fcf18 on gnodet:fix334 into 3f10148563853333bdf63de943a3f496f49dd8eb on simoncozens:master.


Possible rate limit issue/s

Not really sure if this is a rate limit issue.
I've tried looking at the code but I don't see a debug/verbose option.
When running the following:
.\kojirou.exe -l en 58be6aa6-06cb-4ca5-bd20-f1392ce451fb --kindle-folder-mode  --autocrop
It'll get thru a handful 1-3 volumes and then hang, now it just hangs on volume 5 never moving on.
A few times I got the following errors:
Error: covers: mangadex: chapter 0: image 0: stream error: stream ID 17; CANCEL; received from peer
Error: volume 4: pages: mangadex: chapter 23: image 13: stream error: stream ID 259; CANCEL; received from peer
So far it seems stuck on pulling past Volume 5, let me know if you need me to test anything.

Thanks for reporting! This isn't a ratelimit issue though, just a problem caused by me incorrectly implementing some of the cancelling logic. It seems like one of the images in volume 5 is broken, but instead of returning an error, the current code just blocks indefinitely.
I'll try implementing a fix as soon as I can.

The issue should now be fixed.
Note that you still won't be able to download volume 5 correctly, as the upload contains broken images. This is a known issue that I can't really do much about, also see #6 and #24 for the reasoning behind this.
However, now you should be able to get a proper error message, which you can then use to filter out the offending chapters, or load them from a local directory.

@killergoalie Can you confirm the program no longer hangs for you on the latest release?

Just got back from a business trip, give me a few hours and get the latest release pulled.  Thanks

Looks good atleast from understand what's failing now.  Thanks for the quick release.  I'm looking over your code as I'm trying to learn GO at the moment. Thanks again, Also side note, Volumes 5, 6, and 7 have that issue =/

Thanks for the response.
As far as learning Go using this codebase, I think it should be fine, but I would probably try to not replicate the current module structure in your own projects, as it is a bit of a mess right now. Also, note that this is a pre-generics codebase and today you would be able to express a lot of stuff more cleanly by using generics.


How to detect which participant - local or remote goes to "reconnecting" state.

I am working on showing a page to the either side  of one to one room if either of the participant goes to reconnecting state. I observed that as soon as any one participant goes to reconnecting state, useRoomState() gives the roomState as reconnecting almost immediately but as we have to identity which participant is reconnecting so we added the hooks below but still they take around 3-4 seconds to give the result, so this gives the other side a blank screen which is not a good experience.
In case of local Participant, I am using the hook below:
export default function useIsLocalParticipantReconnecting() {
    const [isReconnecting, setIsReconnecting] = useState(false);
    const { room } = useVideoContext();
    useEffect(() => {
        if (room.localParticipant) {
            const handleReconnecting = () => setIsReconnecting(true);
            const handleReconnected = () => setIsReconnecting(false);

            room.localParticipant.on('reconnecting', handleReconnecting);
            room.localParticipant.on('reconnected', handleReconnected);
            return () => {
                room.localParticipant.off('reconnecting', handleReconnecting);
                room.localParticipant.off('reconnected', handleReconnected);
            };
        }
    }, [room]);
    return isReconnecting;
}

In case to detect remote participant reconnection:
const [isReconnecting, setIsReconnecting] = useState(false);
    useEffect(() => {
        const handleReconnecting = () => setIsReconnecting(true);
        const handleReconnected = () => setIsReconnecting(false);

        room.on('participantReconnecting', handleReconnecting);
        room.on('participantReconnected', handleReconnected);
        return () => {
            room.off('participantReconnecting', handleReconnecting);
            room.off('participantReconnected', handleReconnected);
        };
    }, [room]);

@timmydoza  , Can you please  help here and let me know if I am missing something here?  Request your help to let me know if this is not the correct way to identity this state. Thanks !

Thanks for the question @rachnaban! Your implementation looks like it will do the trick. Unfortunately, it does take a few seconds for our signaling servers to detect that that a remote participant is reconnecting. I don't think there's any way around the 3-4 second delay that you mentioned.
Your first hook is basically the same as the useRoomState() hook, with one small difference:
The useRoomState() will change to reconnecting when there has been an interruption in the localParticipant's signaling or media.
Your top hook will change to reconnecting when there has been an interruption in the localParticipant's signaling (but not media).
They are pretty similar. Our React App uses useRoomState() to detect a local reconnection.
The second hook looks good too, but if you want to detect which participant is reconnecting, then you can use the argument provided in the participantReconnecting event:
room.on('participantReconnecting', participant => {
  console.log(participant.identity + ' is reconnecting to the Room');
});

Also, be sure to look at our useIsParticipantIsReconnecting hook: https://github.com/twilio/twilio-video-app-react/blob/master/src/hooks/useParticipantIsReconnecting/useParticipantIsReconnecting.ts
It accepts a participant as an argument, and then uses participant.on('reconnecting', () => {}). It's basically the same as your second hook - it's just an alternative way of doing things. This hook is used by our ParticipantConnectionIndicator component: https://github.com/twilio/twilio-video-app-react/blob/master/src/components/ParticipantInfo/ParticipantConnectionIndicator/ParticipantConnectionIndicator.tsx

Thank you so much, @timmydoza . What i observed is RoomState goes to reconnecting not only when localParticipant goes for reconnection but remoteParticipant as well. I mean if my remote Participant goes for reconnection then roomState is showing as "reconnecting" at both the ends. Is this expected or I am missing something here.

Hmm, that's not the expected behavior of useRoomState(). This hook should only change to reconnecting when the localParticipant is having a connection interruption.
https://media.twiliocdn.com/sdk/js/video/releases/2.7.0/docs/Room.html#event:reconnecting__anchor
Just curious, but are you doing something to test these hooks? Are you doing something to trigger the reconnecting state in order to test the application.

Thanks for your response @timmydoza , If my remote Participant is interrupting his connection , in his console logs , he is getting roomState as reconnecting and at my end also, I am getting "reconnecting" in my console log.  Currently I am testing in this way only.

Sorry about the delay @rachnaban. Are you still experiencing this issue?
I was curious about how you are triggering the reconnecting state.  Or are you using a network throttling tool?

I am closing this issue now as after changing the room type to small group, my issue is resolved. Thanks @timmydoza !


Fix unstaking flows

This PR contains several improvements in terms of updating the staking/unstaking flow:

Ensures that the agent bond on chain matches the selected staking contract.
Collects metadata from staking programs on-chain.
Fixes the staking/unstaking flow to match the discussions in the working document.
Fixes computation of balance on run_service.sh
Removes unused contract ABI JSONs from the folder /contracts and uses the files already available on the packages folder.
Removes the update_service.py script and uses the built-in autonomy command.

Resolves #151


Support updating the public profile

Users have a public profile (sometimes registered via the zonefile) that contains public information like where users store their data for certain apps. Currently, the profile contains a schema definition for Person. However, there is no way for the user to add or change the personal details.
This feature request is a first step to support updating the public profile. It is limited to properties of the Person schema.
Application can request to update the users profile, then the user has to confirm the change via UI like this:

This feature requires:

handling a new type of request (update profile) from apps
adding UI flow to confirm profile updates
re-useing existing functions to upload the updated and signed profile

In connect (see https://github.com/hirosystems/connect/issues/246):

adding new type of request (doUpdateProfile)

//cc @markmhx @yknl

If approved, I am happy to send a pull request

I LIKE THIS FEATURE!

This generally approach looks great to me. We'd love to review a PR with this addition.

Rather than a specific profile updater feature, could this be implemented as a more generic gaia data updater?

@kyranjamie The underlying logic could handle that. The app should then send mime-type, filename and file content, optional encryption key.
This issue is about limited editing feature of the profile because the public profile contains app data that users might not want to mess with in general. Therefore, the flow for this issue would accept only
mime-type = application/json
filename = profile.json
encryption key = undefined
file content follows strictly Person schema

While I think it makes sense to potentially have a Gaia generic function, the profile related stuff does warrant its own methods in my opinion. The profile is a core element of stacks auth and wallet usage, and we should start giving it more focus.
I vote for a specific profile related function before a generic Gaia method.

Here is first proposal for a request token: https://github.com/hirosystems/stacks-wallet-web/pull/2632/files#diff-013e7cf4ed0371a2b480849aa230bf83e4314ff2c959d79818953a93fc460688R30


Sync docs with current status of upgrade command.

Recently the upgrade command in React Native CLI was refactored to check the current and the latest version and give to user link to the Upgrade Helper with the version provided in URL. This PR sync docs with the current job that actually upgrade command does.

Can we get this merged? 🙏


fix compile cloud when cloud/cmake/* changed

Proposed changes
Issue Number: close #xxx

run buildall


Prefer Requirements that were already resolved to get conflicts early

I really don't understand why this integration test now runs into timeout. @matthid did you add this one? Any ideas?

Yes the test was a workaround for #2294 I guess it cannot find a solution in the given time after these changes? This is a difficult puzzle for paket (and I think I have seen this test failing before occasionally)...

Ok thanks. So it must be fixed with the other charges I'm planning.
Am 02.12.2017 13:42 schrieb "Matthias Dittrich"<EMAIL_ADDRESS>
Yes the test was a workaround for #2294
https://github.com/fsprojects/Paket/issues/2294 I guess it cannot find
a solution in the given time after these changes? This is a difficult
puzzle for paket (and I think I have seen this test failing before
occasionally)...
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
https://github.com/fsprojects/Paket/pull/2923#issuecomment-348689665,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AADgNJ3XRML6-r8KgoKKbX7ylkvJq6qnks5s8UW4gaJpZM4QzNS_
.

the thing is: we don't use the set order anymore.
Take a look at getCurrentRequirement - it's calculating the minimal element manually with a loop. Set order is completely ignored


Documentation page throws a 404

https://www.data-marketing-school.com/documentation
Your documentation page is invalid and is throwing a 404 please update it to a valid url or remove it.
Thanks!

Thank you for your feedback, documentation removed for now.


RFC: Unset buildid for reproducible builds

If the same source is built on two different machines, ideally, they should produce the same binary, same image, same layer digests, same config diffids, etc.
This isn't the case today. If you take a recent ghcr.io/google/ko image produced by CI, and try to build the same image from the same commit, using the same Go compiler version, and using the same base image by digest (aided by #592), you still end up with different images.
$ git checkout f8d0aca1abeab6cf3d1800d15acc0592c7c976b7
$ ci=ghcr.io/google/ko@sha256:8fa68c86562684f946e92086a6a4d1f17ee116fff32e157e68f96dbf948b0783
# set base image to golang:1.17@digest in .ko.yaml, based on CI build logs
$ local=$(KO_DOCKER_REPO=gcr.io/imjasonh go run ./ build ./ --platform=all)
...
2022/02/11 11:01:22 Published gcr.io/imjasonh/ko-98b8c7facdad74510a7cae0cd368eb4e@sha256:73d4746ef0423a24b2e36705d5f648f8f6ed8ad2f70a74c43a1d799ea4135936

Note these images have different digests.
Only considering --platform=linux/amd64 for now, their manifests differ in the top layers produced by ko:
$ function manifest-platform-diff {
  diff <(crane manifest $1 --platform=linux/amd64 | jq) <(crane manifest $2 --platform=linux/amd64 | jq)
}
$ manifest-platform-diff $ci $local
7c7
<     "digest": "sha256:52fbe2d8d6f679aaf218c9696bd52cf46a0acf5734e55f477f767fcfc8171fac"
---
>     "digest": "sha256:431bf119280969354ea1b7f01d5057886cde0c5f415a9d6a702fb415b1f9e795"
52,53c52,53
<       "size": 14447415,
<       "digest": "sha256:103dce014357d1447a0ab5bdb2e0820ebe5b3825dc39ec094bec0833641c6f3c"
---
>       "size": 14451848,
>       "digest": "sha256:e73e6295963605d46fa599595e807f93d78035bbab37d5055c284d5fa597fe1b"

Configs diff only in the last diffID:
$ function config-diff {
  diff <(crane config $1 --platform=linux/amd64 | jq) <(crane config $2 --platform=linux/amd64 | jq)
}
$ config-diff $ci $local
91c91
<       "sha256:26b11d7436027c1a8b68d2f263694cb5d004cbaf82cf12fabd3e8bc57b83b675"
---
>       "sha256:a744dfc522c644d427686a3111398e19ce8ba62eb4d1091f7e4cf9773ab2dacc"

The Go binaries in each image:
$ function extract-file {
  crane export $1 - | tar -Oxf - $2
}
$ extract-file $ci /ko-app/ko > cibin
$ extract-file $local /ko-app/ko > localbin

These are different sizes, and have different buildids embedded:
$ go tool buildid cibin
wfsXdWzess1HPZwqxpza/w1W8Gpkeb188ZJ6UexW7/fAQZlgOa-X8oaRDhGERu/OOojyWok9JTAbBpACeWS
$ go tool buildid localbin
fURyWN8_9H1KmClFNJrE/HWiELKLTvC1MFSvyxVmG/dsAR_RpmjFrxsRtRLtDJ/n5VsQTn6NFmq0lhuM8r4

We use buildid to avoid rebuilding/compressing/digesting binaries when KOCACHE is set, but I think maybe if KOCACHE isn't set we should consider also passing -ldflags=-buildid= as recommended here
Or, perhaps making it an opt-in choice like ko build ./ --reproducible?

Is the only diff the buildid?

I believe this was fixed in go1.14
https://github.com/golang/go/issues/33772

It seems like it wasn't fixed.
Compiling two binaries on two different machines with -trimpath resulted in different checksums. Dropping the buildid gave me identical binaries.


Relax lower bound for primitive-unlifted

primitive-unlifted-<IP_ADDRESS> does not compile with GHC 9.2 (https://github.com/haskell-primitive/primitive-unlifted/issues/28), while primitive-unlifted-<IP_ADDRESS> works fine. The changelog suggests "Stay on the 0.1.x.x series unless you need something from this newer version", and this is indeed the case for primitive-extras.

Didn't notice it was a PR and updated it myself. Sorry. Any way, the fix is released.


Support decoding matrix URI scheme

MSC2312: Matrix URI scheme proposal https://github.com/matrix-org/matrix-doc/pull/2312 is nearing completion.
As such, we can expect apps to progressively switch from https://matrix.to links to the matrix: scheme. In order to provide a smooth transition, as well as a fallback when no client is installed
Describe the solution you'd like
I would like matrix to to implement support for parsing matrix: URI as input, and work as usual with the information thus provided:

[ ] define a new matrix.to URL scheme such as https://matrix.to/#/url/matrix%3aroomid%2frid%3aexample.org%3faction%3djoin%26via%3dexample2.org
[ ] Implement parsing for it as specified in MSC2312
[ ] ask visitors if they want to register matrix.to as a protocol handler for matrix: links (can also ease the transition)
[ ] modify web browsers to add matrix.to as a default handler for these links

Describe alternatives you've considered
Falling back to Element.io is another possibility that can be examined, I am not sure which one brings the best onboarding experience. Both are not exclusive.
Additional Context
Users might want to copy-paste a URL from the address bar? In that case, maybe rewrite them to regular matrix.to format?

May I humbly ask if you think this could be ready soon?
Ideally I would like browsers to set matrix.to as a default handler at the same time they whitelist the scheme.

I think supporting a route that will take a percent-encoded matrix: uri makes sense, to support Firefox having a default handler for matrix.to, and to use registerProtocolHandler on matrix.to, both of which use a %s placeholder which will be replaced with the whole URI, including the scheme.
I would suggest https://matrix.to/#/uri/%s, to be in line with MSC 2312 (which calls it uris rather than urls, although they can be both depending on the via params)/

Given that only Firefox seems to offer adding a default handler for matrix:, I don't think it's realistic for clients (or at least Element) to share matrix: links rather than matrix.to. The whole onboarding would just be too broken. This is also why competing services like telegram and whatsapp have their website like matrix.to. The biggest value of the matrix: scheme for now is IMO that we can skip the client list on matrix.to if you already have a client installed.

After parsing the percent-encoded url, we can also rewrite the hash to our canonical url format (see https://github.com/matrix-org/matrix.to/issues/205 as well)


[Update .gitattributes

adda author

learning

In the future, please perform testing like this on your own repositories (testing on this and other public repositories causes a lot of people to receive notifications, which is pretty spammy).


Package is not published as a NPM module

Please publish this latest package on NPM.
When I'm doing "npm view is-boolean-object" I can see only<EMAIL_ADDRESS>Latest tag 1.1.0 is not published. Getting following error
is-boolean-object: No matching version found for<EMAIL_ADDRESS>

Sounds like you have an internal registry that isn’t properly updated.
That command works fine for me, and https://npmjs.com/is-boolean-object clearly shows v1.1.0.


[CI] investigate unstable scenarios – collect samples of flaky CI runs

Describe the bug
The CI occasionally fails for no clear reason, and re-running the pipeline again usually shows no issue.
To reproduce
There is no clear way to reproduce this issue (or issues) at the moment
Difference from expected behavior
We would want the CI to consistently fail or succeed.
Possible solution
Currently we re-run flaky CI runs.
Screenshots
If applicable, add screenshots to help explain your problem.  Otherwise, remove this section.
System on which behavior was encountered
This happens on the Github CI
Codebase information
Happens on recent main commits, unclear since which commit exactly.
Additional context
As a first step towards finding a fix, we probably want to collect samples of failed CI runs to determine what they share in common, and what scenarios are subject to flakiness.
Observed failures

on main, fails:

FAILURE for "Nominal planning: conflict with higher priority" scenario
FAILURE for "ASTM F3411-19 NetRID DSS interoperability" scenario

on a PR, fails:

FAILURE for "Nominal planning: conflict with higher priority" scenario
FAILURE for "ASTM F3411-19 NetRID DSS interoperability" scenario

on a PR

FAILURE for "Nominal planning: conflict with higher priority" scenario
FAILURE for "ASTM F3411-19 NetRID DSS interoperability" scenario

Unrelated to the above, I have had a few cases of scenarios failing locally due to clock skew between the mock_uss and the OS, which runs the qualifier. This concerns scenarios that query the interaction log endpoint of the mock_uss.
The details can be found on #750: if anyone runs into intermittent failures caused by missing interaction log entries, this could be an explanation.


Create persistence layers that persist the Shape Log for a ShapeStream instance for faster loading of shapes (and offline loading)

The Typescript ShapeStream class is responsible for reading the Shape Log from the Electric server and feeding it to other code that'd like to do something with the stream e.g. the Shape class which creates an in-memory materialized view of the Shape as a JS Map.
Shape Logs are cached in Electric (and in an http cache proxy (e.g. nginx/cdn/etc) if you're using one) so they're generally cheap to load. But still, if shapes are larger than a few mbs, persisting shape logs to disk would speed up loading — especially for people on limited mobile networks.
The implementation for this would be:
On Initializing:

ShapeStream checks if there's a persisted log. If so it loads that, the offset, and shapeId for the log.
Then it makes a fetch to the server like normal to catchup with the shape log on the server

On receiving new shape log messages:

ShapeStream would emit these to subscribers and then append them to the persisted shape log.

Compaction — Shape Logs fill up eventually with a lot of redundant information (e.g. 100s of updates to a single row). So periodically compacting the log is necessary to keep reading the log fast. Compaction is basically reading through the log from the start and merging together repeated operations on a single row. When a row is deleted, we can remove all log messages. When compaction is finished, the log is left with one insert operation / row in the shape.
Considerations

benchmark this — sometimes client storage can be slow, particularly on mobile. Sometimes reading from the network is faster than reading from a local cache. We'd need to benchmark carefully (on desktop and mobile) the perf of persisting vs. not. Perhaps it is always faster in which case persisting can be the default. Or perhaps it's always faster on desktop so could be enabled there. Regardless, benchmarking will be a key part of the design process.
what storage option to use? What form should we store them in? Indexeddb & opfs are the two options. Indexeddb is more widely available than opfs so should be the option unless opfs is vastly faster.
Subscribers to ShapeStream wait for a up-to-date control message to let them know that they're caught up with the server and can now inform their subscribers (e.g. a UI component). When reading the persisted log offline, the ShapeStream needs probably to be told it's offline so that it can emit up-to-date when it gets to the end of the persisted log.

I would argue that this functionality should be a layer above ShapeStream. ShapeStream is a nice thin abstraction of the protocol, and perfect for feeding into other stores. If that store has persistence, ShapeStream doesn't need any itself.
Maybe a PersistedShapeStream? It should be composable with a store implementation you could pass it (IndexedDB, OPFS, Node FS)?

Ooo yes! I've been uneasy about throwing this into ShapeStream and persistence is a natural layer to compose above it.
const stream = new ShapeStream()
const streamWithPersistence = new ShapeStreamOPFS(stream)
const shape = new Shape(streamWithPersistence)

We've also talked about not actually storing the log, but just the last value for the keys, since the developer shouldn't build on historical events, since you can never have the guarantees that you get a full history.

I can pick this up - my suggestion is a composable ShapeStreamPersister that accepts a ShapeStream and another instance with a specified Storage interface (perhaps set, get, delete? something sensible), so either we or anyone else can add any storage option (we can start with indexeddb or local storage)
I also agree with Valter, which touches on the compaction mentioned by Kyle above - I think what we should do is materialize the log as we ingest it while also keeping track of the last offset seen, and when it's time to restore the stream from DB the materialized data is converted into a series of inserts (like we do for snapshots in the backend).
Issues regarding FK checks, check constraints etc apply for the compacted log coming from the backend anyway so doing this in the local store should not be a separate issue.

Sounds like a great plan @msfstef!


Please, restore should_have_default_scope in rails-3.1 branch

Following code could be used to ensure, that default scope works as expected:
EmailLetter.scoped.to_sql.should == EmailLetter.unscoped.where(:sent => false, :error => false).to_sql
Usage would be:
it { should have_default_scope.where(:sent => false, :error => false) }
I'll provide pull request if needed.

If you can make it work and pass the specs then have at it. I cut them out somewhere in alpha1 or alpha2 because of the way the arel matching works did not work with default_scopes. The .to_sql conversion doesn't always work everywhere either, at least in the 3.0 series.


ModuleNotFoundError: No module named 'srunner'

I have added SCENARIO_RUNNER_ROOT as senario_runner in the system variables path which I downloaded from Carla git for carla 0.9.12. In this senario_runner we have srunner folder which I am trying to import to scenario_manager.py and in leaderboard_evaluator.py.  I also added ./senario_runner in settings.json of vscode which helped in removing the warning while importing srunner but still when I try to run leaderboard_evaluator.py it gives me this error
Traceback (most recent call last):
  File "C:\CARLA\leaderboard\leaderboard\leaderboard_evaluator.py", line 27, in <module>
    from srunner.scenariomanager.carla_data_provider import *
ModuleNotFoundError: No module named 'srunner'

This is leaderboards_evaluator.py

This is settings.json

OS: [Windows 10]
CARLA Version [CARLA 0.9.12]
Python version [3.8.5]

I also tried carla on ubuntu but I set MESA intel error and it doesn't run. On windows though it runs but I am getting these import errors which I am trying to solve from some days...
Can anyone help me with this... I am stuck on setting carla for a long time. Please help if you know where I am going wrong..
Thank you :)

Hey @chinmay-d, it seems like python isn't finding srunner, have you added your scenario runner folder to the PYTHONPATH?

sorry for being late... Yeah I have tried adding scenario_runner and still I get same error...
Also I dualbooted to ubuntu now and tried following instructions on Leaderboard I have set leaderboard git and also scenario_runner git for carla_0.9.12 in ubuntu and I still get the same error
The error:

VSCode does not show import error:

My ~/.bashrc:

And finally the ./test_run.sh:

I think all my exports are correct and I am getting same error in windows as well as on ubuntu.... If you can find out the error  it will be a lot of help...
Thanks :)

I see. All your images seem fine and unless your scenario-runner_0.9.12 doesn't contain srunner, I can't see the problem. Could you maybe remove the 3 srunner imports and see if the leaderboard ones are imported? With that, we can see if the problem is in the PYTHONPATH, or in the srunner module.

I have commented all the imports containing of srunner and I get this error:

Yeah I think its a PYTHONPATH issue, but can't figure out what's wrong..

I guess, the python version in PythonAPI/carla/dist is 3.7 but the 'carla' env that I made using pyenv has version 3.8.0. As carla 0.9.12 supports python version 3.8 so I used an env with python version 3.8.0. Is this causing the problem?

@glopezdiest  I even tried with env python version 3.7.1 I still get same above error ModuleNotFoundError: No module named 'agents.navigation.global_route_planner_dao' after commenting all the srunner imports

That might be another issue, as we changed the agents for 0.9.12, and without these changes, the leaderboard crashes

@glopezdiest, so I should try cloning leaderboard again?

yea, I made the changes and I still get ModuleNotFoundError: No module named 'srunner' and when I comment all the imports of srunner from all .py files I get this error:

I closed my terminal and opened a new terminal and uncommented all the imports of srunner.... and It worked....
All I did after cloning git of leaderboard when I got support for carla 0.9.12 is cloned git of scenario_runner-0.9.12 and restarted the teriminal.... and now its working..
Thanks @glopezdiest  :)

@chinmay-d  Hello ,can you tell me how to successfuly run this leaderboard_evaluator.py?  thank you

Oh, seems like you updated the PYTHONPATH but didn't refresh the terminal nor sourced the new bash script, so the new PYTHONPATH wasn't being used. It should work for you now.
@GeneralMak Please, open another issue with your specific issues.


Subject: Inquiry on Using Pre-trained Models for Transfer Learning with ClinicaDL

Dear Author
I hope this message finds you well. I am interested in using a pre-trained model you have provided for transfer learning tasks with ClinicaDL. I have successfully downloaded the model, which contains the best_model and performances folders as expected.
However, I have encountered difficulties in implementing this for my own dataset using ClinicaDL's predict command. Specifically, I keep getting an error related to MAPS (Model-Data Architecture), stating that MAPS were not found at the specified directory.
I have verified that the directory structure is correct and even tried upgrading ClinicaDL to the latest version, but the issue persists.
My questions are:
1.Is it possible to use the pre-trained model for transfer learning tasks with ClinicaDL?
2.If yes, could you please provide specific steps or examples on how to properly implement this?
3.Are there any special configurations or files needed to be added to the best_model or performances folders for the transfer learning to work?
Your guidance will be invaluable for me to proceed further in my research. Thank you in advance for your time and assistance.
Best regards.
Feel free to modify this template to suit your specific situation. Good luck!

Hi @mtmmu88,
Thanks for using ClinicaDL,
Have you successfully downloaded it from this address? Yes, it is possible to utilize these pre-trained models for transfer learning tasks with ClinicaDL.
I've personally tested it, and it's working for me. Here's the command line you should use:
clinicadl predict path_to_maps data_group --caps_directory path_to_capsi  --participants_tsv path_to_tsv
If it still doesn't work, please send us the full error message so we can investigate further.
Please note that your dataset must be in a CAPS format, and you should verify the required preprocessing steps in the maps.json file within the results folder (for these models, you need ROI, so you should have masks in your CAPS dataset). If these preprocessing steps haven't been performed, execute the preprocessing pipeline as well as clinicadl extract to obtain tensor versions of the images.
For more detailed instructions, you can refer to our [tutorial] (https://aramislab.paris.inria.fr/clinicadl/tuto/2023/html/index.html)
Bests regards,
Camille

Dear Camille,
Thank you for your previous guidance. Despite following the instructions and recommendations you provided, I continue to face issues when trying to use the pre-trained model with ClinicaDL.
The latest error I'm encountering is: 'MapsManager' object has no attribute 'size_reduction'. This issue persists even after verifying that the data is in CAPS format, checking the maps.json file, and upgrading to the latest version of ClinicaDL.
Additionally, I have downloaded a model from this Zenodo link. Could you confirm if this specific model can be used for transfer learning tasks with ClinicaDL?
Could you please offer further advice or guidance on how to resolve these issues?
Thank you once again for your time and assistance. Your expertise is greatly appreciated as I continue my research.
Best regards

File "/home/zzc/miniconda3/envs/ClinicaDL/bin/clinicadl", line 8, in
sys.exit(cli())
File "/home/zzc/miniconda3/envs/ClinicaDL/lib/python3.8/site-packages/click/core.py", line 1157, in call
return self.main(*args, **kwargs)
File "/home/zzc/miniconda3/envs/ClinicaDL/lib/python3.8/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
File "/home/zzc/miniconda3/envs/ClinicaDL/lib/python3.8/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/zzc/miniconda3/envs/ClinicaDL/lib/python3.8/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/zzc/miniconda3/envs/ClinicaDL/lib/python3.8/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
File "/home/zzc/miniconda3/envs/ClinicaDL/lib/python3.8/site-packages/clinicadl/predict/predict_cli.py", line 113, in cli
predict(
File "/home/zzc/miniconda3/envs/ClinicaDL/lib/python3.8/site-packages/clinicadl/predict/predict.py", line 62, in predict
maps_manager.predict(
File "/home/zzc/miniconda3/envs/ClinicaDL/lib/python3.8/site-packages/clinicadl/utils/maps_manager/maps_manager.py", line 231, in predict
size_reduction=self.size_reduction,
File "/home/zzc/miniconda3/envs/ClinicaDL/lib/python3.8/site-packages/clinicadl/utils/maps_manager/maps_manager.py", line 114, in getattr
raise AttributeError(f"'MapsManager' object has no attribute '{name}'")

Dear ClinicaDL Team,
I am writing to bring to your attention an issue I encountered while trying to run predictions using a pretrained model in ClinicaDL. I followed the instructions in the official documentation but encountered an error.
Here are the details:
Command Used: clinicadl predict --no-gpu /home/zzc/test_adni/INPUT_MAPS_DIRECTORY/maps_exp3/maps /home/zzc/test_adni/DATA_GROUP
Error Message: NotImplementedError: Non-relative patterns are unsupported
I have verified that the maps.json and data.tsv files are in the correct directories as specified in the documentation. I am using the latest version of ClinicaDL and all dependencies are up to date.
Could you please provide some guidance on how to resolve this issue? Your assistance would be greatly appreciated.
Thank you for your time and consideration.
Additional Information:
I have also tried the following steps to resolve the issue:
Moved data.tsv into the DATA_GROUP directory.
Copied maps.json from INPUT_MAPS_DIRECTORY to DATA_GROUP.
Checked all paths and command-line parameters.
Reviewed the official documentation for any additional steps or precautions.
Despite these efforts, the issue remains unresolved. I am particularly interested in using the pretrained model for transfer learning on a new dataset. Could you please provide further guidance?

Hi @mtmmu88 ,
Can you please send us the full error message so we can investigate further ?
Thanks,
Camille

Certainly, here's a sample reply you can use to ask the developers to
review the issue you've posted on their project:
Subject: Request for Review of Issue Posted on Clinica Project GitHub
Repository
Dear Clinica Team,
I hope this message finds you well. I am facing a critical issue while
using the Clinica t1-linear pipeline on my Linux system. The problem is
specifically related to a FileNotFoundError for the 'bias_image' in the
N4BiasFieldCorrection node.
I have documented the details of this issue on your GitHub repository under
the Issues section. The problem is causing significant delays in my
research work, and I would greatly appreciate it if you could take the time
to review the issue and provide any insights or solutions.
Could you please confirm receipt of my GitHub issue and let me know if you
need additional information to diagnose the problem? Your prompt assistance
would be invaluable for the advancement of my research.
Thank you very much for your time and consideration.
Best regards,
mtmmu
camillebrianceau @.***> 于2023年9月22日周五 20:48写道：

Hi @mtmmu88 https://github.com/mtmmu88 ,
Can you please send us the full error message so we can investigate
further ?
Thanks,
Camille
—
Reply to this email directly, view it on GitHub
https://github.com/aramis-lab/clinicadl/issues/476#issuecomment-1731357189,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AVVEHPTKWBCMMC2NU7T2PC3X3WCIRANCNFSM6AAAAAA4WOBOK4
.
You are receiving this because you were mentioned.Message ID:
@.***>


Repo cleaning

The following is a list of suggested actions to clean-up this repo:

[x] rebase executor branch over crystal and/or [dashing],
[x] rebase pubsub over [dashing],
[x] change #19 target branch to [dashing],
[ ] rebase parameters over [dashing],
[ ] remove branches: master
[ ] remove branches:  develop
[ ] remove branches: migration
[x] remove branches: configuration.

Hello, is someone using branches master, develop or dashing migration?


Expecting value: line 1 column 1 (char 0)

Expecting value: line 1 column 1 (char 0)

i don´t know how to solve it

See https://github.com/juzeon/SydneyQt#faq


add pinot-dropwizard to root pom.xml

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 0.00%. Comparing base (59551e4) to head (c03ccf6).
Report is 814 commits behind head on master.

:exclamation:  There is a different number of reports uploaded between BASE (59551e4) and HEAD (c03ccf6). Click for more details.
HEAD has 53 uploads less than BASE

Flag
BASE (59551e4)
HEAD (c03ccf6)

integration
7
1

integration2
3
1

temurin
12
1

java-21
7
1

skip-bytebuffers-true
3
1

skip-bytebuffers-false
7
0

unittests
5
0

unittests1
2
0

java-11
5
0

unittests2
3
0

integration1
2
0

custom-integration1
2
0

Additional details and impacted files
@@              Coverage Diff              @@
##             master   #13723       +/-   ##
=============================================
- Coverage     61.75%    0.00%   -61.76%
=============================================
  Files          2436     2479       +43
  Lines        133233   136985     +3752
  Branches      20636    21324      +688
=============================================
- Hits          82274        0    -82274
- Misses        44911   136985    +92074
+ Partials       6048        0     -6048

Flag
Coverage Δ

custom-integration1
?

integration
0.00% <ø> (-0.01%)
:arrow_down:

integration1
?

integration2
0.00% <ø> (ø)

java-11
?

java-21
0.00% <ø> (-61.63%)
:arrow_down:

skip-bytebuffers-false
?

skip-bytebuffers-true
0.00% <ø> (-27.73%)
:arrow_down:

temurin
0.00% <ø> (-61.76%)
:arrow_down:

unittests
?

unittests1
?

unittests2
?

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

Ready for review

rebased.

Just noticed this PR. I've pulled all modules to the root pom in #13763

Awesome! Thanks.  I'll close this PR.


[Event Request] Table 7320 "Warehouse Shipment Header".DeleteRelatedLines

Describe the request
Please add new event
procedure DeleteRelatedLines()
    var
        WhsePickRqst: Record "Whse. Pick Request";
        WhseComment: Record "Warehouse Comment Line";
    begin
        WhsePickRqst.SetRange("Document Type", WhsePickRqst."Document Type"::Shipment);
        WhsePickRqst.SetRange("Document No.", "No.");
        if not WhsePickRqst.IsEmpty() then
            WhsePickRqst.DeleteAll();

        WhseComment.SetRange("Table Name", WhseComment."Table Name"::"Whse. Shipment");
        WhseComment.SetRange(Type, WhseComment.Type::" ");
        WhseComment.SetRange("No.", "No.");
        WhseComment.DeleteAll();
        //-----------------------------------------------------OnAfterDeleteRelatedLines:BEGIN
        OnAfterDeleteRelatedLines(Rec, xRec);
        //-----------------------------------------------------OnAfterDeleteRelatedLines:END
    end;

    //-----------------------------------------------------OnAfterDeleteRelatedLines:BEGIN
    [IntegrationEvent(true, false)]
    local procedure OnAfterDeleteRelatedLines(var WarehouseShipmentHeader: Record "Warehouse Shipment Header"; xWarehouseShipmentHeader: Record "Warehouse Shipment Header")
    begin
    end;
    //-----------------------------------------------------OnAfterDeleteRelatedLines:END

Additional context
We need do code after this function
Internal work item: AB#556652

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. Please do not reply to this, as we do not monitor closed issues. If you have follow-up questions or requests, please create a new issue where you reference this one.
Build ID: 26527.


Requesting Unique Tags may result in an incomplete list of tags

Describe the bug
If you have Rule One with Tags A and C  and Rule Two with Tags A and B.
If unique tags only is on and

Rule One matches first: Two will not match and results will not be tagged with B.
Rule Two matches first, One will not match and the results will not be tagged with C.

In #204 we parallelize the analysis so the order rules are applied becomes non-deterministic and you will get either behavior 1 or 2 above at random.
To Reproduce
Pass appropriate values to UniqueTagsCheck if any of the tags passed in are included in the set of known tags all tags will be ignored. https://github.com/microsoft/ApplicationInspector/blob/6375d1d73c594645e7eb863e03b6c4bed5164f1a/RulesEngine/RuleProcessor.cs#L324
Expected behavior
Requesting unique tags should return all matched tags, deduplicated.
Proposed Fix
We should check if there are any new tags and grab them, rather than checking for duplicate tags and discarding everything.

in-progress


Azure Event Hub firewall requirements for log streaming

Please, add notice about Event Hub firewall requirements to the Azure Monitor Diagnostic Setting streaming to Event Hub documentation.
If Event Hub uses virtual networks firewall, "Allow trusted Microsoft services to bypass this firewall" option must be activated.
https://docs.microsoft.com/en-us/azure/event-hubs/event-hubs-service-endpoints#trusted-microsoft-services

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 0ca576e1-fafa-3a02-6d29-394f342bc737
Version Independent ID: a6a83de5-41fd-2dce-eb86-cee6c086c07d
Content: Create diagnostic settings to send platform logs and metrics to different destinations - Azure Monitor
Content Source: articles/azure-monitor/platform/diagnostic-settings.md
Service: azure-monitor
Sub-service: logs
GitHub Login: @bwren
Microsoft Alias: bwren

@ivanignatiev Thanks for your feedback. We will review the issue and update the document accordingly.


The extension doesn't work today (Restore YouTube Username from Handle to Custom)

I don't know why, but Restore YouTube Username from Handle to Custom stopped working on normal sessions.
Like this:

Possibly something happened and well, it doesn't work.
Even on incognito stopped working.
I removed, restarted and even reinstalled it. Nothing happens.
Even restarted PC. Still nothing.
Is there any explanation and how to fix this problem? Thank you.

Is there any explanation and how to fix this problem? Thank you.

YouTube is always updating its coding. This is normal as they are doing A/B testing everyday.
Login / Logout your account, or just use another browser.
I cannot reproduce it right now. Nothing I can do.

I have just fixed an issue due to new youtube coding. Try to update to see if it works for you or not.

I have just fixed an issue due to new youtube coding. Try to update to see if it works for you or not.

It works now


nextModel is missing the .50 shade

The .50 shades are not generated by the nextModel.

Never mind.

Although the model performs poorly with very light colours as input.

That's my experience as well. All the 50 shades look pretty much the same.


⚠ RPC server - Arbitrum has degraded performance

In b19990b, RPC server - Arbitrum ($ARBITRUM_RPC_URL) experienced degraded performance:

HTTP code: 429
Response time: 246 ms

Resolved: RPC server - Arbitrum performance has improved in d3d4e89 after 11 minutes.


HSTS setting in Windows Admin Center ver2103

Has anyone performed any actions pertaining to the HSTS vulnerability on Windows Admin Center? No setting(s) that I can find or modify since it's not installed with IIS.
Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: e80c473e-b916-0047-64f6-3c4c0eed7df6
Version Independent ID: b70c774f-8962-0e48-8294-645c03530f40
Content: Windows Admin Center frequently asked questions
Content Source: WindowsServerDocs/manage/windows-admin-center/understand/faq.yml
GitHub Login: @nwashburn-ms
Microsoft Alias: niwashbu

Word from Microsoft is that there is not an implementation yet for HSTS for WAC.  Had to open a case to determine this.  HSTS might be something they integrate in the near futre.


Regard .tikz files as TeX files as well

For those of us, who work with a lot of TikZ graphics, having multiple .latexcfgs to support the .tikz extension for every image in each folder can be quite a hassle.
This could be eased by looking for .latexcfg in a parent directory as in https://github.com/James-Yu/Atom-LaTeX/issues/72.
Otherwise, the common .tikz extension could be supported natively, as already I proposed in https://github.com/James-Yu/Atom-LaTeX/issues/86.
I know this is debatable, so feel free to consider this.

Looks like the change is covered in #89.
Looking for configuration in child folders are troublesome, which was discussed in some previous issues. However, I do think looking in the parent one should be fine, as I cannot think of cases in which performance is a great issue. Meanwhile, I may now focus more in #91 first, and leave this an open question for now 🦍

One side question. Are there any actively developed Atom package that can cover the features in this one? I was away from the table for too long time and wasn't aware of new things 😆

Regarding your side question: for me, this package is still the best one out there. All other similar packages known to me (latex, latex-plus, latextools) rely on a latexmk toolchain exclusively instead of offering plain pdflatex compilation. Also, atom-latex worked best for me out of the box. Anyways I stuck with atom-latex and haven't checked the others in a while. I would hate to see this one fade.

@simon123h
What is your typical TikZ setup like? Do you pre compile each image prior to the main compilation using externalisation?

@ashthespy I like to have my TikZ graphics as separate .tikz files and compile them on their own. I prefer this over externalization. This is why I'd like to compile my .tikz files conveniently.
I see that with externalization from my main document this would most likely not be an issue, as I would then compile my main .tex document.

@simon123h I understand your issue quite well! I am in search for for  way to speed up my build times, as I now just use \input{figures/Image1.tikz} in my code and that doesn't really scale well when there are more than 10 TiKz images.


ci: Generate nightly workflows for alternative runners

They were generated in the root of the repo using the following command:
dagger call -m .github workflows generate export --path .

The new workflows are only configured to run in the context of dagger/dagger repository. These runners will not be available in forks & running the same job with the ubuntu-latest runner doesn't make sense.
The new workflows start with sdk- so that it's easier to group them when looking at metrics. Before this, a special OR filter had to created which was making additional filtering impossible.

I am going to rebase this, fix all the issues & make it follow https://github.com/dagger/dagger/pull/8765
The thing which I care about the most right now is have those workflows run and generate metrics. If we cannot get this merged in the next few days, I am inclined to close this PR and try a different approach.

@gerhard if you have a time constraint, but also want to find consensus with your co-maintainer :) May I suggest opening a separate PR with only the generated workflows that you need - as if they were manually generated. Then you can finish this PR without rushing it, and also get your workflows running right away.

The alternative PR worked well in practice, closing this. We can resurrect in the future if needed.


Error applying environment with 0.6.0

Using the 0.6.0 release, I'm getting the following error message when running tk apply: error: no objects passed to apply.
This doesn't happen with 0.5.0.

I'm taking a look at this now to see if it's an easy fix.


Unable to resolve modules with path starting with ./

Hi all,
When building expo app for ios/android with the following command:
eas build --platform all --non-interactive --auto-submit --clear-cache --no-wait

I get the following error:

Obviously,  this component exists at the correct location:

There's no error for ErrorFallbackScreenwhich is imported the same way.
env
eas-cli/0.53.1 darwin-x64 node-v16.15.0
expo 44.0.6
nx 13.10.3
tsconfig.jsonand tsconfig.app.json files are the one created by the generatecommand.
Would you have any idea how to solve this ?
Thanks !

@xiongemi,  could you please help me solve this issue ? (sorry for tagging 😉, but your help was precious and most appreciated last time )

const { getDefaultConfig } = require('@expo/metro-config');
const { withNxMetro } = require('@nrwl/expo');
const defaultConfig = getDefaultConfig(__dirname);
module.exports = (async () => {
defaultConfig.transformer.babelTransformerPath = require.resolve(
'react-native-svg-transformer'
);
defaultConfig.resolver.assetExts = defaultConfig.resolver.assetExts.filter(
(ext) => ext !== 'svg'
);
defaultConfig.resolver.sourceExts.push('svg');
return withNxMetro(defaultConfig, {
// Change this to true to see debugging info.
// Useful if you have issues resolving modules
debug: false,
// all the file extensions used for imports other than 'ts', 'tsx', 'js', 'jsx'
extensions: [],
projectRoot: __dirname,
watchFolders: []
});
})();

Hi @xiongemi,
so I installed latest version of nrwl/expo and also migrated to SDK 45 but still doesn't work.
When using expo start,  app is working as expected.
any idea ?

the error is about @nrwlworkspace. what version of @nrwl/workspace are you using? if @nrwl/workspace the latest version? if not, could you run nx migrate latest?

Hi @xiongemi,
I use the following versions of the nx packages.

On a blank project the build ran fine. But after copying the src folder, we ran into the above issue.

@xiongemi same project,  working together 😉

I just discover something weird
Everything indicate that the file exists:

Seems file exist, but in expo I got this:

@xiongemi  we found the issue !
Someone added in gitignore *bar( copy/paste from a template I guess).
and thus our NotificationBarcompoenent was ignored from git and also from eas build archive.
This doc was very helpful,  as we could inspect the archive and assert that files were already missing at this point.
https://github.com/expo/fyi/blob/main/eas-build-archive.md
Anyway,  thanks for your support !

lol. no problem. good to hear that problem is resolved.


Unstable test ArchiveFileRollsCorrectly_ExistingArchives

NLog 4.6 rc1
https://ci.appveyor.com/project/nlog/nlog/builds/22222717

will reopen when it's back


Add Dependencies Handler

Addresses #91

This is interesting, but not what I meant for this issue. I intended this to be a change to printUsage in src/main.c, which outputs the current DOME and SDL version to the command line. I wanted the Wren version to be listed here also.


npm-audit-html depends on vulnerable versions of marked

npm-audit-html is being flagged by...npm audit :D
Not a production dependency for me, but wanted to share.
$ npm audit
# npm audit report

marked  1.1.1 - 1.2.9
Severity: moderate
Regular Expression Denial of Service - https://npmjs.com/advisories/1623
fix available via `npm audit fix`
node_modules/marked
  npm-audit-html  >=1.4.2
  Depends on vulnerable versions of marked
  node_modules/npm-audit-html

Heya! I see a PR has been opened for this issue but the tests seem to be failing. Is anyone on the team able to take a look?


Disable or remove error default message on tag

Hello again
I can't find away to remove the error message when i redirect with error message.
Is there away to remove it

has('first_name') ? 'has-error' : '' !!}">       First Name          <input type="text" class="form-control" id="first_name">              //remove this line       {!! $errors->first('first_name', '
:message
') !!}

This is a question related to BootForms.
I guess you could try appending a nulled helpBlock method (->helpBlock(null)), but I never tried it myself. If this doesn't work out, please redirect your question to BootForms issue queue.


Brand details are null

Expected Behavior
All brand params should have values
Actual Behavior
All the brand params and products are none when you run locally
Exception has occurred: NameError
name 'config' is not defined
File "/usr/local/google/home/anaesqueda/Documents/solutions/abcd-detector-external-github/BUGS/abcds-detector/annotations_evaluation/features/b_product_visuals.py", line 101, in detect_annotation
if segment.get("confidence") >= config.confidence_threshold:
brand params are empty {'brand_name': '', 'brand_variations': [], 'branded_products': [], 'branded_products_categories': []..
and it's indeed broken: EXCEPTION:detected_text_in_first_5_seconds() missing 1 required positional argument: 'annotation'.

Fixed in latest PR


GitLab Compatibility (including self-hosted)

We use a self-hosted instance of GitLab in my organization. This tool looks great, but sadly I can't use it. Would be awesome if support for GItLab could be added in a future release.
https://about.gitlab.com/

GitLab is an awesome tool! However, we at @zeit are probably never going to use it. In turn, the push behind supporting this and maintaining the support wouldn't be very good.
I hope you understand... 😊

@leo before I look into extending Release to support GitLab, would this PR be rejected based on the above comment?


Search Transporter form component

The actual form to be added to the manifest, it take the same input at the transporter search API.
Eventually it will need to submit post request to the transporter search endpoint in the form
{
  "epaId": "<search term>",
  "name": "<search term>"
}

It should only submit a request to the back end if the length of either the epaId or the name field has at least three characters.

closed with #186


[ENH] Test clone with all parameters set to non-default values

Bug https://github.com/alan-turing-institute/sktime/issues/2026 was not detected by our test suite, despite the fact that we did have a test that, for all estimators, tests whether clone works.
The problem with the current test is that clone will not fail if a problematic parameter (e.g., overwritten in __init__ is set at its default.
To cover all potentially problematic parameters, a new test is added where before invoking clone all parameters are overwritten at least once to a non-default value.

superseded by testing multiple param settings


Analyzer stuck at 100% CPU on macOS (infinite loop)

When I run flutter analyze in my Flutter project, it runs indefinitely with the Dart process at 100% CPU.
I can workaround this behaviour with this section in the analysis_options.yaml
analyzer:
  exclude:
    - build/**
    - ios/**
    - macos/**

I see the behavior with flutter stable (Dart 2.7.0) and flutter dev (Dart 2.8.0-dev.0.0)
An other issue https://github.com/Dart-Code/Dart-Code/issues/1114 suggests that this may have something to do with a symlink cycle in one of this folders.
I see the same behaviour with the Dart Analysis Server in Intelli, Android studio and VisualStudio Code. (and this is even worth because it seems the exclude section of the analysis_options.yaml is not taken into account?).
I'm available to provide more information if needed.

@scheglov

I opened https://github.com/flutter/flutter/issues/47673 with instructions to reproduce the issue.


Rank services from least common to most common

What services are ubiquitous across libraries (i.e. in 95% or more of the institutions we looked at)? What areas do most/all libraries support? What areas are least supported? What services are rare in libraries (i.e. in 5% or fewer of the institutions)? For assessing which services, will need to collapse across modes for individual services; e.g. a library is considered providing support for data management plans if we found a URL for at least one of the following modes: consult, instruction, web.

Addressed by 77ab9b0


Make it compatible with node 4

Hi there ✋
This plugin is not compatible with node 4 because of the ...asyncChunksSource syntax which throws an error.
I am using this plugin for nuxt.js and I have to choose between dropping node 4 support or removing this plugin (see failed tests). This PR will fix this dilemma 😄
Thank you

@Atinux This should be good to go. Thanks for the PR. We should be getting out a new release within the next week.


ci: Fixed PyPI deploy

Pull Request Checklist

[X] Make sure to read the contributor guide here.
[X] Make sure you are making a pull request against the main branch.
[X] Make sure your pull request title matches the requested format.
[X] Make sure to lint, type-check, and run unit and API tests.

Description
Fixed PyPI deploy auth issue

:tada: This PR is included in version 4.0.1 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


create character network

For the new architecture ideas the character tree concept needs to be handled by a separate network.

One challenge here is to differentiate semantic relationships between sequences. See https://github.com/xh3b4sd/anna/blob/master/doc/diary/2016_03_24.md


fix bugs in custom object

Codecov Report

Merging #269 (dd0c41c) into main (d4d759b) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main     #269   +/-   ##
=======================================
  Coverage   80.18%   80.18%
=======================================
  Files          44       44
  Lines        4865     4865
=======================================
  Hits         3901     3901
  Misses        742      742
  Partials      222      222

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d4d759b...dd0c41c. Read the comment docs.


The data in Incomplete is not used right now

Most of the Incomplete(usize) instances return 0 right now. Here are the possible fixes:

remove the field entirely, and let the calling code manage data aggregation automatically. This is easy (and corresponds to how the current code works).
return a sum type, something like Unknown|Size(usize). This adds more code in pattern matches, but it handles the case where we do not know how much data should be returned, and the case where we know, and the calling parser can augment it

There is still the problem that parsers are just functions, and do not have a value attached for the minimal data size they could need.
Still, returning a needed size is useful in cases where you need to seek, or load a large part of a file in memory, instead of chunking.

Since 232f6766c6c00ecb8cc1e4425240f6e07b5cbaef IResult changed to that signature:
#[derive(Debug,PartialEq,Eq)]
pub enum Needed {
  Unknown,
  Size(u32)
}

#[derive(Debug,PartialEq,Eq)]
pub enum IResult<I,O> {
  Done(I, O),
  Error(Err),
  Incomplete(Needed)
}

This does not change current matches to Incomplete(_), and lets the developer optionally get more info.
The first attempt was to add a field named IncompleteUnknown but it made the compilation time unbearable (adding a lot of match arms in macros).


Can only crop raw image using right corners or right side (Nikon NEF file) after opening

I open a raw image file (NEF), click to edit it, then go to crop the image. Using any of the handles, besides the right corners or right side, you can not crop. Only after one of the right corners has been moved, the other handles (top, left, bottom, and left corners) will work. This does not happen with JPG files.

I do not see this behaviour with NEF's from my D600. What NEFs are you using, and what desktop OS?


language/deep_nesting2_negative_test failing on precompiled runtime after stack depth change

Repro eg on X64 debug on Linux:
python tools/test.py -c precompiler -r dart_precompiled language/deep_nesting2_negative_test

This is a crash, starting after landing of https://dart-review.googlesource.com/c/sdk/+/14920 (itself a relanding), with log
FAILED: precompiler-dart_precompiled debug_x64 language_2/deep_nesting2_negative_test
Expected: Pass
Actual: Crash
Unexpected compile-time error.
Short reproduction command (experimental):
python tools/test.py -c precompiler -r dart_precompiled language_2/deep_nesting2_negative_test
CommandOutput[precompiler]:
Command[precompiler]: DART_CONFIGURATION=DebugX64 out/DebugX64/dart_bootstrap --snapshot-kind=app-aot --snapshot=/b/build/slave/precomp-linux-debug-x64-1-3-be/build/sdk/out/DebugX64/generated_compilations/precompiler/tests_language_2_deep_nesting2_negative_test/out.S --reify-generic-functions --ignore-unrecognized-flags --packages=/b/build/slave/precomp-linux-debug-x64-1-3-be/build/sdk/.packages /b/build/slave/precomp-linux-debug-x64-1-3-be/build/sdk/tests/language_2/deep_nesting2_negative_test.dart
Took 0:00:24.129844

Both the test language/deep_nesting2_negative_test and language_2/deep_nesting2_negative_test are failing, and it differs between configurations of precomp:  it passes on windows simarm64 precomp and fails on mac simarm64 precomp and crashes on linux precomp debug and fails on linux precomp product.

rmacnak@ has reapplied the patch, closing issue.


luci-app-openwisp: improved schema for usability

Without these changes, saving the form of luci-app-openwisp without doing any change generates a configuration which will not work in most cases.
In this patch I took care of providing the right settings to show the default values, avoid saving default values and show the interface names properly. With these changes it will be possible for a user to edit just 1 field without introducing breaking changes.

@mips171 CC

@dibdot @jow- @hnyman could anyone take a look at this? :pray:

@nemesisdesign - merged, thanks!

@nemesisdesign - merged, thanks!

No, thank you Jow!


Write Unit test case for Hive in flutter mobile app

Is your feature request related to a problem? Please describe.
I am using Hive flutter plugin for local storage. I want to write unit test for the same. But I can not Mock the Hive Object and test it.
Describe the solution you'd like
I am using the given below code to store and get data from local storage. When I am trying to write unit test case, i am not able to mock Hive object to test. Please provide some sample to write unit test case for. the below scenario.
'
Future initHive() async {
await Hive.initFlutter();
}
Future storeData(
String storageKey, dynamic value}) async {
var storageBox = await Hive.openBox('data');
storageBox.put(storageKey, value);
}
Future getStoredData(String storageKey) async {
var storageBox = await Hive.openBox('data');
var value = storageBox.get(storageKey);
return value;
}
'
Describe alternatives you've considered
Version

Platform: iOS, Android, Mac, Windows, Linux, Web
Flutter version: 1.12.13+hotfix.8
Hive version: 1.4.1+1

Since Hive works by instantiating a global variable, to mock it you would need to override this variable in your tests. But since it is declared as final, it is not possible :/.
A dirty solution would be creating a new global variable that can be overridden and use it in your code instead of Hive.
// global variable
HiveInterface FooHive = Hive;

Then, in your tests you can:
class FooHiveMock extends Mock implements HiveInteface {}

...

setup(() {
  FooHive = FooHiveMock();
});

Well, actually I have used Mocktail to create a moc of the hive. Now I wanna open the box and get its value but it's not possible.
This is my setUp function for test initializations


PowerShell Script does not work

This script does not work. New-AzMySqlServer is not a recognized commandlet
$Password = Read-Host -Prompt 'Please enter your password' -AsSecureString
New-AzMySqlServer -Name mydemoserver -ResourceGroupName myresourcegroup -Sku GP_Gen5_2 -GeoRedundantBackup Enabled -Location westus -AdministratorUsername myadmin -AdministratorLoginPassword $Password

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 1241adb1-413f-8d98-f4ea-16031d0fac60
Version Independent ID: 405ff59c-6937-8980-1260-ad769c129a66
Content: Quickstart: Create a server - Azure PowerShell - Azure Database for MySQL
Content Source: articles/mysql/quickstart-create-mysql-server-database-using-azure-powershell.md
Service: mysql
GitHub Login: @ajlam
Microsoft Alias: andrela

Hi @wayneme75 - the Az.MySql PowerShell module is still in preview and you'll need to install it separately. You can use the following command:  Install-Module -Name Az.MySql -AllowPrerelease

We will now proceed to close this thread. If there are further questions regarding this matter, please comment and we will gladly continue the discussion.


Always open app with single trade

Coverage decreased (-0.006%) to 31.677% when pulling 5ed637d21e97af940f8220be22ae507ad387487e on qingweibinary:qingwei/debug_trackjs into d2384f279a364fe482dd6de22677a092bf42b4f5 on binary-com:master.

Coverage decreased (-0.006%) to 31.677% when pulling a4873887c4c02a554736490f07542db697b146f4 on qingweibinary:qingwei/debug_trackjs into d2384f279a364fe482dd6de22677a092bf42b4f5 on binary-com:master.


[WIP] test remove pre insert into rule

Be sure to do all of the following checklist to help us incorporate
your contribution quickly and easily:

[ ] Any interfaces changed?

[ ] Any backward compatibility impacted?

[ ] Document update required?

[ ] Testing done
Please provide details on
- Whether new unit test cases have been added or why no new tests are required?
- How it is tested? Please attach test report.
- Is it a performance related change? Please attach the performance test report.
- Any additional information to help reviewers in testing this change.

[ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.

Build Success with Spark 2.1.0, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder2.1/3358/

Build Failed with Spark 2.2.1, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder1/3558/

Build Failed  with Spark 2.3.2, Please check CI http://<IP_ADDRESS>:8080/job/carbondataprbuilder2.3/11625/

Build Success with Spark 2.1.0, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder2.1/3376/

Build Failed  with Spark 2.3.2, Please check CI http://<IP_ADDRESS>:8080/job/carbondataprbuilder2.3/11643/

Build Failed with Spark 2.2.1, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder1/3579/

Build Success with Spark 2.1.0, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder2.1/3402/

Build Failed with Spark 2.2.1, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder1/3605/

Build Failed  with Spark 2.3.2, Please check CI http://<IP_ADDRESS>:8080/job/carbondataprbuilder2.3/11669/

Build Success with Spark 2.1.0, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder2.1/3405/

Build Failed with Spark 2.2.1, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder1/3608/

Build Failed  with Spark 2.3.2, Please check CI http://<IP_ADDRESS>:8080/job/carbondataprbuilder2.3/11672/

Build Success with Spark 2.1.0, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder2.1/3442/

Build Failed  with Spark 2.3.2, Please check CI http://<IP_ADDRESS>:8080/job/carbondataprbuilder2.3/11709/

Build Failed with Spark 2.2.1, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder1/3645/

Build Failed  with Spark 2.1.0, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder2.1/3519/

Build Success with Spark 2.1.0, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder2.1/3520/

Build Failed with Spark 2.2.1, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder1/3724/

Build Failed  with Spark 2.3.2, Please check CI http://<IP_ADDRESS>:8080/job/carbondataprbuilder2.3/11791/

Build Failed with Spark 2.2.1, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder1/3726/

Build Success with Spark 2.1.0, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder2.1/3545/

Build Success with Spark 2.3.2, Please check CI http://<IP_ADDRESS>:8080/job/carbondataprbuilder2.3/11820/

Build Success with Spark 2.2.1, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder1/3753/

@ravipesala : please check.

Build Success with Spark 2.1.0, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder2.1/3560/

LGTM

Build Success with Spark 2.3.2, Please check CI http://<IP_ADDRESS>:8080/job/carbondataprbuilder2.3/11834/

Build Success with Spark 2.2.1, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder1/3767/

Build Success with Spark 2.1.0, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder2.1/3572/

Fixed the SDV test suite also

Build Success with Spark 2.1.0, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder2.1/3576/

Build Success with Spark 2.3.2, Please check CI http://<IP_ADDRESS>:8080/job/carbondataprbuilder2.3/11846/

Build Success with Spark 2.2.1, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder1/3779/

Build Success with Spark 2.2.1, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder1/3784/

Build Success with Spark 2.3.2, Please check CI http://<IP_ADDRESS>:8080/job/carbondataprbuilder2.3/11851/

retest this please

Build Success with Spark 2.1.0, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder2.1/3591/

Build Success with Spark 2.2.1, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder1/3798/

Build Success with Spark 2.3.2, Please check CI http://<IP_ADDRESS>:8080/job/carbondataprbuilder2.3/11866/

@ravipesala / @kunal642 : PR is ready. please merge

LGTM


Pause for certain time after every data row is displayed

Is that possible to pause the animation for 1 or 2 secs after every row of data is displayed?

Yes, it would be possible to add a delay at the end of each period. Maybe a parameter like delay or end_period_delay could work. Not a bad idea. Let me think about it some more.

thanks, it, might become a most sought after feature

The more I think about it, the more it makes sense. I'm going to add it now.

Just added this. Will be in the next release. Thank you @sathyamoorthyrr for the idea!

@tdpetrou give me some tips to add silent audio to the animation
Whenever, I upload the result file to Telegram, it gets rendered as GIF instead of Video as there's no audio

@tdpetrou would 2nd point above meaningful?

@tdpetrou When will these changes be released? 🙄

@tdpetrou When will these changes be released? 🙄

@tdpetrou Been an year. will this be released?

I did this by have two rows with the same data one after an other.  So there is one interval where it animates and one where it sits there and pauses.


Unable to make mongodb restore

Summary
I'm trying to restore mongodb backup made by db-backup container to database which is in the same docker-compose stack.
Steps to reproduce
My docker-compose.yml file:
version: '3.7'
services:
  mongodb:
    container_name: mongodb
    image: mongo:latest
    ports:
      - "27017:27017"
    volumes:
      - ./data:/data/db
    restart: always
    environment:
      MONGO_INITDB_ROOT_USERNAME: ${MONGO_INITDB_ROOT_USERNAME}
      MONGO_INITDB_ROOT_PASSWORD: ${MONGO_INITDB_ROOT_PASSWORD}

  # Database backup
  mongo-backup:
    container_name: mongo-backup
    image: tiredofit/db-backup
    links:
      - mongodb
    volumes:
      - ./backups:/backup
    environment:
      - DB_TYPE=mongo
      - DB_AUTH=admin
      - DB_HOST=mongodb
      - DB_NAME=xxx
      - DB_USER=${MONGO_INITDB_ROOT_USERNAME}
      - DB_PASS=${MONGO_INITDB_ROOT_PASSWORD}
      - DB_DUMP_BEGIN=0500
      - DB_DUMP_FREQ=1440
      - DB_CLEANUP_TIME=10080
    restart: always

What is the expected correct behavior?
Backup is performing normally, so i guess env's are correct and service is able to connect.
Relevant logs and/or screenshots
[tiredofit/db-backup:3.2.1 05:03:26] # restore

## tiredofit/db-backup Restore Script Version 1.0.1
## Visit https://github.com/tiredofit/docker-db-backup/
## ####################################################

1) /backup/zzz_20220407-050000.archive.gz
2) /backup/zzz_20220407-050500.archive.gz
3) Custom
4) Quit
Please select a file to restore: 1

What Database Type are you looking to restore?

    E ) Environment Variable DB_TYPE: 'mongo'
    F ) Parsed Filename Type: 'mongo'

    M ) MySQL / MariaDB
    O ) MongoDB
    P ) Postgresql
    Q ) Quit

** Enter Value (E) | (F) | (M) | (P) :  E

What Hostname do you wish to restore to:

    C ) Custom Entered Hostname
    E ) Environment Variable DB_HOST: 'mongodb'
    F ) Parsed Filename Host: 'mongodb'
    Q ) Quit

** Enter Value (C) | (E) | (F) :  E

What Database Name do you want to restore to?

    C ) Custom Entered Database Name
    E ) Environment Variable DB_NAME: 'xxx'
    F ) Parsed Filename DB Name: 'xxx'
    Q ) Quit

** Enter Value (C) | (E) | (F) :  E

What database user will be used for restore:

    C ) Custom Entered Database User
    E ) Environment Variable DB_USER: 'xxx'
    Q ) Quit

** Enter Value (C) | (E) :  E

What Database Password will be used to restore?

    C ) Custom Entered Database Password
    E ) Environment Variable DB_PASS
    Q ) Quit

** Enter Value (C) | (E) :  C
** What DB Password do you wish to use: xxx

What Database Port do you wish to use?

    C ) Custom Entered Database Port
    E ) Environment Variable DB_PORT: '27017'
    Q ) Quit

** Enter Value (C) | (E) :  E
2022-04-07.05:07:19 [INFO] ** [db-backup-restore] Restoring '/backup/zzz_20220407-050000.archive.gz' into 'mongodb'/'xxx'
Enter password:

2022-04-07T05:07:29.117+0000    error connecting to host: could not connect to server: connection() : auth error: sasl conversation error: unable to authenticate using mechanism "SCRAM-SHA-1": (AuthenticationFailed) Authentication failed.
2022-04-07.05:07:29 [ERROR] ** [db-backup-restore] Restore reported errors

Environment

Image version / tag:  latest
Host OS: OSX

Additional information
If's weird for me too that after passing an ENV DB_PASS i'm still propting about password as the end step of restore process.
I'm missing only a step where user need to pass DB_AUTH=admin (as default in mongo) it might be a case? It's defined in ENV for tis service.

In my case, everything is the same but after press "E" when promted on Database Port, it ask me again and again. It never stops and doesn't work "C" for custom neither.

Same here,
any news on this @tiredofit
Can we help?

I've solved it going back to v3.2.5 and blocking any update.

Hi there, not sure what happened - I remember repairing this before and I then went on a long trip- Its now back to the way it was with newly released 3.3.5 - Please let me know if indeed solved.

Hi.
Still facing the same error:
2022-08-12.17:50:13 [INFO] ** [db-backup-restore] Restoring '/backup/mongo_mongo_20220812-010004.archive.gz' into 'mongo'/'s_2'
Enter password:

2022-08-12T17:50:55.020+0000	error connecting to host: could not connect to server: connection() : auth error: sasl conversation error: unable to authenticate using mechanism "SCRAM-SHA-1": (AuthenticationFailed) Authentication failed.
2022-08-12.17:50:55 [ERROR] ** [db-backup-restore] Restore reported errors

Am i doing something wrong ?

Same issue here.
Looks like mongorestore command should pass correct --authenticationDatabase param

@Yozer you are right, after adding --authenticationDatabase=admin in https://github.com/tiredofit/docker-db-backup/blob/main/install/usr/local/bin/restore#L928 it works. But im not sure, if this is correct solution and will work for others too.


Migrate to OCP4

Overview
A few profiles use issuer-kit code for their apps, in particular:

a2a: this is a development issuer used to test other applications. Only dev and test deployments exist.
As such, in addition to updating the templates, the agent wallets will need to be migrated to the target namespace in OCP4 to ensure a seamless transition.
lsbc: issuer used by an external team (The Law Society of British Columbia). The wallet contents need to be moved to OCP4 for all deployments (dev/test/prod).
prime: issuer used by an external team (MoH Prime). The wallet contents need to be moved to OCP4 for all deployments (dev/test/prod). Additionally, instances of visual-verifier are used and need to be migrated over to OCP4.
prime-sandbox: same as prime, just a sandbox development agent (no test, no prod). Wallet contents need to be moved over to OCP4.
tails: deployments of hosted instances of indy-tails-server. PVC contents need to be moved to OCP4.
default and rev: instances of vc-authn-oidc. Content of both wallet and database need to be transferred over to OCP4.
backup: backup-container (postgres) instance. Data needs to be migrated (it can help migrating wallets/databases as it is already backing them up). Needs to be aligned to the latest backup-container configurations in backup-container.

Templates
This is a list of templates to be updated, assigned to the relevant person.
@wadeking98:

[x] visual-verifier
[x] issuer-wallet
[x] issuer-db
[x] issuer-agent
[x] issuer-api
[x] issuer-web
[x] backup

@esune:

[x] agent-rev
[x] agent
[x] controller-rev
[x] controller
[x] database
[x] wallet
[x] tails-server

Profiles:
This is a list of profiles to be deployed to OCP4, once the templates have been updated, assigned to the relevant person.
@wadeking98:

[x] a2a
[x] lsbc
[x] prime
[x] prime-sandbox

@esune:

[x] default
[x] rev
[x] tails

@wadeking98 I have not yet thought through 100% what to do with the pipelines, but I think as a first step ensuring the components based on issuer-kit respect the same naming convention we put together for essential-services-delivery is a good step. That means the build configurations must be named and labelled the same way (so that they result in exactly the same output) and the deployment configurations must reference those image names.

@WadeBarnes: as discussed, the tails server was setup and is ready to go in all environments. A new sync of the files stored by the old one will be required before we switch over, as well as coordination to ensure the DNS entries are updated correctly when the time comes.

All services have been migrated successfully.
@wadeking98 is in the process of migrating the data for vc-authn and updating the apps that connect directly to it (e.g.: the visual verifiers) to use the new OCP4 urls.
Please note: since we have two instances of vc-authn (one supporting revocation and the other not supporting it) make sure the URL used to update the visual-verifier secret(s) is the appropriate one.

@esune are we free to close this now?

The only thing left to do is turning off the old tails servers - this coming friday. Maybe we should keep this open until we do that and then wrap up everything.

Tails servers have been migrated, so this can be closed now.


How to set socket options for ModbusTCP connection? (e.g. keep-alive)

I have a case where I'm communicating with a ModbusTCP slave device, and its firmware appears to close TCP connections that have been idle for ~5 minutes. Is there a way to access the underlying socket without reaching in too deeply? Looks like I can do this via connection.transport.stream, e.g.
modbus.tcp.connect(port, host, { debug: '...' }, (err, connection) => {
  connection.transport.stream.setKeepAlive(true);
  // ...
});

Is this OK / is this part of the public / stable API from modbus-stream?

Yes, it's OK as long as you ensure you're always using TCP. We'll not break any of those paths, the module looks stable and will not probably change much, just fixes and polishing.