merge ipv4 & ipv6 verification and more IBAN testing

merge ipv4 & ipv6 validation functions, depending on which regex pattern is fit.
add IBAN testing, representing all valid country patterns as represented at:
https://www.iban.com/structure.html

Very nice!


Fix shellescape to work on Tiger

Fix shellescape to work on Tiger
Closes: https://trac.macports.org/ticket/56626
The previous code relied on the fact that we use sh for system and sh is supposed to be a POSIX shell but on Tiger it isn't; Leopard was the first POSIX compliant Mac OS X version

I chose the list of safe characters based on the chart shown here:
https://stackoverflow.com/questions/15783701/which-characters-need-to-be-escaped-when-using-bash#answer-27817504
The test had to be changed from echo to printf because echo would interpret some of the escaped characters.
I verified the test now passes on Tiger and High Sierra.


Enhance Host installation progress for info stageStartedAt and stageUpdatedAt

Considering the data recently received from backend, I am not sure if this change brings much value to the user.
It seems that the stageUpdatedAt works recently just for the writing image stage, since the sub-progress is updated during the process.
Otherwise even long-running stages (like the waiting for control plane) stay untouched long time till reaching next stage.

@andybraren Time passed, thoughts evolved, do we want something like this?

Hm, it's been a while so I've lost some context, but displaying Step started at even if the updated 3 seconds ago part only appears for the Writing image to disk step still seems like it could be somewhat useful to me. WDYT @nirfarkas? Worth the additional text?

LGTM

Rebased


NIOLoopBound should get unchecked variants that are truly free in release mode

NIOLoopBound(Box) is super useful when adopting Sendable but the eventLoop.preconditionInEventLoop() checks everywhere are expensive and -- for repeated accesses -- unnecessary.
There should be unchecked variants of init, get and set value that do eventLoop.assertInEventLoop() instead. Together with @inlinable that should become completely free in release mode (and still check in debug).
Docs:

https://swiftpackageindex.com/apple/swift-nio/main/documentation/niocore/nioloopbound
https://swiftpackageindex.com/apple/swift-nio/main/documentation/niocore/nioloopboundbox

The other option is to use CurrentEventLoop and friends from https://github.com/apple/swift-nio/pull/2228 which only call preconditionInEventLoop() once during initialisation.

The other option is to use CurrentEventLoop and friends from #2228 which only call preconditionInEventLoop() once during initialisation.

Yes, good to have multiple options.

It seems like NIOLoopBoundBox has internal init with an unchecked event loop already. And, NIOLoopBox conforms to @unchecked Sendable. To confirm:

We're not talking about that @unchecked Sendable protocol.
You suggest adding a version of init and value get and set that would still verify the same event loop, but with an assert, so they might blow up.
If the approach below is what you mean, then in release mode, no preconditionInEventLoop() will be invoked. So if something else changes what event loop the code is running on between debug and release, the user might get in trouble. But that's on them, they've used the unchecked variant.

Something like:

// NIOLoopBound

@inlinable
public init(_ value: Value, uncheckedEventLoop: EventLoop) {
    eventLoop.assertInEventLoop()
    self._eventLoop = eventLoop
    self._value = value
}

@inlinable
public var uncheckedValue: Value {
    get {
        self._eventLoop.assertInEventLoop()
        return self._value
    }
    set {
        self._eventLoop.assertInEventLoop()
        self._value = newValue
    }
}

@dnadoba @weissi I've read the proposal in #2228, but I'm not yet experienced enough to quite grasp how I could use it to achieve a similar result.
If the approach above is useful, I'm happy to put this together. If not, let me go work through a few example apps to gain more experience with event loops. Pointers / speed runs towards that particular area are very welcome 🙃

@natikgadzhi yes, thanks, uncheckedValue is exactly what I had in mind.

Thank you swift 😊❤ I 🍏🍎 m glad you are you. Swiftapple. 😊 Good I will look it over and add or git init if it is possible thanks


NPM Example in Readme

Hi There,
Looks great - however I can't for the life of me work out how to get it working in an NPM environment.  No matter what I do, I keep getting 'L.ExtraMarkers undefined' errors.
I'm not a newb but also not experienced in how to do NPM includes that override one another like leaflet and extra-markers.
Would it be possible to include a minimal NPM example in the readme?
Many thanks,
Greg.

OK - So, I figured it out a solution - not necessarily the best or correct.  I'll leave it here for anyone interested. Would still be good to have a formal version in the NPM readme :-).
React/Typescript/ESLint environment...
import L from 'leaflet' // eslint-disable-next-line  (needed as we don't reference ExtraMarkers after the next line so ESLint complains) import * as ExtraMarkers from 'leaflet-extra-markers'; // or else TS complains about the L extension import<EMAIL_ADDRESS> //e.g. using FA icons import 'leaflet-extra-markers/dist/css/leaflet.extra-markers.min.css'; //import the LEM css require( 'leaflet-extra-markers'); //Do the L extension.


Add a wiki: Ages of Conflict Wiki

An anonymous user has submitted a new wiki via our submission form.
Link to origin wiki: aocw.fandom.com
Link to destination (independent) wiki: agesofconflict.wiki
Note from submitter: "The wiki recently migrated off Fandom to Miraheze."

This is an interesting case. City of Heroes is a game that shut down in 2012. We currently redirect cityofheroes.fandom.com to archive.paragonwiki.com (the independent wiki that's since been archived).
There is now an unofficial fork of the game running called[Homecoming: City of Heroes, and they have their own wiki at homecoming.wiki (which is a fork of the archived Paragon Wiki). Likely makes sense to update the redirect, assuming most people are looking for Homecoming info and the info on the original game is retained.

Looks like the Fandom wiki has been closed and is minimally indexed/linked.


OpenCL does not work for standard conda installation of bempp-cl

Hi everybody,
I stuck with a problem on standard conda installation of bempp-cl. In my case, opencl does not work.
At first, I installed the newest version of anaconda from https://www.anaconda.com/products/individual.
Then I installed bempp-cl=0.2.2 into a new conda environment by the command:
conda create -n bempp-cl-env -c conda-forge bempp-cl=0.2.2 notebook matplotlib

If I set bempp.api.DEFAULT_DEVICE_INTERFACE = 'numba' everything works.
If bempp.api.DEFAULT_DEVICE_INTERFACE = 'opencl' (by default) Jupyter kernel appears to have died when I try to assemble the matrix (solve the system). In particular, it is the case for any of tutorials on bempp.com. When I run the same code in python script I can see that it is segmentation fault.
Command bempp.core.opencl_kernels.show_available_platforms_and_devices() returns
0: Portable Computing Language
    0: pthread-Intel(R) Xeon(R) Gold 6144 CPU @ 3.50GHz

so bempp-cl can see only CPU and no GPUs.
Command  print(bempp.core.opencl_kernels.default_device()) returns
<pyopencl.Device 'pthread-Intel(R) Xeon(R) Gold 6144 CPU @ 3.50GHz' on 'Portable Computing Language' at 0x562e2a1c1830>

so default_device is the CPU.
Few months ago, it was no problems with bempp-cl installed in the same way on the same machine. The operation system is CentOS 8. The machine has 32 Intel processors and GPU NVIDIA Tesla V100.
Have you any idea how to fix this?

I had a similar problem recently with Fedora 33. Maybe, it is related. In conda I had to downgrade to POCL Version 1.5 build hecece54_4. Interestingly, this seems not to be an issue on Ubuntu 20.04.

Thank you! Downgrading POCL version from 1.6 to 1.5 fixed the problem. Maybe it makes sense to put a restriction on pocl = 1.6 in the conda package?

We have to investigate this further. I haven't seen this issue on Ubuntu (but need to check more). I am also not sure if it is really POCL related as later builds of POCL 1.5 on Conda do create the issue. The only change is different dependencies in Conda. I will need to bump this up to the POCL developers. But closing this bug report for now as not in Bempp-cl.

Hello Everyone,
I have just run into this problem on a workstation running Ubuntu 18.04.4 LTS. As suggested above, downgrading POCL from version 1.6 to 1.5 fixed the problem.

Hi!
The same issue on Linux Mint 17.3 Rosa.
Downgrading POCL from version 1.6 to 1.5 fixed the problem.


How to add custom view (start and end date)

Hi there,
I'm looking for some solution to get custom view on react-big-calendar. I saw few examples including view of year (https://github.com/jquense/react-big-calendar/issues/1694) but is there any chance to render custom view using start and end date in this library?
Many thanks, Chris

on the same boat, is there a way to pass props to a CustomView?

https://jquense.github.io/react-big-calendar/examples/index.html#customView


Autodeploy to github pages

@tsiq-karold did you have any luck with that?

I'm going to close this because fully automating it would require us to get Sonatype creds into Travis and I don't have the time to figure out how to do all that.  For now, we have a script that can be run locally post-release.


RFC: Support forked backend model

Any thoughts on supporting a model where backend modules are started in a forked process? Metrics would be pushed to the backends over an established pipe, allowing backends to process/post measurements with less impact to the main statsd daemon. For low flush periods, are there any metrics on time drift from backend processing?
The backend pipe should be set non-blocking to prevent statsd from blocking while flushing metrics to a backend that is not able to keep up. Open question is whether the daemon should drop metrics if a backend pipe is blocked over some threshold of time.

After thinking about this and having played with statsite's implementation of this, I don't think we should do this. We would need to add another serialization/deserialization step into StatsD and I don't think the benefit of writing backends in a different language outweighs the cost of that step.


refactor: update CollisionDetection to accept DraggingNode and DroppableContainer[]

Resolves https://github.com/clauderic/dnd-kit/issues/255 and https://github.com/clauderic/dnd-kit/issues/309.
This updates the signature of CollisionDetection to accept DraggingNode and DroppableContainer[] in its arguments, hopefully making it easier to build richer custom collision detection functions. I've also made a small performance improvement to the three included collision detection algorithms, which were making an unnecessary second pass to determine the index of the relevant figure.
The one uncertainty I had writing this was the inclusion of the new DraggingNode type. I chose an additional type because rect for DraggableNode would always be null for the non active node, and this would lead to a bunch of unnecessary non-null assertions in collision detection algorithms.
Looking forward to feedback, and thanks for the great library!

@clauderic Unsure if you'd prefer to write the changeset or if I should include that as well. The bot doesn't really suggest which is better :).

Looks like some unrelated changes were picked up by lint --fix. Happy to remove if you'd like.

Hey @wmain, thanks for getting the ball rolling on this one and opening a PR 🙏  I've been swamped recently and haven't had time to review this yet, but should be able to get to it next week.

Hey @clauderic, anything I can do to help with this? Trying not to rush you, but this is blocking some work we want to do on our project.

Just an update, I'm back from vacation this week, will get this merged before end of week

Thanks @clauderic !

I won't be releasing this to npm on the latest tag just yet, but you can start using it by installing the next tag:

Hi @clauderic ,
I just follow example on storybook (Preset/Sortable/Multiple Containers/Basic Setup) and it takes me lot of time to find out this change not release on the latest tag. Please note this in document or storybook incase anyone want to refer
BTW, thank you for your work.


Feat/bill cpc 1357 user can see time remaining to pay bill

JIRA: https://champlainsaintlambert.atlassian.net/browse/CPC-1357
Context:
As a customer I want to be able to see the time remaining for my unpaid bills.
Acceptance criteria:

Customer should see the days remaining to pay their unpaid or overdue bills.

2.The time remaining should only apply to unpaid or overdue bills. PAID bills should have a bills paid or equivalent instead.
Does this PR change the .vscode folder in petclinic-frontend?:
It does not.
Changes

Changed db loader to load bills instead of bill request dtos.
Changed bill response to have the new field time remaining.
Changed bill response in api-gateway to have the new field time remaining.
Changed bill response in api-gateway to have the vet name and owner name fields.
Fixed old tests and added new ones to the both the bill testing in the API Gateway and the Bill service.
Added new method in EntityDtoUtil  to calculate time remaining.
Added the time remaining feature to the frontend as well as changed some for the styling for the bill status.

Before and After UI (Required for UI-impacting PRs)
 Before

(Took this from a previous pull request so ignore the red squares)
 After

Looks great! Im glad you were able to fix the issue you were facing with the db and updated some tests. The only thing I would suggest changing is switching back the status for a paid bill to "Paid" instead of "Bill is paid" to match the rest of statuses (A user can assume that Paid means that the bill is paid). Also consider changing the name of your PR to fit the naming convention.
Other than that, great work! You also eliminated one of my very minor stories (CPC-1362: Overdue bills should be shown in red) so I'll mark it as resolved.

Woops, didn't mean to do your story. Also did not mean to make them both say that, I accidently put it in both.

Looks great! Im glad you were able to fix the issue you were facing with the db. The only things I would suggest changing are switching back the status for a paid bill to "Paid" instead of "Bill is paid" to match the rest of statuses (A user can assume that Paid means that the bill is paid). Also consider changing the name of your PR to fit the naming convention.
Other than that, great work! You also eliminated one of my very minor stories (CPC-1362: Overdue bills should be shown in red) so I'll mark it as resolved.

Looks great! Im glad you were able to fix the issue you were facing with the db and updated some tests. The only thing I would suggest changing is switching back the status for a paid bill to "Paid" instead of "Bill is paid" to match the rest of statuses (A user can assume that Paid means that the bill is paid). Also consider changing the name of your PR to fit the naming convention.
Other than that, great work! You also eliminated one of my very minor stories (CPC-1362: Overdue bills should be shown in red) so I'll mark it as resolved.

Woops, didn't mean to do your story. Also did not mean to make them both say that, I accidently put it in both.

all good no worries:)


No code implementation about discriminative loss ?

Section 3.4 in your TPAMI version paper,  Learning Category Consistent Features
Is there any code for this part ?

I was also wondering this,
Is it possible for providing the implementation of this discriminative loss?

@LiZhenLiangLee @wqhIris Here's my guess.
The original paper published in TPAMI seems not strict enough to illustrate the implement detail. In fact, from this code, i find this model is a encoder-only model rather than auto-encoder like UNet. The upsample is simple bilinear interpolation, and the upsampled feature is the final segmentation result. Therefore, this main problem to implement the loss function proposed in Learning Category Consistent Features section is there's no feature embedding to "pull" and "push"(i.e. the loss component $L_var$ and $L_dis$).
Here's what i have tried.
I tried to re-implement this metric loss by adding bilinear interpolation followed by a 1x1 conv to the ccnet, while also code the loss function to optimize the feature(embedding) output from 1x1 conv. In fact, i found this loss is not easy to implement, and the training loss of my task seems a little unstable(but actually in convergence). I don't know whether the author of this repo also noticed this problem.


The Oxford Mega Super Meetup Meetup Retro

It would be really lovely/useful/great to get feedback on the first ever Oxford Mega Super Meetup Meetup.
Feel free to go retro style (stop/start/carry-on/etc), or, whatever you want really. Constructive things would be … constructive.
I guess the big question is: we should do this again, right?
(if you have stuff that doesn't belong on this thread - please drop me a DM/email)

I really want the first comment to be about Ben breaking the projector, so here you go.

@spikeheap thanks for pointing that out!  I'd like to point out a message that I put in the organisational slack channel:

…I guess I can nominate anyone in that channel for not helping out? I choose you @spikeheap, thanks for not bothering to help…
Though yeah - projector issues:

hdmi didn't work for some laptops
my mini DisplayPort -> vga adaptor kept on falling off
laptop wasn't at the right resolution

I guess what we could do:

bring a Mini DisplayPort -> HDMI adaptor instead (saves switching inputs)
do actual laptop checks beforehand

The good:

So many people, so many new faces, and IMHO quite a broad/diverse audience
8 talks pulled together with a weeks notice? Awesome.
Another talk about VR: woah man, we should pretty much have a VR talk every time there's an opportunity for a speaker
Having a mic was great.

The less good:

The MC (not pointing fingers) kept breaking the projector
Perhaps should have sent out projector specs (aspect ratio, resolution) to speakers before hand

It would have been nice to record the evening -- even some low-tech GoPro video would have been good I think. A lot of cool moments in the way these short talks are presented is not being captured!

Yes, we should do it again, it was awesome!

Projector - If possible, use a single laptop thats fully charged and set up before people arrive and make sure the speakers understand the mouse setup.
Mic was good but better positioning needed
Drinks / snacks were a bit random / sparse
More seats if possible

🍺 One of the guys I was talking to pointed out that this event was very much like the Oxford Geek Nights of old, and that made me think perhaps the next Oxford Mega Super Meetup Meetup should take place in the Jericho Tavern? Then that would solve the problem of finding a pub to go to afterwards, because you'd already be there.
😺 The talks had a good mix of funny, techy and non-techy, and I think that will keep a good range of people coming.
📹 I miss having the talks recorded. I can try and find out if it's possible to borrow a camera/mic from work. Unfortunately I haven't met any camera people - they tend to be in the other building - so that might be a bit difficult.
💻 Windows laptops work really well with projectors :trollface:

We should go to a quieter pub, or can we just stay at the venue for drinks for a bit longer?

Projector - Ubuntu worked fine with the HDMI! Having one main presentation laptop is fine if people are doing slides, but if you have live code and similar things it can be much trickier. I though Ben held it together really well despite the minor issues.
Pub - Great fun, as always, but agree finding somewhere where there isn't a quiz on at the same time might be good. Did anyone win the cheese raffle?
Overall, great evening!

It's been kind of hard to gather together all tweets/messages/photos/etc - this is partly because it happened on separate meetup pages, but also because #TheOxfordMegaSuperMeetupMeetup is a pretty awkward hashtag.
Almost all announcements went out through @JSOxford, and the JSOxford meetup page seemed like the central source of talks etc.  I guess this is because I was throwing them together at the last moment, so didn't have time to distribute properly.
I really loved the event.  And I also love the principle of a super-meetup - feels like it's got potential to bring together wider/more-diverse groups of people, and by having several tech group organisers involved it means it's always going to be community focussed/based.


Hi Yu meng!

Hi Yu meng:
I read your code and noticed that the code is designed as a full batch training model which might be caused by the part of loss considering global contexts. And this might further cause a massive computation complexity when dealing with large corpus with millions even billions of texts.
Do you have any concern or solution to this problem? Recently I'm interested in the supervised topic model and find your paper is really helpful. Thanks for your excellent work!
Best Weifan.

Another question, I also noticed that both joSH and CatE are implemented with C. Is there any possible limitation for other language like python to reproduce these model?
Best Weifan.


Convert basic enums to simple objects

I see typescript enums being used in a few places in the code, e.g. DEFENCE_TYPES.
There is often a better way to achieve this using the power of types. Refer to
https://www.typescriptlang.org/docs/handbook/enums.html#objects-vs-enums
Why are enums confusing? Because it is not always clear when you are referring to the enum name, and when to the (numeric or string) value. Numeric enums are particularly problematic as name and value are different, whereas String-valued enums seem a bit un-DRY when, as is often the case, the name is exactly the same as the value.
This is a placeholder issue for identifying all enums, and assessing which would be better represented using type narrowing and standard javascript objects. The first of these can then be used to better describe the issue, and a possible alternative. A likely candidate is #355.

Question: Need to decide what to do here @chriswilty

String-valued enums can very easily be converted to standard objects:
const RagStates = {
  RED: 'Error',
  AMBER: 'Warning',
  GREEN: 'Ok',
} as const;
type Rag = keyof typeof RagStates; // 'RED' | 'AMBER' | 'GREEN'
RagStates['AMBER']; // 'Warning'

String-valued enums where the name and value are identical, can be converted to const arrays:
const RagStates = ['RED', 'AMBER', 'GREEN'] as const;
type Rag = typeof RagStates[number]; // 'RED' | 'AMBER' | 'GREEN'

Numeric values of an enum, if genuinely needed, can be represented either by the index within an array ...
const RagStates = ['RED', 'AMBER', 'GREEN'] as const;
type Rag = typeof RagStates[number]; // 'RED' | 'AMBER' | 'GREEN'
RagStates.indexOf('AMBER'); // 1

... or for non-zero-based enums, you can again use a standard object:
const RagStates = {
  RED: 1,
  AMBER: 2,
  GREEN: 3,
} as const;
type Rag = keyof typeof RagStates; // 'RED' | 'AMBER' | 'GREEN'
RagStates['AMBER']; // 2

But remember that transferring the numeric value of an enum between UI and server, or storing that numeric value anywhere for later retrieval, is error-prone because inserting, removing or reordering enum values can very easily break things. It is far better to store the string value, as it has meaning.


Switch to cargo bench

Python is a barrier to running and working on these tests; especially since the scripts look especially complicated. cargo bench offers nice results like this OOTB:

I would recommend moving there and ditching the hacky Python scripts.
(P.S. this is because I want to add getargs 0.5.0 to the tests after it releases, but I won't do that until I can actually run/test the benchmark, which I can't/won't do if it requires Python as a shell-script replacement.)

cargo bench is not sufficient for our needs.  Besides parse time, we also care about binary size, build times, and a little behavior validation.
We could switch the runtime benchmarks to cargo bench, testing end-to-end performance isn't that big of a deal but

We'd need two copies of each test and keep them in sync, one for parse time and one for everything else.
At least at the moment, our only performance use case we care about is passing in a lot of files via  xargs so any extra features or benefits for cargo bench are minimal

Overall, unless we have sufficiently justified additional benxchmark use cases, I'm thinking changing this isn't worth it.

Is there any way for you to support (in any way) using cargo bench for users who may not want or be able to use Python?

We could implement a custom test runner to do all of the behavior of the python script but that is a lot of work for little return.

Would you accept a PR? (If so, could you explain what your requirements are from the test runner?)

If the PR is a custom test runner to re-implement all of the current behavior and continue to provide output fit for the README, I would be open to seeing how well it works out.  Otherwise, not really as it'd run into the problems I had mentioned earlier.
You mentioned all of this is for adding getargs to the benchmark.  That is also a whole separate conversation to have considering our acceptance criteria:

To be included, the crate needs meet one of the following criteria:

10k+ recent downloads
Unique API design

getargs doesn't have the recent download count and it looks to be in the same API design space as lexopt and clap_lex
In looking at getargs API to see if it offers something unique, I'd like to update my statement earlier:

We could switch the runtime benchmarks to cargo bench, testing end-to-end performance isn't that big of a deal

An important aspect of end-to-end benchmarks is that they reflect how most end-users will use the API and care would be needed to make cargo bench benchmarks reflect common real-world use cases.
getargs looks to be optimized for in-memory, &str, operations rather than for interacting with std::env::args_os.  In naive benches, this means that getargs will not do any allocations.  Most argument parsers assume they'll be taking ownership of the result from std::env::args_os and will need to allocations when that isn't the case, like in naive benches.
Results from benchmarks like those included in the getargs repo would need sufficient caveats on what the results are reflecting as most people would assume getargs will be significantly faster for their use cases when in reality it will be more even.

If the PR is a custom test runner to re-implement all of the current behavior and continue to provide output fit for the README, I would be open to seeing how well it works out. Otherwise, not really as it'd run into the problems I had mentioned earlier.

Yes, the PR would be to completely phase out Python and replace it with a custom test runner that performs all of the behavior of the current scripts. AFAIK, this is what I see:

A script to perform the benchmarks and write data to a file in the runs folder
A script to summarize the data from a particular run in a person-readable format

A custom benchmark runner can perform the first task, but the second task would probably be better suited to something like rust-script, so you could run (and tweak) the program at any time without having to perform another benchmark. This would remove the dependency on an external scripting language.
I don't fully understand how the test results are built, though.


fix(cache): new interface

There is a need for better control of the cache. So i suggest this interface which is not uncommon[0].
this requires tweaking so that caches operate with expiration time and not modification time.
examples:
// fetch cache item by key, returns empty array if absent
$data = $cache->get('data', []);

if ($response->code === 429) {
    // cache this for 60 seconds
    $cache->set('response', $response, 60);
} else {
    // cache this forever
    $cache->set('response', $response);
}

// delete all items
$cache->clear();

// delete this cache item by key
$cache->delete('response');

i dont think we need the purgeCache method. Its purpose is to delete old/expired cache items for space managment issues mainly. It's also being constantly called which clogs the cpu on high traffic.
thoughts?
@em92 @Mynacol
[0] https://www.php-fig.org/psr/psr-16/

i dont think we need the purgeCache

What is difference between purgeCache and clear?

What is difference between purgeCache and clear?

purgeCache deletes items older than $seconds while clear() deletes all items unconditionally.

An alternative is to change from public function loadData() to public function loadData(int $seconds) which can encapsulate the expiry logic inside instead of it leaking out.

i think this might interfere with the custom cache timeout option _cache_timeout

see https://github.com/RSS-Bridge/rss-bridge/pull/3594


agents protocol proposal

Multiple things here:
Orchestration improvements

Ping requests: When the worker register itself to the controller, we setup a ping interval, if the ping is too late, the controller will invalidate this worker (See 5. for retry)

The Python framework will automatically create an HTTP endpoint for health check (for k8).  This will easily be configurable by providing an async function to our users. (It will defaults to a basic OK response)
NOTE: When combining the health checks to the new pipeline API, we could automatically check if a specific pipeline is stuck. (This is important to detect issues with other providers like ElevenLabs/Deepgram/OAI...)

Better job dispatching:

It's up to the users to know how/when scaling up the workers but Cloud needs to be smarter on which worker the job should be dispatched. (e.g We want to avoid situation where we have a lot of nodes that handles only one job.)
Cloud needs to know of much jobs one Worker could theoretically handle

Users must define a function that returns the current load of a worker, this value is sent on each PingRequest and when accepting a job (Load is [0;1])
Users must define the cost of a job, we don't know if a job will use CPU/GPU or bandwidth, so we need this "abstract" and configurable cost that determines how much it'll will increase the load. (Cost is [0;1])
Depending on the job, the cost may be different, so the cost must be reported when accepting a job
Then Cloud will determine which worker is the best to assign a specific job:
1. Send a request to every worker,
2. Determine the best worker with the current load and the returned cost
3. How much answers should we wait for tho? this needs to be as fast as possible

When a worker becomes overloaded, it can request to migrate a specific job with a MigrateRequest, Cloud will then try to reassign the job to another worker (This should happens in the worst scenario)

Automatic retry of jobs. Cloud and OSS must automatically detect when a job is failing and send a AvailabilityRequest with resuming=true to another worker.
This could happens when the agent participant disconnects without sending JobState.JS_SUCCESS, or when the ping interval is too late.
Also when we invalidate a worker due to health checks failing, all jobs inside this worker must be resumed.
To know if a specific job can be resumed, this is specified inside the AvailabilityResponse (should_retry field).
This also applies when a media node crashes or becomes unavailable.

Third party workers

Agent that can connect to any project: e.g DirectAI or our own transcription agent

After thinking, we don't need a general subdomain like agents.livekit.cloud. What we're doing today is OK.
The main difference is how the jobs are dispatched, I propose to introduce the concept of namespaces.
It has the goal to resolve multiple issues (multiple jobs for one room & third party workers where a company can connect to rooms of other projects).The namespaces in Cloud are handled differently from OSS. A namespace is optional, when NONE, it is the same as if the namespace was tied to a single project (the worker only receive jobs from the project):

OSS:

Namespaces are created/removed on the fly when a worker registers using a specific namespace
Or another option for OSS is to just configure namespaces inside the yaml configuration file.

Cloud:

A namespace must be unique across all our project, they must be created inside our Cloud dashboard (name may be generated?).

namespace can have different properties:
1. cloud global means that this namespace can be used by other projects.
On the Cloud dashboard, users can enable/disable a specific namespaces
1. Only the project who created a namespace can create a Worker listening to this specific namespace.
2. We don't show the list of existing namespaces, the users must put it inside an input box, and when trying to activate it we show the name of the project that created this namespace (like LiveKit or DirectAI)

Miscellaneous improvements

Moving JobSimulation to the SFU (today I fake this inside the Python framework, but I want to reduce the code there)
Job can send metadata to Cloud so we can show some information on our dashboard.

Added the load of a job on the JobStatusUpdate message.
I think both WorkerStatusUpdate and JobStatusUpdate will be sent periodically


It would be cool to have script for starting a container.

Reply to this email directly, view it on GitHub
https://github.com/docker/docker/issues/29792, or mute the thread
https://github.com/notifications/unsubscribe-auth/AAdcPMHAV-Di4wg3GBcqZu15AgG4Asc3ks5rNQRDgaJpZM4LYLli
.

@justincormack Cool, thanx.


Don't hide console on no_editor layout

This allows to create exercises where the user can still play with code before going to next exercise

Console is actually being displayed, but it doesn't work.


NTR: flagellum attachment zone organization

Accession: GO:xxxxxx
Name: flagellum attachment zone organization
Ontology: biological_process
Synonyms: FAZ organization, flagellum attachment zone organisation
Definition: A process that is carried out at the cellular level which results in the assembly, arrangement of constituent parts, or disassembly of a flagellum attachment zone. FAZ is a network of cytoskeletal and membranous connections responsible for the lateral attachment of the cilium to the cell body in some trypanosomatid species.
is_a: GO:0006996 ! organelle organization
Source: GO:0120119, PMID:26776656

Hi @vanaukenk,
Wondering whether you had any chance to look at this term?
Thanks,
Achchuthan

@Achchuthan - the term has now been added and should be available for use in the next ontology release.


Fix plans, bump version, and update changelog

Coverage remained the same at 82.242% when pulling 31fa8e8d516fd6da9f4eba6dfb530a75bad12837 on mparke:fix-plan-miscalculation into da0432c430546e76fe119c072a194b68f175bdb3 on linode:master.


bug fix

Assigned Issue Number ?
#709
Language Opted ?
C++
Number of files added or updated ?
1
Name of these files ?
Maths_And_Stats => Series => Lucas_Series => Lucas_Series.cpp
Checklist

[x] I've read the contribution guidelines.
[x] I've referred the correct issue number.
[x] I've fill up this entire template correctly.

Bug fix for #723 (just changed one number so that the number of outputs is the same as the number input)

Major conflicts while merging :warning:


checkberry gpcheckperf series updates are used to solve the problem that cbdb 1.5.2 version gpcheckperf does not display disk information

fix #421 cbdb 1.5.2 version gpcheckperf does not display disk information

Change logs

Changes to fix gpcheckperf failure with an exception when sometime time command output
has a comma separator instead of a dot.

Before this patch, while running gpcheckperf utility the buffer was set by default for the underlying gpnetbenchClient utility as 32Kb. It led to problem with receiving annoying and misleading warnings about connections between hosts.
Use '--buffer-size' flag with size in kilobytes to set buffer size, which will be used at gpnetbenchClient. It is an optional parameter. The default value is 8Kb.

etc.
Why are the changes needed?
The open source database gp has made a series of updates to the gpcheckperf tool, including

Replace scp with rsync
Fixing gpcheckperf failure on -V with -f optio
gpcheckperf: Fix memory calculation function
gpcheckperf - Update a Python 3 reference of "python" to "python3"
gpcheckperf: incorporating parity changes from 6X PR
gpcheckperf: fixing string parsing in parseMultiDDResult()
gpcheckperf: add buffer size parameter
Fixing if the time command has comma in the output

Does this PR introduce any user-facing change?
no
How was this patch tested?
Contributor's Checklist
Here are some reminders and checklists before/when submitting your pull request, please check them:

[ ] Make sure your Pull Request has a clear title and commit message. You can take git-commit template as a reference.
[ ] Sign the Contributor License Agreement as prompted for your first-time contribution(One-time setup).
[ ] Learn the coding contribution guide, including our code conventions, workflow and more.
[ ] List your communication in the GitHub Issues or Discussions (if has or needed).
[ ] Document changes.
[ ] Add tests for the change
[ ] Pass make installcheck
[ ] Pass make -C src/test installcheck-cbdb-parallel
[ ] Feel free to request cloudberrydb/dev team for review and approval when your PR is ready🥳

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.0 out of 5 committers have signed the CLA.:x: Ed Espino:x: Annu149:x: piyushc01:x: jnihal:x: red1452Ed Espino seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.


Single select custom html label

Add named scope for label, and add an example in the docs
This pull request is related to https://github.com/shentao/vue-multiselect/issues/377 and  https://github.com/shentao/vue-multiselect/issues/371 and https://github.com/shentao/vue-multiselect/issues/82
I am not sure what naming methods to use so feel free to rename and edit whatever you see fit

This looks good! Thanks a lot. I will investigate it this weekend and hopefully merge it and do a release (finally).


Add script validators

Script validators should validate various things passed to them.
Hair ids, npc ids, etc etc.
Should default to true -- trust the scripts.
Will be useful for various between-version issues.

Should default to true -- trust the scripts.

Once this is changed, this issue can be closed. I don't believe I defaulted to true on #8 .


Trouble building with Ubuntu 12.04 and ROS Hydro (cxx11 support and boost issues)

I am not sure if this is actually an issue or just a result of my Linux ineptitude so please be gentle :)
Here is the progress I have made...
After some minor hiccups I was able to get a build going that made it all the way to kinect2_bridge before failing.  The error I got was
[100%] Building CXX object iai_kinect2/kinect2_bridge/CMakeFiles/kinect2_bridge.dir/src/kinect2_bridge.cpp.o
/home/hemes/ros_ws/src/iai_kinect2/kinect2_bridge/src/kinect2_bridge.cpp: In member function ‘void Kinect2Bridge::run()’:
/home/hemes/ros_ws/src/iai_kinect2/kinect2_bridge/src/kinect2_bridge.cpp:372:62: error: no matching function for call to ‘libfreenect2::SyncMultiFrameListener::waitForNewFrame(libfreenect2::FrameMap&, int)’
/home/hemes/ros_ws/src/iai_kinect2/kinect2_bridge/src/kinect2_bridge.cpp:372:62: note: candidate is:
/usr/local/include/libfreenect2/frame_listener_impl.h:54:8: note: void libfreenect2::SyncMultiFrameListener::waitForNewFrame(libfreenect2::FrameMap&)
/usr/local/include/libfreenect2/frame_listener_impl.h:54:8: note:   candidate expects 1 argument, 2 provided
make[2]: *** [iai_kinect2/kinect2_bridge/CMakeFiles/kinect2_bridge.dir/src/kinect2_bridge.cpp.o] Error 1
make[1]: *** [iai_kinect2/kinect2_bridge/CMakeFiles/kinect2_bridge.dir/all] Error 2
make: *** [all] Error 2
Invoking "make" failed

After digging some I realized that libfreenect2 was compiling without cxx11 support and thus the member function causing the error was never defined.  See libfreenect2/frame_listener_impl.h:50
...
	#ifdef LIBFREENECT2_THREADING_STDLIB
	  bool waitForNewFrame(FrameMap &frame, int milliseconds);
...

Turns out that the gcc that ships with 12.04 (Version 4.6) does not have cxx11 support so I tried upgrading to 4.8.  This solved my cxx11 support issue and I was able to build libfreenect2 again and got rid of the above error.
However, my shiny new version of gcc now causes some issues with the version of boost in 12.04 (Version <IP_ADDRESS>).  I now get a plethora of errors regarding use of deleted functions.  Some representative output is as follows:
[ 80%] Built target depth_registration
[100%] Building CXX object iai_kinect2/kinect2_bridge/CMakeFiles/kinect2_bridge.dir/src/kinect2_bridge.cpp.o
In file included from /home/hemes/ros_ws/src/iai_kinect2/kinect2_calibration/src/kinect2_calibration.cpp:37:0:
/opt/ros/hydro/include/cv_bridge/cv_bridge.h: In constructor ‘cv_bridge::CvImage::CvImage(const Header&, const string&, const cv::Mat&)’:
/opt/ros/hydro/include/cv_bridge/cv_bridge.h:77:54: error: use of deleted function ‘std_msgs::Header_<std::allocator<void> >::Header_(const std_msgs::Header_<std::allocator<void> >&)’
     : header(header), encoding(encoding), image(image)
                                                      ^
In file included from /opt/ros/hydro/include/sensor_msgs/CameraInfo.h:51:0,
                 from /home/hemes/ros_ws/src/iai_kinect2/kinect2_calibration/src/kinect2_calibration.cpp:34:
/opt/ros/hydro/include/std_msgs/Header.h:55:8: note: ‘std_msgs::Header_<std::allocator<void> >::Header_(const std_msgs::Header_<std::allocator<void> >&)’ is implicitly deleted because the default definition would be ill-formed:
 struct Header_
        ^
/opt/ros/hydro/include/std_msgs/Header.h:55:8: error: use of deleted function ‘boost::shared_ptr<std::map<std::basic_string<char>, std::basic_string<char> > >::shared_ptr(const boost::shared_ptr<std::map<std::basic_string<char>, std::basic_string<char> > >&)’
...

I saw references that this issue was addressed in revision <IP_ADDRESS>.  If I try to update boost, however, apt-get wants to remove all of ROS Hydro...  is there any hope besides building ROS from source with newer versions of gcc/boost?
I am in this predicament because ROS-Industrial is still in Hydro.  I believe they have an Indigo release scheduled for some time before April of this year but I would like to get this going before that.

Have you tried gcc 4.7? I think I used that one on Hydro.
If it is a dedicated machine that runs the Kinect 2, you could try to run Indigo.

With commit b58979eb7bade5836f9826f64fd665df35cc54d2 I added code that should make it possible to run kinect2_bridge even if libfreenect2 is not compiled with CXX11. So it should work with gcc 4.6.

I tried with gcc 4.7 and am getting the same boost errors.  I can try sans CXX11 support and see what I get but I am thinking I might also just make the jump to Indigo at this point and possibly try to use the indigo development branch for all of the ROS Industrial stuff... might be esasier that way and then I'll be totally ready for when ROS-I releases in Indigo.

OK, so I rebuilt everything using gcc 4.6 using your modified fork of libfreenect2 (still have to make the revision in issue #29 to libfreenect2 CMakeLists.txt) and the devel branch of iai_kinect2.  Everything builds but FYI the output of rosdep install -r --from-paths . is now
ERROR: the following packages/stacks could not have their rosdep keys resolved
to system dependencies:
registration_viewer: Cannot locate rosdep definition for [kinect2_bridge]
kinect2_calibration: Cannot locate rosdep definition for [kinect2_bridge]
kinect2_bridge: Cannot locate rosdep definition for [depth_registration]
Continuing to install resolvable dependencies...
#All required rosdeps installed successfully

The protonect executable runs but when I run rosrun kinect2_bridge kinect2_bridge I get a segfault with the following output:
hemes@jake:~/ros_ws/src/iai_kinect2$ rosrun kinect2_bridge kinect2_bridge
[DepthRegistration::New] OpenCL registration method not available!
[DepthRegistration::New] OpenCL registration method not available!
[OpenGLContext] failed to initialize glew for the current context!
Segmentation fault (core dumped)

Here is the backtrace:
(gdb) bt
#0  0x0000000000000000 in ?? ()
#1  0x00007ffff6264e1b in bindToUnit (unit=33984, this=<optimized out>)
    at /home/hemes/src/libfreenect2/examples/protonect/src/opengl_depth_packet_processor.cpp:216
#2  allocate (new_height=4240, new_width=352, this=0x67ede8)
    at /home/hemes/src/libfreenect2/examples/protonect/src/opengl_depth_packet_processor.cpp:228
#3  libfreenect2::OpenGLDepthPacketProcessorImpl::initialize (this=0x67ec90)
    at /home/hemes/src/libfreenect2/examples/protonect/src/opengl_depth_packet_processor.cpp:375
#4  0x00007ffff62638b9 in libfreenect2::OpenGLDepthPacketProcessor::OpenGLDepthPacketProcessor (this=0x689a80, parent_opengl_context_ptr=<optimized out>,
    debug=false)
    at /home/hemes/src/libfreenect2/examples/protonect/src/opengl_depth_packet_processor.cpp:683
#5  0x00007ffff62527cc in libfreenect2::OpenGLPacketPipeline::createDepthPacketProcessor (this=0x679ea0)
    at /home/hemes/src/libfreenect2/examples/protonect/src/packet_pipeline.cpp:108
#6  0x00007ffff62528f1 in libfreenect2::BasePacketPipeline::initialize (
    this=0x679ea0)
    at /home/hemes/src/libfreenect2/examples/protonect/src/packet_pipeline.cpp:43
#7  0x00007ffff6252aaf in libfreenect2::OpenGLPacketPipeline::OpenGLPacketPipeline (this=0x679ea0, debug=<optimized out>)
    at /home/hemes/src/libfreenect2/examples/protonect/src/packet_pipeline.cpp:101
#8  0x0000000000410551 in Kinect2Bridge::Kinect2Bridge(double, bool, int, int)
    ()
#9  0x0000000000407a13 in main ()

I am going to start working on an Indigo build but will keep my Hydro install around in case you care to see this through...
Thanks,
Brett

I am getting this same last error with my Indigo build:
hemes@cake:~/ros_ws/build/iai_kinect2$ rosrun kinect2_bridge kinect2_bridge
[DepthRegistration::New] OpenCL registration method not available!
[DepthRegistration::New] OpenCL registration method not available!
[OpenGLContext] failed to initialize glew for the current context!
Segmentation fault (core dumped)

It seems that the OpenCL depth registraion method is getting called despite me haveing Eigen.  The catkin_make output looks OK...
hemes@cake:~/ros_ws$ catkin_make -DCMAKE_BUILD_TYPE="Release"
Base path: /home/hemes/ros_ws
Source space: /home/hemes/ros_ws/src
Build space: /home/hemes/ros_ws/build
Devel space: /home/hemes/ros_ws/devel
Install space: /home/hemes/ros_ws/install
####
#### Running command: "make cmake_check_build_system" in "/home/hemes/ros_ws/build"
####
-- Using CATKIN_DEVEL_PREFIX: /home/hemes/ros_ws/devel
-- Using CMAKE_PREFIX_PATH: /home/hemes/ros_ws/devel;/opt/ros/indigo
-- This workspace overlays: /home/hemes/ros_ws/devel;/opt/ros/indigo
-- Using PYTHON_EXECUTABLE: /usr/bin/python
-- Using Debian Python package layout
-- Using empy: /usr/bin/empy
-- Using CATKIN_ENABLE_TESTING: ON
-- Call enable_testing()
-- Using CATKIN_TEST_RESULTS_DIR: /home/hemes/ros_ws/build/test_results
-- Found gtest sources under '/usr/src/gtest': gtests will be built
-- Using Python nosetests: /usr/bin/nosetests-2.7
-- catkin 0.6.11
-- BUILD_SHARED_LIBS is on
-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-- ~~  traversing 5 packages in topological order:
-- ~~  - iai_kinect2 (metapackage)
-- ~~  - depth_registration
-- ~~  - kinect2_bridge
-- ~~  - kinect2_calibration
-- ~~  - registration_viewer
-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-- +++ processing catkin metapackage: 'iai_kinect2'
-- ==> add_subdirectory(iai_kinect2/iai_kinect2)
-- +++ processing catkin package: 'depth_registration'
-- ==> add_subdirectory(iai_kinect2/depth_registration)
-- Eigen found (include: /usr/include/eigen3)
-- Could NOT find OpenCL (missing:  OPENCL_LIBRARIES OPENCL_INCLUDE_DIRS)
-- CPU based depth registration enabled
-- OpenCL based depth registration disabled
-- +++ processing catkin package: 'kinect2_bridge'
-- ==> add_subdirectory(iai_kinect2/kinect2_bridge)
-- Using these message generators: gencpp;genlisp;genpy
-- +++ processing catkin package: 'kinect2_calibration'
-- ==> add_subdirectory(iai_kinect2/kinect2_calibration)
-- Using these message generators: gencpp;genlisp;genpy
-- +++ processing catkin package: 'registration_viewer'
-- ==> add_subdirectory(iai_kinect2/registration_viewer)
-- Using these message generators: gencpp;genlisp;genpy
-- Boost version: 1.54.0
-- Found the following Boost libraries:
--   system
--   filesystem
--   thread
--   date_time
--   iostreams
--   serialization
-- checking for module 'openni-dev'
--   package 'openni-dev' not found
-- checking for module 'openni-dev'
--   package 'openni-dev' not found
-- checking for module 'openni-dev'
--   package 'openni-dev' not found
-- looking for PCL_COMMON
-- looking for PCL_KDTREE
-- looking for PCL_OCTREE
-- looking for PCL_SEARCH
-- looking for PCL_SURFACE
-- looking for PCL_GEOMETRY
-- looking for PCL_SAMPLE_CONSENSUS
-- looking for PCL_FILTERS
-- looking for PCL_FEATURES
-- looking for PCL_SEGMENTATION
-- looking for PCL_IO
-- looking for PCL_REGISTRATION
-- looking for PCL_KEYPOINTS
-- looking for PCL_RECOGNITION
-- looking for PCL_VISUALIZATION
-- looking for PCL_PEOPLE
-- looking for PCL_OUTOFCORE
-- looking for PCL_TRACKING
-- looking for PCL_APPS
-- Configuring done
-- Generating done
-- Build files have been written to: /home/hemes/ros_ws/build
####
#### Running command: "make -j12 -l12" in "/home/hemes/ros_ws/build"
####
Scanning dependencies of target kinect2_calibration
Scanning dependencies of target depth_registration
Scanning dependencies of target viewer
[ 20%] Building CXX object iai_kinect2/kinect2_calibration/CMakeFiles/kinect2_calibration.dir/src/kinect2_calibration.cpp.o
[ 40%] [ 60%] Building CXX object iai_kinect2/depth_registration/CMakeFiles/depth_registration.dir/src/depth_registration.cpp.o
Building CXX object iai_kinect2/depth_registration/CMakeFiles/depth_registration.dir/src/depth_registration_cpu.cpp.o
[ 80%] Building CXX object iai_kinect2/registration_viewer/CMakeFiles/viewer.dir/src/viewer.cpp.o
In file included from /usr/include/eigen3/Eigen/Core:328:0,
                 from /usr/include/eigen3/Eigen/Geometry:4,
                 from /home/hemes/ros_ws/src/iai_kinect2/depth_registration/src/depth_registration_cpu.h:22,
                 from /home/hemes/ros_ws/src/iai_kinect2/depth_registration/src/depth_registration_cpu.cpp:18:
/usr/include/eigen3/Eigen/src/Core/products/CoeffBasedProduct.h: In instantiation of ‘struct Eigen::internal::traits<Eigen::CoeffBasedProduct<const Eigen::Matrix<double, 4, 4>&, const Eigen::Matrix<double, 4, 1>&, 6> >’:
/usr/include/eigen3/Eigen/src/Core/DenseBase.h:41:34:   required from ‘class Eigen::DenseBase<Eigen::CoeffBasedProduct<const Eigen::Matrix<double, 4, 4>&, const Eigen::Matrix<double, 4, 1>&, 6> >’
/usr/include/eigen3/Eigen/src/Core/MatrixBase.h:48:34:   required from ‘class Eigen::MatrixBase<Eigen::CoeffBasedProduct<const Eigen::Matrix<double, 4, 4>&, const Eigen::Matrix<double, 4, 1>&, 6> >’
/usr/include/eigen3/Eigen/src/Core/products/CoeffBasedProduct.h:113:7:   required from ‘class Eigen::CoeffBasedProduct<const Eigen::Matrix<double, 4, 4>&, const Eigen::Matrix<double, 4, 1>&, 6>’
/home/hemes/ros_ws/src/iai_kinect2/depth_registration/src/depth_registration_cpu.cpp:155:39:   required from here
/usr/include/eigen3/Eigen/src/Core/products/CoeffBasedProduct.h:67:37: warning: logical ‘and’ applied to non-boolean constant [-Wlogical-op]
       CanVectorizeRhs = RhsRowMajor && (RhsFlags & PacketAccessBit)
                                     ^
In file included from /usr/include/eigen3/Eigen/Core:284:0,
                 from /usr/include/eigen3/Eigen/Geometry:4,
                 from /home/hemes/ros_ws/src/iai_kinect2/depth_registration/src/depth_registration_cpu.h:22,
                 from /home/hemes/ros_ws/src/iai_kinect2/depth_registration/src/depth_registration_cpu.cpp:18:
/usr/include/eigen3/Eigen/src/Core/Assign.h: In instantiation of ‘struct Eigen::internal::assign_traits<Eigen::Matrix<double, 4, 1>, Eigen::CoeffBasedProduct<const Eigen::Matrix<double, 4, 4>&, const Eigen::Matrix<double, 4, 1>&, 6> >’:
/usr/include/eigen3/Eigen/src/Core/Assign.h:500:111:   required from ‘Derived& Eigen::DenseBase<Derived>::lazyAssign(const Eigen::DenseBase<OtherDerived>&) [with OtherDerived = Eigen::CoeffBasedProduct<const Eigen::Matrix<double, 4, 4>&, const Eigen::Matrix<double, 4, 1>&, 6>; Derived = Eigen::Matrix<double, 4, 1>]’
/usr/include/eigen3/Eigen/src/Core/PlainObjectBase.h:411:46:   required from ‘Derived& Eigen::PlainObjectBase<Derived>::lazyAssign(const Eigen::DenseBase<OtherDerived>&) [with OtherDerived = Eigen::CoeffBasedProduct<const Eigen::Matrix<double, 4, 4>&, const Eigen::Matrix<double, 4, 1>&, 6>; Derived = Eigen::Matrix<double, 4, 1>]’
/usr/include/eigen3/Eigen/src/Core/products/CoeffBasedProduct.h:196:7:   required from ‘Eigen::CoeffBasedProduct<Lhs, Rhs, NestingFlags>::operator const PlainObject&() const [with LhsNested = const Eigen::Matrix<double, 4, 4>&; RhsNested = const Eigen::Matrix<double, 4, 1>&; int NestingFlags = 6; Eigen::CoeffBasedProduct<Lhs, Rhs, NestingFlags>::PlainObject = Eigen::Matrix<double, 4, 1>]’
/home/hemes/ros_ws/src/iai_kinect2/depth_registration/src/depth_registration_cpu.cpp:155:39:   required from here
/usr/include/eigen3/Eigen/src/Core/Assign.h:49:19: warning: logical ‘and’ applied to non-boolean constant [-Wlogical-op]
                   && (int(Derived::Flags) & int(OtherDerived::Flags) & ActualPacketAccessBit),
                   ^
/usr/include/eigen3/Eigen/src/Core/Assign.h: In instantiation of ‘struct Eigen::internal::assign_traits<Eigen::Matrix<double, 4, 1>, Eigen::Matrix<double, 4, 1> >’:
/usr/include/eigen3/Eigen/src/Core/Assign.h:500:111:   required from ‘Derived& Eigen::DenseBase<Derived>::lazyAssign(const Eigen::DenseBase<OtherDerived>&) [with OtherDerived = Eigen::Matrix<double, 4, 1>; Derived = Eigen::Matrix<double, 4, 1>]’
/usr/include/eigen3/Eigen/src/Core/PlainObjectBase.h:411:46:   required from ‘Derived& Eigen::PlainObjectBase<Derived>::lazyAssign(const Eigen::DenseBase<OtherDerived>&) [with OtherDerived = Eigen::Matrix<double, 4, 1>; Derived = Eigen::Matrix<double, 4, 1>]’
/usr/include/eigen3/Eigen/src/Core/Assign.h:520:123:   required from ‘static Derived& Eigen::internal::assign_selector<Derived, OtherDerived, false, false>::run(Derived&, const OtherDerived&) [with Derived = Eigen::Matrix<double, 4, 1>; OtherDerived = Eigen::Matrix<double, 4, 1>]’
/usr/include/eigen3/Eigen/src/Core/PlainObjectBase.h:618:105:   required from ‘Derived& Eigen::PlainObjectBase<Derived>::_set_noalias(const Eigen::DenseBase<OtherDerived>&) [with OtherDerived = Eigen::Matrix<double, 4, 1>; Derived = Eigen::Matrix<double, 4, 1>]’
/usr/include/eigen3/Eigen/src/Core/Matrix.h:288:31:   required from ‘Eigen::Matrix<_Scalar, _Rows, _Cols, _Options, _MaxRows, _MaxCols>::Matrix(const Eigen::Matrix<_Scalar, _Rows, _Cols, _Options, _MaxRows, _MaxCols>&) [with _Scalar = double; int _Rows = 4; int _Cols = 1; int _Options = 0; int _MaxRows = 4; int _MaxCols = 1]’
/home/hemes/ros_ws/src/iai_kinect2/depth_registration/src/depth_registration_cpu.cpp:155:39:   required from here
/usr/include/eigen3/Eigen/src/Core/Assign.h:49:19: warning: logical ‘and’ applied to non-boolean constant [-Wlogical-op]
Linking CXX shared library /home/hemes/ros_ws/devel/lib/libdepth_registration.so
[ 80%] Built target depth_registration
Scanning dependencies of target kinect2_bridge
[100%] Building CXX object iai_kinect2/kinect2_bridge/CMakeFiles/kinect2_bridge.dir/src/kinect2_bridge.cpp.o
In file included from /usr/include/pcl-1.7/pcl/visualization/pcl_visualizer.h:52:0,
                 from /usr/include/pcl-1.7/pcl/visualization/cloud_viewer.h:39,
                 from /home/hemes/ros_ws/src/iai_kinect2/registration_viewer/src/viewer.cpp:32:
/usr/include/pcl-1.7/pcl/visualization/area_picking_event.h: In member function ‘bool pcl::visualization::AreaPickingEvent::getPointsIndices(std::vector<int>&) const’:
/usr/include/pcl-1.7/pcl/visualization/area_picking_event.h:64:30: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
           if (nb_points_ == -1)
                              ^
Linking CXX executable /home/hemes/ros_ws/devel/lib/kinect2_bridge/kinect2_bridge
Linking CXX executable /home/hemes/ros_ws/devel/lib/kinect2_calibration/kinect2_calibration
[100%] Built target kinect2_bridge
[100%] Built target kinect2_calibration
Linking CXX executable /home/hemes/ros_ws/devel/lib/registration_viewer/viewer
[100%] Built target viewer

I am out of time for today, otherwise I would have dug a bit more.  Hope this at least gives you an idea of what is going wrong.

The segfault comes from the OpenGL processor, but in Protonect it is working. I will try it on my system later.
And the other problem is that the OpenCL registration is used even if it is not enabled in depth_registration. I will look at this too.

I found out why glewInit() is not initializing. glfwInit() has to be called before glewInit(). This is done in Protonect but not in libfreenect2.
The depth_registration problem should be fixed with 999b30cce61d1ed753228bee1b5ac75b61f1fc92.
@christiankerl: should is be like this, that the caller has to execute glfwInit() before using libfreenect2, or should the opengl_depth_packet_processor / pipeline instantiation be responsible for this?

Indeed 999b30cce61d1ed753228bee1b5ac75b61f1fc92 fixes the depth_registration problem.
As for the glfwInit issue, I went ahead and added a glfwInit() call in kinect2_bridge.cpp (right before the first libfreenect2 calls) just to try and test the original Hydro issues I was having and everything ended up working.  I am able to run the bridge, viewer, and calibration in both Hydro and Indigo.
So in conclusion it seems b58979eb7bade5836f9826f64fd665df35cc54d2 solves the original  12.04+Hydro / no CXX11 issue when the distro-default gcc 4.6 is used.
Feel free to close or rename this issue however you see fit depending on how you end up handling the remaining glfwInit() issue.
Thanks so much for your work; this package will be very useful for my purposes.
Brett

In the devel branch in commit efd26792504ffe88aae8f7f6d53698e8b4af5a8f I added glfwInit() to kinect2_bridge.


adding license tests to ruby structure tests

License tests are automated tests to check that all packages installed in the image conform to Google's open source licensing stardards. Check out https://github.com/GoogleCloudPlatform/runtimes-common/blob/master/structure_tests/licenses_test.go (and the corresponding README) for more details.

hmm. maybe we should also have a structure test just verifying this file is here in the first place?

If you're talking about /nodejs/LICENSE, then the test will fail if the file isn't there.

also, wait a minute. this is testing the /nodejs/LICENSE in the ruby image?

I was wondering the same thing. The Ruby base image downloads and installs NodeJS. But why are we checking LICENSE when the the NodeJS is installed from nodejs.org?

The entire point of the files option in the license tests is to check anything that is installed, but not through apt. It's to keep everything consistent, and to remind people to check stuff that's wget'ted or curled in the Dockerfile.


Queries from charts for async databases are being executed in the server instances, not the workers

Charts for databases with Asynchronous query execution aren't being executed in the worker nodes as expected.
How to reproduce the bug

Add a datasource in a setup with Async Queries configured via Celery;
Enable the following options in Advanced > Other > Engine Parameters:

{"pool_pre_ping":true,"echo_pool":"debug","echo":true}

Enable the option Advanced > Performance > Asynchronous query execution;
Add a sample chart in a dashboard.
Check the logs of both the webserver and the worker instances;

Expected results
I expected the queries to be executed in the worker instance. For instance, when running through SQL Lab that's what happens. The unexpected behavior occurs when using dashboards.
Actual results
We see the connection being established in the webserver instance, not the worker (<IP_ADDRESS> stands for the IP of my database with data to be charted, not the Superset database)
pod-app-hostname 2022-11-03 19:19:30,871 DEBUG sqlalchemy.pool.impl.NullPool Created new connection <_mysql.connection open to '<IP_ADDRESS>' at 0x55beb238bc50>
pod-app-hostname DEBUG:sqlalchemy.pool.impl.NullPool:Created new connection <_mysql.connection open to '<IP_ADDRESS>' at 0x55beb238bc50>
pod-app-hostname INFO:sqlalchemy.engine.base.Engine:SHOW VARIABLES LIKE 'sql_mode'
pod-app-hostname 2022-11-03 19:19:30,873 INFO sqlalchemy.engine.base.Engine SHOW VARIABLES LIKE 'sql_mode'
pod-app-hostname 2022-11-03 19:19:30,873 INFO sqlalchemy.engine.base.Engine ()
pod-app-hostname INFO:sqlalchemy.engine.base.Engine:()
pod-app-hostname 2022-11-03 19:19:30,877 INFO sqlalchemy.engine.base.Engine SHOW VARIABLES LIKE 'lower_case_table_names'
pod-app-hostname 2022-11-03 19:19:30,878 INFO sqlalchemy.engine.base.Engine ()
pod-app-hostname INFO:sqlalchemy.engine.base.Engine:SHOW VARIABLES LIKE 'lower_case_table_names'
pod-app-hostname INFO:sqlalchemy.engine.base.Engine:()
pod-app-hostname INFO:sqlalchemy.engine.base.Engine:SELECT DATABASE()
pod-app-hostname 2022-11-03 19:19:30,881 INFO sqlalchemy.engine.base.Engine SELECT DATABASE()
pod-app-hostname INFO:sqlalchemy.engine.base.Engine:()
pod-app-hostname 2022-11-03 19:19:30,882 INFO sqlalchemy.engine.base.Engine ()
pod-app-hostname 2022-11-03 19:19:30,884 INFO sqlalchemy.engine.base.Engine show collation where Charset = 'utf8mb4' and Collation = 'utf8mb4_bin'
pod-app-hostname INFO:sqlalchemy.engine.base.Engine:show collation where Charset = 'utf8mb4' and Collation = 'utf8mb4_bin'
pod-app-hostname 2022-11-03 19:19:30,885 INFO sqlalchemy.engine.base.Engine ()
pod-app-hostname INFO:sqlalchemy.engine.base.Engine:()
pod-app-hostname 2022-11-03 19:19:30,887 INFO sqlalchemy.engine.base.Engine SELECT CAST('test plain returns' AS CHAR(60)) AS anon_1
pod-app-hostname INFO:sqlalchemy.engine.base.Engine:SELECT CAST('test plain returns' AS CHAR(60)) AS anon_1
pod-app-hostname 2022-11-03 19:19:30,887 INFO sqlalchemy.engine.base.Engine ()
pod-app-hostname INFO:sqlalchemy.engine.base.Engine:()
pod-app-hostname 2022-11-03 19:19:30,889 INFO sqlalchemy.engine.base.Engine SELECT CAST('test unicode returns' AS CHAR(60)) AS anon_1
pod-app-hostname INFO:sqlalchemy.engine.base.Engine:SELECT CAST('test unicode returns' AS CHAR(60)) AS anon_1
pod-app-hostname INFO:sqlalchemy.engine.base.Engine:()
pod-app-hostname 2022-11-03 19:19:30,889 INFO sqlalchemy.engine.base.Engine ()
pod-app-hostname INFO:sqlalchemy.engine.base.Engine:SELECT CAST('test collated returns' AS CHAR CHARACTER SET utf8mb4) COLLATE utf8mb4_bin AS anon_1
pod-app-hostname INFO:sqlalchemy.engine.base.Engine:()
pod-app-hostname 2022-11-03 19:19:30,891 INFO sqlalchemy.engine.base.Engine SELECT CAST('test collated returns' AS CHAR CHARACTER SET utf8mb4) COLLATE utf8mb4_bin AS anon_1
pod-app-hostname 2022-11-03 19:19:30,891 INFO sqlalchemy.engine.base.Engine ()
pod-app-hostname DEBUG:sqlalchemy.pool.impl.NullPool:Connection <_mysql.connection open to '<IP_ADDRESS>' at 0x55beb238bc50> checked out from pool
pod-app-hostname 2022-11-03 19:19:30,895 DEBUG sqlalchemy.pool.impl.NullPool Connection <_mysql.connection open to '<IP_ADDRESS>' at 0x55beb238bc50> checked out from pool
pod-app-hostname DEBUG:sqlalchemy.pool.impl.NullPool:Pool pre-ping on connection <_mysql.connection open to '<IP_ADDRESS>' at 0x55beb238bc50>
pod-app-hostname 2022-11-03 19:19:30,895 DEBUG sqlalchemy.pool.impl.NullPool Pool pre-ping on connection <_mysql.connection open to '<IP_ADDRESS>' at 0x55beb238bc50>
pod-app-hostname 2022-11-03 19:19:30,901 DEBUG sqlalchemy.pool.impl.NullPool Connection <_mysql.connection open to '<IP_ADDRESS>' at 0x55beb238bc50> being returned to pool
pod-app-hostname DEBUG:sqlalchemy.pool.impl.NullPool:Connection <_mysql.connection open to '<IP_ADDRESS>' at 0x55beb238bc50> being returned to pool
pod-app-hostname 2022-11-03 19:19:30,902 DEBUG sqlalchemy.pool.impl.NullPool Connection <_mysql.connection open to '<IP_ADDRESS>' at 0x55beb238bc50> rollback-on-return
pod-app-hostname DEBUG:sqlalchemy.pool.impl.NullPool:Connection <_mysql.connection open to '<IP_ADDRESS>' at 0x55beb238bc50> rollback-on-return
pod-app-hostname 2022-11-03 19:19:30,903 DEBUG sqlalchemy.pool.impl.NullPool Closing connection <_mysql.connection open to '<IP_ADDRESS>' at 0x55beb238bc50>
pod-app-hostname DEBUG:sqlalchemy.pool.impl.NullPool:Closing connection <_mysql.connection open to '<IP_ADDRESS>' at 0x55beb238bc50>
Screenshots
If applicable, add screenshots to help explain your problem.
Environment
(please complete the following information):

browser type and version: Google Chrome 107.0.5304.87
superset version: 2.0.0
python version: Python 3.8.12
node.js version: Not installed in the Docker instance
any feature flags active:

FEATURE_FLAGS = {
    'ENABLE_TEMPLATE_PROCESSING': True,
}

Checklist
Make sure to follow these steps before submitting your issue - thank you!

[ x ] I have checked the superset logs for python stacktraces and included it here as text if there are any.
[ x ] I have reproduced the issue with at least the latest released version of superset.
[ x ] I have checked the issue tracker for the same issue and I haven't found one similar.

Additional context
I'm using an image derived from apache/superset:2.0.0 and tried to connect both to Hive and to MySQL.

Hi, we are facing the same issue. @diogenes1oliveira  did you find any fix or workaround?

Enabling AQE for a database only applies to SQL Lab - for charts you'll need to enable the GLOBAL_ASYNC_QUERIES feature flag and add corresponding config (see docs here: https://github.com/apache/superset/blob/master/CONTRIBUTING.md#async-chart-queries)

Hi @dingbat, is there any doc or guide which shows how to use GLOBAL_ASYNC_QUERIES? I tried setting this to true, but, I started getting getting jwt token error. After setting GLOBAL_ASYNC_QUERIES_JWT_SECRET to some random 32 bit number, I am getting these types of logs continuously-
<IP_ADDRESS> - - [15/Feb/2024:06:59:18 +0000] "GET /api/v1/async_event/ HTTP/1.1" 403 24 "http://IP/dashboard/45/?native_filters_key=RkL3s6EjCpRaxau_afgK1VBnmRh5Qx1pfj8z4UlRXoDyVCKAq6IWEhlVxvXEbzu5" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/<IP_ADDRESS> Safari/537.36"

Hi @shauryagoel, did you fix the issue related to setting up the GLOBAL_ASYNC_QUERIES flag in superset_config.py file for charts asynchronous query executions task in Celery?

Hi @shauryagoel, did you fix the issue related to setting up the GLOBAL_ASYNC_QUERIES flag in superset_config.py file for charts asynchronous query executions task in Celery? If yes, can I know the solution?

You have to set same value for GLOBAL_ASYNC_QUERIES_REDIS_CONFIG in superset_config.py as in config.json that is used by superset-websocket
For example:
GLOBAL_ASYNC_QUERIES_REDIS_CONFIG = {
    "port": 6379,
    "host": "redis",
    "db": 1,
    "ssl": False
}

Also wondering if there is a guide on using GLOBAL_ASYNC_QUERIES and monitoring/tuning async query performance. It looks like the docs at https://github.com/apache/superset/blob/master/CONTRIBUTING.md#async-chart-queries were moved.

@Tim-Hodge the GAQ feature was effectively in beta, and is about to go under a revamp. See SIP-143 for more details/discussion/voting (CC @villebro re: revisiting docs as part of the SIP... happy to help here if needed).


[WIP]workflow: fix incorrect files number for optimizer test

We use nginx:1.23.3 to test the optimizer, but sometimes the number of files accessed is not equal to 44. Normally, the optimizer works fine if the number of files accessed is greater than 0.

Codecov Report
Patch and project coverage have no change.

Comparison is base (397914d) 29.65% compared to head (a011473) 29.65%.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #446   +/-   ##
=======================================
  Coverage   29.65%   29.65%
=======================================
  Files          39       39
  Lines        4080     4080
=======================================
  Hits         1210     1210
  Misses       2713     2713
  Partials      157      157

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

This PR could solve the above problem. But I think the more sensible solution is to check why the number of files accessed is incorrect.
Just wait.

The reason why the test sometimes fails is that nginx generates a temporary file that may not be captured by the fanotify server.
/etc/nginx/conf.d/sed5PCVAq

It replaced by a simple workload echo hello.


Deadlock on Mix_PlayChannel

Today I encountered a strange SDL Mixer issue on my Pixel 6 Pro Android device.
If an alarm notification occurs while my SDL indie game app is running, the SDL Mixer system gets irreparably harmed.
Post-notification, calls to the SDL Mixer system stall waiting to obtain a mutex lock.
I tried rearranging interactions with SDL Mixer in a few ways, but nothing works. It seems be out of my hands as the app developer.

What are the callstacks of the other threads in the application? In particular, it would be helpful to know what else already has the audio mutex locked.

@ebarlas
can you check, if it's using aaudio or opensles ?
(put some trace in src/audio/aaudio or src/audio/opensles )
If it's aaudio, maybe this is a duplicate of https://github.com/libsdl-org/SDL/issues/4985 ?

@ebarlas,
Also, if you can reproduce it, try to force a back-end (aaudio or opensles) by forcing them in src/audio/SDL_audio.c
(&aaudio_bootstrap, or &openslES_bootstrap,)
to see if this is a back-end dependent issue

Here's a more complete view of the two relevant threads. After the alarm overlay, with sound and vibration, a deadlock ensues. I paused the debugger to freeze program execution at this point.
Also, I forgot to note that I'm using SDL_Mixer 2.6.3.

I'm going to try toggling the backends next, @1bsyl .

When I switch to openslES the problem is non-existent!
All of the Android defaults have worked well across SDL libraries, so I haven't toggled them. However, do you recommend that I disable the AAudio driver?

That seems odd that the DRMP3 mixing process isn't releasing the lock. Can you verify that it actually completes in this case? Maybe the lock is taken somewhere else and then not released properly?

I had the same thought. Indeterminate audio processing would be strange. I'll try to annotate the code more to gather more timing info. But from everything I've gathered so far, that application thread is completely stalled there.

I don't exactly remember why, because of previous issue, I use this patch for aaudio:
comment out DetectBrokenState + re-open
diff --git a/src/audio/aaudio/SDL_aaudio.c b/src/audio/aaudio/SDL_aaudio.c
index 76fe4e710..68f8c402d 100644
--- a/src/audio/aaudio/SDL_aaudio.c
+++ b/src/audio/aaudio/SDL_aaudio.c
@@ -212,6 +212,16 @@ static void aaudio_PlayDevice(SDL_AudioDevice *_this)
     aaudio_result_t res;
     int64_t timeoutNanoseconds = 1 * 1000 * 1000; /* 8 ms */
     res = ctx.AAudioStream_write(private->stream, private->mixbuf, private->mixlen / private->frame_size, timeoutNanoseconds);
+    if (res < 0) {
+        if (res == AAUDIO_ERROR_DISCONNECTED) {
+            ctx.AAudioStream_close(private->stream);
+            ctx.AAudioStreamBuilder_openStream(ctx.builder, &private->stream); // reopen stream (mem leak probably)
+            ctx.AAudioStream_requestStart(private->stream); // start new stream
+            res = ctx.AAudioStream_write(private->stream,  private->mixbuf, private->mixlen / private->frame_size, timeoutNanoseconds);
+            SDL_Log("SDL AAudio play: DISCONNECTED / RECONNECTED");
+        }
+    }
+
     if (res < 0) {
         LOGI("%s : %s", __func__, ctx.AAudio_convertResultToText(res));
     } else {
@@ -475,6 +485,7 @@ void aaudio_ResumeDevices(void)
 */
 SDL_bool aaudio_DetectBrokenPlayState(void)
 {
+#if 0
     int i;

     /* AAUDIO driver is not used */
@@ -515,7 +526,7 @@ SDL_bool aaudio_DetectBrokenPlayState(void)

         (void) captureDevice;
     }
-
+#endif
     return SDL_FALSE;
 }

I'm using the latest SDL 2 release: 2.6.3.
Yes, main loop is blocked as shown on SDL mixer API call. I haven't instrumented the code yet, so it's hard to make specific claims about timing on the main and audio threads.
I'll add that shortly and report back.

I reproduced the issue with a bit of extra logging in mixer.c to paint a clearer picture.
I added the following snippets at the beginning and end of void mix_channels(...) called by the internal audio processing thread:
SDL_Log("mix_channels enter thread=%lu, ticks=%d", SDL_ThreadID(), SDL_GetTicks());
...
SDL_Log("mix_channels exit thread=%lu, ticks=%d", SDL_ThreadID(), SDL_GetTicks());

and the following at the beginning and end of int Mix_PlayChannel(...) called by the application:
SDL_Log("Mix_PlayChannel enter, thread=%lul, ticks=%d", SDL_ThreadID(), SDL_GetTicks());
...
SDL_Log("Mix_PlayChannel exit, thread=%lu, ticks=%d", SDL_ThreadID(), SDL_GetTicks());

The results are striking.
The Android alarm fired at about SDL-ticks 34000. Calls to mix_channels in the audio processing thread immediately increase about 100x. Prior to SDL-ticks 34000, mix_channels was called about 40x per second. Subsequently, it is called about 4000x per second.
So the audio processing workload is literally starving the application thread of access.
$ egrep "ticks=[0-9]{5}" sdl_log2.txt | grep "ticks=33" | wc -l
      44

$ egrep "ticks=[0-9]{5}" sdl_log2.txt | grep "ticks=36" | wc -l
    4046

sdl_log2.txt

mix_channels() is callback of SDL_OpenAudioDevice, set from SDL_mixer/src/mixer.c
It's called from SDL/src/audio/SDL_audio.c ( line ~592), so this seems to loop to much.
In normal case, the delay should come from end to the function, 3rd if-block. (unless you need to convert sample, that would use the 1 if-block).
is this still happening when the bug occurs ?
 642         } else { /* writing directly to the device. */
 643             /* queue this buffer and wait for it to finish playing. */
 644             current_audio.impl.PlayDevice(device);
 645             current_audio.impl.WaitDevice(device);

WaitDevice, has no implementation (but also for other backend),
so what about aaudio_PlayDevice() ?
Is there an error triggered inside ?  (you need to enable the LOGI define in src/audio/aaudio/SDL_aaudio.c)
I haven't been able to reproduce the issue, by using a timer alarm. this still loops normally.
because the error case should be handle with a Delay.
there is a underflow block also, that could give informations


Fix bug: use scoped GoogleCredentials

I tried to write a test for this, but couldn't think of a straight forward way.

Codecov Report

Merging #1160 into master will increase coverage by 0.06%.
The diff coverage is 0%.

@@             Coverage Diff              @@
##             master    #1160      +/-   ##
============================================
+ Coverage     52.06%   52.13%   +0.06%
  Complexity     1588     1588
============================================
  Files           268      268
  Lines         12947    12947
  Branches       1501     1501
============================================
+ Hits           6741     6750       +9
+ Misses         5696     5687       -9
  Partials        510      510


feat: support setting replicas and resource limits of Gateway

Description:

Testing done:

Affected area:

Functional Area

New Functionality
[ ]

CI System
[ ]

CLI Tool
[ ]

Certificate Management
[ ]

Control Plane
[ ]

Demo
[ ]

Documentation
[ ]

Egress
[ ]

Ingress
[ ]

Install
[ ]

Networking
[ ]

Observability
[ ]

Performance
[ ]

SMI Policy
[ ]

Security
[ ]

Sidecar Injection
[ ]

Tests
[ ]

Upgrade
[ ]

Other
[ ]

Please answer the following questions with yes/no.

Does this change contain code from or inspired by another project?

Did you notify the maintainers and provide attribution?

Is this a breaking change?

Has documentation corresponding to this change been updated in the fsm-docs repo (if applicable)?

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (7230d46) 46.84% compared to head (4092e3a) 46.84%.

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@              Coverage Diff              @@
##           release/v1.2     #179   +/-   ##
=============================================
  Coverage         46.84%   46.84%
=============================================
  Files               199      199
  Lines             15232    15232
=============================================
  Hits               7136     7136
  Misses             8050     8050
  Partials             46       46

Flag
Coverage Δ

unittests
46.84% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


[patch] Hotfix project info

Patch the docs link, institute name, and copyright info

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

0 of 0   changed or added relevant lines in 0 files are covered.
2 unchanged lines in 1 file lost coverage.
Overall coverage decreased (-0.2%) to 94.688%

Files with Coverage Reduction
New Missed Lines
%

logger.py
2
92.0%

Totals

Change from base Build<PHONE_NUMBER>:
-0.2%

Covered Lines:
410

Relevant Lines:
433

💛  - Coveralls


E2E:Fix failing Cypress tests on SAML and mark as unread

Summary
Fix failing Cypress tests on SAML and mark as unread
Ticket Link
none, failing on Cypress daily

/update-branch

/update-branch

Will try to auto merge this PR once all tests and checks are passing. This might take up to an hour.


Stuck at "waiting for MoveJ action server..."

🐳   user/ros2_foxy_desktop/~/Projects/dev_ws $  ros2 run ros2_execution ros2_execution.py --ros-args -p PROGRAM_FILENAME:="cubePP" -p ROBOT_MODEL:="irb120" -p EE_MODEL:="schunk"
--- Cranfield University ---
(c) IFRA Group
ros2_RobotSimulation --> SEQUENCE EXECUTION
Python script -> ros2_execution.py
[INFO] [1709237997.941264838] [ros2_program_param]: PROGRAM_FILENAME ROS2 Parameter received: cubePP
[INFO] [1709237997.951131662] [ros2_robot_param]: ROBOT_MODEL ROS2 Parameter received: irb120
[INFO] [1709237997.960076382] [ros2_ee_param]: EE_MODEL ROS2 Parameter received: schunk
Waiting for MoveJ action server to be available...
@IFRA-Cranfield @seemalasif @MikelBueno @benkyd

Hi, have you found a solution for this? I'm stuck at the same thing..

Hi, which launch file are you using? For me it works if I use the "_interface" launch files


 It has since diverged significantly through many refactors.
One thing that has never worked is the scalar upgrade verb when using GitHub as the source. The reason is that we package our releases differently than VFS for Git does, so the upgrader fails to find the Git installer and Scalar installer (instead, we have two .zip files, one for each platform). Since we are moving to another system for the typical upgrade pattern, we should just delete the GitHub upgrader. The scalar upgrade verb will remain for the NuGet feed, when configured. This resolves #351.
An issue was found in VFS for Git where the service would restart constantly when checking for an upgrade. The root cause was a problematic Environment.Exit() call. See microsoft/vfsforgit#1668 for an equivalent change.
Another annoying thing is the verbose output during scalar clone that repeats the given options. In the case of a non-GVFS protocol clone, some of these just don't make sense. Delete this output for clarity.
Finally, scalar clone failed miserably when using a non-standard URL such as file://. Fix that by doing some URL scanning in advance. If the URL does not start with http:// or https://, we will not even try the GVFS protocol and will move on to a native Git clone. We also avoid any protocol prefix (X://) that is not http://, https:// or ssh://. For SSH, the protocol prefix is not required, so we do not require a protocol prefix. Resolves #378.
Since this is a grab-bag of things, please review commit-by-commit. For the GitHub upgrader removal, consider ignoring whitespace changes.

I'm not sure I'm understand the subtleties of the ProductUpgrader commit and GH vs NuGet changes, but everything else looks good.

I deleted the GitHubUpgrader which never worked for Scalar. It may have lost some test coverage because the GitHubUpgrader was used as a mock for the orchestrator. I did manually verify that scalar upgrade will show an error message (but fail gracefully) if we haven't configured a private NuGet feed.


[tool_crash] _TypeError: (#0      _gitCommitDate (package:flutter_tools/src/version.dart:387:4))

Command
flutter --version

Steps to Reproduce

...
...
...

Logs
_TypeError: (#0      _gitCommitDate (package:flutter_tools/src/version.dart:387:4))
#0      _gitCommitDate (package:flutter_tools/src/version.dart:387:4)
#1      _FlutterVersionGit.frameworkCommitDate (package:flutter_tools/src/version.dart:495:37)
#2      FlutterVersion.toString (package:flutter_tools/src/version.dart:225:99)
#3      FlutterCommandRunner.runCommand.<anonymous closure> (package:flutter_tools/src/runner/flutter_command_runner.dart:347:30)
<asynchronous suspension>
#4      AppContext.run.<anonymous closure> (package:flutter_tools/src/base/context.dart:153:19)
<asynchronous suspension>
#5      FlutterCommandRunner.runCommand (package:flutter_tools/src/runner/flutter_command_runner.dart:295:5)
<asynchronous suspension>
#6      run.<anonymous closure>.<anonymous closure> (package:flutter_tools/runner.dart:119:9)
<asynchronous suspension>
#7      AppContext.run.<anonymous closure> (package:flutter_tools/src/base/context.dart:153:19)
<asynchronous suspension>
#8      main (package:flutter_tools/executable.dart:90:3)
<asynchronous suspension>

[☠] Flutter (the doctor check crashed)
    ✗ Due to an error, the doctor check did not complete. If the error message below is not helpful, please let us know about this issue at https://github.com/flutter/flutter/issues.
    ✗ Null check operator used on a null value
    • #0      FlutterValidator._getFlutterUpstreamMessage (package:flutter_tools/src/doctor.dart:701:86)
      #1      FlutterValidator.validate (package:flutter_tools/src/doctor.dart:568:20)
      #2      Doctor.startValidatorTasks.<anonymous closure> (package:flutter_tools/src/doctor.dart:272:72)
      #3      asyncGuard.<anonymous closure> (package:flutter_tools/src/base/async_guard.dart:111:32)
      #4      _rootRun (dart:async/zone.dart:1399:13)
      #5      _CustomZone.run (dart:async/zone.dart:1301:19)
      #6      _runZoned (dart:async/zone.dart:1826:10)
      #7      runZonedGuarded (dart:async/zone.dart:1814:12)
      #8      runZoned (dart:async/zone.dart:1760:12)
      #9      asyncGuard (package:flutter_tools/src/base/async_guard.dart:109:3)
      #10     Doctor.startValidatorTasks (package:flutter_tools/src/doctor.dart:264:9)
      #11     DoctorText._validatorTasks (package:flutter_tools/src/doctor.dart:784:60)
      #12     DoctorText._validatorTasks (package:flutter_tools/src/doctor.dart)
      #13     DoctorText._runDiagnosis (package:flutter_tools/src/doctor.dart:788:53)
      #14     DoctorText.text (package:flutter_tools/src/doctor.dart:780:36)
      #15     DoctorText.text (package:flutter_tools/src/doctor.dart)
      #16     _createLocalCrashReport (package:flutter_tools/runner.dart:262:51)
      #17     _handleToolError (package:flutter_tools/runner.dart:224:31)
      <asynchronous suspension>
      #18     AppContext.run.<anonymous closure> (package:flutter_tools/src/base/context.dart:153:19)
      <asynchronous suspension>
      #19     main (package:flutter_tools/executable.dart:90:3)
      <asynchronous suspension>

[✗] Android toolchain - develop for Android devices
    ✗ Unable to locate Android SDK.
      Install Android Studio from: https://developer.android.com/studio/index.html
      On first launch it will assist you in installing the Android SDK components.
      (or visit https://flutter.dev/docs/get-started/install/linux#android-setup for detailed instructions).
      If the Android SDK has been installed to a custom location, please use
      `flutter config --android-sdk` to update to that location.

[✗] Chrome - develop for the web (Cannot find Chrome executable at google-chrome)
    ! Cannot find Chrome. Try setting CHROME_EXECUTABLE to a Chrome executable.

[✓] Linux toolchain - develop for Linux desktop
    • clang version 16.0.6
    • cmake version 3.28.3
    • ninja version 1.11.1
    • pkg-config version 2.1.0

[!] Android Studio (not installed)
    • Android Studio not found; download from https://developer.android.com/studio/index.html
      (or visit https://flutter.dev/docs/get-started/install/linux#android-setup for detailed instructions).

[✓] IntelliJ IDEA Ultimate Edition (version 2019.3)
    • IntelliJ at /home/thomas/idea-IU-193.5662.53
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart

[✓] Connected device (1 available)
    • Linux (desktop) • linux • linux-x64 • Manjaro Linux 6.7.7-1-MANJARO

[✓] Network resources
    • All expected network resources are available.

! Doctor found issues in 4 categories.

Flutter Application Metadata
No pubspec in working directory.

Hi @tha-uep, do you have git installed on your system?

Of course

Hi @tha-uep, what version of git do you have installed on your system?

Hi @danagbemava-nc, my git version is 2.44.0 actually

Without knowing what version of Flutter this stack trace came from, it's not easy to troubleshoot what went could have went wrong here. If you happen to recall what version of Flutter you were using when you ran into this bug, feel free to let me know and I can reopen this.


Doc Outline for Developers / Partners / Community Sprinters

What sort of resources and information would partners need for using the new Advanced Mapping feature in Batch Data Import?

Purpose of this outline
We are trying to structure the documentation we are refreshing in a way that is useful for developers , partners, techies to understand the considerations they need to make when integrating their own products, custom implementations, community projects.
A similar sprint project tried to address a similar issue for data import, here it is for reference:
https://drive.google.com/open?id=1H-5OXdgl-sZRQ5C4pZyHiOFcJvAhaG9H .
This project highlighted some concerns the developer/partner and customer community has about integrating with NPSP and what's missing in the current product and its documentation.
Gaps in the current documentation
There are currently two docs in the power of us hub that address the NPSP data import tool, added to files: NPSP Advanced User's Guide to Importing Data and NPSP Administrator's Guide to Importing Donor Data.
Those docs are great for documenting the basic the use cases for importing data into NPSP and creating batch data import templates but they fall short of addressing the automation of imports via API integration, which could prove beneficial for partners, developer and integration enthusiasts. There is a section in the advanced guide called "Additional Automation Using Custom Apex Code" but

it suggests that the only way of creating automation is via Apex
"This feature requires Apex coding and should only be used by a developer"
there is no mention of an API that can be used by a system integrator to programmatically start the batch process.

This content currently lives in a section called import data, there is no section in power of hus hub that explicitly mentions integration in the title and includes the topic of programmatic integration using the Import tool (there is one integration section but it's called integration and general administration and it seems to lump together topics that have little to do with APIs and programmatic integration)
Suggestions for improvement

Create an explicit section for integration in power of us hub
Create content for developers that shows them what to do to use NPSP Data Import tool, an example from the previous sprint Sprint Project: Data Capture Standards for NPSP (Tutorial - Test Data Import Technology)
Update documentation to include create an infographic to depict how the NPSP Data Import Tool works.


Allow finding intermediate solutions with Minizinc

I am trying to find all intermediate solutions for an optimization problem. CPM_minizinc does not allow this, as solve() will only return the last solution, and solveAll() does not work for optimization problems. I did not encounter this problem for the CPM_ortools model, where I can provide a callback to solve() to store all intermediate solutions myself

For some additional context, I pass intermediate_solutions=True to the solve() function to get multiple solutions. I am then met with the Warning: multiple solutions found, only returning last one log

The easiest solution would be to simply have an argument change this behavior of solve(). I am willing to make a PR for this if this sounds good. Let me know

For those looking for a workaround, I extended the CPM_minizinc class to store objective values and timestamps the following way:
class MinizincSolver(CPM_minizinc):
    """
    Wrapper around CPM_minizinc that allows capturing all (intermediate) solutions,
    because CPM_minizinc abandons all but the last solution
    """

    objective_values: List[int]
    objective_values_timestamps: List[float]

    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)

        self.objective_values = []
        self.objective_values_timestamps = []

    def _post_solve(self, mzn_result):
        solutions = mzn_result.solution if is_any_list(mzn_result.solution) else [mzn_result.solution]
        for sol in mzn_result.solution:
            self.objective_values.append(sol.objective)
            self.objective_values_timestamps.append(time.time())
        return super()._post_solve(mzn_result)

I then call solver.solve(intermediate_solutions=True), and use solver.objective_values and solver.objective_value_timestamps to access the information I need

Hi Tim,
I think that is a very elegant solution.
To have something to main CPMpy would indeed be best with a callback, but if the above solution suits you well then I think we can leave it at that?

Hi Tim,
I think that is a very elegant solution.
To have something to main CPMpy would indeed be best with a callback, but if the above solution suits you well then I think we can leave it at that?

Yes, this works for me. We can leave it at that, but it may be nice for future users. However, I am not sure how common this requirement is


Bug: When using the Enter or Ctrl+Enter hotkey, the last state of the entered value is not saved in the dialogs

Fixed by 0bded38 commit


Git tags for releases?

Would it be possible to get git tags for releases? I can see that the latest version is 0.2.2 according to pypi, but in order to build from source I'm not sure which commit that corresponds to. Having releases tagged helps when packaging augmax for other package managers, eg. nixpkgs.

for context, I'm working on getting augmax packaged here: https://github.com/NixOS/nixpkgs/pull/160946

Hi Samuel, very cool! I fully support you in packaging the library.
The git tags exist, they just weren't pushed -- I forgot that you need to push git tags separately :sweat_smile: Will push them tonight when I'm on my dev machine.
Cheers,
Konrad

The tags are now live @ https://github.com/khdlr/augmax/tags :)

Awesome, thank you so much, @khdlr! augmax is now packaged in nixpkgs: https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/python-modules/augmax/default.nix. I'll update to use the git tags on the next release/whenever I go to update the derivation again!


Logger factory does not handle gossip logs

In order to transfer my logger to dragonboat I use logger.SetLoggerFactory, but the gossip service keeps writing logs to stdout.
Dragonboat version
v3.3.4
Expected behavior
Logs from gossip written to my logger.
Actual behavior
Logs from gossip written to stdout.
Steps to reproduce the behavior
Simple example of wrapper code:
package dragonboat

import (
	"sync"

	"github.com/lni/dragonboat/v3/logger"
)

var (
	onceLogger sync.Once
)

var _ logger.ILogger = (*loggerWrap)(nil)

type loggerWrap struct{}

func (l *loggerWrap) SetLevel(_ logger.LogLevel) {
	// custom logger code
}

func (l *loggerWrap) Debugf(_ string, _ ...interface{}) {
	// custom logger code
}

func (l *loggerWrap) Infof(_ string, _ ...interface{}) {
	// custom logger code
}

func (l *loggerWrap) Warningf(_ string, _ ...interface{}) {
	// custom logger code
}

func (l *loggerWrap) Errorf(_ string, _ ...interface{}) {
	// custom logger code
}

func (l *loggerWrap) Panicf(_ string, _ ...interface{}) {
	// custom logger code
}

func initLogger() {
	onceLogger.Do(func() {
		logger.SetLoggerFactory(func(_ string) logger.ILogger { return &loggerWrap{} })
	})
}

In this example all logs is ignored, but gossip write logs to stdout.

@lebovski you need to add some code for dragonboat to plumb memberlist's (the gossip library used in dragonboat)  logging to the logger used by dragonboat.
you can write a logger wrapper for memberlist and specify it here -
https://github.com/hashicorp/memberlist/blob/619135cdd9e5dda8c12f8ceef39bdade4f5899b6/config.go#L212
this logger should just pass all logging info to your above loggerWrap used in dragonboat.
Any chance you can help to look into the problem and send a PR? Will really appreciate that.

I prepared pr, please review:
https://github.com/lni/dragonboat/pull/217

already resolved #217


GEDI L2B VEDA Guidelines

In reference to: https://github.com/orgs/NASA-IMPACT/projects/29/views/6?pane=issue&itemId=32193528
Added rendering info, new sections (Run This Notebook, About the Data and Additional Resources), and added headers throughout the tutorial since they were missing.

super minor suggestion unrelated to the PR content in itself @smk0033, in case you had not noticed. If you link the related issue through the project board, and the issue gets moved from the board, I believe the link gets broken. E.g. the issue link in this PR is broken https://github.com/MAAP-Project/maap-documentation/pull/219.
So maybe to ensure the link is preserved for anyone looking at the PR in the future, either of the following ? could be better (which all amount to directly link to the issue rather than the board it was added to)

https://github.com/MAAP-Project/maap-documentation/issues/243
#243
or better, in any issue on the right there is a button to link a pull request. Could use that as well !

Thanks, @emileten! I had no idea those links were breaking😅 thanks for pointing that out and recommending fixes for them!


Can't find folders...

Expected Behavior
I should be able to type "Screens..." and Zazu returns me "Screenshots" folder in ~/Dropbox/Screenshots
Actual Behavior
nothing. Suggest to find on google
Steps to reproduce the behavior

open zazu
type the name of one folder you use a lot

Versions

Zazu: latest
OS: macos high sierra
GIT:

Please, include a link to gist of most applicable ~/.zazu/log/ file.

Closed here, opened there https://github.com/tinytacoteam/zazu-file-finder/issues/40


WIP: General Issue: glossary

Background Information
Based on discussions on Slack and #30 we want to start a glossary of key terms in the project.
Definition of Done
No response

Hi @kamilsi, @rammprasad, @edgar-manukyan:
May I make a request? Can we start by trying to clarify these terms in the glossary?

dataset
domain
raw as in raw_dataset (as used in hardcode_no_ct())
target as in target_dataset (as used in hardcode_no_ct())
topic

I am aware of https://www.cdisc.org/kb/articles/domain-vs-dataset-whats-difference. However, I think it still warrants clarification.
Take the case of the domain definition.

Domain: A collection of logically related observations with a common, specific topic that are normally collected for all subjects in a clinical investigation.

If we were to apply the concept of domain to R's iris data set, then I guess we could call it a domain in the sense that the iris data frame is a collection of related observations with a common topic, i.e. plant leaves (not plant species, right?). So even if we split the iris data into two data frames, the set of the two data frames would still be that one domain whose topic is about leaves, right? So one domain is typically materialized as one dataset, but it needs not to. Real life examples would help here.
Then, the CDISC definition of dataset:

A collection of structured data in a single file.

It is, perhaps, not so obvious either... To start, because of the reference to a file. I am guessing that the original intention was to refer to the implementation on a computer, be it a file, an object in memory, database, etc.. Right? It feels like the idea is to say that the domain definition corresponds to the conceptual idea of a data set, and that the dataset is the actual instantiation of that concept on a computer.
Regarding topic, can we say that it equates with the concept of observational unit in tidy data?


Enable deploy_advanced.py verification with the recommendation from the author

Contributing on top of PR #77 , followed the recommendation (Link) provided by the author and propose a solution in this PR.

this looks lovely! Thank you!!