Create Documentation

We need to create some proper Documentation for the ENO repo

Created PR https://github.com/Nordix/eno/pull/21


Duplicate matcher does not create a warning

Subject of the issue
When a duplicate matcher is introduced, a warning should be produced by RSpec.
Your environment

Ruby version: 2.5.3
rspec-expectations version: 3.8.4

Steps to reproduce
RSpec::Matchers::DSL::Matcher.new(:duplicate_matcher, {}, self, nil)
RSpec::Matchers::DSL::Matcher.new(:duplicate_matcher, {}, self, nil)

Expected behavior
It should create a warning
Actual behavior
There is no warning for the user about the duplicate matcher.

I have created a PR that does just that. But, I'm afraid that the specs everywhere are breaking because RSpec raises an error when creating a warning. Is there an alternative method that I can use that will allow RSpec to gracefully inform the user about this.

I don't think using config.raise_on_warning = false would be a good idea as it would suppress all warnings.

Closing in preference to discussion on #1136 itself.


[github] add codeowners

Description
PR adds codeowners files for automatic team assignment for PRs.
Checklist
All Submissions

[x] Do all unit tests pass locally? Attach a log. N/A
[x] Have you formatted the code using clang-format? N/A

onemkl-admin is not defined in https://github.com/oneapi-src/oneMKL/blob/develop/CONTRIBUTING.md. Please consider to change the group or align in different sources.

I've changed the group.


Are variables supported when using KQL with Azure Resource Graph

[Enter feedback here]
I am pretty sure the answer is no , if so it would be helpful to explicitly call this out in an "important" section or similar
thx
Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 18b7fed4-6794-fbad-831e-402ae74df56e
Version Independent ID: ddb188c3-adc5-79c0-a1a6-59a1878bdcfe
Content: Understand the query language - Azure Resource Graph
Content Source: articles/governance/resource-graph/concepts/query-language.md
Service: resource-graph
GitHub Login: @timwarner-msft
Microsoft Alias: timwarner

@andyjballgit
Thanks for bringing this to our attention. We will investigate it further and update you shortly.

Hi @andyjballgit. You are correct that Azure Resource Graph doesn't support the let operator (ref: https://learn.microsoft.com/en-us/azure/governance/resource-graph/concepts/query-language#supported-kql-language-elements). Because we list the supported KQL operators on this page, I don't see value in calling out let at this point. Thanks, Tim

#please-close


doc-nits warning

Does nobody in the project run doc-nits any more?
 mk doc-nits
perl ./util/find-doc-nits -c -n -l -e
*** WARNING: Verbatim paragraph in NAME section at line 8 in file doc/man3/TS_RESP_CTX_new.pod

Perhaps capturing stderr and exit 1 if not emptiy during a CI build?
At any rate, it seems that deleting line 8 from that file fixes the noise.

?? I'm not seeing this on master ??
/usr/bin/perl ../openssl/util/find-doc-nits -c -n -l -e
$

We still run a CI that does doc-nits.

This probably depends on Pod::Checker version.  I recall that there was a version that was unusually verbose with some warnings.

The main point, which got lost in my intro snark, is that the warnings are not captured and do not affect the status.  Shrug. Close this if you want.

I've done a fix for the specific problem.
I didn't realise we could exit with a success after producing output on stderr.

Warnings are not errors, that's true.  That's how Pod::Checker sees it, according to the manual:
https://metacpan.org/pod/Pod::Checker#RETURN-VALUE

I didn't realise we could exit with a success after producing output on stderr.

Uhm...

If you really want to check if there were any warnings, and have our script behave differently, you need to use the full interface, described here:
https://metacpan.org/pod/Pod::Checker#INTERFACE
The podchecker function (which is what we currently use) simply returns $checker->num_errors(), but with the full interface, you have access to $checker->num_warnings() as well.

Thanks, I'll give this a whirl tomorrow and see what explodes.

I tried to do this before and gave up, but @paulidale you're better at perl than I am.  Thanks.  I think the project wants to do this for the same reason it uses -Werror in the code.

If you're up for an adventure, try warning level 2. :wink:
(and no, we shouldn't use that as a standard, it will make certain documentation impossible)

If you're up for an adventure, try warning level 2. 😉
(and no, we shouldn't use that as a standard, it will make certain documentation impossible)

I did.  And then undid the mod and walked slowly away :)

And then undid the mod and walked slowly away :)

Wisely done

... but @paulidale you're better at perl than I am.

No b****y way, my Perl is minimialistic.


Puzzle for ways around the Brain Idol

(Mind Idol?)
Those that respect it, they know the mess they're in. Those that don't, don't know the mess they're in.

get crocked half from delivering pot or whatever
the other piece of it. Not sure where

Crocked half should now trigger the Thought Idol paying a bit more attention to you.


Setup github actions for production

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.0 out of 2 committers have signed the CLA.:x: zacparker41:x: blam23You have signed the CLA already but the status is still pending? Let us recheck it.


Fix for #3365 on armel tizen

CC: @Dmitri-Botcharnikov

https://github.com/dotnet/corert/issues/3365

Thanks!


Infer fetch function argument types from key array

If passing an array as query key, all of the values implicitly get passed to the async function as args.
Is there a way to infer the types as such:
 const { isLoading, data, error } = useQuery(
    ['myKey', { id: 1 }],
    (key, { id }) => { /* id should be inferred as type number */ }
  );

Currently, it seems that id is inferred as type any which doesn't seem corrext

The type connection between the query key and the query function parameters has been removed because it conflicts with how react-query actually works. The eventual query function parameters can be different from the query key because they can be filtered or transformed with queryFnParamsFilter and parameters can be added in the case of infinite queries (fetchMoreVariable). The implication of this is that the query function parameters will need to be typed manually.
When not using this functionality, this should also be possible:
useQuery(['key', filters], () => fetch(filters))

I stumbled upon this today as well. Even if I type my query fetch function explicitly, I will not get a type error if the key doesn't match. To pick up the previous example:
 const { isLoading, data, error } = useQuery(
    ['myKey', { id: 1 }],
    fetchFn
  );

function fetchFn(_: string, params: { id: string }) {
    return axios(...)
}

I would expect to see a type error here somewhere because in the queryKey, I pass in id: number, but in the fetch function, i declare that id should be a string...
It seems like there is no correlation whatsoever between the queryFunction and the queryKey on type level

There is indeed no correlation on type level because there is also no correlation on runtime level. If they were correlated then they could be linked, but this is not the case due to the dynamic nature of React Query. In my previous comment I explained a bit more about this.

Yep, unfortunately, if you are using a detached function (not inline like @boschni suggested), then you'll have to manually type your params to your function.


The Now ruler is sticked to the top of Today – should it not move down according to time?

Not sure what you mean, can you elaborate?


Bower package only includes demo/

I have installed the bower package however it only seems to contain the demo/ directory and none of the .sass code.
I can see that the src/ directory is excluded in the bower.json config file but can't see any mention of the dist/ directory so I am not sure why exactly it is not being installed.

Fixed using the instructions here - https://github.com/milligram/milligram-scss/issues/4

@bencoveney sorry guy!!!
I've been out for a long time. I am updating all repositories during these days and everything will return to normal.


Fixed Issue #1

just contributed to open source

LGTM
Thanks for your contribution @Mohitkumar6122!
Your code style completely follows the style guide. Nice job!
All the tests are passing. Nice job!

thanks for opportunity @dantharejabot


[Event Request] codeunit 1013 "Job Jnl.-Post Batch" - OnBeforeDeleteNonRecJnlLines new parameters

Could we please have additional parameters in event OnBeforeDeleteNonRecJnlLines in Codeunit 1013?
Event:
OnBeforeDeleteNonRecJnlLines(JobJnlLine, JobJnlLine2, JobJnlLine3, JobJnlBatch, IsHandled);
Thank you and kind regards

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. We will update this issue with information about availability.

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. We will update this issue with information about availability.


Feign Request Compression doesn't work with ApacheClient with Hystrix and Ribbon Disabled

I am unable to get the Feign request compression enabled with Apache client and feign request compression properties. I am able to get the compression working alternatively without ApacheClient by adding the request header in the RequestInterceptor with Content-Encoding as "gzip". The same however doesn't work with ApacheClient.
I am using Spring cloud Release - Dalston SR4 with spring-cloud-netflix-core-1.3.4
Sample Test case shown below.
@RunWith(SpringRunner.class)
@SpringBootTest(classes=FeignCompressionUnitTest.Application.class,  value={
"feign.compression.request.enabled=true",
"feign.compression.response.enabled=false",
"feign.hystrix.enabled=false",
"ribbon.eureka.enabled=false",
"opencpu.url=http://localhost:4444/ocpu/library/lib1/R/",
"feign.compression.request.min-request-size=100"
})
public class FeignCompressionUnitTest {
@Autowired
private OpenCPUClient openCPUClient;

@Test
public void testFeign(){
    openCPUClient.calculateReward(input());
}

@Configuration
@SpringBootApplication(scanBasePackages = "mw.analytics", exclude = {
    LiquibaseAutoConfiguration.class
})
@EnableFeignClients(clients = OpenCPUClient.class)
public static class Application {
}

When i add the feign-httpclient dependency, the ApacheHttpClient automatically adds the Accept-Encoding: gzip, default even when feign.compression.response.enabled is set to false.

As a workaround, I was able to disable response compression by creating following beans and request compression by extending ApacheHttpClient and enabling the compression.
`
@Bean
public HttpClient httpClient(){
return HttpClientBuilder.create()
.setMaxConnPerRoute(10)
.setMaxConnTotal(20)
.disableContentCompression()
.build();
}
@Bean
public ApacheHttpClient apacheHttpClient(HttpClient client) {
    return new ApacheHttpClient(client);
}

`


Accept donations :-)

This tool helped me numerous times. Are you accepting small private donations?

That's really sweet of you, but no I don't need nor want donations. Just the fact that this tool is useful to someone is good enough for me :)
I'm pretty sure there are billion other charities with a valid mission statement which would gladly accept your donations :D
cheers


Reserve Auction v1.0: add tests when the protocol fee is non-zero

Description

Towards https://github.com/ourzora/v3/pull/122 as a part of the Community Audit.
Motivation and Context

When an auction is settled, there are payouts to these respective parties:

Royalty Payout
Protocol Fee (optional)
Finders Fee (optional)

The integration tests cover the Royalty Payout case and the Optional Finders fee, but not the optional Protocol Fee.
This adds new assertions to the Reserve Auction v1.0 when the protocol fee is 1 basis point (bps), the equivalent of 0.01%:
https://github.com/ourzora/v3/blob/dfba6b3036d3098f6dde6d722931ac3d12b47b33/contracts/test/modules/ReserveAuction/V1/ReserveAuction.integration.t.sol#L82-L84
I chose this number based on the existing assertions defined in test/auxiliary/ZoraProtocolFeeSettings.t.sol.
How has this been tested?

In my development environment, I ran yarn test and confirmed all tests pass before opening this PR.
Checklist:

[x] The module includes tests written for Foundry
[x] The module is documented with NATSPEC
[x] The documentation includes UML Diagrams for external and public functions
[x] The module is a Hyperstructure

Reviewer Notes 👓
👋 @kulkarohan -- I'd like your evaluative feedback on this approach:

Are there any other edge cases that we should test for, that I missed?
Are there any stylistic changes that the linter didn't pick up that should be implemented?

Please let me know: I'm open to make any additional changes based on your input. Thank you! 🙇♂

this looks great @almndbtr. i cloned your repo and ran forge test (your description mistakenly says yarn) and can confirm they pass.
this is a necessary addition that we should get into the repo asap and i cant think of any relevant low hanging fruit to add, so let's go ahead and merge. @kulkarohan @tbtstl either of you want to take a glance and merge if nothing comes up?

@almndbtr ah of course. for some reason yarn test didnt work for me at first but just confirmed it does.

lgtm!


RuntimeError: Cannot find plugin to use

File "openvino_tiny-yolov3_test.py", line 245, in
sys.exit(main_IE_infer() or 0)
File "openvino_tiny-yolov3_test.py", line 174, in main_IE_infer
plugin = IEPlugin(device=args.device)
File "ie_api.pyx", line 387, in openvino.inference_engine.ie_api.IEPlugin.cinit
RuntimeError: Cannot find plugin to use :
Can you also provide a step by step guide for setting up the openvino in raspberry pi and how to run this project in raspberry pi?

打开

非常感谢你。
它工作，是的，这是一个电源问题。

请问是怎么解决的

@jxdgjs

If trying the above does not work, try self-powered USB Hub.

@jxdgjs

如果尝试以上操作无效，请尝试使用自供电USB Hub。

ok, thanks

ok,thanks

Hi.
File "openvino_tiny-yolov3_MultiStick_test.py",line363.
I changed  "camera_width = 320 camera_height = 240" into "camera_width = 640 camera_height = 480".
Why it runs more slowly (30 FPS to 20 FPS)?
And at the same time, it has the following problem (about the "self-powered USB Hub"):
E: [xLink] [    172632] dispatcherEventSend:889    Write failed header -1 | event USB_WRITE_REQ
E: [xLink] [    172867] dispatcherEventReceive:308	dispatcherEventReceive() Read failed -4 | event 0x6a6fee20 USB_READ_REL_RESP
E: [xLink] [    172867] eventReader:256	eventReader stopped
E: [ncAPI] [    172867] ncGraphQueueInference:3538    Can't send trigger request
E: [watchdog] [    174029] sendPingMessage:164	Failed send ping message: X_LINK_ERROR
E: [watchdog] [    175029] sendPingMessage:164	Failed send ping message: X_LINK_ERROR
E: [watchdog] [    176029] sendPingMessage:164	Failed send ping message: X_LINK_ERROR
E: [watchdog] [    177029] sendPingMessage:164	Failed send ping message: X_LINK_ERROR
..............................................................................................................
Is it also a power supply issue? It works normally in "320*240".  Is it possible to work with larger values such as "1280 * 720" ?

@PINTO0309
Hi.
File "openvino_tiny-yolov3_MultiStick_test.py",line363.
I changed "camera_width = 320 camera_height = 240" into "camera_width = 640 camera_height = 480".
Why it runs more slowly (30 FPS to 20 FPS)?
And at the same time, it has the following problem (about the "self-powered USB Hub"):
E: [xLink] [ 172632] dispatcherEventSend:889 Write failed header -1 | event USB_WRITE_REQ
E: [xLink] [ 172867] dispatcherEventReceive:308	dispatcherEventReceive() Read failed -4 | event 0x6a6fee20 USB_READ_REL_RESP
E: [xLink] [ 172867] eventReader:256	eventReader stopped
E: [ncAPI] [ 172867] ncGraphQueueInference:3538 Can't send trigger request
E: [watchdog] [ 174029] sendPingMessage:164	Failed send ping message: X_LINK_ERROR
E: [watchdog] [ 175029] sendPingMessage:164	Failed send ping message: X_LINK_ERROR
E: [watchdog] [ 176029] sendPingMessage:164	Failed send ping message: X_LINK_ERROR
E: [watchdog] [ 177029] sendPingMessage:164	Failed send ping message: X_LINK_ERROR
..............................................................................................................
Is it also a power supply issue? It works normally in "320*240". Is it possible to work with larger values such as "1280 * 720" ?

@jxdgjs

Why it runs more slowly (30 FPS to 20 FPS)?

It is natural that it becomes later if the field of view to be processed becomes 4 times.
Width x2 , Height x2

And at the same time, it has the following problem (about the "self-powered USB Hub"):

It is probably the limit of USB 2.0.
It is not a logic problem but a hardware problem.

Is it also a power supply issue?

If you are using a self-powered USB Hub correctly, it is not a USB Hub's power problem.

Is it possible to work with larger values such as "1280 * 720" ?

If the 640x480 does not work properly, it will not work properly as well.
I do not know the type of hardware you are using, the number of NCS2/NCS, OS, so I can not answer accurately. I am not Esper.
If you are using RaspberryPi 3, please use a high-quality adapter that can supply "3A" absolutely.
If the power supply voltage of the main unit becomes unstable, such an error will occur.
Also, the use of Virtualbox is not strongly recommended.

@jxdgjs

Why it runs more slowly (30 FPS to 20 FPS)?

It is natural that it becomes later if the field of view to be processed becomes 4 times.
Width x2 , Height x2

And at the same time, it has the following problem (about the "self-powered USB Hub"):

It is probably the limit of USB 2.0.
It is not a logic problem but a hardware problem.

Is it also a power supply issue?

If you are using a self-powered USB Hub correctly, it is not a USB Hub's power problem.

Is it possible to work with larger values such as "1280 * 720" ?

If the 640x480 does not work properly, it will not work properly as well.
I do not know the type of hardware you are using, the number of NCS2/NCS, OS, so I can not answer accurately. I am not Esper.
If you are using RaspberryPi 3, please use a high-quality adapter that can supply "3A" absolutely.
If the power supply voltage of the main unit becomes unstable, such an error will occur.
Also, the use of Virtualbox is not strongly recommended.

I am using RaspberryPi 3B+(ubuntu), a NCS2, a USB camera, a USB Hub (it is 2.0 and it is not self-powered).
When it is "camera_width = 320 camera_height = 240"  in the file "openvino_tiny-yolov3_MultiStick_test.py",  the display window is small, so I changed it into "640 * 480" to let the display window become bigger.(I use my computer to connect the RaspberryPi as a monitor and try to display on full screen).
Before it changed, it works normally. After changed, it works more slowly (20FPS) and the power supply issue occurs. What might be the reason?
Thanks for your reading and analysis.

@jxdgjs

(it is 2.0 and it is not self-powered)

As the throughput increases, device load, especially power consumption increases.
First of all, please purchase the self-powered USBHub to isolate the problem.
You should isolate the problem one by one.

@jxdgjs

(it is 2.0 and it is not self-powered)

As the throughput increases, device load, especially power consumption increases.
First of all, please purchase the self-powered USBHub to isolate the problem.
You should isolate the problem one by one.

Ok, thanks for your answers.

Who can help to provide frozen_tiny_yolo_v3.xml and relevant bin file download url . Just for testing

Just run the download script below.
$ cd lrmodels/tiny-YoloV3/FP16
$ ./download_tiny-yolov3lrFP16.sh

or
$ cd lrmodels/tiny-YoloV3/FP32
$ ./download_tiny-yolov3lrFP32.sh

@PINTO0309
Thank you very much indeed

@hvard07
Can't see very clear about FPS.  14 FPS ?


fix: return value of packument fn is not a manifest object

The packument function returns a Promise that resolves to the packument object, not a manifest object.

Thanks, but I don't feel this really matters. I prefer the previous naming which is sightly more readable to me.


[AutoPR sql/resource-manager] Add Swagger for Short Term Retention Policies

Created to sync https://github.com/Azure/azure-rest-api-specs/pull/2860

(message created by the CI based on PR content)

This PR has been merged into https://github.com/Azure/azure-sdk-for-python/pull/2285


HTTP Redirect Example is Invalid

What happened:
https://gateway-api.sigs.k8s.io/guides/http-redirect-rewrite/#redirects example includes a backendRef.
What you expected to happen:

[ ] The included example should not include a backendRef
[ ] Webhook validation should prevent an HTTPRoute that specifies a redirect filter from specifying a backendRef (until/unless we resolve https://github.com/kubernetes-sigs/gateway-api/issues/1185)

cc @rainest

/help

/assign
/help cancel

/remove-help


Update to latest bevy main

Updates library to work with the latest bevy main

@mcobzarenco Thank you! Looks like this just needs a clippy fix and we can merge.

@aevyrie Ooops, sorry about that, should have checked -- I've fixed it and pushed force

No worries. Thanks!


Fix panic on OIDC implicit login flows other than auth0

This fixes a regression introduced in #1502.
This issue only happens on non-desktop environments.
https://jira.mesosphere.com/browse/DCOS-60349

@mesosphere-ci run integration tests

@makkes : I've updated the commit, it was a nil pointer dereference.

@janisz : indeed it'd be nice to move some code to other methods as it grew quite big. I've started to move some part of it. As the loginWithBrowser method should not attempt to login multiple times in case it uses the login server and the token is incorrect, it cannot be extracted as in your diff.

There is no test or comment explaining it :(

There is no test or comment explaining it :(

https://github.com/dcos/dcos-cli/blob/253a400266d1d5816b2e4524046fc691fd302033/pkg/login/flow.go#L192-L194

there is no need to retry
But still it we can retry.

I'll follow-up in another PR


Add ETH Docker Logs dashboard

Adds a new dashboard to Grafana that allows for people to view their Execution, Consensys, Validator, and MEV-boost logs, searchable by time and viewed on one page.
Also adds loki logging to the mev-boost container, so we can show logs for it too.
https://grafana.com/grafana/dashboards/18700-eth-docker-logs/

Ref: https://github.com/eth-educators/eth-docker/issues/1381

I tested this with a clean install btw.

Looks great, thank you!

This is awesome! Required some manual effort to make it work on my end since my containers are not prefixed with eth-docker due to different COMPOSE_PROJECT_NAME / folder name.

Can you think of a way to make this dashboard independent of the project name?

We could probably change the filtering to be a regex instead of a hard coded container name
@nflaig - can you provide more detail on what changes you had to make?

@gorillamania it is pretty simple, just updating the container name in label filters


Bundle update

Coverage remained the same at 91.81% when pulling 3306daa641900667f306f781c2f2a2eebb8f31d7 on bundle into 742a2435f7de56a1f39d5bf22a60ce85bd21d878 on master.


Add basic i18n support

Very simple translation management system.
Use tr('yourstring') when you deal with a string that needs to be translatable.
If you have dynamic variables, you can use tr('Title level {level}', {level: yourvar}).
Use tr.register({string: translation}) to populate the locale.
A new editor option is added, i18n, which, when present, will be used to register new localized strings.
So to add localized strings, one would do something like:
new Minilslate.simpleEditor(element, {
    i18n: {
        'Paragraph': 'Paragraphe',
        'Title level {level}': 'Titre niveau {level}',
        …
    }
}

Piiing ;)

hey @olivier-m any chance to get that merged on trunk (or reviewed) ? :)


Updating icon-font element removing i

A common bad pattern around the Web has been to use i for icon fonts. Semantically the i element meant italics, but since it's more correct to use em, the i has largely become unused except for icon fonts. This is a bad pattern nonetheless.
At edX we are moving away from the i element for icon fonts and instead using span. All markup and classes may remain the same; only the element changes.
This is follow-up work done by @ssemenova in the platform. It contains one update and one addition:

updates terminology for previous icon-font checks to specify span rather than i
adds new checks to ensure the use of span instead of i (will fail on <i> encounters)

Version bumps the package for use in platform.
Reviewers

[ ] @benpatterson

FYI @cptvitamin

Confirmed that this test works. I ran it on Master (using the commit hash in our package.json) and it didn't find any errors (thanks to @ssemenova). I manually included an incorrect usage on the LMS Dashboard to see if it could get caught, and sure enough!

@benpatterson Can you holla at this review?

👍


Add Layout Settings

UI Button Location
Add a setting button as last action of normal menu (Default AppBar).
Settings
View Mode

Grid.
List.

Spacing Mode

Compact.
Confortable.

The compact mode was implemented and the comfortable is the default one.
View mode is not a priority right now, I think lists fits pretty well, so I'll not mark this as closed yet.


AngularJS not rendering anymore

@thoop
we are using prerender.io for caching our angular website pages, but this is not working anymore.
It is not rendering pages, and removing the scripts tag even before that
https://metadesignsolutions.com is our website.
http://service.prerender.io/https://metadesignsolutions.com returns a blank page...
Can you please help ?

FYI... the page gets render if i test it through local prerender installation
http://localhost:3000/render?url=https://metadesignsolutions.com

Just responded to your email. Let's continue the conversation there!


dbt_deps question

@tomasfarias I'm playing around with trying to use the dbt_deps operator to install packages during mwaa run. So far I've been able to get the deps operator to run correctly, but the downstream task, dbt_run will fail due to not being able to find the packages that were just installed? Do I need to hand off the dbt_packages folder via xcom somehow?
I'm happy to add some documentation for the next person, I'm just hoping to get pointed in the right direction.
Thanks again for making this library!

i think i found the answer to my first question, but it prompts another. If anyone else finds this, if you want to use the deps operator, you need to read this
So my follow up questions are:

Is this flow circular?
Can a zipped dbt project work with this flow, or does it need to be uncompressed?

pull s3 hosted dbt code from repo
run dbt deps in mwaa
push back to 1? Or can/should we aim for another bucket so that flows are acyclic ?
next run will have deps installed from prior run?

hey @samLozier! Seems like you found the answer to your first question, but just to add more details:
As Tasks in Airflow are supposed to run independently, we cannot ensure that the artifacts produced by one task will be available to a downstream task[1]. At least, not without us saving the artifacts somewhere: which is what we do when running DbtDepsOperator with an S3 project URL. This way we can have a task run dbt deps, and push back the artifacts to the original S3 project URL for downstream tasks to pull.
The solution is not perfect, as we don't detect when a dependency is already available (to save some work). As an alternative: you could also run dbt deps as part of your CI/CD pipeline and just push the dbt_packages directory together with the rest of your project to S3.
With that clarified, let's go to your questions:

Is this flow circular?

I'm not seeing any circular flows: essentially, DbtDepsOperator does:

Create temporary directory.
Fetch project from S3 project URL into temp dir.
Run dbt deps in temporary directory.
(Optional) zip (if the project was zipped).
Push project back to S3 project URL.

May I ask for clarification?
Subsequent runs will repeat this flow as there is currently no way for DbtDepsOperator to check if the dbt_packages are already installed. We could add this if you would find it useful. Alternatively, you could write a branch task that checks the S3 Project URL first to see if the deps are there, and skip DbtDepsOperator if they do.

Can a zipped dbt project work with this flow, or does it need to be uncompressed?

Both zipped and uncompressed should be supported, we use zipped!

push back to 1? Or can/should we aim for another bucket so that flows are acyclic ?

Pushing to a different bucket/S3 path is not supported. There is no technical limitation to adding support for something like this. It would require you to have two copies of your dbt project: one basic (w/o deps installed) project to use with DbtDepsOperator and one full project to use with downstream tasks. DbtDepsOperator would then pull from the basic project and push to the full project.
However what advantage would this workflow bring over just using one project? I'm personally not seeing the value but I'm more than willing to support other workflows if the need is there!

next run will have deps installed from prior run?

As long as your S3 project URL stays the same (and the files were not manually deleted and/or modified), you should expect deps to be installed from a previous run.
Hoping some of these answers help you! Thank you for your reports.
1: If you are running with a local executor or running only 1 worker with a remote executor, you could probably ensure this, but MWAA, and most production deployments, will not be using local executors.

There were a lot of points, so I replied inline below. Happy to clarify further if it would be helpful.

hey @samLozier! Seems like you found the answer to your first question, but just to add more details:
As Tasks in Airflow are supposed to run independently, we cannot ensure that the artifacts produced by one task will be available to a downstream task[1]. At least, not without us saving the artifacts somewhere: which is what we do when running DbtDepsOperator with an S3 project URL. This way we can have a task run dbt deps, and push back the artifacts to the original S3 project URL for downstream tasks to pull.
The solution is not perfect, as we don't detect when a dependency is already available (to save some work). As an alternative: you could also run dbt deps as part of your CI/CD pipeline and just push the dbt_packages directory together with the rest of your project to S3.

this is what I've been doing, but I was curious to test the operator and see if there was a benefit. It sounds like maybe not.

With that clarified, let's go to your questions:

Is this flow circular?

I'm not seeing any circular flows: essentially, DbtDepsOperator does:

Create temporary directory.
Fetch project from S3 project URL into temp dir.
Run dbt deps in temporary directory.
(Optional) zip (if the project was zipped).
Push project back to S3 project URL.

May I ask for clarification?

Yes, but circular all I meant was that it appears to read and then write back to the same directory. While unlikely, this could in theory create a situation where an error that corrupts the project in airflow gets pushed back to s3 and then corrupts all subsequent runs of the same dag. I view the ci flow uni directional: push code > test code> push code to s3 > airflow reads from s3 > airflow runs dbt. In a way, the current structure could allow the code to modify itself which, IME, could create a tricky to find bug.
^^ this is what I was thinking when I asked the initial question.
Now that I know that it doesn't check for dependencies prior to subsequent runs, I'm less worried about it, but still view it as a possible edge case.

Subsequent runs will repeat this flow as there is currently no way for DbtDepsOperator to check if the dbt_packages are already installed. We could add this if you would find it useful. Alternatively, you could write a branch task that checks the S3 Project URL first to see if the deps are there, and skip DbtDepsOperator if they do.

The latter may be interesting, but the former is actually what I was thinking should be avoided.

Can a zipped dbt project work with this flow, or does it need to be uncompressed?

Both zipped and uncompressed should be supported, we use zipped!

so it pulls from s3, unzips, runs, re-zips and pushes back to s3? Based on your comment this is now my understanding, I just wanted to be 100% clear.

push back to 1? Or can/should we aim for another bucket so that flows are acyclic ?

Pushing to a different bucket/S3 path is not supported. There is no technical limitation to adding support for something like this. It would require you to have two copies of your dbt project: one basic (w/o deps installed) project to use with DbtDepsOperator and one full project to use with downstream tasks. DbtDepsOperator would then pull from the basic project and push to the full project.
However what advantage would this workflow bring over just using one project? I'm personally not seeing the value but I'm more than willing to support other workflows if the need is there!

I was just thinking about the issue I raised above. IMO this is not worth spending time on since it can be avoided by just running dbt deps in CI

next run will have deps installed from prior run?

As long as your S3 project URL stays the same (and the files were not manually deleted and/or modified), you should expect deps to be installed from a previous run.
Hoping some of these answers help you! Thank you for your reports.
1: If you are running with a local executor or running only 1 worker with a remote executor, you could probably ensure this, but MWAA, and most production deployments, will not be using local executors.

Yes, but circular all I meant was that it appears to read and then write back to the same directory. While unlikely, this could in theory create a situation where an error that corrupts the project in airflow gets pushed back to s3 and then corrupts all subsequent runs of the same dag.

Ah, yeah, I see what you mean. Of course, errors while executing dbt are safe as the operator will just fail without pushing anything back to S3, but a bug on the operator itself could certainly cause issues. Even though I don't think such a bug exists, we could extend our unit test suite to avoid introducing bugs. For example, We currently only check for the existence of files after the operator runs, without reviewing the rest of the project.
Moreover, before pushing anything back, we could also ensure that DbtDepsOperator only modified files in your dbt packages directory. This way, in the worst case, we corrupt dbt packages that can be easily cleaned and re-downloaded.
I can use this issue to track development towards more testing for the operator and making this pre-push check.

so it pulls from s3, unzips, runs, re-zips and pushes back to s3? Based on your comment this is now my understanding, I just wanted to be 100% clear.

Exactly.

I'm doing some development of project safety checks over at: https://github.com/tomasfarias/airflow-dbt-python/tree/safety-checks-for-deps-and-docs.
Branch also extends unit testing to check for modified files when running DbtDepsOperator.

Just brainstorming here, but is it possible to pass the deps directory via xcom? IMO that might offer a few benefits. It would remove any "write back to source" issues, and it would make the state of the deps directory more intentionally ephemeral so that artifacts of a prior dbt deps run couldn't cause any issues with a subsequent node.
One thing i'm unclear about (and I haven't had time to check the code) is how exactly the files are copied to s3. EG is it a file by file copy only, or does it empty the directory first and then copy over the newly generated files?

but is it possible to pass the deps directory via xcom?

I'd say it depends on your XCom backend: if you are using the default Airflow metadata database, that may not be the best place for a directory of unknown size, leaving aside the issues of serializing a directory as JSON.
However, if you are using a custom XCom backend, then it may be very doable. In fact, S3 may be used as a custom backend and then it would be pretty much like what we are doing right now (but with the XCom API in front): see this example from Astronomer that even uses the same S3Hook we do here.
Personally, I think there is some value in supporting this for custom XCom backends, but leaving it up to the user to implement an XCom backend that works for them and can handle directories.

One thing i'm unclear about (and I haven't had time to check the code) is how exactly the files are copied to s3. EG is it a file by file copy only, or does it empty the directory first and then copy over the newly generated files?

We do a file by file copy, replacing files in case of conflict. Relatively safe as if you have S3 versioning enabled, but not so much if you don't so I can consider making the replace optional and ignoring files that already exist in S3.

This issue has been open for quite a while and there have been no new questions.
I've taken some feedback to produce more unit testing to ensure we maintain a project status when running DbtDepsOperator, and have also gone with the idea of supporting optional replacing (by adding a new parameter). PR #31 will be closing this issue as it implements said testing and feature.
As also discussed in this issue, we may later add support for custom backend, but first I'll need to look into it a bit more (PRs and new issues are very welcomed!).
To avoid de-railing the issue too much, feel free to open a new issue if you have any more questions @samLozier. Thanks for your input!


Add primary publishing organisation to organisations

Currently Organisations don't have primary publishing organisation set. This PR changes Whitehall to set a primary publishing organisation. It also adds a rake task to update links for all organisations.

Why are we doing this? It seems like unnecessary duplication without a bit more context.
It's the sort of thing (because orgs are so ubiquitous in the system), that in future people will stumble across this and wonder why orgs have a link back to themselves, so having the initial requirement in the commit message might help.

Currently, users of the content data app can't filter organisation pages by primary publishing organisation. Adding primary publishing organisation allows us to do this. This commit message hints at this.

@sihugh we would like to prevent adding custom logic to apps for specific document types. The primary organisation being in links is standard for most document types and we'd like to maintain this consistency.


Personalization when user logs in

@astridu commented on Mon Aug 24 2020
When user logs in, we should know who they are and have the ability to show personalizations throughout the site (like their name, for example).

@justinlittman commented on Tue Aug 25 2020
What are the requirements beyond the user's name?

@astridu commented on Wed Aug 26 2020
Just putting the user's name on the dashboard and the ability to know that it's them and which institution they are from. I put the other requirements into separate tix.

@justinlittman commented on Thu Aug 27 2020
Thanks!

@jermnelson commented on Mon Sep 21 2020
@astridu what would the URL be for this functionality? https://sinopia.io/dashboard?

From a comment https://github.com/LD4P/sinopia/issues/296#issuecomment-696327306, we want the dashboard to be available after the cataloger logs in to be at the https://sinopia.io/dashboard URI.


add contribution guidelines

Should include
I think as a minimum

[ ] Contribution Licence terms
[x] Code of Conduct
[x] Coding Standards
[x] PR Standards

My suggestions;
To be compatible with Racket I'd suggest the following for Licence terms;

By making a contribution, you are agreeing that your contribution is licensed under the Apache 2.0 and MIT licenses. Those licenses are available in this repository in the files LICENSE-APACHE.txt and LICENSE-MIT.txt.

Follow the GitHub guidance on Code of Conduct.
Follow the Racket Style Guide and Contribution Guide for the last two.

What would be APACHE allowed and MIT don't?

I’m suggesting following the dual licensing lead of racket

Done


Reference to the hadoop-etl-udfs repo

Please note that User guide in this repo refers to the "create_kerberos_conn.py" file from the now archived repository hadoop-etl-udfs.
Now it just looks weird.
Maybe maintainer later even decides to remove the hadoop-etl-udfs repo, who knows ...
Please consider hosting the "create_kerberos_conn.py" file in an active project.

For file udf_debug.py please see ticket https://github.com/exasol/virtual-schemas/issues/481.


Implement a test suite in Jest

As of right now, the only plausible testing mechanism is provided by calling npm run demo-test on 0.3.0 and above and  check for errors. I'd like to implement a more robust testing system, using Jest or similars.

Implemented as of 0.6.0


Examples question

Hi,
In your Examples directory, the source code listed there contains the header, a license from Facebook, which includes their Patent Provision clause, but no other indication where the code originates from.  Can you clarify from which projects this code comes from?
Thanks

This has been resolved in the last update. This project has nothing to do with facebook.


Optimize language server performance

Description
This PR speeds up all features in the language server, at the cost of a small performance hit during AST traversal.
The first optimisation is the Position type in sway-types/src/span.rs. This has been changed to take a reference instead of using clones. The line_col() method was getting called a lot in order to convert between the span's start and end to the lsp_types::Range type that is used in LSP. As such, we were calculating this for every token in the language server when iterating over the TokenMap for all of the features. Removing the clones led to a 40% speed up.
The last optimisation introduces a new TokenIdent type. This is similar to sway_types::Ident but holds data that is more useful for LSP, specifically, a PathBuf and Range. We are now precomputing these during AST traversal and using the new TokenIdent as the key into the TokenMap. This avoids needing to pass the source engine around to convert SourceId's to paths and also avoids needing to continually compute the range more than once.
I think the small performance hit we take moving this computation to the traversal stage is worth the insane speed increases we get for all other LSP features. I have a plan in #4959 to parallelize traversing the AST's using the rayon crate in a follow up PR that overcomes this performance regression.
Here is the result of running cargo bench with these optimisations applied.

Type
Previous Duration
Current Duration
Speedup
Improvement %

on_enter
572ms
567ns
1008818.34x
100.0%

code_lens
34ms
16.059µs
2117.19x
99.95%

idents_at_position
438ms
3.281ms
133.50x
99.25%

prepareRename
433ms
6.635ms
65.26x
98.47%

token_at_position
439ms
6.76ms
64.94x
98.46%

definition
433ms
6.726ms
64.38x
98.45%

hover
442ms
7.389ms
59.82x
98.33%

document_symbol
447ms
7.49ms
59.68x
98.32%

semantic_tokens
464ms
24.48ms
18.95x
94.72%

completion
893ms
51.03ms
17.50x
94.29%

parent_decl_at_position
437ms
42.49ms
10.28x
90.28%

tokens_at_position
437ms
42.87ms
10.19x
90.19%

code_action
444ms
56.97ms
7.79x
87.17%

documentHighlight
441ms
115ms
3.83x
73.92%

rename
436ms
115.35ms
3.78x
73.54%

inlay_hints
15ms
9.75ms
1.54x
35.0%

compile_and_traverse
1.795sec
1.9715sec
0.91x
-9.83%

tokens_for_file
10ms
11.37ms
0.88x
-13.7%

Related to: #4901
Checklist

[x] I have linked to any relevant issues.
[x] I have commented my code, particularly in hard-to-understand areas.
[ ] I have updated the documentation where relevant (API docs, the reference, and the Sway book).
[x] I have added tests that prove my fix is effective or that my feature works.
[ ] I have added (or requested a maintainer to add) the necessary Breaking* or New Feature labels where relevant.
[x] I have done my best to ensure that my PR adheres to the Fuel Labs Code Review Standards.
[x] I have requested a review from the relevant team or maintainers.

Removing the clones led to a 40% speed up.

Brilliant!

The line_col() method was getting called a lot in order to convert between the span's start and end to the lsp_types::Range type that is used in LSP. As such, we were calculating this for every token in the language server when iterating over the TokenMap for all of the features.

I am also having this problem elsewhere. Maybe we should have a data struct offset -> line number somewhere? crates.io has some interval trees. Not sure which one would be the best fit.

What do you think about the Position type taking a SourceId instead? Would that provide any benefits or just complicate things for you?

I'm not sure I understand the proposed benefit of doing that. One of the goals of this work is to get away from the Position and SourceId types as quickly as possible. This way we only need to compute the Range once. We also eliminate the need to pass around the source engine throughout the server to get the required PathBuf.
Would be keen to hear your thoughts on if you see a better way forward though!

Maybe we should have a data struct offset -> line number somewhere? crates.io has some interval trees. Not sure which one would be the best fit.

Yeah that could be interesting! Ping me if you come across anything worth looking into


Combobox: Custom current value display

Describe the feature in detail (code, mocks, or screenshots encouraged)
The search input is currently tightly coupled to the selected item. When an item is selected, the input is always set to the item's label, causing the list to be filtered by the item's label.
This is undesirable when using the Combobox with an external API, where the user might have selected the wrong item and would expect to still have their search results shown if they re-open the result list. It also results in an extra API call for no reason, every time a selection is made. On top of that, it creates a disconnect between the input's value and the current selection, making it possible for the user to modify the input, with the underlying selection remaining the same.
I would like to have the option of a separate slot for the current value, and a slot for the search input. This way, when the user makes a selection, their search input and results are retained, while the selection is shown. Then, if they click on the current value, their search input and associated results could be displayed again. If the user does not change their selection and blurs away from the combobox, their selection would be shown once again.
What type of pull request would this be?
Enhancement
Provide relevant links or additional information.
In this screenshot, I selected Mango, then typed Apple, then clicked away. This creates a state where it looks like I've selected Apple, but Mango is still selected. The input text still reads Apple, even if I click into the field and back away from it.

Closing as fixed in bits-ui@next

I’m facing the same issue in the latest version, 1.0.0-next.74. When selecting 'Mango', typing 'Apple', and then clicking away, it creates a state where it appears that 'Apple' is selected, even though that is not actually the case. Can you guide me on how to fix this?


neutralize path separator when used in service name

tree.Path must distinguish parts in a path vs names using the path separator char.
This quick fix replaces path separator with another char I don't expect anyone would use for resource name
closes https://github.com/docker/compose/issues/11336

a nicer approach is to redefine tree.Path as a slice of parts, so that parts can be any string. I'm looking into this, but can't guarantee a PR to be ready short-terms

:ghost: funny touch


add docs for createAction.match

Writing docs is not my strong suit, so if you have any better ideas, I'm all for it :)
Also, I'm not a user of redux-observable myself, so maybe @on3iro has a better idea for a redux-observable example here?

Drat, I actually meant to do this on the docs tweak branch I was working on a few minutes ago, and forgot :)
Thanks!

Travis failure is due to formatting - just re-run Prettier.
Also, can you add a brief mention that the type guards might be particularly useful in a custom middleware as well?  I was actually just dealing with this a couple days ago.

Travis failure is due to formatting - just re-run Prettier.

:+1:

Also, can you add a brief mention that the type guards might be particularly useful in a custom middleware as well? I was actually just dealing with this a couple days ago.

Good idea, added a sentence :)

Thanks!

@on3iro - while this is already merged, if you have a better idea for a redux-observable example, a PR is very welcome!

@phryneas the example looks good to me :+1:


tf.train.batch and shuffle_batch undetermined for multi queue input with multiple threads

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow):
Custom code, see below.

OS Platform and Distribution (e.g., Linux Ubuntu 16.04):
OSX

TensorFlow installed from (source or binary):
pip install tensorflow == 1.2

TensorFlow version (use command below):
('v1.2.0-rc2-21-g12f033d', '1.2.0')

Python version:
Python 2.7.10

Bazel version (if compiling from source):
n/a

CUDA/cuDNN version:
n/a

GPU model and memory:
n/a

Exact command to reproduce:

a_fifo = tf.train.slice_input_producer([np.arange(30)],
                                        shuffle=False,
                                        name='a')[0]
b_fifo = tf.train.slice_input_producer([np.arange(30)],
                                        shuffle=False,
                                        name='b')[0]
train_batch = tf.train.batch([a_fifo, b_fifo],
                            batch_size=5,
                            capacity=18,
                            num_threads=2,
                            name='batch')
at, bt = train_batch
with tf.Session() as sess:
    threads = tf.train.start_queue_runners(sess=sess)
    sess.run(tf.global_variables_initializer())
    try:
        for i in range(150):
            av, bv = sess.run([at, bt])
            assert list(av) == list(bv), "queues derailed!"
    except Exception as e:
        print(("Exception in training: {}").format(e))
        print("__",i,"__")
        print(av)
        print(bv)

Output:

Exception in training: queues derailed!
__ 11 __
[26 25 27 28 29]
[25 26 27 28 29]

Describe the problem
Both queues should be dequeued synchronously, as it does when num_threads =1 in the batch statement. With num_threads > 1 the two input queue's will derail, as shown in the example above. Same issue appears also in train.shuffle_batch.
Source code / logs
See code above.

@BastiaanBergman we are moving to tf.data, which is a much nicer interface. Would you be open to that?

I really liked tf.data but found it not working for some other reason, therefore reverted back to the queues. Will retry some day.

We are deprecating the queues in favor of tf.data and would really like to hear about what the problem is.
@mrry maybe we can suggest a solution?

Yes, the difficulty of getting deterministic results is one of the reasons why we are recommending that people use tf.data instead of the queue-based APIs. For example, here's what your program would look like in tf.data:
a_dataset = tf.data.Dataset.range(30).repeat()
b_dataset = tf.data.Dataset.range(30).repeat()
batched_dataset = tf.data.Dataset.zip((a_dataset, b_dataset)).batch(5)
iterator = batched_dataset.make_one_shot_iterator()
at, bt = iterator.get_next()

with tf.Session() as sess:
    try:
        for i in range(150):
            av, bv = sess.run([at, bt])
            assert list(av) == list(bv), "queues derailed!"
    except Exception as e:
        print(("Exception in training: {}").format(e))
        print("__",i,"__")
        print(av)
        print(bv)

Yes its nice that way. I still have to look and see what made me move away from dataset. I think it was issues with switching between test and train sets on the same graph. But I had issues with that in queue's too and eventually settled with separate graphs for training and testing anyway.
Are there multiple threads working on this dataset version? Would I be save with one thread on queue's? Or might I still have compromised 'deterministic results'? ;-) Thanks anyway, it's been awesome learning tf.

There are a couple of ways to switch between different datasets with a single Iterator (search for "reinitializable iterator" and "feedable iterator" in this guide), but most people find it easier to construct multiple graphs for these cases.
The code example I showed does not create additional threads, but you can add them wherever you'd like by adding a .prefetch(buffer_size) transformation. Although it doesn't make much sense for the concrete example, here's how you'd add prefetching threads to the a_dataset and b_dataset computations:
a_dataset = tf.data.Dataset.range(30).repeat().prefetch(10)
b_dataset = tf.data.Dataset.range(30).repeat().prefetch(10)
batched_dataset = tf.data.Dataset.zip((a_dataset, b_dataset)).batch(5)

If you have an expensive Dataset.map() in your program, you can also parallelize that by adding num_parallel_calls=N to process N elements at once.
All these options add parallelism without comprising on determinism. There is an explicitly "sloppy" transformation called tf.contrib.data.sloppy_interleave() that can produce non-determinstic results (but can be useful to make I/O blockages), and we've considered adding a non-deterministic parallel map (to deal with stragglers), but it's generally been more pleasant to work with deterministic APIs.
By contrast, the num_threads > 1 versions of the queue-based APIs are inherently racy, because they issue multiple Session.run() calls in parallel, and mutate queue state (dequeuing from the source queue, enqueuing to the destination queue) without any coordination. Setting num_threads = 1 ought to help, but it conflicts with tf.train.shuffle_batch(), because there can still be a race between a single producer thread and a single consumer thread when capacity != min_after_dequeue, leading to unrepeatable results.

Closing due to lack of activity.


The Sleuth Kit

Could you please add The Sleuth Kit, or parts of it such as fls, icat, istat?
https://www.sleuthkit.org/sleuthkit/download.php

I am currently working on the 1.1 version and The Sleuth Kit is already in. Thank you!

Closed 51392d0a4a2cd7c01a4e2ec9f8b8db2362e6b83d


EventHub batch processing

It seems like Azure Functions' EventHub binding doesn't allow processing of more than one message at a time, when custom webjob does. This slows down processing dramatically. Are there any plans to add a way to process batch of event hub messages?
Signature for Azure Functions binding:
Run(string myEventHubMessage, TraceWriter log)
Something like:
Run(string[] myEventHubMessages, TraceWriter log)

This is already supported. In a C# function, you can bind to EventData[] events or string[] events. If this doesn't work for you, please reopen the bug with full details.

According to this article https://docs.microsoft.com/en-us/azure/azure-functions/functions-bindings-event-hubs, only next signature is supported:
public static void Run(string myEventHubMessage, TraceWriter log)
It is supported in WebJobs itself, but not in Azure Functions' plugin for EventHubs, seems like to me.

The docs don't enumerate all the possible signatures that are supported (many are). The docs should be improved I agree to list the possibilities.
When you say "seems like to me" do you mean you actually tried binding to EventData[] events and it doesn't work?

I will try and let you know.

It works, but requires "WindowsAzure.ServiceBus":"3.4.2" nuget package to be imported. Thanks.


Clamp the maximum memory of the grenad sorters

This PR clamps the memory usage of the grenad sorters to a reasonable maximum. Grenad sorters are opened on multiple threads at a time. This can result in higher memory usage than expected, even though it shouldn't consume more than the memory available.

/benchmark indexing

@Kerollmops does this PR fix https://github.com/meilisearch/meilisearch/issues/4152?

/benchmark indexing

IMO, considering the RAM we give to our big client, I would not be shocked if we went as up as 1GiB 👀

Seems ok to me, as #4350 will probably change many things. This PR is here to reduce the memory crashes not to speed up the engine. We can work on improving the performances in a lot of other domains.
bors merge

bors merge


Support for Network Area project creation

Hi,
Looks like the CLI doesn't have support to create projects using Network Area, I would like to have that option.
Thanks

Hi @tchelovilar,
Thanks for reaching out! We have the integration of the network and network area planned for the next couple of weeks. We will let you know here when those are integrated.


Spring cleanup and restructure - dev to master

Most of what's included is referenced in https://github.com/openaps/docs/pull/114 - please see there for notes if you're unfamiliar with the bulk dev changes.
Creating this PR to trigger discussion around anything major that needs to be done before this overtakes master. IMO, dev is good enough as-is, although we know it needs some work on the flow especially as it relates to NS setup; it'd be better to get this to master to get more eyes on what needs improved.
cc @bewest @jasoncalabrese @scottleibrand for comments, fixes in addition to feedback from everyone else. If no major objections (including lack of thumbs up), I'll merge this on Tuesday April 19 (after 4 days).

:+1: on this being better than what's in master

Fantastic.  This has become the "new default" for the time being, so I'd love to see it merged to master ASAP.  If there's anything about dev we're unhappy with as the new defaults, let's figure it out and fix it; since it's been available and we seem to be increasingly depending on it, I think you can feel free to merge/release to master anytime.
Big :+1: .

Merging spring cleanup!


Feature Request: open player with single-click in system tray icon

Currently, when minimized to the system tray, it is necessary to double-click the player to open it. Instead of a double-click, please allow the player to be opened with a single-click.

This changes from OS to OS, we try to follow the standard behaviour on each OS which at least on windows is double click to open.

What about placing an option in the settings and letting the user decide?


Fix prepare_package.dart not updating base_url

The issue was that the parameter newBucket of the _updateMetadata() method was always being passed false, even for the new bucket manifest, and this value was used to fill in the base_url value.
Fixes https://github.com/flutter/flutter/issues/81870

LGTM

Can you please approve :)


Consolidate rendering for animted emotes (native/BTTV)

Feature Request
Hey there.
Now that Twitch has integrated animated emote rendering natively in the their app (play animation for a few seconds then freeze), how about reusing the same rendering pipeline for animated BTTV emotes.
Larger chats with lots of emote spam tend to lag out weaker Android processors, so freezing animations could help lead to an improvement in quality of life.
In contrast, an option to not freeze native animated emotes could be interesting for people with high power Android devices.
Therefore, I propose exploring ways to consolidate animated emote rendering for both native and BTTV emotes and add an additional option that lets users choose if they want unlimited animation / stop animation after some time / static images only.

I am aware of the performance issues when a lot of emotes are used, but never figured out why. The mod is generating "normal" Emote Objects, though I had to do some trickery to get the animations to work in the first place. I'll have a look at what has changed and how (or if) a better integration is possible.

Let's hope this will also fix #152. Thank you for bringing this up again! This will land in 0.14.0-beta once I can rebase to the new twitch version.

v0.14.0-beta was just released!


make globus and paramiko optional dependencies

Description
Place globus and paramiko into optional dependencies list, due to rare usage and paramiko's incorrect binary installs of its unix .so file.
Changed Behaviour
This PR doesn't affect any user workflow or functionality.
Fixes
Fixes #2218
Type of change
Choose which options apply, and delete the ones which do not apply.

Code maintenance/cleanup

Thank you for your feedback, @benclifford! I made changes according to your comment about removing dependencies from requirements.txt as well.
As you said some checks have failed indeed, can you please have a look, so that I proceed with fixes. Thank you!

It looks like you figured out the approach other parts of the Parsl codebase use to work with optional modules.
I can see that the tests are failing at the moment, with this error:
ImportError while loading conftest '/home/runner/work/parsl/parsl/parsl/tests/conftest.py'.
parsl/__init__.py:22: in <module>
    from parsl.app.app import bash_app, join_app, python_app
parsl/app/app.py:12: in <module>
    from parsl.dataflow.dflow import DataFlowKernel
parsl/dataflow/dflow.py:25: in <module>
    from parsl.channels import Channel
parsl/channels/__init__.py:2: in <module>
    from parsl.channels.ssh.ssh import SSHChannel
parsl/channels/ssh/ssh.py:27: in <module>
    raise OptionalModuleMissing(['paramiko'], "OAuthSSHChannel requires paramiko module.")
E   parsl.errors.OptionalModuleMissing: The functionality requested requires optional modules ['paramiko'] which could not be imported, because: OAuthSSHChannel requires paramiko module.

That's happening because this block of code in ssh.py raises an exception at import time if the optional module is missing - but we need this file to be importable even if the optional module is missing:
if _paramiko_enabled:
    class NoAuthSSHClient(paramiko.SSHClient):
        def _auth(self, username, *args):
            self._transport.auth_none(username)
            return
else:
    raise OptionalModuleMissing(['paramiko'], "OAuthSSHChannel requires paramiko module.")

Perhaps you could try removing that else raise section - so that if the optional module is available, the NoAuthSSHClient will be defined, and if it is not, then it will silently be skipped.
Then, the only use of NoAuthSSHClient is protected by a different OptionalModuleMissing, that you have added at the start of SSHChannel.__init__


Add example invocation with jostler

This change updates the docker-compose.yaml to run the ndt-server with jostler in a local mode.
This configuration demonstrates:

generation of ndt-server result schemas for ndt5 and ndt7 for use by jostler
configuration of ndt-server to output results that are jostler/autoloader compatible (i.e. uncompressed json)
configuration of jostler using local output mode, simulating upload to GCS bucket with a local directory.

Note: the jostler timeouts and size limits are for illustration purposes only, and not suitable for a production deployment.
This type of configuration could be used by a third-party running the ndt-server with the intent to share the data with M-Lab.

This change is

@cristinaleonr FYI.

Pull Request Test Coverage Report for Build 1655

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.04%) to 81.68%

Totals

Change from base Build 1652:
0.04%

Covered Lines:
1828

Relevant Lines:
2238

💛  - Coveralls

Thank you!


Stop Rx.Subject/Observable from emitting like event.stopPropagation do

Probably this is out of the Observable Pattern and can not be even considered, can be a feature request or I can't found in the docs the right function to do it, so, here is my example:
var source = new Rx.Subject();

source.map((data) => {console.log('map1'); return data;})
	.subscribe((data) => {
		console.log('subscribe1', data);
		if(someCondition) {
			source.stop(); //????????
		}
	});

source.map((data) => {console.log('map2'); return data;})
	.subscribe((data) => {
		console.log('subscribe2', data);
	});

So, when I call source.next("Hello World"); only the first subscriber will be notified. Of course this will fail in source.stop() because the stop function does not exist but it is just to elaborate my question.
I know that I don't have a chain where the value goes like source => subscribe1 => subscribe2. Right now I have source => subscribe1 and then source => subscribe2. But in the console it prints
map1
subscribe1
map2
subscribe2

so there is a sequence. So, exist a way to say: "Hey, do not emit any more for now", like event.stopPropagation do?

You could try something like:
(Getting rid of the maps which seem to do nothing - if you want to observe with no side effects, try do.
var source = new Rx.Subject();

// filtered source will only "fire" if some condition is true
var filteredSource = source.filter((data) => someCondition);

// by subscribing to source, I get everything
source.subscribe((data) => {
		console.log('subscribe1', data);
	});

// by subscribing to filteredSource, I only get those events which match `someCondition`
filteredSource((data) => {
		console.log('subscribe2', data);
	});

I was hopping that RxJS has an option/fucntion so the sequence could be stopped, filter does not suggest it can block the item from getting to the next observer at all, at least the name of the function doesn't :-). Your answer is kind of a hack, but your idea help me to accomplish what I wanted. Thanks

I think the difficulty here is it's not really a "propagation", per se. It's more notifiying a list of observers that have no hierarchy.

Closing for now as some answers been provided.

Looks like some bad practice. If event fired on some channel ALL it's subscribers should be called (otherwise one can expect hardly understandable bugs). And in my understanding it's better not to assume any order of called subscribers. So that it could happen that 2nd subscriber will be called first (that should be implementation dependant). With knowing the order you breaking encapsulation.
In this case running subscribers separately is correct solution. Create another stream for 2nd subscriber. E.g. with filter (as it was already proposed) or takeUntill (if you need to unsubscribe 2nd subscriber) operator.


Support of submitting massive number of video shots to the system

Is your feature request related to a problem? Please describe.
I try to use the DRES system for AVS annotation.
I have a pool of video shots needed to submit to the DRES system for evaluation, e.g., 10k video shots for each query.
Currently, I use the API  to submit the video shots. However, I cannot submit too many at a time and it is time-consuming for submitting the pool.
Describe the solution you'd like
I wander if  the system can have the upload button at the front (UI) for submission. Thus, I can submit the pool by uploading a file. The file lists the video shots.
Describe alternatives you've considered
allow submissions of larger batch size. Currenctly, it is less than 300?
Additional context
Can I change the batch size of submission before building DRES? if so, where I can change.
Thanks a lot.

Thanks for your issue.

I try to use the DRES system for AVS annotation.

I assume you are trying to evaluate Ad-hoc Video Search (AVS) system(s) ? DRES is for evaluations, not to annotate a dataset (and export this information).
As for now, DRES is primarily build for interactive evaluation campaigns, where AVS submissions tend to be less than 2.5k submissions simultaneously -- we never tried to have 10k submissions so far.
Before considering this feature request, we'd need some more information:

How do you plan to judge these 10k submissions? The typical AVS setup (and the task type preset) are set for manual judgement, which seems to be off-limits with 10k submisisons.
How did you upload the submissions currently?

I wander if the system can have the upload button at the front (UI) for submission

The frontend (UI) of DRES is designed and build for these use cases:

Designing and building the evaluation (admin)
Orchestrating / directing the evaluation (admin)
Manual judgement of incoming submissions in certain cases (judge)
Getting information abou the (ongoing) evaluation, particularly the task (participant)

Intentionally, there is no UI for submissions, as this should be facilitated using the REST API for clients - did you try this? (There is an OpenApi UI for this: https://editor.swagger.io/?url=https://raw.githubusercontent.com/dres-dev/DRES/master/doc/oas-client.json )


Add JSPM config

This will make it easier to install and use Foundation Sites with JSPM. Let me know if you have any questions.
http://jspm.io/
/cc @guybedford

This would be nice to slip into 6.2 on Friday, could you open a new pull request that targets the 6.2-stable branch instead? That's where we're making final adjustments to the patch. Thanks :)

Done – closing.


To make use of recently checkin keywords

Purpose of this issue is to adopt recently checkin keywords

[ ] Explore the new keywords what was merged

[ ] Adopt into the GUI Header, Where it is applicable.

This issue is for all keywords?  Or was this supposed to be keywords in some specific file(s)?

I think it is done.


Reformat Tweet

As discussed on Twitter chat, I had a few suggestions:

Put title in the Tweet
Move URL up so easier to click (currently very close to Retweet and Like buttons)

I also changed the length calcs, and moved the hashtags to it's own line.
Old:

Running in dev mode. Following tweet would be sent
Tweet length: 300
🦖 Random MDN 🦖
This chapter describes the DOM Event Model. The Event interface itself is described, as well as the interfaces for event registration on nodes in the DOM, and event listeners, and several longer … #webdev
https://developer.mozilla.org/en-US/docs/Web/API/Document_Object_Model/Events

New:

Running in dev mode. Following tweet would be sent
Tweet length: 268
🦖 Random MDN: Events and the DOM 🦖
https://developer.mozilla.org/en-US/docs/Web/API/Document_Object_Model/Events
This chapter describes the DOM Event Model. The Event interface itself is described, as well as the interfaces for event registration on nodes in the DOM, and event listeners, and several longer …
#webdev

Thanks so much @bazzadp. 🙇♂


Error raised when fiting after lr_find

When doing fit_one_cycle after lr_find, I get the following error:

This happened when I trained a learning model and then used it at a classifier, the same way as in
# Training Language Model
dls_lm = TextDataLoaders.from_csv(path, csv_fname=train,valid_pct=0.1,text_col="text",is_lm=True)
learn = language_model_learner(dls_lm, AWD_LSTM, metrics=[accuracy, Perplexity()], path=competition_google_drive, wd=0.1).to_fp16()

# Training Unfreezing Training
learn.fit_one_cycle(1, 1e-2)
learn.unfreeze()
learn.fit_one_cycle(10, 1e-3)
learn.save_encoder('encoder_finetuned')

#Training Classifier

dls = TextDataLoaders.from_csv(path, csv_fname=train,valid_pct=0.1,text_col="text",label_col="target",text_vocab=dls_lm.vocab)

learn = text_classifier_learner(dls, AWD_LSTM, drop_mult=0.5, path=competition_google_drive, metrics=accuracy)
learn = learn.load_encoder('encoder_finetuned')

learn.lr_find()

learn.fit_one_cycle(1, 1e-3)

learn.save('finetuned_stage_1')

learn.freeze_to(-2)
learn.lr_find()

#The line below raise an error
learn.fit_one_cycle(1, 1e-6)

Thanks for flagging (and for the detailed reproducer). This should be fixed now.


Update backport:prev-minor to backport to all previousMinor versions

We want to change the behavior of release_note:fix backports to go back to all minor versions.  There's two options:

Leave backport:prev-minor as is and advocate for using backport:current-major.  This label never made it to the Kibana repository but it is supported.
Update backport:prev-minor to backport to all minor versions.  Update backport:all-open to backport to all minor versions.  Remove backport:current-major.

I took the second approach here.  It's slightly less semantic (may no longer be a single version) but will see immediate adoption.  I don't see a use case for backporting to a single minor version when multiple are available.

Update backport:prev-minor to backport to all minor versions. Update backport:all-open to backport to all minor versions.
Would this mean the two labels are going to do the same thing, or is there a typo in the phrasing?

typo

Update backport:all-open to backport to all minor versions and 7.17


[plot] Remove some old deprecated methods/arguments

This PR removes some (not all) deprecated (at the very least since a year) methods/signals/arguments in PlotWidget and PlotWindow.
As a side effect, it closes #1677

LGTM


meshery-perf crashing

Current Behavior
CrashLoopBackOff
Expected Behavior

Using mesheryctl to install on the k3d cluster, should run perfectly fine, but both meshery-perf and meshery-cillium is crashing.
Reference to cilium-adapter crashing, #71
Screenshots/Logs

kubectl logs meshery-perf-794bb4f4c7-4tf6l -n meshery
time="2022-09-05T22:20:51Z" level=info msg="Component listening at port: 10013" app=meshery-perf
time="2022-09-05T22:20:51Z" level=info msg="Registering static workloads with Meshery Server..." app=meshery-perf
time="2022-09-05T22:20:51Z" level=info msg="Registering latest components with Meshery Server" app=meshery-perf
time="2022-09-05T22:20:51Z" level=info msg="Registering latest workload components for version " app=meshery-perf
time="2022-09-05T22:20:51Z" level=info msg="Failed to generate components for version " app=meshery-perf
time="2022-09-05T22:20:51Z" level=error msg="mkdir /templates/oam/workloads: no such file or directory" app=meshery-perf code=1013 probable-cause="Invalid Path or version passed in static configuration.URL passed maybe incorrect.Version passed maybe incorrect" severity=2 short-description="error creating components" suggested-remediation="Make sure to pass correct configuration.Make sure the URL passed in the configuration is correct.Make sure a valid version is passed in configuration"
panic: interface conversion: error is *fs.PathError, not *errors.Error

goroutine 41 [running]:
github.com/layer5io/meshkit/errors.GetCode({0x287ee80, 0xc00025e090})
        /go/pkg/mod/github.com/layer5io/meshkit@v0.5.5/errors/errors.go:90 +0x90
github.com/layer5io/meshkit/logger.(*Logger).Error(0xc000010018, {0x287ee80, 0xc00025e090})
        /go/pkg/mod/github.com/layer5io/meshkit@v0.5.5/logger/logger.go:69 +0xbb
main.registerCapabilities({0x25fa974, 0x5}, {0x28d9090, 0xc000010018})
        /build/main.go:142 +0xf5
created by main.main
        /build/main.go:98 +0x590

Environment

Host OS: Ubuntu WSL2
Platform: Kubernetes
Meshery Server Version: stable-v
Meshery Client Version: stable-v
Meshery Adapter Version: stable-v

Contributor Guides and Resources

🛠 Meshery Build & Release Strategy
📚 Instructions for contributing to documentation

Meshery documentation site and source

🎨 Wireframes and designs for Meshery UI in Figma
🙋🏾🙋🏼 Questions: Layer5 Discussion Forum and Layer5 Community Slack

Close as this was solved by PR #6178

Thank you.


When will the code be released?

Dear Sir/Madam, I am very interested in your research work, I would like to know when you will release the code?

Same here, looking forward to your code!

Thanks for your interest in our work. We are cleaning the code recently and will release the code within 1~2 weeks.

Same same here, looking forward to your code!

Is it possible for your team to open source DetZero? @superkoma

Hm.

Hm.


FIRInstallations: recover mac os tests

Here are the changes required to run FIRInstallations tests against mac os.

I'm satisfied with your responses and good to merge. Thanks!


Client-side security

I just wanted to know if it is possible and if so, how to deny calls to Roles.addUsersToRoles from the client? Thank you in advance!

Client-side security is automatic by default.
In Meteor apps, only the profile field of the Meteor.users collection is editable from the client unless you explicitly grant access to other fields yourself.  Since Roles also uses Meteor.users to store permission information, client-side changes are also not permitted.
You can see this in action by trying to manually use Roles.addUserstoRoles from the browser console.  Log in as the normal user at this example app and try to add 'view-secrets' permission.  You'll see console error messages similar to 'access denied'.


Is Windows CTRL_CLOSE correctly handled?

As far as I know, in Windows, handling the CTRL_C event is easier, since it can be listened to and then handled in an asynchronous manner by the program.
But from what I have seen, this seems not to work for CTRL_CLOSE signals. This is because, when this signal is received by a windows console app, the thread of the running program is paused (although the rust thread might still have time to catch the signal and run a small piece of code), and the registered CtrlHandler function is called in a new thread. But whatever code rust can still run before its thread is paused will not be well defined behaviour, since it is dependent on the system latency to pause the thread.
On the other hand, windows allow functions defined by the SetConsoleCtrlHandler entity to run up to 5 seconds before they are forcefully terminated. The rust thread, however, will just have whatever latency the os has in pausing the thread, which is naturally much shorter.
I.e. the mechanism of the the CTRL_CLOSE (also CTRL_SHUTDOWN and maybe others) signal in windows is much different from the CTRL_C. Whereas the SIGINT/SIGTERM/SIGHUP in UNIX all work in a similar way amongst themselves and can be asynchronously handled in the rust main thread.
I can be wrong, but this was my understanding of the topic.
Check this example in the rust-ctrlc crate:
https://github.com/Detegr/rust-ctrlc/blob/master/src/platform/windows/mod.rs
Maybe this Go-related github issue can also be useful, it is where I learned about the issue:
https://github.com/golang/go/issues/41884#issuecomment-706695923
(Also, while in Linux all the applications run in the same subsystem, in Windows, apps can run be of the Console or GUI types, and they run in different subsystems. By default Rust compiles as Console app, which opens a terminal while running. But from what I can gather, GUI apps receive different signals than those in Console. For example, WM_CLOSE instead of CTRL_CLOSE. I think these can be handled the same way as CTRL_C. Maybe it could be interesting to support GUI-app signals as well.)

I'm not entirely sure if I understand what should be done different... What exactly would be the scenario where the current code does not work, and what effect would that have?

GUI Apps should handle the WM_CLOSE signal in the GUI code, I don't think this crate would be the correct place to deal with this.

What I mean is that I think that the CTRL_CLOSE signal handling in this app won't grant any time for a graceful shutdown to happen. tokio might have time to catch the signal, but the thread may be frozen shortly after.
Instead, unlike the CTRL_C methodology, for CTRL_CLOSE events you might have to implement something similar to what was done in that file from the rust-ctrlc crate I mentioned above.

I'm not sure what you mean with the "similar to the rust-ctrlc crate".
If it is true that only the function defined in SetConsoleCtrlHandler runs and everything else will be paused indefinitely, then there is nothing we can do. This is for async systems, and if the async reactor stops running, then we cannot work around that, unless I misunderstand something here.

#[cfg(windows)]
let mut signal = tokio::signal::windows::ctrl_close().expect("Failed to create signal handler");

let ctrl_c = tokio::signal::ctrl_c();

// Keep the program running and handle signals
tokio::select! {
    _ =
        signal.recv()
     => {
        let _ = File::create("empty_async_signal.txt");
        cleanup_ports(&gateway, external_port);
    }
    _ = ctrl_c => {
        let _ = File::create("empty_async_ctrlc.txt");
        cleanup_ports(&gateway, external_port);
    }
}

As an example, I initially tried to run this section of code. In case of ctrl_c, the whole code is run and the program shuts down properly. but in the ctrl_close case, only the file creation was executed, and not the cleanup_ports function. If I commented the file creation line, then cleanup_ports executes but only partially, not fully (probably because the thread is frozen before it has time to finish).

Again, if this is how it works, I'm not sure how to handle it correctly. This mechanism of windows might simple be incompatible with async programs then.

I created a small repo to test this:
https://github.com/Astlaan/rust-ctrl-close-handler/blob/main/src/main.rs
It seems I was wrong. The main rust thread is not frozen when the handler function is called.
To test it, you have to do cargo build and then double click on the executable on windows (not sure if that's your OS?)
The main thread saves a file every second. Then click on the X button to close, and you can see that not only the handler starts saving the files (prefixed by m_), but the main thread also doesn't stop saving them.
So it seems the async app can still cleanup after itself directly. I think you only have to set up a sleep in the handler, so that the handler doesn't return before the time is up. Or, more correctly, some sort of flag to that the handler thread knows when it is time to return.
Ofc, it should be clear to the user that the app only has 5 seconds to clean itself before being forcefully terminated.
As for the Gui apps, they can be fully without using this console handler. I.e. using the same method you used so far. Should be easy to extend support, probably you only have to add listening for signals relative to GUI apps.

I'm unsure what you mean with "set up a sleep in the handler". Which handler are you referring to?

For reference, the function that you are referring to is probably this one:
https://github.com/tokio-rs/tokio/blob/10e23d1c621ab38aadf2cefba1120494cff615f0/tokio/src/signal/windows/sys.rs#L109
I don't have any control over this function, this is deeply embedded in tokio itself.

So do you think this change was a mistake from my side?
https://github.com/Finomnis/tokio-graceful-shutdown/commit/3ac73fe24eb6aaaef478f7a4e7f8e2bf81af2d08#diff-6c2f9810ae3295fc04958cc626d918ac033edf18290ed77858201ab570c538d8R34
Should I revert CTRL_CLOSE and CTRL_SHUTDOWN?
I don't think I can properly write code that handles those two; the only option left might be to not handle them at all and have the user of this crate take control over this himself. Although that would probably collide with tokio's implementation; you might be correct that this problem needs to be escalated to the tokio devs.

Lets see. In the tokio file that you sent, it seems that the handler function simply registers an event and then returns right away. Which would cause the program to be shut down.
unsafe extern "system" fn handler(ty: u32) -> BOOL {
    let globals = globals();
    globals.record_event(ty as EventId);

    // According to https://docs.microsoft.com/en-us/windows/console/handlerroutine
    // the handler routine is always invoked in a new thread, thus we don't
    // have the same restrictions as in Unix signal handlers, meaning we can
    // go ahead and perform the broadcast here.
    if globals.broadcast() {
        1
    } else {
        // No one is listening for this notification any more
        // let the OS fire the next (possibly the default) handler.
        0
    }
}

According to:
https://learn.microsoft.com/en-us/windows/console/handlerroutine
It seems that if 1 is passed, it means that the handler handled the shutdown, and it is ready to be shutdown. If 0 is passed, "next handler function in the list of handlers for this process is used". Also from the https://learn.microsoft.com/en-us/windows/console/setconsolectrlhandler :

When a console process receives any of the control signals, its handler functions are called on a last-registered, first-called basis until one of the handlers returns TRUE. If none of the handlers returns TRUE, the default handler is called.

But in the function, they call broadcast() which "Returns true if an event was delivered to at least one listener". It seems that the way tokio implements it doesn't even give a change for the program to do anything with the signal, since it broadcasts it to the listeners, and if they exist, the handler function returns a true, which would terminate the program right away?

PS: Apparently I also wasn't able to get it to work with the ctrlc crate.
https://github.com/Detegr/rust-ctrlc/issues/122#issue-2742433547

Not sure how to respond now; if you suspect that this is a Tokio issue, your should report it there. I can't really change it here.


Add support for using remote libvirt instances.

This is primarily useful for the case where users wish to run the Factory within a virtual machine but are able to grant the Factory the ability to create additional VMs on an external infra like RHEV-M or vSphere.
Both Oz and Factory have hard-coded assumptions that virt image files are local and that the libvirt VMs are run under KVM.
Some adventurous users have demonstrated that we can partly work around this by accessing remote RHEV-M image storage via NFS shares.  However, in the general case we may have to allow for the case where we build an image remotely then move the image content back to the Factory server for further modification.
This further modification, both by Oz and within the Factory, uses libguestfs.  The libguestfs code gracefully deals with a lack of fast local virt (and it works).  However it is of course slower than it would be in the presence of KVM.  As far as we know, there is no "remote API" mechanism for libguestfs.  We need to use it and it needs to run its "mini guest" locally.
An entirely different approach to this issue would be to actively investigate "nested" virtualization.  "Nested KVM" (that is, kvm running efficiently within a kvm guest) is in active development.  VMWare also claim some level of efficient support for nesting hypervisors.

Bump - This remains relevant and is an RFE worth trying to work on.