Manual routes management via API

I have a use-case where a large number of hostnames (or subdomains) (dozens to a few hundreds) need to be set for a particular consul service. While the number of hostnames is a problem and unwieldy to put into the job file, the hostnames can be added and remove programmatically as customers are provisioned.
I couldn't find any documentation on how to do this via the API, but figured out GET/PUT to /api/manual on the fabio UI port works for this.
I have a request similar to https://github.com/fabiolb/fabio/issues/364#issuecomment-332803388 where more control over adding and removing routes for a given service would be ideal. I looked for a DELETE operation on the API to use when a hostname is added or removed programmatically, but one does not exist.
I also need to provide a route between a host (foo.example.com) and the service (app) when hostnames are added or removed programmatically. Is this supported?
Is this the correct syntax?
route add app foo.example.com/ app

The idea of the config language approach is that the routing table has a human readable text format where multiple fragments are just concatenated and parsed into a runtime representation. Right now, one fragment is generated from the service tags in consul and the other fragment comes from /fabio/config in the consul KV store. This approach makes parsing simple and robust but doesn't lend itself to API driven partial updates.
Another reason there is no real fabio API for this is that consul already provides an API and a client to manage all this. The idea of the API is from the time when I wanted to support multiple backends besides consul. Given external constraints that doesn't seem to be happening any time soon.
My suggestion is to write a script that generates the routing table and store that under /fabio/config in the consul KV store instead of, or in addition to using service tags. Update the full routing table on every change. This keeps the process simple.
This could be as simple as
routes.py | consul kv put fabio/config

Depending on the size of the routing table there may be some space and/or performance issues. See #343. A single consul KV entry can hold only 512KB and it takes some time to parse lots of them.
The first problem can be addressed by supporting multiple nodes or treat /fabio/config as a directory and parse all subnodes in order. For the parsing speed I need to write a better parser.
The routing syntax is
# host:port is the address app listens on
route add app foo.example.com/ http://host:port

See: https://github.com/fabiolb/fabio/wiki/Routing#config-language
Please let me know if that helps and whether the consul space constraint is an issue for you.

To find out on which ip addresses and ports app is running you can run a consul or DNS query
dig @localhost -p 8600 srv app.service.consul +short

curl -i localhost:8500/v1/catalog/service/fabio

Depending on how you implement this you could use the consul watches for this:
https://www.consul.io/docs/agent/watches.html

Thanks @magiconair!
The current design certainly makes sense and I like the suggestion of regenerating the routing table with a script in consul. Will try that.
Using multiple consul kv nodes under fabio/config is really interesting and has solid semantics for those of us that work with Consul regularly. I don't see the 512KB as a limiting factor right now.
However, having to watch/manage the dst (http://host:port) is probably the bigger problem for my use case. I think of this more as a "manual route" or a "static route" vs an "override" and a way to provide the same routes as the urlprefix- tags we use when registering consul services.
It sounds like this is not supported today because the dst needs to be a URL and I am trying to use the service name (for dynamic IP:PORT lookup from consul):
route add app foo.example.com/ app

Would this be possible?:
route add app foo.example.com/

It says send requests to foo.example.com/ to the service named app in consul.

Now I get it. fabio could allow you to provide a template which fills in the host:port so that you just provide the service id and fabio pulls the address itself. That's an interesting case and shouldn't be too hard to add.

@magiconair Yup, I think so! I was just trying to replicate what we do in consul service tags with urlprefix to wire up routes.

Incremental route updates
The issue with incremental route updates is that they are unbounded. So just
keeping a list of all changes and applying them every time won't work.
fabio uses Consul for service discovery and coordination so pushing an
external routing table into a separate Consul KV node seems like the simplest
thing to do.
Consul has a 512KB limit on KV store nodes which translates into 4-5k targets
uncompressed. With gzip this can probably be pushed 2-5x so you'd end up with
10-25k routes of an external routing table. And then you can always have
multiple KV nodes to get around that limit as long as your Consul cluster can
keep up with the write rate.
Depending on the size of your routing table this may also require fixing #343.
Generating route targets
If you generate an external routing table but still want to refer to services
in Consul then fabio can generate them for you. However, the current list of
services and their addresses is not stored and hidden in the registry/consul
package.
I can see several approaches:

The registry/consul package caches that list and exposes it.
This would tie this approach to the consul implementation

Services are defined separately as part of the routing table which extends the config language.
This would decouple the routes from the services. A bit like normalizing an SQL data model.

service svc-a <IP_ADDRESS>:5000 proto=http
service svc-a <IP_ADDRESS>:5000 proto=http
service svc-a <IP_ADDRESS>:5000 proto=http
route add svc-a /foo
route add svc-a /bar

fabio expands route add defintions without a target to all known targets.
This would be shorter but feels like magic and also depends on order.

route add svc-a /foo http://<IP_ADDRESS>:5000
route add svc-a /foo http://<IP_ADDRESS>:5000
route add svc-a /foo http://<IP_ADDRESS>:5000
...
route add svc-a /bar # would expand to three previously defined targets

I think the last option is the last invasive but the second option looks like the cleanest solution.
We could do the last option first as a stepping stone to the second option.

@magiconair https://github.com/fabiolb/fabio/pull/417 looks good to get around the consul 512KB limitation.
Option 2 above ("Services are defined separately as part of the routing table") looks interesting.
This would allow a service to be targeted for routing even if it was not in the consul service catalog, right?
If svc-a is already registered in consul, could it be defined like this?
service svc-a http://${ADDR}:${PORT} proto=http
or
service svc-a :${PORT} proto=tcp

Is some kind of a marker needed for the parser to target?
service svc-a consul:http://${ADDR}:${PORT} proto=http
or
service svc-a consul::${PORT} proto=tcp

Option 1 above (registry/consul) seems really interesting even in light of tighter coupling to consul. This could be a competitive and usability advantage (vs something like nginx+, etc.) if I understand what you are proposing.
Stepping back for just a minute...
We have access to really good tooling (git2consul, etc) these days for using consul KV as a configuration store backed by git repositories and have already worked out handling of multiple-regions and multiple environments.
If Fabio gave us the ability to define all routes in consul and also use consul services by name, I would try to use it in lieu of the urlprefix tags.
Here is a real-world example of an api service block in a nomad job that we use within 2 different regions and 2 environments:
service {
    name = "api"
    port = "http"
    tags = [
        "http",
        "urlprefix-api.example.com/",
        "urlprefix-api.us1.example.com/",
        "urlprefix-api.eu1.example.com/",
        "urlprefix-api.example-dev.com/",
        "urlprefix-api.us1.example-dev.com/",
    ]
}

I'm not sure how other users are handling this. Maybe they are templating the job file on submit to nomad, or outside of nomad, have some business logic that provides the appropriate routes as tags for the region and environment. I'm just registering everything for convenience, but it adds weight to the routing table and feels a bit clunky. I also might want the developer of the service to not have to worry about the routing if another team is handling it.
As we discussed earlier in this issue, the urlprefix routes shown are just the static ones that are always present. Subdomain and CNAME routes can be added, changed, or removed for one of our applications dynamically and that's where idea of having Fabio route to healthy members of a service by service name comes in.
Ideally, these route commands (all stored in fabio/config/api or similar) could replace the urlprefix tags from the example and handle the subdomain/CNAME routes:
route add svc-a api.example.com/ tags "http"
route add svc-a api.us1.example.com/ tags "http"
route add svc-a subdomain.example.com/ tags "http"

These route commands omit the <dst> parameter, which is currently required by the parser.
As I understand it, the remaining piece is how to use consul service entries when <dst> is not provided, or as you noted, allow some kind of templating that indicates the service host and port should be obtained from consul.
I prefer the lack of a <dst> parameter to indicate a dynamic lookup, but the templating version could look like:
route add svc-a api.example.com/ http://${ADDR}:${PORT} tags "http"
route add svc-a api.us1.example.com/ http://${ADDR}:${PORT} tags "http"
route add svc-a subdomain.example.com/ http://${ADDR}:${PORT} tags "http"

The goal of the config language is to provide a textual representation of the routing table so that humans and machines can read it. Otherwise, manual changes would require a different syntax. The urlprefix- tag is already a different representation of the same route add command with a slightly different syntax.
urlprefix-/foo strip=/foo

becomes
route add svc-a /foo http://<IP_ADDRESS>:5000 opts "strip=/foo"

When I've developed fabio, nomad didn't really exist and we've deployed self-registering services. The idea was that services know which routes they can handle and by deploying them you make them routable. This way you never have to store a routing table in a separate system (e.g. Puppet) and coordinate deployment of the routing table with the deployment of the service. (routing table before services or after? How to handle roll-back?) You can just forget about this part altogether.
However, different setups require different solutions and if you want to decouple service discovery from defining the routes then having the config language being able to express that would help.
This is why I think option 2 is the cleanest solution since it decouples the two concepts which also means that you can update the service list without knowing about the routing and vice versa.
urlprefix-/foo strip=/foo

becomes
service add svc-a <IP_ADDRESS>:5000
route add svc-a /foo opts "strip=/foo"

Now you can easily integrate different service discovery mechanisms or update the routing or do both. However, when you update the routing manually you have to coordinate this again for every deployment.
I don't think that there would need to be marker to the service name itself but it may be helpful to indicate the origin of the service definition, e.g.
service add svc-a <IP_ADDRESS>:5000 src=consul

Ah, I think I've closed that one prematurely since this refers to multiple issues. #417 only addresses the 512kb limit. @stevenscg shall we pick the discussion up again if this is still relevant?


chore: update validator management policy to be parallel

Fire these puppies up in parallel, faster

#9087
#9086  👈
master

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @Maddiaa0 and the rest of your teammates on  Graphite

Merge activity

Oct 8, 2:12 PM EDT: @Maddiaa0 merged this pull request with Graphite.


GH1775: Fix CakeScript tool resolution

Address Executing assembly issue
Add support for Cake script on Cake.CoreCLR
Remove no longer needed special .NET Core ToolPath in integration tests
fixes #1775

@gep13 rebased.

@cake-build/cake-team rebased.

@devlead your changes have been merged, thanks for your contribution 👍


Add Reproduction Results for Anserini Repo Steps

Environment Details:
-Operating System: Ubuntu 22.04.4 LTS
-Java Version: Upgraded to JDK 21 from JDK 11 to resolve compatibility issues that were initially causing build failures.
Experience:
The process followed the outlined steps without significant issues, except for an initial JDK version mismatch. Once the JDK was updated to version 21, all scripts executed flawlessly, and the expected results were successfully reproduced. The step-by-step guidance provided was clear and easy to follow.
Suggestions for Improvement:
Considering the JDK version mismatch issue I encountered, it could be beneficial to include a specific note about verifying JDK version requirements in the setup documentation, similar to the approach taken in the Pyserini documentation.

Environment Details: -Operating System: Ubuntu 22.04.4 LTS -Java Version: Upgraded to JDK 21 from JDK 11 to resolve compatibility issues that were initially causing build failures.
Experience: The process followed the outlined steps without significant issues, except for an initial JDK version mismatch. Once the JDK was updated to version 21, all scripts executed flawlessly, and the expected results were successfully reproduced. The step-by-step guidance provided was clear and easy to follow.
Suggestions for Improvement: Considering the JDK version mismatch issue I encountered, it could be beneficial to include a specific note about verifying JDK version requirements in the setup documentation, similar to the approach taken in the Pyserini documentation.

Where in the documentation should we clarify?
We already have these in the README:

Where else would you suggest we update the docs?


[GHSA-42xw-p62x-hwcf] Improper Access Control in Apache Derby

Updates

References

Comments
Add two patches:
https://github.com/apache/derby/commit/a2027c64e185a9ce46929f352e2db03371c1f95
https://github.com/apache/derby/commit/4da5b2db5f3a60c1fa8ef616d88a7efe28b0c9d.
the commit msgs show their intention.

👋 This pull request has been marked as stale because it has been open with no activity.  If you do nothing this pull request will be closed eventually by the stale bot. Please see CONTRIBUTING.md for more policy details.

👋 This pull request has been marked as stale because it has been open with no activity.  If you do nothing this pull request will be closed eventually by the stale bot. Please see CONTRIBUTING.md for more policy details.


Tests that visit multiple pages in a single test get stale data from the IOC

If you create a test that visits multiple pages, like the example below, on the second request many objects in the IOC are still hanging around from when they were created on the first page. This can have side effects.
Example unit test:
// visit a page (first request)
$this->visit('/example');

// do something that results in a second page request
$this->submitForm('Do Something', ['email' => 'bad email address]);

// check I'm back on example page, but I can see errors on the form
$this->onPage('/example');
$this->see('Some error message about bad email address');

In my case I had a form builder class. It's constructor was passed the request, which it used to access the flashed errors in the session. On the second page request, my form builder from the first page request was still in the ioc and was returned. This referenced 'out of date' session error data, resulting in the tests failing (no error messages were displayed, as there were no form errors in the first request).
I think everything in the ioc would need to be cleared out when the test framework attempts to start a new page request to ensure there are no dependencies on the original request or session in existence.

You need to refresh the application.

I think this is a valid issue.
Before following each redirect, the app instance should be refreshed by default.

I don't agree. That could kill performance. We don't want to refresh twice.

Then your test is not reflecting normal behaviour.
In a normal user-agent, you will get a fresh application for each request.

This of course will break other aspects of the tests, like sessions.
But I believe we should think of a way to mitigate this.

(1) For very specific tests, you can use:
$this->refreshApplication();

This resets the application state between each request, ensuring that no data or headers persist across calls.
(2) However, if possible, it’s better to split the requests into separate focused tests. This approach keeps your test suite cleaner and more maintainable.


AntPathMatcher Bug

Prerequisites

[X] I have checked the FAQ, and issues and found no answer.

What version of pinpoint are you using?
master/v1.7.3/v1.7.2/1.8.X etc.
Describe the bug

com.navercorp.pinpoint.bootstrap.util.spring.AntPathMatcher.AntPathStringMatcher#matchStrings

org.springframework.util.AntPathMatcher.AntPathStringMatcher#matchStrings

The bug in the picture above is clear and obvious , the judgement should be if (this.variableNames.size() != matcher.groupCount()).

What did you do to trigger the bug?
The Bug will be trigger by calling this method com.navercorp.pinpoint.bootstrap.util.spring.AntPathMatcher#extractUriTemplateVariables.
Expected behavior
As described.

Ohh, There is another Bugs in AntPathMatcher .
    @Override
    public Map<String, String> extractUriTemplateVariables(String pattern, String path) {
        Map<String, String> variables = new LinkedHashMap<String, String>();
        boolean result = doMatch(pattern, path, true, variables);
       // Should Be if(!result)
        if (result) {
            throw new IllegalStateException("Pattern \"" + pattern + "\" is not a match for \"" + path + "\"");
        }
        return variables;
    }

Ohh, There is another Bugs in AntPathMatcher .
    @Override
    public Map<String, String> extractUriTemplateVariables(String pattern, String path) {
        Map<String, String> variables = new LinkedHashMap<String, String>();
        boolean result = doMatch(pattern, path, true, variables);
       // Should Be if(!result)
        if (result) {
            throw new IllegalStateException("Pattern \"" + pattern + "\" is not a match for \"" + path + "\"");
        }
        return variables;
    }

@geekMessi Hi~
First of all, thank you for your First of all, thank you for your contribution..
I analyzed this bug in detail.
This is a bug in Spring, but pinpoint is not affected by this bug.
pinpoint does not use TemplateVariable, so it is not affected.
https://github.com/naver/pinpoint/blob/master/bootstrap-core/src/main/java/com/navercorp/pinpoint/bootstrap/util/AntPathMatcher.java#L53
    @Override
    public boolean isMatched(String path) {
        if (path == null) {
            return false;
        }
        return this.springAntMatcher.match(pattern, path); <-- Pinpoint uses only this api.
    }

https://github.com/naver/pinpoint/blob/master/bootstrap-core/src/main/java/com/navercorp/pinpoint/bootstrap/util/spring/AntPathMatcher.java#L160
    @Override
    public boolean match(String pattern, String path) {
        return doMatch(pattern, path, true, null);  <--- always null
    }

    protected boolean doMatch(String pattern, String path, boolean fullMatch, Map<String, String> uriTemplateVariables) {
        if (path.startsWith(this.pathSeparator) != pattern.startsWith(this.pathSeparator)) {
            ret
        //...
     }


docker container doesn't have permission to access customdashboards.yml

I ran docker-compose and got hit with this error:
logger=server t=2023-01-26T04:28:27.37+0000 lvl=eror msg="Server shutdown" error="*provisioning.ProvisioningServiceImpl run error: Failed to create provisioner: Failed to read dashboards config: could not parse provisioning config file: customdashboards.yml error: open /etc/grafana/provisioning/dashboards/customdashboards.yml: permission denied", *provisioning.ProvisioningServiceImpl run error: Failed to create provisioner: Failed to read dashboards config: could not parse provisioning config file: customdashboards.yml error: open /etc/grafana/provisioning/dashboards/customdashboards.yml: permission denied
Here's my grafana service in docker-compose:
  grafana:     image: teslamate/grafana:latest     restart: always     environment:       - DATABASE_USER=teslamate       - DATABASE_PASS=teslamodely2023!       - DATABASE_NAME=teslamate       - DATABASE_HOST=database     ports:       - 3000:3000     volumes:       - teslamate-grafana-data:/var/lib/grafana       - ./Teslamate-CustomGrafanaDashboards/customdashboards.yml:/etc/grafana/provisioning/dashboards/customdashboards.yml       - ./Teslamate-CustomGrafanaDashboards/dashboards:/TeslamateCustomDashboards
I checked the perms on the Teslamate-CustomGrafanaDashboards dir and verified that all the files have full permissions.
What gives?

For those who has the same issue, try this: https://community.grafana.com/t/new-docker-install-with-persistent-storage-permission-problem/10896/5
I tried solution 2, it worked.

Hey! I'm glad you found a solution for this. It usally happens when your user (uid) is not the same as the user running docker service (with no privilegies on your home path). Other solution for this, is to use an absolute path where the users have the correct permissions as explained here https://github.com/jheredianet/Teslamate-CustomGrafanaDashboards#using-other-path-for-repository
Cheers!


Add VS2019 to CI

This PR adds Visual Studio 2019 to the CI.

Hi @amyspark! Since you seem to be pretty knowledgeable on SIMD programming and the MSVC toolchain, you may have an idea what breaks the Visual Studio 2019 builds here. There are only two failures:

one is in Vc::exp having a larger error now. You had a look at Vc::exp recently as well, so I figured you might also know what is going on here. It might also just be different floating point optimizations done by MSVC. The new results are a few ULPs off.
the other is a utility function. And it just fails for SSE.

If you have any clue on what is happening here, I would be super happy! Thank you very much!

@bernhardmgruber Vc::exp's float implementation is a direct port of the double-precision one, which needs (to begin with) an adjustment to its boundary values. (This is in #295.) The rest, I can have a look at them later!

@bernhardmgruber, re the Vc::exp failures,  it seems there's an issue with either the implementation on common/exponential.h, Vc::ldexp, or both. On my end, I've needed to enable /fp:strict and replace the ldexp call with the following, to make those tests pass:
        for (size_t i{0}; i != x.size(); i++) {
            x[i] = z[i] * std::powf(2, n[i]);
        }
        // x = ldexp(z, n); // == z * 2ⁿ

I'll continue later with the utils error.

@bernhardmgruber codegen bug, please insert AddCompilerFlag("/d2SSAOptimizer-") in the test suite flags for MSVC. Reported here: https://developercommunity.visualstudio.com/t/AVX-codegen-bug-on-Vc-with-MSVC-2019/1470844?space=62&entry=myfeedback

@bernhardmgruber, can you trigger a CI check again? I've been able to confirm Microsoft's assertion that the SSE bug is fixed in 16.10.4.

I see we now have a clean fail on MSVC, I'll check on those during the week.

AVX failures are confirmed and already marked as "under investigation" by MS.
With the same compiler version, I cannot reproduce the SSE failures here. I can only think it may be down to the runner's CPU vendor -- @bernhardmgruber, do you have an Intel box to test with, since mine is a Ryzen 1st gen?

Note that the AppVeyor runs do complete successfully, although with a slightly older MSVC; it's another inconsistency to tally...

Note that the AppVeyor runs do complete successfully, although with a slightly older MSVC; it's another inconsistency to tally...

AppVeyor completes successfully, because it builds with VS2017. Btw.: I remove AppVeyor (and travis) from the CI in #293.

I am starting to wonder if the failing unit tests can ever be correct. There is no requirement on the precision of std::exp. I just dumped the reference results of the failing test math_avx.testExp, which call std::exp on random numbers, into text files with VS2019 and GCC11. They are different. So if the implementation of std::exp is different in libstdc++ than in MSVC, how can a single Vc::exp compare equal to the reference results with two different standard libraries?
Maybe we should just declare that Vc tries to reproduce the results of libstdc++ and set much higher epsilons in the unit tests when a different standard library is used.

I am starting to wonder if the failing unit tests can ever be correct. There is no requirement on the precision of std::exp. I just dumped the reference results of the failing test math_avx.testExp, which calls std::exp on random numbers, into text files with VS2019 and GCC11. They are different. So if the implementation of std::exp is different in libstdc++ than in MSVC, how can a single Vc::exp compare equal to the reference results with two different standard libraries?
Maybe we should just declare that Vc tries to reproduce the results of libstdc++ and set much higher epsilons in the unit tests when a different standard library is used.

If any precision issues arise with MSVC (especially in the presence of /fp:precise and lower), those should be reported ASAP. I know they've been running into many bugs with their SSA optimizer in 2019.

If you can share the Godbolt, I can download and run it here. I've got 2017, 2019, the 2022 preview in my box, and also various MinGW flavors for further comparison.

If any precision issues arise with MSVC (especially in the presence of /fp:precise and lower), those should be reported ASAP. I know they've been running into many bugs with their SSA optimizer in 2019.

I reached the point where I really start to believe this can be a compiler bug now. I tried printing the constants where the tests would fail and ended up here:
int main() {
    const auto v = 0x1.26f8f4p+3f; // 9.2178897857666015625f
    const auto r = std::exp(v);
    printf("%f %a", r, r);
    // gcc               10075.780273
    // VS2019            10075.780273

    const auto vv = Vc::Vector<float, Vc::VectorAbi::Avx>(v);
    const auto rr = Vc::exp(vv);
    printf("%f %a", static_cast<float>(rr[0]), static_cast<float>(rr[0]));
    // gcc               10075.780273
    // VS2019            10075.780273
}

That is all correct. But if the float value 0x1.26f8f4p+3f occurs in the unit test math_avx/testExp<simd< float, AVX>>:
    for (size_t i = 0; i < 100000 / V::Size; ++i) {
        V x = (V::Random() - T(0.5)) * T(20);
        V reference = x.apply([&b](T _x) { return std::exp(_x); });
        FUZZY_COMPARE(Vc::exp(x), reference) << ", x = " << x << ", i = " << i;
    }
    // VS2019 unit tests Vc::exp = 10075.77539, reference = 10075.78027

the result is wrong. While playing around, I introduced the following side effect:
    for (size_t i = 0; i < 100000 / V::Size; ++i) {
        V x = (V::Random() - T(0.5)) * T(20);
        V reference = x.apply([](T _x) {
            if (std::abs(_x - 9.21789) < std::numeric_limits<T>::epsilon() * 10)
                printf("");
            return std::exp(_x);
        });
        FUZZY_COMPARE(Vc::exp(x), reference) << ", x = " << x << ", i = " << i;
    }
    // VS2019 test passes

This should not change the result, but it somehow makes the test pass. So I guess the call to printf somehow puts a barrier into the instruction flow for the optimizer and the bug is not triggered.
I don't know how to continue from here. @amyspark if you want you can try to reproduce this even better and file a bug report to MS. Otherwise, I am also fine to just live with the issue and increase the error margins for MSVC in the unit tests. We can revisit this in the future and see if a future version of VS fixes this.

@bernhardmgruber I'll certainly work on a test case for MS. Did you check with the /d2SSAOptimizer- flag? You could add it to the tests if(MSVC AND CMAKE_CXX_COMPILER_VERSION VERSION_GREATER_EQUAL 19.20).

Did you check with the /d2SSAOptimizer- flag?

I just tried that and indeed, the tests pass successfully then.

I just updated the bug report at Microsoft's with the new test case.

LGTM 🎉

@bernhardmgruber, Karen Huang from Microsoft needs further information as to the processor's specs you are using.
As I cannot reproduce this anymore on my Ryzen box, perhaps you can comment in the bug report?

@amyspark I commented on the bug report. Thx raising my awareness!


Fuzzy Finder not working with WSL2 directory

Oni Version:
0.3.6
Neovim Version (Linux only):
n/a
Operating System:
Windows 10 Home Insider Preview Build 18922
Issue:
I am using the new Windows Subsystem for Linux (WSL2) on Windows and when I open a linux directory in Oni, the Fuzzy Finder menu does not work.
Expected behavior:
Type ctrl+p with a linux directory open. I should be able to see the files in the open directory and type the names of files to find what I want.
Actual behavior:
The Fuzzy Finder menu shows files outside of the opened directory, from the Windows system drive. The shown files change each time the menu is opened. The menu shows a loading spinner at the bottom. The menu becomes unresponsive. I cannot find any files from the opened directory.
Steps to reproduce:

Install a WSL2 operating system - Ubuntu in this case
Open Oni
Press ctrl+shift+p and select Workspace: Open Folder
Navigate to (for example) \\wsl$\Ubuntu\home\pete\my-project
Press ctrl+p to open the Fuzzy Finder menu

Mapping a network drive gets around this issue.


Composition for test methods within the UI

The tech writers (internal or external) will need a convenient way to draft test methods and verify they work.
In the short term this can be done by converting the test view to an editable text area. In the future, we may want to give people a more convenient way to do it.

Fixed by #927.


Bump targetSdk version to 34

Overview

After August 31, 2024, any app releases or updates will require targetSdk version 34, but this app has targetSdk version 33.
The definition can be found here.

https://github.com/DroidKaigi/conference-app-2024/blob/9b090cfd47b6427c9d3e55da618701d7738ddcd6/build-logic/src/main/kotlin/io/github/droidkaigi/confsched/primitive/AndroidGradleDsl.kt#L38

We should be able to launch the app without crashing. However, it doesn't need to be perfect.

Link

https://support.google.com/googleplay/android-developer/answer/11926878?hl=en&sjid=11494528915384632195-AP

🙋♀

Thanks! Assigned 👍


add a Lazy EMPTY_ACCOUNT as pub static

Description
This pull request introduces improvements inspired by the Ethereum execution specification:

Introduces a Lazy pub static EMPTY_ACCOUNT, initialized on first access, which is now utilized in the is_empty method for more efficient account emptiness checks.
Unit tests have been added to validate the correctness and functionality of the enhanced is_empty method.

These changes enhance the codebase by optimizing account emptiness checks and providing thorough test coverage.

I don't think this is correct, because now the bytecode_hash field must be Some for is_empty to return true. this is likely why the is_empty function is not just a comparison with a const value in the first place.

@Rjected My bad, I had forgotten this case. I just fixed it, if you agree I think we can merge this because I put the unit tests to cover these two cases of hash None and KECCAK_EMPTY to no longer cause this confusion.


Add templates module and update version

This pull request adds a new templates module and updates the version to 0.4.1. The templates module provides a function to get prompt templates with customizable input variables. This update enhances the functionality of the PromptLayer library.

I am approving this, but one thing that comes to mind here is if we do the input_variable parsing in template.get and then we require the customer to do a track.prompt it seems like a bit of redundant work on the customer end.

For that we need a request_id. Do you think we should expose a new function like templates.run which does all of this automatically? Basically library version of evaluations? With the new system it becomes extremely easy to do it.


How to decide the focal length

Hi, I was wondering how should we decide focal length when we are in various situation, for example sometimes very close or far away
And here in ROMP, we rescale the image into 512. When we apply to a different camera which from what I see the focal length is [1265, 1265] while the image has different size. If I want to project the 3D mesh onto the image,. How should I change the focal length?

Hi, long time no see~
I am seeking the solution to get the camera parameter estimation involved too. I guess maybe SLAM / SFM - related method can help with that.
The focal length f is calculated as
f = (H / 2)  / (tan(FOV/2))

where H is the height of image, and FOV is the field of view (angle).
You can first calculate the FOV with given focal length (like [1265, 1265]) and original image height H,  and then re-calculate the f for our H=512.

Hi, long time no see~ I am seeking the solution to get the camera parameter estimation involved too. I guess maybe SLAM / SFM - related method can help with that.
The focal length f is calculated as
f = (H / 2)  / (tan(FOV/2))

where H is the height of image, and FOV is the field of view (angle). You can first calculate the FOV with given focal length (like [1265, 1265]) and original image height H, and then re-calculate the f for our H=512.

Great! Thanks for that.
By the way, does your image.py support multi-GPU? I think the logic is only for single gpu and if we use multi-gpu, there will be errors. I will try to fix this.
And, does save_mesh_stand_on_img support running on server without desktop? My laptop is Windows so it's sometimes annoying to get the environment set up, for which I prefer to run it on server. But it would crash with only one message :"aborted" before we call image_result = plt.show(camera=camera_pose) in vedo_visualizer.py

Why does it not support multi-GPU? Anything wrong in logic?
save_mesh_stand_on_img uses vedo for rendering, which need visulization hardwares which most servers don't have.


Improve Documetation in Documentation folder

There iare few markdown files in the documentation folder. It would be great if its content is better designed and improved.
Just a suggestion - you can add a table to LanguageSupport.md file similar to this one - (link)

Hey Soham, I'd like to work on this! could you please assign me this?

Ya sure!
Comment here if any help is needed.

Good day, can you assign me this as well? I could make some edits to your motive.md file. Thank you.

Good day, can you assign me this as well? I could make some edits to your motive.md file. Thank you.

Good day @ThomasGCarpenter . Feel free to solve the issue and make a pull request for it.
Keep hacking!


Create user configuration to set a timeout for debugger connection.

This is a feature request from the Python VS Code extension team, in response to customer feedback.
Due to startup times on various customer setups, the debugger startup time can become too long, and the debugger fails with a popup window like this:

The issue is caused by a combination of:

A customer's terminal can take a large amount of time to startup (Group policy settings, antivirus).
A Python environment can take a little extra time to start up.

See related issues: #60633, Microsoft/vscode-python#2752
Reproduce this issue
VS Code: 1.31.1
Python extension: 2019.2.x
On Windows:

Install the Python extension
Create a workspace, add a single file repro.py, with the following content:

def my_method():
    print(123)
    return 123

if __name__ == "__main__":
    my_method() # place breakpoint on this line.

Set your shell to Powershell (Open the command palette, type Terminal: Select Default Shell).
Open the terminal window (should be Powershell).
Open or create the file described by the $PROFILE environment variable.
Add the following code to the Profile file:

Start-Sleep -Seconds 20

Place a breakpoint on the call to my_method() (line 6).
Ensure all terminal windows are closed by clicking on the Kill Terminal icon until they are all gone.
Press F5 to start the debugger.

If the timeout has not triggered, change the Start-Sleep -Seconds x to a higher value of x.

There is no "Timeout waiting for debugger connection" message anyway in VS Code. Are you sure that this Alert is originating from VS Code and not from the Python extension?


Unchecked cast causes crashes with models that do not implement HasPieces

Pluckable Arrows 1.1.2
From https://github.com/itsmeow/betteranimalsplus/issues/293 @Chipotle4
Not sure why this crash happened considering ModelReindeer extends EntityModel and it appears you have a mixin, but there's a crash here
https://github.com/DeflatedPickle/PluckableArrows/blob/91b9756d1edb04219eaab686c1dd56ace9748433/src/main/kotlin/com/deflatedpickle/pluckablearrows/mojank/StuckObjectsFeatureRenderer.kt#L51
Here's the crash log
https://github.com/itsmeow/betteranimalsplus/files/7947473/crash-2022-01-26_22.43.52-client.txt

Thanks for forwarding this my way, sorry it was reported to you! I believe that 1.1.2 didn't have the EntityModel mixin so the fix might be to update though it's quite an oversight on my part to not check before casting


Explicitly setting VS compiler cores and BuildInParallel to true.

CMake build command Explicitly setting VS compiler cores and BuildInParallel property to true.
The build command would look something like this:
cmake --build . --config Debug --target ALL_BUILD -- /m:10 /p:BuildInParallel=true /p:GenerateFullPaths=true

And would be more consistent with the way we set cores(i.e -j 10) when using any other generators that are not using the VS compiler.

@ca-mtg the hard coded build flags were removed, since the user can manually add them via buildArgs.
Going to close this one for now.


ISSUE-544 add call unique id to handler kwargs

At the level of ChargePoint.handle_call, currently the library only passes the message payload to the _on_action, and _after_action handlers.
For some features like logging the messages being treated by those handlers, it would be nice to pass the Call.unique_id as well.
:warning: This will be a breaking change though as all the handlers defined by the library clients should always define call_unique_id in the kwargs, hence the label For inclusion in release 1.0.0
Issue Link: https://github.com/mobilityhouse/ocpp/issues/544

I share your need for allowing users to access the request id. Currently, this library is lacking at that point.
But I'm hesitant about this PR. It is a breaking change to the main API: the handler.  So potentially, it breaks a lot of code for users of this library.
I'm thinking out loud here and considering some alternative implementations:

Would it be possible to inspect the interface of a handler and only pass the unique id to the handlers when an argument unique_id is part of the signature?
Alternatively, in Flask, users have access to the request context. Maybe we can do something similar using ContextVars..

I'm going to be unavailable during the Holidays, so I might respond in a few weeks.

I share your need for allowing users to access the request id. Currently, this library is lacking at that point.
But I'm hesitant about this PR. It is a breaking change to the main API: the handler. So potentially, it breaks a lot of code for users of this library.
I'm thinking out loud here and considering some alternative implementations:
1. Would it be possible to inspect the interface of a handler and only pass the unique id to the handlers when an argument `unique_id` is part of the signature?

2. Alternatively, in Flask, users have access to [the request context](https://flask.palletsprojects.com/en/3.0.x/reqcontext/). Maybe we can do something similar using[ ContextVars.](https://docs.python.org/3/library/contextvars.html).

I'm going to be unavailable during the Holidays, so I might respond in a few weeks.

@OrangeTux makes sense.
I applied the suggestion of checking the signature, should not be a breaking change now.


Clock + Current Weather Widget

Hi there!
I've been using and enjoying your app for the last few days. A modern, maintained and minimal weather app was sorely needed. :)
Would you consider creating a widget that acts like the normal "clock" widget, with the addition of the current weather underneath it?
I understand that the current widget is resizable and displaying more/less information depending on its size, but the clock/weather widget is what a lot of people expect, as that's how a lot of OEMs handle it.

Hello! Thank you for the suggestion and the kind words, I'm very glad you're enjoying the app :)
I will definitely consider adding this, it seems natural that a weather app should have a combined clock widget.


[BUG] Unstable performance of SAML integration test case in CI

What is the bug?
SAML integration test case is unstable during the CI run
How can one reproduce the bug?
By running the integration test in CI.
What is the expected behavior?
The performance of SAML integration test case should be more stable.
What is your host/environment?
Github Actions
Do you have any screenshots?

@RyanL1997 This looks like a duplicate of an issue I just opened - would you like to pick one that we use to track this issue and close the other one as a duplicate? https://github.com/opensearch-project/security-dashboards-plugin/issues/1193

Aside - I love that you immediately filed an issue on this test failure!

Closing this now. Linking it to #1193 for tracking the same issue.


Update chart grafana to 8.0.2

This PR contains the following updates:

Package
Update
Change

grafana (source)
patch
8.0.1 -> 8.0.2

Release Notes

grafana/helm-charts (grafana)
v8.0.2
Compare Source
The leading tool for querying and visualizing time series and metrics.
What's Changed

[grafana] Adding configmap annotation by @jmiszczak83 in https://github.com/grafana/helm-charts/pull/3064

New Contributors

@jmiszczak83 made their first contribution in https://github.com/grafana/helm-charts/pull/3064

Full Changelog: https://github.com/grafana/helm-charts/compare/k8s-monitoring-1.0.13...grafana-8.0.2

Configuration
📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
🔕 Ignore: Close this PR and you won't be reminded about this update again.

[ ] If you want to rebase/retry this PR, check this box

This PR has been generated by Renovate Bot.

Edited/Blocked Notification
Renovate will not automatically rebase this PR, because it does not recognize the last commit author and assumes somebody else may have edited the PR.
You can manually request rebase by checking the rebase/retry box above.
⚠ Warning: custom changes will be lost.


Updated SWAG sync to reference the top-level account status field

More reliable SWAG syncing.

Looks good to me.  Might want to lock the requirements.txt swag-client>=0.4.1


Team Services Plugin

Hi everyone
I just wanted to tell you, that I created a plugin for TFS and Visual Studio Team Services which will create a badge on the build process using shields.io.
You can grab it here: https://marketplace.visualstudio.com/items?itemName=chwebdude.badge-creator
Thanks
Fabrice

👍


reduce use of depset.to_list()

I'm not sure how much of a difference this will make. WDYT?
I noticed copy_to_directory_action consuming 3-5s in the angular-cli analysis phase which I think this will improve? Also just reduces some duplicate code 🤷
EDIT:
Results: seems like this reduces the analysis time of the angular-cli e2e tests by at most 5% 🤷

I don't remember all of the context on this one. Lets go over it together on Monday


[core] Prettier everything possible

Formats all possible files (adds css,html,jsx,md,ts). Closes #14792
@mbrookes Don't know how the translations work. We should make sure before merging new translations that yarn prettier is run before.

Don't know how the translations work.

I'm afraid, running prettier on the source markdown may break the existing translations. Prettier is also messing up indentation on code samples. I'm not sure it's a good idea.

I'm afraid running prettier on the source markdown may break the existing translations.

In what way?

Prettier is also messing up indentation on code samples in the translated files.

Could you point to an example and what you mean by "messing"? I don't want to get into a code style argument. The point of a code formatter is to get rid of those.

In what way?

When a source string changes, Crowdin removes the translation from the target file. Someone has to go into the editor, and either resubmit the previous translation, or create a new one. I'm not sure what changes it will ignore, (e.g. same string with a different line-break mid sentence might be okay?) or which changes require retranslation.
I tagged one with "Oops!". This isn't a code style argument.

In what way?

Matt has more context than me, but I will try to give the essence of what would happen. Crowdin keeps the translation up to date at the expense of non translated content. Meaning, it's better to have documentation up to date than translated. I would expect a single space change to invalidate the translations. So, it might invalidate a large part of the translations. In order to help the translations effort, two tools are at our disposition: MT and TM (machine translation and translations memory). We might be able to recover from the changes, but it's probably a laborious long hand work.

Which is only an issue once, no? Or does it also invalidate if a translated file changes?

Ideally, we would try one markdown change and see how that goes. Maybe we should complete this Prettier effort post v4 release? How much time it's gonna take us to recover from the changes, weeks? The sooner we can leverage the i18n for SEO (next subdomain does not) and rewards all the translators by putting their effort live, the better.

Well it's not like we can't serve the docs without it but looking how it uncovered broken translations I think it is quite useful to get this right before v4.

uncovered broken translations

Who said it uncover broken translations?

uncovered broken translations

Who said it uncover broken translations?

https://github.com/mui-org/material-ui/pull/15341#discussion_r275146336

I think that we should ignore the translated files from the prettier generation. Crowdin detects the changes from the translated files and merges them back to the original translations. In our case, it would create many conflicts. We can't do it (Matt correct me if I'm wrong).

Did you just ask me for proof if it detected a bug and then just ignored that?

Did you just ask me for proof if it detected a bug and then just ignored that?

It's great, we should fix the problem 👍. It doesn't change the other issues we have.

It doesn't change the other issues we have.

So far I only got feedback for one issue that's was not answered by @mbrookes yet.

Which is only an issue once, no? Or does it also invalidate if a translated file changes?

Waiting for resolution here.
What are the other issues?

What are the other issues?

@eps1lon I'm aware of the following:

Reformating the markdown might make use lose a large chunk of the translation effort. Something we can't recover, or partially. This is a transient issue.
Formatting the translated files conflict with the translations done. It might invalidate them. To check with Matt. This is a permanent issue.

If Prettier changes the formating rules every now and then.

What is the evidence for that and why is pinning the version if that happens as a possible fix ignored?

What is the evidence for that

Since you are using Prettier, have you never seen the format change when upgrading Prettier?

is pinning the version if that happens as a possible fix ignored?

Who ignored it? It's something we need to be aware of.

Since you are using Prettier, have you never seen the format change when upgrading Prettier?

Maybe once but I don't care about formatting as long as everybody is on the same page

Who ignored it? It's something we need to be aware of.

Why? CI will tell us.

Why? CI will tell us.

We need to know the consequences of our choice. If we decide to introduce Prettier, we need to know the cost with i18n.

@mbrookes Could you clarify if crowdin invalidates the translation if the translated files change?
It's been weeks now with broken translated docs and I'd like to fix this.

@eps1lon Could you demonstrate that it doesn't?
I suggest setting up a Crowdin trial project, and including a file or files that contain examples of everything that prettier changes. Make sure that there are examples sandwiched mid-sentence and mid paragraph (where appropriate), as well as the beginning and end of a line, especially for lines that will be reformatted.
Translate the sample files, approve the translations, then run prettier against them, and sync the changes. What percentage is still translated, and what percentage is still approved?

Sounds like a plan. Will also check out if crowd-in support some sort of post-processing.

@eps1lon Could you demonstrate that it doesn't?

Test project: https://github.com/eps1lon/crowdin-prettier and https://crowdin.com/project/crowdin-prettier
If you format the translated files it will try to sync them again i.e. if you automatically open PRs if new translations are added it will immediately open a new one after you merged formatted translations.
It will, however, not display the file as not translated. If we don't automatically open PRs with translations we should be fine.

It will, however, not display the file as not translated

But... README-de.md doesn't contain anything to test?

But... README.md doesn't contain anything to test?

I don't follow. There are some files where format is less important, yes. We can limit markdown formatting to docs pages only if that helps.

We can limit markdown formatting to docs pages only if that helps.

I think that we should absolutely avoid formatting the translated markdowns. It's the only exception I can think of.

I don't follow.

We've discussed this already, but just to recap: You wish to use Prettier to format the markdown files. There are two potential side effects:

The impact on the translations of reformatting the source files. If Crowdin interprets the reformatted files as containing new or changed strings, the translations will be invalidated.

The impact of reformatting the translated markdown. Crowdin will see this as a diff from its output format, and resubmit the translations.

The purpose of the proposed test was to determine the impact to the existing translations of the different reformatting "algorithms" that prettier applies to the source markdown.
I suggested using a file or files containing examples of all the things that prettier would reformat, translate those files, then run prettier against the source files to test the impact of them being reformatted on the translations.
Instead you have used a two-line README that is unchanged when running Prettier against it, and declared the translations unaffected.

@mbrookes Can we try with a single page of the documentation? It would avoid the need to set up a whole different Crowdin project and more importantly, test it end to end?

@oliviertassinari One page is not guaranteed to contain all of the things that prettier changes. @eps1lon has already set up the project, he just needs to populate it with meaningful test content.

Instead you have used a two-line README that is unchanged when running Prettier against it, and declared the translations unaffected.

For future reference: I never made such claims. The project is not set up this way and I didn't declare anything but crowdin did.


Weblate is closing spurious "HTML" tags

In source texts like this one, weblate seems to be appending a close HTML tags to its translation, e.g. </h1></book-title> because there are pseudo-tags like <h1> in the text.
Steps to reproduce:

Remove </h1></book-title> from the translation
Save it and go back, or use the save and stay option
See that the part you just removed it is still there

@14NGiestas these strings are not spurious as they are from minibooks.md from contributing section, (https://fortran-lang.github.io/webpage/en/community/minibooks/?highlight=book+title) (1.1 Writing your mini-book in markdown section)
Thanks and Regards,
Henil Shalin Panchal
CC @awvwgk

These are not html tags as they are just placeholders for examples in making minibooks.

Thus, I would like to request to please close this issue as it seems resolved.
Thanks and Regards,
Henil Shalin Panchal

Weblate has been intended only for providing the UI for the translations, using automatic translations may  cause similar bugs , see also #121 .
Thanks and Regards,
Henil Shalin Panchal
CC @awvwgk

We should review the contributing guidelines carefully. Some information is currently outdated (#120) and most of it is rather brief.

Weblate has been intended only for providing the UI for the translations, using automatic translations may cause similar bugs , see also #121 .

Maybe I wasn't clear, I know they are just placeholders, but it seems Weblate sees it as HTML tags and automatically appends a close tag to it.
I keep removing such close tags, but somehow they keep appearing back, so I believe there is something going on with some Weblate configuration.

I've updated the issue description with some steps to reproduce the behavior.
I guess this explains why the Chinese translator changed <book-title> to, just `book-title` so it avoids that kind of trouble with HTML escaping.

Maybe I wasn't clear, I know they are just placeholders, but it seems Weblate sees it as HTML tags and automatically appends a close tag to it.

I removed the safe-html flag from all strings for the webpage. We shouldn't have any raw HTML showing up for translation anyway, in case they do, I would consider it a bug. The <h1> tag in the minibook guide should be removed, because it is currently incorrect, we explicitly create the first header rather than using frontmatter for this purpose.

I just hope that this don't create another issue, a security one, where someone could inject script tags in the translation in which is called XSS (cross-site scripting) sometimes, this can be used to do all sorts of evil things.
If this isn't the case, then I think it's solved now.

@awvwgk to resolve the XSS issue we should maybe keep restricted access to the translations (like github auth on weblate so that we could be able to find user just incase ) or switch to the manual way earlier.
As it seems that there is no other method to resolve the XSS bug.
Thanks and Regards,
Henil Shalin Panchal
CC @awvwgk @14NGiestas

I will be updating the minibooks.md and also adding the po files in the upcoming PR by this weekend.
Thanks and Regards,
Henil Shalin Panchal
CC @awvwgk

I think the first thing that we should try is to limit the access, submissions, edits, etc. to Webplate and restrict it to a small number of people. At a later time we could try and sanitise the input to Webplate, but that might be unnecessary for now.


Replace usage of 'mtl' with 'transformers'

Issue Number

Overview

[ ] I have replaced usage of 'mtl' monad transformers with the equivalent from 'transformers'

Comments

There's no strong feeling about using one or another. I noticed we were using ExceptT from mtl but StateT from transformers in various places in the code. So for consistency, I've opted for using the building blocks from transformers. mtl is built on top of transformers anyway now, but we do not use any of the classy stuff from mtl at the moment (and there's no clear reason we will in the near future?).

let's stick with this one and only pull 'mtl' if we feel it becomes a need

👍

it is good to choose one (the same for the lens), the slimmest one but good enough to service out needs. Started using only transformers primitives in my PRs

Although how would you change this? (from #83) https://github.com/input-output-hk/cardano-wallet/blob/9bcbc098f1f64e66ba26f29663c2f493e3eeab08/test/integration/Test/Integration/Framework/Request.hs#L75-L87

You would just remove the MonadReader and directly work with a ReaderT (which is already what we do anyway because GHC can't figure out the right type to pick otherwise).


Add max attribute for input date and option to show or hide config button on the table

Looks good though I cannot run demo site locally (unrelated to this change).


Inline suggestion does not work when there is a .hg folder

Describe the bug
It seems inline suggestion does not work in projects using mercurial, which has a .hg folder, as the files twinny uses have the extension of the SCM folder, which is removed in: https://github.com/rjmacarthy/twinny/blob/ef38e356186c19ffad43d67c3cf5cdd933da4990/src/extension/file-interaction.ts#L144
The Error is:
2024-04-16 10:27:29.269 [error] [rjmacarthy.twinny] provider FAILED
2024-04-16 10:27:29.269 [error] CodeExpectedError: cannot open file:///raid/standards/base-setups/risebase/production/scripts/auto_inactive.py.hg. Detail: Unable to read file '/raid/standards/base-setups/risebase/production/scripts/auto_inactive.py.hg' (Error: Unable to resolve nonexistent file '/raid/standards/base-setups/risebase/production/scripts/auto_inactive.py.hg')
at r.$tryOpenDocument (vscode-file://vscode-app/usr/share/code/resources/app/out/vs/workbench/workbench.desktop.main.js:2163:62448)
To Reproduce
Steps to reproduce the behavior:

Have a mercurial repo (install mercurial, hg init)
create file
trigger inline completion
See error in extension host output

Expected behavior
The inline completions should work

Thanks for the report. Will investigate for future release.


Complexity warnings do not show up

I have a file with some complex functions that trigger a warning in flake8:
❯ flake8 src/analytics/calcs.py --ignore=E501,E741,D202,W291,W503,Q000 --max-complexity=10 -vv 2>flake8.log
src/analytics/calcs.py:222:1: C901 'volatility_decay' is too complex (13)

However, I can't seem to get the warning to show up with flakeheaven. Here's my config in pyproject.toml:
[tool.flakeheaven]
max_line_length = 99
# check cyclomatic complexity (mccabe). Currently not working with flakeheaven
max_complexity = 10

# make output nice
format = "grouped"

[tool.flakeheaven.plugins]
"flake8-*" = ["+*", "-Q000", "-D202", "+C901"]

I ran the following from the shell:
❯ flakeheaven lint src/floatdata/analytics/vol_decay_calcs.py -vv 2>flakeheaven.log

which does not report the C901 despite explicitly adding it. Passing --max-complexity=10 also gives no result.
Here's the diff view of the output logs

:
The Extended default select list seems like the crucial bit that's missing.

Activate C901 using mccabe instead of  flake8-*.
Invalidate cache and run again.

Thanks for the tip about clearing the cache, that was the ticket!


USB Timeout on OSX

Hi, under linux the demo script for HR monitor works perfectly, but under OSX I get the following error one second after starting.
Listening for HR monitor events (120 seconds)...
Exception in thread Thread-1:
Traceback (most recent call last):
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 810, in __bootstrap_inner
self.run()
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 763, in run
self.__target(*self.__args, **self._kwargs)
File "build/bdist.macosx-10.11-intel/egg/ant/core/event.py", line 50, in EventPump
buffer += evm.driver.read(20)
File "build/bdist.macosx-10.11-intel/egg/ant/core/driver.py", line 77, in read
data = self._read(count)
File "build/bdist.macosx-10.11-intel/egg/ant/core/driver.py", line 235, in _read
return self._epIn.read(count).tostring()
File "build/bdist.macosx-10.11-intel/egg/usb/core.py", line 402, in read
return self.device.read(self, size_or_buffer, timeout)
File "build/bdist.macosx-10.11-intel/egg/usb/core.py", line 988, in read
self.__get_timeout(timeout))
File "build/bdist.macosx-10.11-intel/egg/usb/backend/libusb1.py", line 833, in bulk_read
timeout)
File "build/bdist.macosx-10.11-intel/egg/usb/backend/libusb1.py", line 936, in __read
_check(retval)
File "build/bdist.macosx-10.11-intel/egg/usb/backend/libusb1.py", line 595, in _check
raise USBError(_strerror(ret), ret, _libusb_errno[ret])
USBError: [Errno 60] Operation timed out

Could you test by changing this line
https://github.com/SamyCookie/python-ant/blob/dev/src/ant/core/event.py#L52
by
if e.errno in (60, 110): # timeout ?

works fine :) That fixed it!


Investigate BrowserStack integration

This will need resource management similar to what karma-browserstack-launcher has (using the npm browserstack module). Otherwise running this with too many browsers will immediately fail with a message:
UnknownError: N sessions are currently being used. Please upgrade to add more parallel sessions.

What's still missing is Protractor to update the tests results via the API
{
    uri: 'https://www.browserstack.com/automate/sessions/' +
          sessionId + '.json',
    method: 'PUT',
    json: { "status": status }
}


Copy from computes generated columns when used as partition key and pk.

A generated column was not taken into account as a partition key if it
was also a primary key. This resulted in the generated column not being
computed when copying data from a source, and the import would fail as the
rows would have a null primary key.

Codecov Report

Merging #5487 into master will decrease coverage by 0.18%.
The diff coverage is 87.5%.

@@             Coverage Diff              @@
##             master    #5487      +/-   ##
============================================
- Coverage     82.74%   82.56%   -0.19%
+ Complexity    11407    11380      -27
============================================
  Files          1363     1363
  Lines         44475    44478       +3
  Branches       5575     5575
============================================
- Hits          36801    36722      -79
- Misses         5083     5145      +62
- Partials       2591     2611      +20

Impacted Files
Coverage Δ
Complexity Δ

.../crate/planner/statement/CopyStatementPlanner.java
92.22% <87.5%> (-0.89%)
22 <0> (ø)

...n/java/io/crate/exceptions/JobKilledException.java
0% <0%> (-66.67%)
0% <0%> (-1%)

...arch/indices/recovery/BlobStartPrefixResponse.java
66.66% <0%> (-33.34%)
3% <0%> (-2%)

...in/java/io/crate/lucene/match/CrateRegexQuery.java
43.47% <0%> (-30.44%)
4% <0%> (-4%)

...crate/executor/transport/TransportShardAction.java
72.97% <0%> (-21.63%)
8% <0%> (-2%)

...a/io/crate/exceptions/ContextMissingException.java
81.81% <0%> (-18.19%)
1% <0%> (-1%)

...o/crate/analyze/SafeExpressionToStringVisitor.java
83.33% <0%> (-16.67%)
7% <0%> (-1%)

...ration/reference/doc/lucene/IpColumnReference.java
72.72% <0%> (-13.64%)
5% <0%> (ø)

...ration/collect/collectors/OrderedDocCollector.java
90% <0%> (-10%)
5% <0%> (-1%)

.../java/io/crate/analyze/symbol/ParameterSymbol.java
87.5% <0%> (-6.25%)
6% <0%> (-1%)

... and 25 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7206053...69b768a. Read the comment docs.


Powershell Intergrated Console Won't Run Powershell Command after ps module version was updated

Extension version: 2020.6.0
VS Code version: Code 1.47.3 (91899dcef7b8110878ea59626991a18c8a6a1b3e, 2020-07-23T13:12:49.994Z)
OS version: Windows_NT x64 10.0.18363
Steps to Reproduce:

Update AZ powershell module in a seprate powershell window
Attempt to run any command from Az.Compute module in the Intergrated Treminal
Receive following errors

PS C:\GitRepo\Labdeploy> Publish-AzVMDscConfiguration
Publish-AzVMDscConfiguration: The 'Publish-AzVMDscConfiguration' command was found in the module 'Az.Compute', but the module could not be loaded. For more information, run 'Import-Module Az.Compute'.
PS C:\GitRepo\Labdeploy> Get-AzVM
Get-AzVM: The 'Get-AzVM' command was found in the module 'Az.Compute', but the module could not be loaded. For more information, run 'Import-Module Az.Compute'.

System Info

Item
Value

CPUs
Intel(R) Core(TM) i7-8650U CPU @ 1.90GHz (8 x 2112)

GPU Status
2d_canvas: enabledflash_3d: enabledflash_stage3d: enabledflash_stage3d_baseline: enabledgpu_compositing: enabledmultiple_raster_threads: enabled_onoop_rasterization: disabled_offprotected_video_decode: enabledrasterization: enabledskia_renderer: disabled_off_okvideo_decode: enabledviz_display_compositor: enabled_onviz_hit_test_surface_layer: disabled_off_okwebgl: enabledwebgl2: enabled

Load (avg)
undefined

Memory (System)
15.92GB (5.85GB free)

Process Argv

Screen Reader
no

VM
0%

Does this issue occur when all extensions are disabled?: Yes/No

What does Import-Module Az.Compute say?

/extPowershell

Import-Module Az.Compute doesn’t throw any errors, but when trying to run an AZ command, getting same error as above.
I tried various combinations of Import-Module/Restart Integrated Shell/Restart VS code and only restarting the whole computer worked
Michael Resnick | Premier Field Engineer - US PFE - East Region
michael.resnick<EMAIL_ADDRESS>(o)<PHONE_NUMBER>  (c)<PHONE_NUMBER>
Premier Support: 1.800.936.3100 | Premier Online: https://premier.microsoft.comhttps://premier.microsoft.com/
<EMAIL_ADDRESS>https://twitter.com/AzureSupporthttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftwitter.com%2FAzureSupport&data=02|01|Michael.Resnick%40microsoft.com|64ccece7594a4936f83808d7a8f9380f|72f988bf86f141af91ab2d7cd011db47|1|0|637163660759497841&sdata=1sFlBVPr%2B9nm%2F%2BJNHTzBkr%2FsLOmgLAksNstTqNg7nqQ%3D&reserved=0
https://twitter.com/MSFT365Statushttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftwitter.com%2FMSFT365Status&data=02|01|Michael.Resnick%40microsoft.com|64ccece7594a4936f83808d7a8f9380f|72f988bf86f141af91ab2d7cd011db47|1|0|637163660759497841&sdata=S05MkNQN5RJN5%2Fo59OtO88m%2B9wL1xDbJMJuJ2l7YCnI%3D&reserved=0
If you have any feedback about my work, please let either myself or my manager, Daryl Tutton  (Daryl.Tutton<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>From: Daniel Imms<EMAIL_ADDRESS>Sent: Monday, August 10, 2020 4:35 PM
To: microsoft/vscode<EMAIL_ADDRESS>Cc: Mike Resnick (PFE<EMAIL_ADDRESS>Author<EMAIL_ADDRESS>Subject: Re: [microsoft/vscode] Powershell Intergrated Console Won't Run Powershell Command after ps module version was updated (#104084)
What does Import-Module Az.Compute say?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmicrosoft%2Fvscode%2Fissues%2F104084%23issuecomment-671576056&data=02|01|Michael.Resnick%40microsoft.com|9eb5a7b52145412d4dfc08d83d6cd135|72f988bf86f141af91ab2d7cd011db47|1|0|637326884866817635&sdata=DemWwotDjYzb48YnFyfeuNBO7IpqR8a0wcPD1mVtKE8%3D&reserved=0, or unsubscribehttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAGNHKNN7DZHXJJGVDGJAZ7LSABKWJANCNFSM4PVTW3XA&data=02|01|Michael.Resnick%40microsoft.com|9eb5a7b52145412d4dfc08d83d6cd135|72f988bf86f141af91ab2d7cd011db47|1|0|637326884866827629&sdata=lGgrQL5cfpm%2BqosoNCc%2FIVQQnjH%2BYJw%2F9UqjHDx771g%3D&reserved=0.


Crash rates 3x in 1.10.7 in the beta channel reported over 1.9.1+hotfix.4

Subject pretty much sums it up --- I don't see obvious culprits yet in the crash logs.
/cc @zanderso can someone on the tools team please investigate?

As discussed offline, the crash rate is still below our threshold, and there are no new/unexpected crashes, so I'll go ahead and close this in favor of the issues we already have filed.


Modules not setting individually when setting them with list comprehension

Hello everyone,
I encountered a strange behaviour when trying to set several modules with different attributes using pythonic features. I reproduced a minimal example here :
import jax.numpy as jnp
import haiku as hk

class TestModule(hk.Module):

    def __init__(self, to_add):
        super(TestModule, self).__init__()
        self.to_add = to_add

    def __call__(self, x):

        return x + self.to_add

to_add_list = [1*jnp.ones((1,)), 2*jnp.ones((1,)), 3*jnp.ones((1,))]

func_list = [hk.without_apply_rng(hk.transform(lambda x: TestModule(to_add)(x))) for to_add in to_add_list]

for func in func_list:

    print(func.apply(None, jnp.ones((1,))))

This outputs [4.], [4.], [4.], instead of the desired [2.], [3.], [4.]
In this snippet, we would like to set 3 modules which add 1, 2 and 3 to the input, and when using them, all the three modules are set to add 3 (only the last one should do this).
I got this issue with haiku 0.0.6 and 0.07 and upgrading jax/jaxlib did not change anything. I have almost no idea about why it behaves this way, and defining the functions under jax.jit or pmap do not fix either.

Haiku modules should be seen purely as functions. They do not save any data, including e.g. to_add.
Since self.to_add is not a trainable parameter, the only way to access it inside TestModule.__call__ is by using state. See the official documentation for haiku.transform_with_state.

Rethinking of this for unknown reason : the issue may originate from the use of lambda function to define this list of modules.
import jax.numpy as jnp
import haiku as hk

class TestModule(hk.Module):

    def __init__(self, to_add):
        super(TestModule, self).__init__()
        self.to_add = to_add

    def __call__(self, x):

        return x + self.to_add

to_add_list = [1*jnp.ones((1,)), 2*jnp.ones((1,)), 3*jnp.ones((1,))]

def build_module(to_add):

    def f(x):
        return TestModule(to_add)(x)

    return f

func_list = [hk.without_apply_rng(hk.transform(build_module(to_add))) for to_add in to_add_list]

for func in func_list:

    print(func.apply(None, jnp.ones((1,))))

This snippet works as intended!

Haiku modules should be seen purely as functions. They do not save any data, including e.g. to_add.

I can't imagine NN-layers working without saving any metadata such as the entry/output size or number of features etc. Maybe your comment was more about keeping mutable arguments in module properties, however JAX array are immutable.


Add face_mask detector models

#47
Detector models are trained for face/mask detection task.
For detecting face and face-mask we need a frozen graph trained on a customized dataset of face and face-mask, extracted from WIDER face dataset.
The validation set contains 2818 images (Faces = 2020, face_mask = 798)
X86 model mAP on validation set  = %76.74
edgetpu quantized model mAP on validation set = % 68

Looks good.


Question: how to customise @expose

I want to storage all views' endpoints in one class to make them easy to customise.
But in this library expose decorator is every where but it hasn't all the functionality of Flask's App method add_url_rule , that can take endpoint and view_func arguments, that are useful. Also, expose doesn't have any reference to add_url_rule inside, so I don't know how to enhance it.
Can you help me?

I have found some interesting code inside AdminViewMeta.__init__ and BaseView.create_blueprint methods, but It seems to be hard to change them: all View classes inherit from them.

PR

is this PR to be able to tune the default '/admin' endpoint to stg else ? i have another plugin using 'admin' blueprint and therefore have a blueprint collision. or did i misunderstand your point.  thanks for you answer.

@jeanphilippegoldman You can change the endpoint by specifying a value for the url parameter when initialising, like: admin = admin.Admin(app, url="/stuff", name='Example App')


Add the possibility to import schemas

Problem
Eventually, as the data grows, the number of models also increases. When relate to them is important, doing so in a single schema file can be stressful.
Some might argue that one could make separate files already in Prisma's current state, but with this it is not possible to guarantee consistency in relationships as currently, especially when considering the use of Prisma's VSCode extension (which helps a lot in this).
So, it would be very important to implement a method that makes importing schemas possible, maintaining consistency between everything.

Suggested solution
The import could be done using something like @import(filepath), present somewhere in the schema you are importing.
The way it works could be how is in SASS (or even CSS), where, in a way, we can establish the main file and even "omit" the files that should not be compiled.

Thanks for the suggestion, I think this is part of https://github.com/prisma/prisma/issues/2377 (where importing individual files is one of multiple approaches).


[WIP][Compute] Add vm extension publish, vm extension publish-version

https://github.com/Azure/azure-cli/issues/11219

This checklist is used to make sure that common guidelines for a pull request are followed.

For new extensions:

[ ] My extension description/summary conforms to the Extension Summary Guidelines.

If this PR is for a new extension or change to an existing extension, use the following to try out the changes in this PR:
docker run -it microsoft/azure-cli:latest
export EXT=<NAME>
pip install --upgrade --target ~/.azure/cliextensions/$EXT "git+https://github.com/qwordy/azure-cli-extensions.git@11219#subdirectory=src/$EXT&egg=$EXT"


Regression loss functions missing

Feature request description
Regression algorithms missing.
Solution proposed
Implement loss functions like MSE, RMSE, etc.
Additional context
Refer to docs.fast.ai, PyTorch and Keras docs for implementation

Regression feature implemented in v<IP_ADDRESS>


Adapt tests to new test framework

Adapt legacy tests framework to new system.
Now every "fake" file have a "get_credentials" method. "msrestazure" becomes a needed dependencies for legacy testing as well.
The Wiki is then accurate for both ARM and Legacy tests
Thanks to @schaefi to point this lake of consistency :)

Coverage remained the same at 57.864% when pulling 71ebc5c5968f486dcc171c76eb6fe2f852c7afc0 on lmazuel:test_framework_update into 2b1980c6b7bcaf15f22f2216a21b217e2224915d on Azure:dev.


Apply to multiple versions

The ability to update a base table isn't limited to current, so this inclusion is misleading. This should be in all versions of SQL Server, subject to the limitations in the CREATE VIEW item.

@way0utwest : Thanks for your contribution! The author(s) have been notified to review your proposed change.

@way0utwest - thanks for clarifying this one!
#sign-off


Add the ability to configure the max connections and max per route on PoolingHttpClientConnectionManager with different values

Followup to https://github.com/spring-cloud/spring-cloud-netflix/pull/795 if you are not happy with the config names:
zuul.hostRoutingFilter.maxTotalConnections and zuul.hostRoutingFilter.maxPerRouteConnections
let me know.
This breaks backwards compatibility as zuul.max.host.connections is not applied any more and SimpleHostRoutingFilter ctors has changed.

Totally agree, sorry for the delay. Renamed the classes accordingly.

@AndreasKl I need you to rebase rather than merge, otherwise there are extra commits.

Done.

Squashed and merged with some polish via aa2d0eb4425c935218834aa2487dd0036b794583.


Use head instead of render nothing: true

Why: head has been the recommended option since Rails 4.
render nothing: true has now been deprecated in Rails 5.1

Looks like there's a 💥 on CI. It is having trouble installing nokogiri in ruby 2.0.0. How committed are we to supporting that ruby version?

I don't think we are committed. The identity-idp app requires Ruby 2.1 or higher I believe.


Inspector should discover Wikipedia article through Wikidata

If a feature is tagged with wikipedia, the inspector will show a multimedia excerpt of the Wikipedia article. It should do the same if the feature is only tagged with wikidata but not wikipedia. For example, this relation is tagged wikidata=Q3275007; since Q3275007 on Wikidata links to the article “Lunken Airport” on the English Wikipedia, the inspector should show an excerpt from that article, including the following image:

To determine the Wikipedia article associated with a Wikidata QID, we could use the Wikidata API or Hub.

And if there’s no Wikipedia article, the same API call we’re making to Wikipedia would yield a similar excerpt (image plus description) if we point it to the Wikidata API instead.

Would you recommend we deemphasize the need for Wikipedia links altogether? (I'd love that)

Would you recommend we deemphasize the need for Wikipedia links altogether? (I'd love that)

Deemphasizing is probably the right word. I wouldn’t go so far as to start removing them yet, but if we have a fallback to find the Wikipedia content using wikidata, then there’s much less need to promote wikipedia tagging on new features.

@ZeLonewolf implemented an extension for Chrome and Firefox that automatically resolves QIDs in the openstreetmap-website inspector to human-readable blurbs from Wikidata. We should integrate the script that powers this extension into ohm-inspector.

Would my extensions work out of the box if I allowed it to run on OHM?

Most likely.

One thing you may run into is dealing with CORS issues. The plugin avoids that problem with permissions declarations that the browser knows to allow.

Do you mean that the Wikidata API might disallow requests from within openhistoricalmap.org? ohm-inspector already makes requests to the Wikipedia API, which is essentially the same thing.


feat/openFiles: open files upon click in task details and my organisations document section

Functionality:
feat/openFiles
Solution:
open files in new tab upon click in Task details and My Organisation documents section.
Risk level:

[ ] high
[x] medium
[ ] low

How to test:

firebase emulators:start
ng serve
auto setup
go to my organisations document section. Upload a file if not one is uploaded.
click on the document icon of the respective file to view it. The file opens in a new tab.

Please give demo on this


add try_ctx_for_window function

This is necessary for bevy-inspector-egui to support multiple windows. Could you maybe release a small non-breaking version so that I can implement this without waiting for the next big release?

What if we just change ctx_for_window to return an option? I'm personally fine with making a new minor release with this breaking change. I think it's easier to do so rather than making a patch release just to remove the try flavour later.

In most cases ctx_for_window would be unwrapped if it returned an option. When creating a new window and doing  .add_system_set(SystemSet::on_update(AppState::Done).with_system(ui_second_window.system())) the UI system will only run when the window exists.
My use case is for adding a ui system to a Plugin which runs even if the window is not set up yet.

I think having both methods is fine, because the panicking version will be used the most, and the try_ version exists as a fallback for the more remaining cases. Same with vec[idx] vs vec.get(idx).

Sounds fair. And thanks for the PR, I'll release the new version today.


Another one...

Hi! Thanks again for this great PR :-)
I didn't know about math.lossyCast, cool to discover that!


tone js defs file added

Full definitions for Tone.js audio library version 4

@lukephills thank you your contributions.
please check this document. http://definitelytyped.org/guides/contributing.html#tests


Add an npmignore file so that npm publish will work

What's going on with the tests? chrome is not found?

LGTM

Yeah. AFAICT something changed in the test environment. Not sure what, but retesting passing commits shows them failing now, so it doesn't appear to be an issue with wct.


quota limit on cli deployment

I keep getting this :
The specified capacity '1' of account deployment is bigger than available capacity '0' for UsageName 'Tokens Per Minute (thousands) - GPT-35-Turbo'. (Code: InsufficientQuota)
How to overcome it ?

You need to reduce the TPM of your existing deployments before you can make a new one, or try deploying to a different region. You can edit the TPM in the studio:
For each of your existing deployments, edit them and slide the slider down:


docs: deploy flannel with NET_RAW

Description

NET_RAW is a capability that docker has given containers by default forever. However, it's not needed for the majority of pod workloads, and is a security hole for workloads that don't need it.
CRI-O dropped NET_RAW by default in 1.18
I believe pods that need capabilities should ask for them, and we shouldn't let every pod have caps they don't need.
This PR adds NET_RAW to flannel's requested capabilities. For Docker, it will be a no-op; for CRI-O, the pod will successfully come up
partially fixes issue reported here
Todos

[ ] Tests
[ ] Documentation
[ ] Release note

Release Note

Status: Downloaded newer image for golang:1.10.3

Makefile:100: recipe for target 'gofmt' failed

make: *** [gofmt] Error 1

The command "make test" exited with 2.

seems unrelated

actually, is the same PR as mine.. https://github.com/coreos/flannel/pull/1305

Dang @kubealex I should have searched... a step ahead of me yet again


.update() with undefined values doesn't return

Waterline version: 0.11.6 (via dogwater)
Node version: 6.9.1
NPM version: 3.10.9
Operating system: Mac OS X El Capitan 10.11.6
Adapter: sails-postgresql ^0.11.4

Issue:
When passing in undefined to the values parameter of .update(), the function never returns and seems unresponsive (e.g. no error is thrown).
As soon as it has a non-null values param, it works fine.
Code
Person.update(params.uuid, payload);

Docs
The docs state that values is required but no error seems to get thrown.

Suggested Solution
Throw an error if the values is passed in as undefined

No worries, I've given up on Waterline since it tries to do far too much without even doing the basics well first. 👋


Add new product(s): products/meilisearch.json

Add new product(s) from unicef/publicgoods-candidates

Closing as a duplicate of #35 (this one was missing the repositoryURL field)


Use kubernetes kyverno: v1 19.8 the image built as the basic image is always in the waiting state during operation

The basic image used is kubernetes kyverno: v1 19.8 the image built in 19.8 can't get up in the pod and is always in the waiting state

Running to this step will get stuck

You're not up because the Calico network isn't up；
kyverno redirect-registry policy is invalied:
Kubefile:

raw docker version:

redirect-registry policy is invalied:

We have replaced the network plug-in calico - > hybridnet. It seems that this plug-in will also affect kyverno


docs: add preact devtools option

Description (required)
Documents the addition of a devtools option for the preact integration.
Related issues & labels (optional)

Closes #
Suggested label: merge on release

For Astro version: 4.8. See astro PR #10938.

Lunaria Status Overview
🌕 This pull request will trigger status changes.

Learn more
By default, every PR changing files present in the Lunaria configuration's files property will be considered and trigger status changes accordingly.
You can change this by adding one of the keywords present in the ignoreKeywords property in your Lunaria configuration file in the PR's title (ignoring all files) or by including a tracker directive in the merged commit's description.

Tracked Files

Locale
File
Note

en
guides/integrations-guide/preact.mdx
Source changed, localizations will be marked as outdated.

Warnings reference

Icon
Description

🔄
The source for this localization has been updated since the creation of this pull request, make sure all changes in the source have been applied.

D'oh! Our <Since /> component is hard-coded to say "astro" because it wasn't available for these other pages in our previous setup. Let me see whether it's easy enough for me to update the component to take an additional prop, or whether we just use Markdown to "fake" this (like we used to)

Updating branch and pointing to 4.8.0 branch!


Optimization of database storage

Data size stored in mysql server (on aeon-db2) is approaching 400GB
This will hit the storage size limit of the aeon-db2 server, at which point we will need to request for a disk size increase.
I'd like to discuss potential solutions to better engineer/optimize for storage space in our next data architecture meeting.
The bulk of the storage is in the streams schema, where we extract and store the data from relevant streams. Among these, the majority of the storage is at UndergroundFeederEncoder (the wheel data) (~100GB) and SpinnakerVideoSourceVideo (~80GB)

I'm sharing here the current size (in GB) of each schema in the aeon-db2 database
{'aeon_streams':<PHONE_NUMBER>16,
 'aeon_test_streams': 13.5774208,
 'aeon_test_block_analysis': 0.000180224,
 'aeon_archived_exp02_acquisition': 42.648174592,
 'aeon_test_tracking': 0.216285184,
 'aeon_worker': 0.092962816,
 'aeon_tracking': 32.597950464,
 'aeon_test_worker': 0.001687552,
 'aeon_archived_exp01_analysis': 0.010092544,
 'aeon_test_acquisition': 0.010010624,
 'aeon_analysis': 0.00188416,
 'aeon_archived_exp02_subject': 0.000835584,
 'aeon_archived_exp02_worker': 0.000131072,
 'aeon_report': 4.9152e-05,
 'aeon_block_analysis': 23.938367488,
 'aeon_archived_exp02_report': 0.002752512,
 'aeon_device': 3.2768e-05,
 'u_thinh_aeonfix': 0.00475136,
 'aeon_subject': 0.000638976,
 'aeon_archived_exp02_streams': 0.000507904,
 'aeon_archived_exp01_acquisition': 0.10944512,
 'aeon_archived_exp02_qc': 24.861868032,
 'aeon_test_subject': 0.000278528,
 'aeon_archived_exp02_analysis': 4.933648384,
 'aeon_lab': 0.000442368,
 'aeon_archived_exp01_lab': 0.000425984,
 'aeon_test_qc': 9.8304e-05,
 'aeon_archived_exp01_subject': 0.000294912,
 'aeon_archived_exp01_report': 0.041025536,
 'aeon_archived_exp01_qc': 26.919780352,
 'aeon_test_analysis': 0.304234496,
 'aeon_archived_exp02_lab': 0.000442368,
 'aeon_acquisition': 0.121913344,
 'aeon_archived_exp02_tracking': 9.951330304,
 'aeon_qc': 4.260446208,
 'aeon_archived_exp01_tracking': 1.720942592,
 'aeon_test_lab': 0.000442368}

Ah yes, I'm happy downsampling the wheel data to 50 hz as something quick we can do immediately

@ttngu207 proposed 2 possible ways to downsample the wheel data:

downsample as part of ingestion and store this in processed (but this is inconsistent with the current "read-only" way)
downsample and store in processed before ingestion

@ttngu207 feel free to expand/correct my comment here

Hi @jkbhagatio ,
We discussed this issue a bit more with @lochhh and @anayapouget today.
Downsampling the wheel data to 50Hz is a good immediate step we can take to optimize the storage. On this, I think there are 2 ways to accomplish this

Add logic to do down sampling at the step of ingestion with DataJoint.

Pros: simple, easy
Cons: mismatch in how the data in files are stored vs. how it's stored in DataJoint. For all of the tables in streams, we have been maintaining this consistency, breaking it here is something to think about

Create another set of the downsampled wheel data files in processed folder - this will get ingested

Pros: zero change needed in the current ingestion routine, and maintaining the consistency in files and ingested data
Cons: additional routine needed to do the downsampling and store downsample files in processed (perhaps together with, or after, the data copy step)

Technically, at the streams level, we want to maintain consistency between the ingested data with the "read" data - i.e. the dataframe returned by the StreamReader class (not strictly the data in the raw files).
So, perhaps a 3rd option to consider is to update the StreamReader for the wheel data to have a downsampling keyword argument (or downsampled_rate), and we use this option for ingestion.
This approach would also require zero change in the current ingestion flow, only need to update the reader in aeon.schemas.schema, and still maintain consistency between data returned by the reader and data ingested

@jkbhagatio @glopesdev
Let us know your thoughts on this, if this is a reasonable approach, I'll start the implementation.
Currently, data ingestion for social0.4 is paused due to this storage problem

@ttngu207 for wheel data, number 3. sounds preferable for now, especially since it avoids data duplication.
For the video CSV it's a bit harder to imagine a compressed scheme, especially since it seems from the numbers that the stored data is already in binary form. The only other option would be to save timestamps only and table indices and pull data on demand post-query if needed.
I can push to implement the downsampling parameter ASAP for wheels since it seems this would unblock ingestion the fastest.

@glopesdev agreed, I also like option 3 best. The video CSV is definitely trickier, we can deal with that later
It would be great if we can prioritize the downsampling parameter in the io.reader.Encoder, that would allow ingestion to resume.

what is the underlying database engine? innodb? is page compression enabled? you can also get savings depending on the underlying file system.

@ttngu207 @glopesdev, I'm also happy with option 3 mentioned above

@jkbhagatio from our meeting today we were converging on a downsampling strategy and API for the wheel data. Since the encoder reports absolute wheel position we can simply decimate the data to 50 Hz by sampling the raw stream at fixed 20ms intervals inside each 1h chunk (no need for filtering).
This would be implemented as a property at the level of the reader.Encoder class so it can be toggled in the schema (or we can set the default to True).
For the video metadata one possibility we discussed to immediately gain some space is to drop the hw_counter column from the ingested data. We should be able to reconstruct dropped frames at both the TTL and software levels by comparing the camera timestamp with the harp timestamp directly, and it would give us back around 1/3 of the space.
If this sounds good I can submit a PR soon.

@glopesdev this sounds good regarding the changes to Encoder, happy for that immediately. Regarding the video metadata, can we discuss the hw_counter in a bit more detail?

@ttngu207 @jkbhagatio @lochhh  I have a working implementation of the downsampling operation at https://github.com/SainsburyWellcomeCentre/aeon_mecha/commit/3340df25977b1c0826dce02848e5713fcb68c618.
However, I am now conflicted as to whether having downsample be a property of the reader / stream is a good idea. The reason is because of testing. The few tests we have of the low-level API were actually using encoder data on a chunk which has non-monotonic timestamps and after resampling the behavior is changed.
This makes sense, but at the same time made me more aware that some of the more subtle issues with timestamping came from carefully inspecting this low-level raw data and I would like to retain the ability to easily query the raw data if necessary.
The problem is that if we implement downsample as a property which is frozen at the level of the schema stream, then whoever calls aeon.load with that stream does not have any choice to see the raw data, since the flag is set at schema creation time.
Therefore I would like to provide a way to override reader behavior at aeon.load time as well, when necessary. This made me think that a general way to support this behavior could be to simply have readers accept optional **kwargs arguments and have aeon.load forward any kwargs that it receives in its call to the reader which is passed in the function argument.
This would solve my dilemma with the encoder testing and future debugging, but also in general gives us a straightforward way to have options in readers which can be toggled at aeon.load time, rather than only by configuring the reader at schema creation time.
Any thoughts on this?

@glopesdev Thanks, the implementation looks good to me.
I'm fine with downsample be an additional argument for load, but it seems a bit one-off and may break the elegancy of the current implementation/usage of .load() (or not)
How about downsample still being a property of the reader, and add the get/set method for this property so it can be changed on the fly before calling the load() (and not frozen at the time of schema creation)?

I've reviewed the PR, allowing any additional kwargs to be passed on into the load() is also very clean and potentially very useful!

Hi @glopesdev , this makes sense to me conceptually, but I might suggest using a specific arg name (that will be a dict) instead of kwargs, to differentiate args that will be passed to load alone vs. args that will be passed to a read via load.

@jkbhagatio can you review directly in the PR? Just easier to iterate the code implementation there.

Done when considering #407 and #438 combined


Unable to set Adjacent Divs

I have the following code:

                PLay Area

```

it's creating gap between 2 DIVs. As you can in image, there is a gap between black and Red DIV.

![screen shot 2015-03-19 at 5 23 10 pm](https://cloud.githubusercontent.com/assets/273196/6729970/82421ffe-ce59-11e4-84a7-538bc654c3cd.png)

You should have a

 to place your columns in. Also, without knowing how the "top2" or "left" classes are styled it would be hard to tell what is wrong.


Make Viz links work the same way as Table links

Currently only StroomLink() is exposed to viz  and doesn't understand dashboard links

Visualisations can now link in the same ways that tables can. The following functions are now available:
stroom.link(href, target)
stroom.dashboard(uuid, params, queryOnOpen)
stroom.annotation(annotationId, streamId, eventId, title, subject, status, assignedTo, comment)
stroom.stepping(id, partNo, recordNo)
stroom.data(id, partNo, recordNo, lineFrom, colFrom, lineTo, colTo)
The functions can be called via d3 handlers, e.g.
bar.on("click", function(d, i) {
   stroom.dashboard(null, "user=" + seriesData.key);
});


Reading and writing on S22 using kwrite()

hello friends
I've tried the exploit on Galaxy S22 and it works well!
but after a lot of works and trying to access root and selinux permissive, I realized that I can't change or even read some special parts of kernel memory such as the offset of SELINUX_ENFORCING_BOOT.
every time that I tried to call kwrite() or even kread() for the content of this address the kernel crashed.
at first I was thinking that the problem is related to my special kallsyms offset so I tried some others like SELINUX_STATE , SELINUX_SETUP but now I understand that the problem is related to some special kallsyms offset .
can any body explain more about my problem I am a newbie in this field and I can use any help.

I've figured out that the problem is from offsetting in my program.


Allow SSR to render unescaped inlined script tags.

I have a feature request  - In order to fully replace templating engine server should be able to output raw unescaped contents for some tags like <script>. Currently all the transcluded content or the content provided by v-text/v-html is automatically escaped and the script becomes unusable.
Example of usage - output json-ld:
<script type="application/ld+json">
{
  "@context": "http://schema.org",
  "@type": "Person",
  "name": "John Doe",
  "jobTitle": "Graduate research assistant",
  "affiliation": "University of Dreams",
  "additionalName": "Johnny",
  "url": "http://www.example.com",
  "address": {
    "@type": "PostalAddress",
    "streetAddress": "1234 Peach Drive",
    "addressLocality": "Wonderland",
    "addressRegion": "Georgia"
  }
}
</script>

Also have the same issue here and it's a blocking issue for me. Any updates to this? Or, if you can point me to what files to look into I can look to building a PR for this.

Also have the same issue here and it's a blocking issue for me. Any update plane to this? Now,
I can only insert the script after the page is rendered to html

+1 I would like this too, it's a major blocker for us right now.

I would love this feature too. I had a very quick and brief look into the compiler:
https://github.com/vuejs/vue/blob/dev/src/compiler/parser/html-parser.js#L64
It seems like it was intended to skip endTags/plainTexts, but that does not seem to work. The replacement takes place when handleStartTag is called. Shouldn't it be !lastTag && !isPlainTextElement(lastTag) instead of !lastTag || !isPlainTextElement(lastTag) anyway?
Whatever the cause might or might not be, if someone makes a PR, should this behaviour be configurable? This kind of  stuff seems like a breaking change and the escaping could be seen as security-measurement.

Is there any workaround/progress on this? Pretty essential to provide metadata for SEO.