x/vgo: import package not yet uploaded to a remote location

$GOPATH/src/github.com/prometheus/prombench
├── cmd
│   └── prombench
│       ├── go.mod
│       └── main.go
├── provider
│   └── gke
│       └── gke.go

runing vgo build inside ../cmd/prombench fails as
main.go imports github.com/prometheus/prombench/provider/gke
which is not yet uploaded to the remote repo since this is a WIP so can't yet upload it to upstream.
vgo: import "github.com/prometheus/prombench/cmd/prombench" ->
        import "github.com/prometheus/prombench/provider/gke"<EMAIL_ADDRESS>open<EMAIL_ADDRESS>no such file or directory

I tried adding replace "github.com/prometheus/prombench/provider/gke" v0.0.0 => "../../provider/gke"  to go.mod , but didn't make any difference.
not sure what should be the behaviour here, but I see this as a quite common scenario

it looks like the first time vgo build ignores the replace directive.

Do you have a go.mod at the repo root? If not I think you need one, e.g. mine is for myitcv.io. Yours would be for github.com/prometheus/prombench)

If you intending for github.com/prometheus/prombench/provider/gke to be a module, then it will also need a go.mod (in provider/gke). In this case the replace directive from cmd/prombench would be => ../../provider/gke
otherwise, the replace directive will be => ../../ (because provider/gke is a package within the main module

If you are able to point us towards a branch of code that would probably help to diagnose.

here is the WIP
https://github.com/krasi-georgiev/prombench/tree/vgo

Ok I've just updated my earlier comment - my understanding is that you still need a go.mod in the root of your repo. And depending on whether github.com/prometheus/prombench/provider/gke will be a module or not it will also require a go.mod
So assuming you have:
$GOPATH/src/github.com/prometheus/prombench
└── go.mod
├── cmd
│   └── prombench
│       ├── go.mod
│       └── main.go
├── provider
│   └── gke
│       └── gke.go

I think there is actually a problem here because a vgo build in cmd/prombench ends up querying github.com when resolving github.com/prometheus/prombench/provider/gke; whereas I believe it should first walk to the repo root because that module (github.com/prometheus/prombench) is specified as a replace and indeed is a prefix of github.com/prometheus/prombench/cmd/prombench. I'll let @rsc comment on this though.
In the meantime, if you move your go.mod to the repo root, per https://github.com/myitcv/prombench/tree/vgo, then vgo build makes more progress (before failing somewhere in k8s land):
vgo: resolving import "github.com/pkg/errors"
vgo: finding github.com/pkg/errors (latest)
vgo: adding github.com/pkg/errors v0.8.0
vgo: resolving import "cloud.google.com/go/container/apiv1"
vgo: finding cloud.google.com/go (latest)
vgo: adding cloud.google.com/go v0.0.0-20180409203830-29f476ffa9c4
vgo: resolving import "google.golang.org/genproto/googleapis/container/v1"
vgo: finding google.golang.org/genproto (latest)
vgo: adding google.golang.org/genproto v0.0.0-20180409222037-51d0944304c3
vgo: resolving import "gopkg.in/alecthomas/kingpin.v2"
vgo: finding gopkg.in/alecthomas/kingpin.v2 (latest)
vgo: adding gopkg.in/alecthomas/kingpin.v2 v1.2.6-gopkgin-v2.2.6
vgo: resolving import "gopkg.in/yaml.v2"
vgo: finding gopkg.in/yaml.v2 (latest)
vgo: adding gopkg.in/yaml.v2 v1.2.1-gopkgin-v2.2.1
vgo: resolving import "k8s.io/api/apps/v1beta1"
...

success!
for some reason vgo tries to use  old  version of some packages which caused build failures,but after adding the latest commit hashes manually it builds ok now.
if I comment out these 2 it tries to use older versions for some reason.
"golang.org/x/oauth2" v0.0.0-20180411000000-921ae394b9430ed4fb549668d7b087601bd60a81
"gopkg.in/yaml.v2" v0.0.0-20180411000000-5420a8b6744d3b0345ab293f6fcba19c978f1183

feel free to close the issue if you think it behaves as expected.
module "github.com/prometheus/prombench"

require (
	"cloud.google.com/go" v0.0.0-20180409203830-29f476ffa9c4
	"github.com/alecthomas/template" v0.0.0-20160405071501-a0175ee3bccc
	"github.com/alecthomas/units" v0.0.0-20151022065526-2efee857e7cf
	"github.com/googleapis/gax-go" v0.0.0-20180411000000-de2cc08e690b99dd3f7d19937d80d3d54e04682f
	"github.com/pkg/errors" v0.8.0
	"go.opencensus.io" v0.8.0
	"golang.org/x/oauth2" v0.0.0-20180411000000-921ae394b9430ed4fb549668d7b087601bd60a81
	"google.golang.org/api" v0.0.0-20180410000351-9f7560f3b05b
	"google.golang.org/genproto" v0.0.0-20180409222037-51d0944304c3
	"google.golang.org/grpc" v1.11.3
	"gopkg.in/alecthomas/kingpin.v2" v1.2.6-gopkgin-v2.2.6
	"gopkg.in/yaml.v2" v0.0.0-20180411000000-5420a8b6744d3b0345ab293f6fcba19c978f1183
	"k8s.io/api" v0.0.0-20180411000000-acf347b865f29325eb61f4cd2df11e86e073a5ee
	"k8s.io/apimachinery" v0.0.0-20180411000000-19e3f5aa3adca672c153d324e6b7d82ff8935f03
	"k8s.io/client-go" v0.0.0-20180411000000-78700dec6369ba22221b72770783300f143df150
	"k8s.io/kube-openapi" v0.0.0-20180410221256-abc8da71b985
)

actually for yaml it is clear that the v2 doesn't have go.mod so probably that is why it defaults to v1 , but for oauth2 not sure why.

I think there is actually a problem here because a vgo build in cmd/prombench ends up querying github.com when resolving github.com/prometheus/prombench/provider/gke; whereas I believe it should first walk to the repo root because that module (github.com/prometheus/prombench) is specified as a replace and indeed is a prefix of github.com/prometheus/prombench/cmd/prombench. I'll let @rsc comment on this though.

I've given this a bit more thought, and wonder whether what we need here, in addition to the replace directive, is the ability to specify the equivalent of a go-import meta tag override (which by definition would be at the package level):
metaoverride (
    // specify local directory
    "github.com/prometheus/prombench/provider/gke" => "github.com/prometheus/prombench/provider/gke" dir ../../

    // or specify git URL per regular go-import meta tag definition
    // the import path on the RHS required to specify the import
    // path at the root of the directory/URL
    "github.com/prometheus/prombench/provider/gke" => "github.com/prometheus/prombench" git https://github.com/myitcv/prombench
)

Resolutions would first check such overrides before hitting "the internet".
I'm fairly sure this idea isn't novel and that it will have been considered en route to the replace directive so would appreciate any pointers/explanations on where it falls down.

isn't the problem here that vgo tries to download packages before it concideres the replace directive?
what if vgo build checks the replace and if the file/folder exists than no need to try and download.

from my perspective without knowing the technical implementations it seems that metaoverride and replace overlap its meanings.
I mean shouldn't  replace  always affect what package is used?

Many packages can live within a module. And indeed modules can live within the repo of other modules (i.e. they are submodules).
My understanding is that replace operates on modules. In working out what module a package lives inside (and the package itself could be the module), vgo goes through a resolution step. Hence why I don't think the overlap you describe exists.
Actually, the concept of a go-import meta tag override only applies in the case of custom domains (which is my situation, hence the bias). But this idea of a go-import-esque package-level override could apply regardless of whether we're talking a custom domain, code host etc.

I would need to spend a bit  more time using vgo and see few more examples to fully understand this idea.

I'm sorry for coming to this conversation late, but @krasi-georgiev can you say a little more about why you want to structure prometheus/prombench as multiple modules?

I need the same thing, in this case, because of a domain that is down. There is an import path in one of our dependencies for "leb.io/hashland/keccakpg" and "leb.io" has been down since May. I'd love to remap it to the GitHub path, but vgo tries to download before evaluating the replacement:
vgo: resolving import "leb.io/hashland/keccakpg"
FindRepo: Get https://leb.io/hashland/keccakpg?go-get=1: dial tcp <IP_ADDRESS>:443: connect: connection
refused
FindRepo: Get https://leb.io/hashland?go-get=1: dial tcp <IP_ADDRESS>:443: connect: connection refused
FindRepo: Get https://leb.io?go-get=1: dial tcp <IP_ADDRESS>:443: connect: connection refused
vgo: import "github.com/faceit/matcher-go/p2p" ->
        import "github.com/libp2p/go-libp2p" ->
        import "github.com/libp2p/go-libp2p-circuit" ->
        import "github.com/libp2p/go-libp2p-host" ->
        import "github.com/libp2p/go-libp2p-interface-connmgr" ->
        import "github.com/libp2p/go-libp2p-net" ->
        import "github.com/libp2p/go-libp2p-peer" ->
        import "github.com/multiformats/go-multihash" ->
        import "leb.io/hashland/keccakpg": Get https://leb.io/hashland/keccakpg?go-get=1: dial tcp 104.131
.190.18:443: connect: connection refused

@rsc actually I don't want multiple modules , I just thought that the structure of the project should be as per my first comment https://github.com/golang/go/issues/24804#issue-313209688 . So After moving go.mod in the project root it all works except a small nit as in my comment https://github.com/golang/go/issues/24804#issuecomment-380434025

Hi,
I'm still testing vgo and trying to understand how I should work with it, so please forgive me if I'm missing something.
My use case, related to this issues, is:

write down some code to explore an idea
not able to set-up a repository online now (e.g. offline)
don't want to set-up a repository online now (e.g. I don't want to create and online repo every time I explore an idea)

I am not able to run vgo build successfully until all the following happen

I create the repo online
I'm able to connect to it
I upload the code at least once (otherwise I obtain no Go source files)

even if I run vgo build on the module (github.com/brunetto/vigo).
Nothing works until vgo is able to "see" the source code online. No way to run it fully local.
A reference repo is here: https://github.com/brunetto/vigo.

@brunetto, your repo is very close. The only missing bit is in cmd/vigo/go.mod there is no "require" statement asking for the root repo. The "replace" only kicks in if the requirement exists (because in general replace could be replacing an indirect requirement, not just things directly written in the current effective go.mod).
If I do this in go.mod then it works fine:
module github.com/brunetto/vigo/cmd/vigo

replace github.com/brunetto/vigo v0.0.99 => ../../../vigo

require github.com/brunetto/vigo v0.0.99

I just made up v0.0.99 - obviously that doesn't exist online right now.
Every command does print go: finding github.com/brunetto/vigo v0.0.99 but I'll get rid of that print (and the network lookup). It was already on my list.


Cosmos DB does not return connection string

az cosmosdb list-connection-strings -g inventory-hub -n mycosmosdb03302018

{
  "additionalProperties": {},
  "connectionStrings": null
}

Empty connections string

See also #4419 as another report.

Thank you @mindlessroman. Closing this issue as it is a duplicate of #4419.


Fix for touchscreen laptops: input could be switched between mouse an…

This makes sense - thanks.


When creating a new account, "Continue" button doesn't work when getting to the "Set Recovery" phase

Problem
I rebuilt the Wallet Docker image to work inside Kurtosis and things seem to work at first, until I get to "Set Recovery". Clicking the "Continue" button doesn't do anything, and nothing shows up in the Network tab or Console.

Expected Behavior
Proceeds to the next screen where it shows me my passphrase
Steps to reproduce

Install Kurtosis if not done already: https://docs.kurtosistech.com/installation.html
Run kurtosis module exec kurtosistech/near-kurtosis-module:2022-02-09
Open the returned Wallet URL
Try to create an account

@mieubrisse thank you for reporting this issue. We had a bug that made it into the testnet wallet but shipped a fix earlier today: https://github.com/near/near-wallet/pull/2466.
Please try using the latest version of the master branch as this issue should be solved now.

Hi @mieubrisse, we can absolutely look at incorporating Docker image builds into our upcoming CI pipeline. Are you currently relying on the Docker image built from near-wallet/packages/frontend without any custom behavior on top of it?


Regression in ceph-cache plugin deployment

Commit https://github.com/datashim-io/datashim/commit/079b99e568556ffd17ff83e7d068972a917c1edc#diff-97be71743a1ddb84c9e09d0d0797677eb7b5656d5d9de9569ec0a50add9ba5fe introduced 2 regressions in the ceph-cache plugin deployment files:

Makefile does not have | kubectl apply -n $(NAMESPACE) -f -  for deployment. Hence make deployment does not actually deploy
The image quay.io/datashim/ceph-cache-plugin:latest-amd64 gives a 401 Unauthrorised exception

Fixed makefile regression as part of #158. The unauthorised exception has still to be dealt with

Ceph cache plugin is out of date. This issue is no longer relevant


EX4300 guage appears to be off by a factor of 10.

Hi,
In the lab I have a MX104---EX4600---EX4300 in a string and collecting analytics from all 3 devices (they should give me roughly the same values).  I have noticed that the guage for bps for EX4300 out by a factor of 10. The attached file has screen caps of what I'm seeing, which hopefully is attached.
EX4300bpsguageproblem.docx
Respective show version and configuration below.
root@ex4300-01> show version
fpc0:
Hostname: ex4300-01
Model: ex4300-24p
Junos: 14.1X53-D30.3
JUNOS EX  Software Suite [14.1X53-D30.3]
JUNOS FIPS mode utilities [14.1X53-D30.3]
JUNOS Crypto Software Suite [14.1X53-D30.3]
JUNOS Online Documentation [14.1X53-D30.3]
JUNOS EX 4300 Software Suite [14.1X53-D30.3]
JUNOS Web Management Platform Package [14.1X53-D30.3]
JUNOS py-base-powerpc [14.1X53-D30.3]
---------------snip
root@ex4300-01> show configuration services
analytics {
export-profiles {
exp-prof {
stream-format json;
interface {
statistics {
traffic;
queue;
}
}
}
}
resource-profiles {
res-prof {
queue-monitoring;
traffic-monitoring;
}
}
resource {
interfaces {
ge-0/0/0 {
resource-profile res-prof;
}
}
}
collector {
address <IP_ADDRESS> {
port 50020 {
transport udp {
export-profile exp-prof;
}
}
}
}
}

I think it is a query problem for BPS graph. Could you please change it to:
SELECT derivative("value", 1s) FROM "jnpr.analyticsd" WHERE ("type" = 'traffic-stats.rxbps' OR "type" = 'traffic-stats.txbps' )AND "device" =~ /$host_regex$/ AND $timeFilter GROUP BY "device", "interface", "type" fill(null)
And:
SELECT derivative("value", 1s)*8 FROM "jnpr.jvision" WHERE ("type" = 'egress_stats.if_octets' OR "type" = 'ingress_stats.if_octets') AND "device" =~ /$host_regex$/ AND $timeFilter GROUP BY "device", "interface", "type"

Michael
Mark took a snapshot of his database recently (attached)
We should be able to reproduce that ourselves now
We used this procedure to do the backup, the link also explains how to do restore
https://docs.influxdata.com/influxdb/v0.10/administration/backup_and_restore/
I've been trying to look at it but hard to find time these days :( if you have the opportunity to look at it, it will be great
Thanks
Damien
marc_db_20160331.db.save.tar.gz

Fixed Grafana zooming issue with JTI.


Test vectors

Description
This PR adds tests for the vectors layer, improves the doc strings and cleans up the code a tiny bit, but doesn't change any functionality.
One question is - should the input parameters be
edge_width, edge_color, and length for the width, color, and the multiplicative length factor for the vectors or should they be something else. They used to just be width, color, and length but I added edge_ to make the parameters the same as for the points and shapes layer, though you could argue that for the points layer the parameters do different things and that in the vectors layer we don't have a face and an edge so it is just confusing. I'm open to suggestions - personally I like the consistency - but we can change it. Thoughts @bryantChhun @kevinyamauchi @jni?
Type of change

[x] Bug-fix (non-breaking change which fixes an issue)

How has this been tested?

[x] adds napari/layers/vectors/tests/test_vectors.py

Final checklist:

[x] My PR is the minimum possible work for the desired functionality
[x] I have commented my code, particularly in hard-to-understand areas
[x] I have made corresponding changes to the documentation
[x] I have added tests that prove my fix is effective or that my feature works

I'm ok with "edge_".  It's clear enough to me that this shape isn't exactly like the others.


Remove "name" attribute completely from replaced input field

In Internet Explorer 11 a submitted form with an input which has an empty name attribute is sent like this:
http://example.com/foo?bar=baz&=06%2F12%2F2016
Note the &=
Can the name attribute be completely removed from the replaced input field when hiddenName is used?

Hi man, i had the same problem on IE8 and EDGE.
I added this line; "hiddenName: undefined," to the pickadate() function and now it works (i dont receive a 501 or 503 HTTP response code).
Now, in the POST request, i have this:
input=22%2F03%2F2017&input_submit=22%2F03%2F2017
(sorry for my bad english)

I think this might be a solution for a different issue? I'm using hiddenName but not name.


Setup

relax linting to airbnb-base
remove createTag()
set up sidekick for helix 3

I am not sure why we would remove createTag, but fine with me for now.

I went back and forth on this one. I thought it's the only function that's somewhat debatable as it's merely a shorthand for document.createElement and not really needed for the site to work. I personally appreciate the efficiency of it, but I also understand it's a matter of taste.


Use current token as default query

Currect only "A previously run command" uses existing input as fzf's default query. I think this should apply to all functions.
For variable searching I did some extra work for leading $:
https://user-images.githubusercontent.com/44045911/103555474-4feac880-4eeb-11eb-918c-97878fa43180.mov

Can you rebase this on master so we can test more confidently? Thanks!

Can you rebase this on master so we can test more confidently? Thanks!

Also, I want to thank you from the bottom of my heart for working on hard on making fzf.fish a better plugin. I've learned a lot from your valuable suggestions and more advanced-shell techniques. I feel bad that our review/feedback/improvement cycles are quite long and also want to give you more of a voice as you've demonstrated your shell prowess, so what do you think about setting up some place where we can chat in real-time? Do you use Gitter or Twitter (or even FB) for chats?

Also, I want to thank you from the bottom of my heart for working on hard on making fzf.fish a better plugin. I've learned a lot from your valuable suggestions and more advanced-shell techniques. I feel bad that our review/feedback/improvement cycles are quite long and also want to give you more of a voice as you've demonstrated your shell prowess, so what do you think about setting up some place where we can chat in real-time? Do you use Gitter or Twitter (or even FB) for chats?

Also, I want to thank you from the bottom of my heart for working on hard on making fzf.fish a better plugin.

I also want to thank you for creating such a amazing plugin and making valuable reviews!

I feel bad that our review/feedback/improvement cycles are quite long

That's just how open source works 😄 In fact we are quite fast according to my experience.

so what do you think about setting up some place where we can chat in real-time? Do you use Gitter or Twitter (or even FB) for chats?

I only use Telegram for IM. And let's keep PR related discussion on GitHub.

Also, I want to thank you from the bottom of my heart for working on hard on making fzf.fish a better plugin.

I also want to thank you for creating such a amazing plugin and making valuable reviews!

I feel bad that our review/feedback/improvement cycles are quite long

That's just how open source works 😄 In fact we are quite fast according to my experience.

so what do you think about setting up some place where we can chat in real-time? Do you use Gitter or Twitter (or even FB) for chats?

I only use Telegram for IM. And let's keep PR related discussion on GitHub.

Well, it sure is wonderful to partner with you. What gave you such an interest in this plugin to pour in so much work into it?
Previously, I chatted with @IlanCosman who also helped a ton on Gitter but sure, let's just keep conversations on GitHub since the feedback loop is fast enough already for you.

Well, it sure is wonderful to partner with you. What gave you such an interest in this plugin to pour in so much work into it?
Previously, I chatted with @IlanCosman who also helped a ton on Gitter but sure, let's just keep conversations on GitHub since the feedback loop is fast enough already for you.

Well, it sure is wonderful to partner with you. What gave you such an interest in this plugin to pour in so much work into it?

Because it's easier than writing yet another one... Just kidding. I use it so why not make it better.

Well, it sure is wonderful to partner with you. What gave you such an interest in this plugin to pour in so much work into it?

Because it's easier than writing yet another one... Just kidding. I use it so why not make it better.

Note for posterity: I had struggled with whether to merge this or not because I wasn't sure this change was worth maintaining if it's not very discoverable and it makes fzf.fish's behavior more complex, a bit more unpredictable than always inserting the user's selection. However, I decided to merge because

most users will probably have an empty current token when they trigger fzf, so this won't affect them
those who do have a current token when they trigger fzf ended up having to go back and modify their commandline because the current token would become one word with what fzf.fish inserts into the commandline
there's already a strong precedence of replacing the current token with autocomplete and suggestions being built into fish
this behavior can be easily documented with new gifs

Note for posterity: I had struggled with whether to merge this or not because I wasn't sure this change was worth maintaining if it's not very discoverable and it makes fzf.fish's behavior more complex, a bit more unpredictable than always inserting the user's selection. However, I decided to merge because

most users will probably have an empty current token when they trigger fzf, so this won't affect them
those who do have a current token when they trigger fzf ended up having to go back and modify their commandline because the current token would become one word with what fzf.fish inserts into the commandline
there's already a strong precedence of replacing the current token with autocomplete and suggestions being built into fish
this behavior can be easily documented with new gifs


Improve UX to configure TCP buffer sizes

In VerneMQ one wants to configure the TCP buffer sizes for the MQTT client connections, the VMQ Inter-Cluster connections, and the Erlang distribution connections.
The buffer sizes for the MQTT client connections are currently taken directly from the OS, to be precise VerneMQ tries to get the max allowed buffer Size from the OS. Because the OS as well as VerneMQ internally allocates memory for the buffer we often observe Out-of-Memory issues caused by too big TCP buffers. Currently the buffer size could be adjusted using a custom auth_on_register plugin that returns the tcp_opts modifier, or by globally setting the vmq_server.tcp_listen_options using an advanced.config config file. E.g.
[{vmq_server, [
          {tcp_listen_options,
          [{sndbuf, 4096},
           {recbuf, 4096}]}]}].

The buffer sizes for the VMQ Inter-Cluster connections are by default taken directly from the OS. Those buffer sizes could be globally adjusted like above via the advanced.config or specifically for the cluster connections via setting the vmq_server.outgoing_connect_opts also in the advanced.config e.g.
[{vmq_server, [
          {outgoing_connect_opts,
          [{sndbuf, 4096},
           {recbuf, 4096}]}]}].

While not a direct TCP configuration the Erlang distribution buffer sizes are also important and can be adjusted in the vernemq.conf via erlang.distribution_buffer_size which will add the +zdbbl flag to the generated vm.args. Obviously, this setting doesn't affect a custom vm.args that you might have deployed in /etc/vernemq/vm.args. Note: the current Docker images rely on a custom vm.args file, therefore all the Erlang VM related configurations (like this one) don't work at the moment.

Can we close this after #1169 was merged? /cc @dergraf

Looks like this one could be closed @ioolkos ?

@codeadict jep, thanks


Drop codegen from generator packages towards something more generic

This is only relevant for the schema plugin, and even then it still can generate files, though not "code".
How about "@graphback/schema-graphqlcrud" or "@graphback/schema-plugin"

How about "@graphback/schema-graphqlcrud" or "@graphback/schema-plugin"

+1 for "@graphback/schema-graphqlcrud". Another suggestion along the lines is "@graphback/graphqlcrud-plugin".

Let's close this - since the plugin still does generate a schema file it's fine where it is and make sense.


Tests & Release

Hi Dan,
Have you considered how testing and bundling might work in this setup?
I ask because tsc doesn't currently support bundling so it seems like we'd need to bring in some other tool in order to support testing and bundling.

Definitely something that will have to be figured out. I doubt I'll do anything along those lines in this project, but in a "real life" project it'll be a requirement. For testing, they now have some documents on the http://angular.io site if you're interested. For bundling, I've been playing around with JSPM but there are some other options out there (WebPack, etc.).


Add Tracer.Flush() on all platforms?

There are two important use-cases for Tracer.Flush():

process shutdown (graceful or ungraceful)
tests

That said, a Tracer that does no buffering has no need for a Flush() method, so in that sense we would be making (potentially inaccurate) assumptions about Tracer implementations.
OT users could alternatively manually maintain a reference to the actual Tracer impl type and call Flush() on it directly (in signal handlers, in unit tests, at the end of main(), etc)... In my mind, though, buffering will be so widespread that Tracer.Flush() is worth factoring into the main API, even if it's a noop for some impls.
See also: https://github.com/opentracing/opentracing-go/issues/66

I like having Flush, but could be persuaded otherwise. I don't know of a use case that cannot be solved without public Flush() method.

If there are no objections, I propose that we add Tracer.Flush() to the spec (and the various platforms)... Let's wait >= 48 hours to see if there are concerns.
I'll cc some folks who might care: @yurishkuro @tschottdorf @dkuebric @michaelsembwever @adriancole @wladekb @bcronin

any asynchronous implementation in Tracer should be providing and connecting its own shutdown hooks. and client code needs to be able to configure the asynchronicity, ie when writing crash-only software.
i can only see a need for this at the test level, and then this becomes test-pollution.
in most languages i'm pretty sure that tests can access non-api methods (via package-protected scope or via subclassing).
but i have no strong objections.

I'm fairly neutral on this, but overall it seems like a net gain.

As I've been implementing tracers in several languages now, I am now inclined to say that we shouldn't require Flush() and leave this function to the implementation. Main reasons:

as was already stated, there doesn't seem to be a strong production use case for this, at least not one that cannot be solved without such official method
if we did require it, we would need to debate which exact async API such method would implement (blocking, future, callback, etc.)
the overall semantic meaning of flush() is not clear, at least vs. Close(). With Close() the tracer can stop accepting/reporting new spans, and therefore the "flush" step is well-defined. But with plain flush(), especially called in the middle of the run, tracer may still continue receiving spans, so when is flush() supposed to return?

I'm swayed by @michaelsembwever and @yurishkuro's arguments and – after doing some more digging into our own (non-public) code – I'm realizing that the sort of "process exit and signal handler" code can be designed to have access to the OT impl, thus allowing it to provide a "flush" method as needed.
I can still see us adding something like this in a while (given enough evidence of its ubiquity in implementations), but there's no urgency.
So: I will close this issue in 48h unless there are objections :)

Resolved unless new information arises: Tracer.Flush() (or Close() or similar) are the responsibility of implementations and not part of the OT API proper.


HandleCallback not fireing.

In your WinBleTest.cpp I changed the Service and Characteristics to those for the device that I am testing.
In the main.cpp I can enumerate all of the services and all of the characteristics.
In the testSerialService I can setValue for a Tx characteristic in the service that is writeable.
However, when it gets to the part in testSerialService I never get any descriptors. I get a descriptors.size() of 2 so I go into that if (descriptors.size() > 0) block but nothing ever is reported. In fact, the HandleCallback never fires.
Do you still get the HandleCallback for your device? or did something change between VS2017 and VS2019 etc?
Edit Addition:
I  started from scratch and opened and compiled in VS2017 with the v141 Toolset and SDK 10.0.18362.0.  I changed the UUIDs to those for my device and ran.  Again, the HandleCallback never fired.
In case this explains anything since I don't have the data for your device, my main service is FFF0 in which the FFF4 characteristic is for returns (Rx) and only has the IsNotifialbe set to true. The FFF5 is the characteristic for sending commands (Tx) and only has the IsReadable and IsWrteable set to true.  If you getValue for this Tx service it will only return the last command sent via setValue.
So, because my Tx does not have an IsNotifiable, I get a txit and a rxit and get a chacteristicTx for the UUID_FFF5Tx_CHARACTERISTIC and a chacteristicRx for the UUID_FFF4Rx_CHARACTERISTIC.  I send the command to turn the power on to the characteristicTx.  I cannot enumerateBleDescriptors for the Tx since it is not IsNotifiable so, from that point on I use the characteristicRx since it is the receiving characteristic.  Everything else remains the same as your testSerialService.  I get no errors it is just that HandleCallback never fires and no descriptors are every listed.

The firing of the callback depends on the type of characteristic the test device exposes. The characteristic has to be notifiable inorder to generate a callback. As I have said this code is only POC I already know that the TX characteristic is notifiable so I don't actually check in the console app, in a production quality application all the characteristic attributes should be validated.

Derek,
On mine, the Tx characteristic is not notifiable and gives the appropriate error if I try to get its descriptors.  I can see from the displayServices cout reports that the Tx is only writeable and readable.  The Rx is only notifiable.  But when I try to get the descriptors for that Rx it goes through
enumerateBleDescriptors()
descriptor->getValue()
registerNotificationHandler(callback) and
setIsSubscribeToNotification()
with no errors or problems but the actual  callback (HandleCallback) is never triggered.  That is why I was wondering if you were still able to get a callback or if something has changed since you last tested it.
I can also run the MS Bluetooth LE Explorer app and see the same characteristic properties as I see with WinBle and I can see the Rx characteristic value change when I enter a write value in the Tx characteristic.  I just can't see that Rx value change in the WinBle because the callback never fires.  Everything I have read says it is supposed to fire automatically.  I have followed the code via Debug and all goes according to design only the callback never fires.  I was wondering if you had any guesses.  As I said the Rx is notifiable and the WinBle acknowledges that and proceeds accordingly.  Is there maybe another command line that is needed to trigger the callback?
Ed
At 06:08 AM 7/19/2022, you wrote:

The firing of the callback depends on the type of characteristic the test device exposes. The characteristic has to be notifiable inorder to generate a callback. As I have said this code is only POC I already know that the TX characteristic is notifiable so I don't actually check in the console app, in a production quality application all the characteristic attributes should be validated.

Reply to this email directly, https://github.com/DerekGn/WinBle/issues/14#issuecomment-1188919419view it on GitHub, or https://github.com/notifications/unsubscribe-auth/AIURENH67CRBCTASAKC62F3VU2EDDANCNFSM53477ONQunsubscribe.
You are receiving this because you authored the thread.Message ID: @.***>

--
This email has been checked for viruses by AVG.
https://www.avg.com

Derek,
Yes my Rx characteristic IsNotifiable and your code checks for that and will throw and exception if it is not.  The code also registerNotificationHandler(callback) for the characteristics without problems and also sets the descriptor->setIsSubscribeToNotification();... I'll keep playing with it.
Thanks again.
At 09:22 AM 7/19/2022, you wrote:

The TX and RX characteristic may be the wrong way around in this code. Whats important is the characteristic attributes being notifiable. Note you might actually need to enable notifications from a characteristic. Again this is device specific in my case the characteristic has notification enabled.

Reply to this email directly, https://github.com/DerekGn/WinBle/issues/14#issuecomment-1189118600view it on GitHub, or https://github.com/notifications/unsubscribe-auth/AIURENFSMO2YVUOMOSVSG33VU2237ANCNFSM53477ONQunsubscribe.
You are receiving this because you authored the thread.Message ID: @.***>

--
This email has been checked for viruses by AVG.
https://www.avg.com

I found the solution but it was not anything that I would have guessed.
In my continued Google search I came across this stackOverflow post
https://stackoverflow.com/questions/56923982/bluetooth-le-setting-characteristic-to-byte-array-sends-wrong-values
In it the OP makes a comment that "Incidentally, even though the characteristic is marked as Writable: true, Writable-no-response: false, I need to set the flags to no-response in order for the values to get sent."
Just for the hell of it I changed the flag in setValue()  BluetoothGATTSetCharacteristicValue() from BLUETOOTH_GATT_FLAG_NONE to BLUETOOTH_GATT_FLAG_WRITE_WITHOUT_RESPONSE and BOOM!  the callback fired and I got several Received callback data: responses.
Who would have guessed?  Jeeeeze!
Thanks again for posting this code.  I would have never even gotten this far without it.
Ed
At 09:22 AM 7/19/2022, you wrote:

The TX and RX characteristic may be the wrong way around in this code. Whats important is the characteristic attributes being notifiable. Note you might actually need to enable notifications from a characteristic. Again this is device specific in my case the characteristic has notification enabled.

Reply to this email directly, https://github.com/DerekGn/WinBle/issues/14#issuecomment-1189118600view it on GitHub, or https://github.com/notifications/unsubscribe-auth/AIURENFSMO2YVUOMOSVSG33VU2237ANCNFSM53477ONQunsubscribe.
You are receiving this because you authored the thread.Message ID: @.***>

--
This email has been checked for viruses by AVG.
https://www.avg.com

Derek,
One last question while I have you on the line.  Do you know if there is a way to verify that the BTLE device still has a connection before issuing a setValue or would I have to go through all of the enumerate devices, services, etc. each time to re-establish a connection?  My program will be issuing setValue commands at unspecified intervals and I have noticed that sometimes I will come back with a setValue and nothing happens.  But if I precede the setValue with the whole enumerate sequence that you have in main then it connects.  I was wondering if there was a shorter way of doing this.
Ed
Ed
At 04:00 AM 7/20/2022, you wrote:

Welcome to the world of Bluetooth, dispute there being a certification program and a set of tests that devices that are Bluetooth compliant must pass there are unfortunately many reasons why a device does not behave as expected. Sometimes its because the device was never certified, later firmware changes break compatibility with the certification etc. There are also issues with the Bluetooth drivers which cause compatibility issues. I have direct experience of Bluetooth issues caused buy how a chip set vendors driver interpreted the specification. These drivers + chip sets are used in millions of devices.
So you will always find examples of devices + chipsets that don't behave as expected even though they are qualified Bluetooth products.

Reply to this email directly, https://github.com/DerekGn/WinBle/issues/14#issuecomment-1190016147view it on GitHub, or https://github.com/notifications/unsubscribe-auth/AIURENGMIO6QMMR5MYRNUBLVU654VANCNFSM53477ONQunsubscribe.
You are receiving this because you authored the thread.Message ID: @.***>

--
This email has been checked for viruses by AVG.
https://www.avg.com

The opening of  the device handle specifies the device should be present. I am not sure if there is a way to determine if the device is still connected. It sounds like the BLE connection is being dropped, hence why the enumeration of the devices works.
Maybe the device handle can be queried to see if the device is still present, but off the top of my head I would not be 100% sure unless I looked into it again.

I tested by getting a device handle via
HANDLE hDevice = bleDevice.getBleDeviceHandle(bleDevice.getDeviceIntstanceId());
After the 1st testSerialService and its call to setValue I slept  for 10 seconds and then checked if the handle was till there.  If it was I skipped the enumerateBleServices etc.
Apparently on disconnecting the handle is not automatically cleared.  Regardless, that would just add additional lines of code that would not be necessary since bleDevice.enumerateBleServices() runs if it is still connected or not and would have to be run if it was not connected so just running it before any call to testSerialService would be the simplest thing to do.
From testing, I have determined that only these 2 lines need to be run to re-connect.
bleDevice.enumerateBleServices();
displayServices(bleDevice);
I test for the device being turned off during the process so I can just rely on the original device enumerator still being good.
However, since bleDevice is instantiated in main I cannot put these 2 lines at the top of the what was your testSerialService.  I have to keep the 2 lines in main right before any new setValue (or actually before any call to testSerialService) .  I tried passing the bleDevice to the testSerialService but (maybe I did it work) that didn't work.  Can you think of a way that bleDevice could be referenced in a function away from main?
At 01:15 PM 7/21/2022, you wrote:

The opening of the device handle specifies the device should be present. I am not sure if there is a way to determine if the device is still connected. It sounds like the BLE connection is being dropped, hence why the enumeration of the devices works.
Maybe the device handle can be queried to see if the device is still present, but off the top of my head I would not be 100% sure unless I looked into it again.

Reply to this email directly, https://github.com/DerekGn/WinBle/issues/14#issuecomment-1191793052view it on GitHub, or https://github.com/notifications/unsubscribe-auth/AIURENEVYL6MTSXSLDFSWPTVVGHTXANCNFSM53477ONQunsubscribe.
You are receiving this because you authored the thread.Message ID: @.***>

--
This email has been checked for viruses by AVG.
https://www.avg.com

I was able to pass bleDevice by reference by adding ,BleDevice &bleDevice to the parameter list of the testSerialService().  That simplifies me having to add the two line before each and every call to testSerialService.
Thanks again..I'm slow but I eventually get there ;-)
At 01:15 PM 7/21/2022, you wrote:

The opening of the device handle specifies the device should be present. I am not sure if there is a way to determine if the device is still connected. It sounds like the BLE connection is being dropped, hence why the enumeration of the devices works.
Maybe the device handle can be queried to see if the device is still present, but off the top of my head I would not be 100% sure unless I looked into it again.

Reply to this email directly, https://github.com/DerekGn/WinBle/issues/14#issuecomment-1191793052view it on GitHub, or https://github.com/notifications/unsubscribe-auth/AIURENEVYL6MTSXSLDFSWPTVVGHTXANCNFSM53477ONQunsubscribe.
You are receiving this because you authored the thread.Message ID: @.***>

--
This email has been checked for viruses by AVG.
https://www.avg.com


Allow Analytics example to work better within cookbook site iframe

This isn't optimal but I'm struggling for a better solution.  The "Analytics API" demo appears wrong because the table gets populated after page load.
I'm really hoping there's an easier, cleaner solution that I'm not seeing.

This isn't optimal but I'm struggling for a better solution.  The "Analytics API" demo appears wrong because the table gets populated after page load.

I'm really hoping there's an easier, cleaner solution that I'm not seeing.

I think we might be able to fix this by adding width: 100% to '.main-inner' and 'iframe' in src/css/style.css
I'm not able to test because for some reason my local builds aren't displaying the site properly

Height is more so the issue -- how would modifying width help?

Height is more so the issue -- how would modifying width help?

Sorry, totally meant height. Updating comment.
I tried this out in a branch and it seems to work, see https://github.com/TimAbraldes/serviceworker-cookbook/commit/74af6447ec166c4e4dc459e1161c55e24b8c86d8

I implemented what you have here but the JSON Cache demo, for example, gets cut off.  The problem is that using height: 100% on the IFRAME is it's 100% of the current page, not the IFRAME's contents. :/


Define a simple interface to be able to implement authentication and authorization.

Right now we just should able to implement a basic MySQL auth implementation, with users and passwords on a config file, but that interface must be prepared to do integrations in the future with LDAP, and Kerberos.
Also, audit logs showing who is executing which query should be taken in mind too.

Vitess already defines authentication mechanisms and comes with several methods:

Static: The currently used mechanism. Right now on start it add only one user but users could be loaded from a json file: https://github.com/src-d/gitbase/blob/master/vendor/gopkg.in/src-d/go-vitess.v1/mysql/auth_server_static.go#L35
LDAP: It already comes with an ldap auth diver: https://github.com/src-d/gitbase/blob/master/vendor/gopkg.in/src-d/go-vitess.v1/mysql/ldapauthserver/auth_server_ldap.go

I have to check if it's possible to save hashes of passwords with salt instead of plain text. It seems feasible as the salt is sent by the server:
https://github.com/src-d/gitbase/blob/master/vendor/gopkg.in/src-d/go-vitess.v1/mysql/auth_server_static.go#L183-L186

Also, it seems feasible to change the authentication method. Right now it uses mysql_native_password and some new software complains as it tries to use a newer one caching_sha2_password.
https://dev.mysql.com/doc/refman/8.0/en/caching-sha2-pluggable-authentication.html


insufficient quota to match these scopes

Bug report
We're getting Error creating: insufficient quota to match these scopes: [{PriorityClass In [system-node-critical system-cluster-critical]}] once we have upgraded to 1.8.5
General Information

cilium versions 1.8.5
K8s version 1.17.12-gke.1504

We were using 1.8.3 on a 1.17 cluster and we made a fresh cluster and bumped the version to 1.8.5 on the install.
EDIT: Just tried it with cilium 1.8.3 and got the same error, I now suspect it's GKE?
EDIT EDIT: it looks like the priorityClasses are getting created:
joshua_knarr@cloudshell:~ (ips-non-prod-mydf)$ kubectl get priorityclass
NAME                      VALUE        GLOBAL-DEFAULT   AGE
system-cluster-critical<PHONE_NUMBER>   false            4h1m
system-node-critical<PHONE_NUMBER>   false            4h1m

EDIT EDIT EDIT: Turning it to false in the helm chart (whatever the option name is) doesn't disable the behavior.
How to reproduce the issue

provision a 1.17.12-gke.1504 GKE cluster
Remember to set your NATIVE_CIDR and install cilium with:

helm install cilium https://github.com/cilium/charts/raw/master/cilium-1.8.5.tgz --namespace cilium --create-namespace --set global.nodeinit.enabled=true --set nodeinit.reconfigureKubelet=true --set nodeinit.removeCbrBridge=true --set global.cni.binPath=/home/kubernetes/bin --set global.gke.enabled=true --set config.ipam=kubernetes --set global.nativeRoutingCIDR=$NATIVE_CIDR --set nodeinit.restartPods=true --set global.hubble.enabled=true --set global.hubble.metrics.enabled='{dns,drop,tcp,flow,port-distribution,icmp,http}'

Still trying to nail down the behavior - the org I'm working in has a seriously locked down cluster for compliance reasons, this does not happen on generic GKE clusters. I believe it's an org policy issue. My humble suggestion is to just allow us to turn off the feature if we're not allowed to use it with the caveat that this will allow pod pre-emption.

helm install cilium https://github.com/cilium/charts/raw/master/cilium-1.8.5.tgz --namespace kube-system --create-namespace --set global.nodeinit.enabled=true --set nodeinit.reconfigureKubelet=true --set nodeinit.removeCbrBridge=true --set global.cni.binPath=/home/kubernetes/bin --set global.gke.enabled=true --set config.ipam=kubernetes --set global.nativeRoutingCIDR=$NATIVE_CIDR --set nodeinit.restartPods=true --set global.hubble.enabled=true --set global.hubble.metrics.enabled='{dns,drop,tcp,flow,port-distribution,icmp,http}'
Fixes the issue?

Yeah running this and specifying --namespace kube-system does resolve my error.


mongoaudit unattended (not interactive)

thanks for mongoaudit. Gave me some nice inputs for our config.
Is it possible to run mongoaudit unattended?

Hi @GETandSELECT,
Thanks a lot for your feedback.
We are considering separating the logic from the UI so that it can be run unattended, which we understand can be quite interesting for some use cases.

This would definitely be a great addition. Especially if you are running a larger fleet of servers and want to set up periodic unattended scans.


[Bug] can not catch the error occur in webWorker

Describe the Bug
When an error occur in web worker task, i can not be catch. because the handle function in decodeTask file is async, need to catch it with await it.
Steps to Reproduce
while an error occur in webWorker, such as 'target array for image does not have the same length as the decoded image length'
The current behavior
can not catch the error occur in webWorker.
The expected behavior
catch the error occur in webWorker.
OS
Windows11
Node version
v18.17.1
Browser
Chrome  116.0.5845.180

if the error says it is not the same length I guess it is good that we are not catching it and throwing explicitly. Why do you want to catch it?

if the error says it is not the same length I guess it is good that we are not catching it and throwing explicitly. Why do you want to catch it?

I want a way to know if the web worker is processing correctly. If errors can be caught, corresponding actions can be performed, such as displaying errors on the interface


GUIT-1045 Set JWT into cookie

Already tested with OPC UA Browser GUIT-1045-branch, and I will open that PR after this one is merged, thanks.


Message history encryption?

Do you want to demonstrate how to encrypt the message history in DynamoDB in this example?
https://github.com/langchain-ai/langchain/blob/aad3d8bd47d7f5598156ff2bdcc8f736f24a7412/libs/community/langchain_community/chat_message_histories/dynamodb.py#L51

Hi Justin - The DynamoDB table is already encrypting data at-rest as the SSE specification is set to True.
For double encryption, you can specify a KMS key ID when calling LangChain's DynamoDBChatMessageHistory method in the agent's chat class.


Feedback for: docker-cloud/getting-started/deploy-app/12_data_management_with_volumes.md

File: docker-cloud/getting-started/deploy-app/12_data_management_with_volumes.md
$ docker-cloud service redeploy redis

This command does not reset the counter to zero. Is this because redeploying now reuses the existing container volumes by default? If I do the same thing at https://cloud.docker.com there's a toggle switch "Reuse existing container volumes?". If I turn this off then the counter is set to zero. I'm not sure how this would be done through the CLI so I couldn't submit a change request.

Found it!
docker-cloud service redeploy redis --not-reuse-volumes

Works. Will submit a change request.

This tutorial doesn't seem to work because redeploying the service doesn't reset the counter. It appears the volume is set up from the beginning?

Hey @sardjv I don't think the --not-reuse-volumes flag is correct. (I believe it's --reuse-volumes=false_

Hm, it looks like I need to update the API docs - hold tight I'm asking around.

Relates to #1181

Thanks again @sardjv !

@sanscontext what's the correct flag then: --not-reuse-volumes or --reuse-volumes=false?


Update Readme.md with fixed link

Required items, please complete
Change(s):

Removed full-stop from https://openai.com/.api/pricing/

Reason for Change(s):

The link was broken due to the full-stop.

Testing Completed:

Yes

Checked that the validations are passing and have addressed any issues that are present:

Yes

Thank you. Do you know why this is still showing as review required @v-mchatla?


Allow taxon export

We would like to allow users to export taxons in order to make it easier to
create tagging spreadsheets.
Those spreadsheets need a mapping between taxon names and their content IDs. The
CSV file we are generating gives us exactly that, together with the link type.
This way, we can download a list of user-selected taxons into a format expected
by the tagging spreadsheet.
More information in trello: https://trello.com/c/gRoOrJoo/121-spike-build-functionality-into-content-tagger-which-makes-creating-a-tag-importer-spreadsheet-easier
It looks like this:

@mobaig thoughts?

This looks solid, but I'm not sure we actually need a CSV export. If we just render an HTML table with the three required fields the user can just copy paste that straight from the browser window into Google Sheets.
Taking that approach allows us to achieve basically the same result with less code. What do you reckon?

@mobaig I thought about a few options - the HTML table, the Google API and then this CSV exporter.
The google API added a bit more complexity, so I discarded it for now.  I thought that adding an HTML table for the purpose of copy/pasting taxons had the potential to be less clear than exporting a CSV that Excel users are already used to work with.
However, if you think it's easier to have an HTML table I can go ahead and change that part of the PR.
By the way, is the pre-selection of taxons ok (the checkboxes)?

Opening a new PR soon with a different approach


Run on new robot ?

Hi,
Great work ! And thank you very much for sharing the code and pretrained weights !
After browsing the repo and project page, I am a little bit confused on how I could apply octo to a new robot.
I would like to run it on a humanoid robot (Reachy https://www.pollen-robotics.com/), which has a geometry that is quite different from the robot arms you showcase.
I guess I would need to collect data for fine tuning and provide a description of the robot to the system somehow, as well as write some kind of interface ? But it is not clear to me what are the expected formats for the data, the robot description (URDF?) etc.
Could you give an overview of the required steps ?
Thank you vey much !

Thanks for your interest and the great question!

Yes, you need to record some expert demonstrations and store them in the RLDS format. If you have an existing dataset, you can check this example on how to do the conversion.
Once you have a target dataset, you can start by trying finetuning to your robot, by specifying your observation and action space. A great example on how to do this is this notebook where we show how to finetune to a bimanual setup with joint control.

Does this answer your question?

Yes thanks !
We probably try it out in the following months, what amount of expert demonstration data would you consider sufficient for fine tuning on a new robot ?

Hmm that's very dependant on the task, the action and observation space etc.  The finetuning experiments in the technical report used around 100 in-domain demos.


Refreshing the RubberNugget webapp triggers a hard crash

Refreshing the webapp, or accessing the webapp menu link run.html causes the RubberNugget to crash.
Running other edit and save functions work fine.

The getFileList() function created a new String array every time it was called.  For temporary mitigation, I've moved the array outside the scope of this function and just reset all the values to blank when getFileList() is called.
Download the latest binary for version 1.0.3 to fix this issue.


Resurrecting Swift bindings / how much of the core C++ API can run on iOS?

I'm directly addressing TensorFlow's admins and maintainers of the core C/C++ code with this issue.
Before starting out, I am not using TensorFlow for inference on iOS. I am doing training, which is orders of magnitude more difficult to support. It has never been done on a high-level, feature-complete ML framework except for DL4S (MPSGraph and ArrayFire don't count). Using an iPhone or iPad for training may seem strange, but I have numerous reasons for doing so (leaving out those reasons for brevity).
The existing TensorFlow training API is only compatible with macOS, Linux, and Windows. I am concerned with the C API used internally by Swift for TensorFlow. iOS can't run Python, which means it certainly can't run the TensorFlow in this repository. However, Swift for TensorFlow brings an opportunity to take Python out of the equation. C and C++ can run on iOS, and I assume a lot of the code is platform-agnostic. Has anyone ever succeeded in training TensorFlow models on an Android device?

Side note: I know Swift for TensorFlow is "archived", and I'm fixing that right now. In fact, the Metal backend is part of S4TF's resurrection. I have had extensive conversations with several members of the original S4TF team (6 so far) and I know about the obstacles I will face.

Apple released a PluggableDevice implementation of GPU acceleration for Macs. However, it's closed-source, missing several ops it could support, and shuts out low-end Intel Macs. I'm looking for universal training support just like DL4S. Is there any chance that I could reuse the C TensorFlow API as a component of my Metal backend for iOS?
This would save time and make DirectML acceleration for Swift for TensorFlow easier. Also, there are several ops like Cholesky grad which aren't in Metal Performance Shaders or Accelerate. I don't have the slightest clue how to implement them in CPU code. I assume that TensorFlow's core C API contains CPU kernels for those obscure ops.
One more thing - if I could use PluggableDevice for my backend, it would be much more in line with TensorFlow and less likely that my S4TF branch would diverge from main and eventually die. I'm approaching this  with the objective of realistically resurrecting S4TF. Whether TensorFlow's owners can change their mind and officially support my work has a massive impact on its long-term viability. How can I contact the people who have authority over this?

Another side note: Apple just responded to me about an issue with syntax coloring in Xcode. I spent three months waiting for them without hearing back, and it should not have taken that long. I fear that Google will treat me the same way (part of the reason I'm not reaching out through private email), but hopefully it's going to be different this time.

After scanning the core C/C++ code, there’s no mention of iOS - outside of TensorFlow Lite (again, not my goal). However, iOS and macOS are very similar, and the same can be said for Linux and Android. Are there any fundamental barriers to bringing it to iOS using macOS?
For example, loading a specific dynamic library at runtime, JIT compiling CPU code, relying on the structure of the macOS file system in a way that deviates from generic Darwin.
If I do find out how to modify the macOS code to run on iOS, how much of a chance is there for TensorFlow to merge my additions into main? I expect you might also want something similar for Android to make it equal and consistent.
I would be okay with keeping Swift for TensorFlow exclusive to desktop platforms initially, as long as I could guarantee it’s possible to train on iOS in the future. There are other problems (Swift autodiff) that currently prevent deployment on iOS anyway, which won’t be resolved for several months.

CC: @sachinprasadhs

@rohan100jain the information in this issue should be helpful: https://github.com/tensorflow/swift-apis/issues/1190

@saeta I think you're the person I should be talking to.

Thank you so much @philipturner for reaching out!

If it was archived for some other more serious or not publicly disclosable reason, I imagine that my request will be very controversial and require a lot of discussion internally at Google. That's why I'm reaching out to you and building support in the community so that Google can't say no 😄

The decision to archive S4TF was based on (among many other things) private discussions between two companies. Suffice it to say, while no one wanted to shut down S4TF, it was (unfortunately) not reasonable to continue forward with the project as envisioned at Google. Sorry!
That said, I'd be delighted for you to take over the project and move things forward in whatever way you envision (feel free to fork it!). There will likely be a number of choices you'll have to make. S4TF evolved from targeting a library of kernels (i.e. TF's kernel library) to targeting XLA. (The reasons for doing this are beyond the scope of this comment, but I'd be happy to share.) The problem with this approach (historically) on iOS has been that JIT compilation was forbidden (it seems that this might have changed in the last year or so); as a result, I don't know of anyone who has tried getting XLA to work on iOS. (Related to this is your autodiff strategy; I see @ProfFan has already mentioned the associated caveats.) This affects what C APIs you need, and what portion of the Core TensorFlow codebase must be ported to work on iOS.
Although I don't want to dissuade you, I'd be remiss if I didn't mention that I believe you can do some forms of on-device training with TFLite (although this won't save you from having to author the model in Python).

If I can't use PluggableDevice, I'm planning to rip out XLA on most platforms and replace it with an open backend for MPSGraph and DirectML. The reason I need TensorFlow/Google's support is for publicity and maintainability purposes. I need to preserve the public API and a fork is destined to die.

The decision to archive S4TF was based on (among many other things) private discussions between two companies. Suffice it to say, while no one wanted to shut down S4TF, it was (unfortunately) not reasonable to continue forward with the project as envisioned at Google. Sorry!

If possible, could you tell what company made Google shut down S4TF? Was it Apple going for Python TensorFlow? Given how many people are dissatisfied by it shutting down, there needs to be an answer for why that decision happened.

Would it be acceptable if Google sponsored a genuine successor that reused 90% of the code base but changed its name to omit TensorFlow? I have a problem with caching TensorFlow binaries, as my GitHub account doesn't allow uploading files larger than 25 MB (x10 binaries are 40 MB). So, without how it was before, I have to stay on TensorFlow 2.4.0 for platforms it will be the backend for (Google Colab and CUDA), using the last one-off builds Google ever made of x10.
There also might be licensing issues with that approach because of the way I'm setting up the successor. Is it possible to transfer ownership of the repository to another entity or allow me to preserve the commit history on a new repository without the restrictions of it being a GitHub fork? At a minimum, could there be one final addition to the S4TF README citing my Swift-Colab and successor as spin-off projects like PythonKit and AutoDiff?

I anticipate that what I'm about to hear next will not be very pleasing. In advance:
I knew all along that Google probably wasn't going to support this. Nonetheless, I pushed it 100% down the rabbit hole to guarantee they wouldn't help me. Moving forward, my objective will be more difficult, but it should still be possible. If I find the right people, I should be able to acquire the resources I need to bring Swift for TensorFlow back to life.

@saeta when I made this post asking for Google’s support, I didn’t realize the following:

I can post files larger than 25 MB on GitHub, using binary downloads from releases.
I can make my own releases on a fork.
I can enable issues and discussions on a fork made from S4TF.

This knowledge effectively solves my problems. Still the open-source community needs an answer for why Google stopped working on S4TF.


TagBot trigger issue

This issue is used to trigger TagBot; feel free to unsubscribe.
If you haven't already, you should update your TagBot.yml to include issue comment triggers.
Please see this post on Discourse for instructions and more details.
If you'd like for me to do this for you, comment TagBot fix on this issue.
I'll open a PR within a few hours, please be patient!

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/65087

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/65364

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/68314

Triggering TagBot for merged registry pull request
This extra notification is being sent because I expected a tag to exist by now, but it doesn't.
You may want to check your TagBot configuration to ensure that it's running, and if it is, check the logs to make sure that there are no errors.

Triggering TagBot for merged registry pull request
This extra notification is being sent because I expected a tag to exist by now, but it doesn't.
You may want to check your TagBot configuration to ensure that it's running, and if it is, check the logs to make sure that there are no errors.


[HOLD for payment 2023-12-06] [ECARD] CRITICAL: Transactions - Enable MoneyRequestView Edit Transactions Logic

Design Doc Section
https://docs.google.com/document/d/1iELOXPOAcnuIPZJ3XbO1S8_3qHaM31Ptf8dR4LRZMyE/edit#bookmark=id.qpl4zgrq5w2x
Overview
Goal:
We need to Hold on updates to EditMoneyRequest to handle card transactions. Once that’s complete, we will add the edit logic for non reimbursable transactions to NewDot.
Upwork Automation - Do Not Edit

Upwork Job URL: https://www.upwork.com/jobs/~0149be352f2615a7b7
Upwork Job ID:<PHONE_NUMBER>508114432
Last Price Increase: 2023-10-04

HOLD on edit functionality
cc @luacmartins

This is on Hold still

still on hold

on hold still, but close!

I'm going ooo until Monday Nov 27th so going to assign another BZ teammate to monitor this until I'm back. Thanks!
Status: On hold but working on https://github.com/Expensify/App/pull/31353

Looks like the blocker label has been removed from https://github.com/Expensify/App/issues/32077 and @grgia is working on a solution.
I think we're good to continue working on this job!

prepping for payment!
Payouts due:
Issue Reporter: NA
Contributor: NA
Contributor+: $500 @akinwale (in Upwork)
Eligible for 50% #urgency bonus? N
Upwork job is here.

Design Doc Section
https://docs.google.com/document/d/1iELOXPOAcnuIPZJ3XbO1S8_3qHaM31Ptf8dR4LRZMyE/edit#bookmark=id.qpl4zgrq5w2x
Overview
Goal: We need to Hold on updates to EditMoneyRequest to handle card transactions. Once that’s complete, we will add the edit logic for non reimbursable transactions to NewDot.
Upwork Automation - Do Not Edit

I checked your job posted on Upwork https://www.upwork.com/jobs/~0178d694337026c11f , FYI its private job i.e, its details are not visible in upwork. You may need to correct the access while generating JOB via bot :)
Hi, I'm in interested in the project, let me know when we can have a quick discussion call?

Hi @codewithnimesh! Thanks for reaching out - this job is complete but feel free to apply to any jobs that have a Help Wanted label in our Expensify/App repo - https://github.com/Expensify/App/issues?q=is%3Aopen+is%3Aissue+label%3A"Help+Wanted"

@akinwale I paid you in Upwork based on the payment structure here - https://github.com/Expensify/App/issues/28836#issuecomment-1841940306
Closing this but let me know if I'm missing anything!


Facter: error while resolving custom fact "install_server__check_service": No such file or directory - pgrep -f "dtserver"

On every puppet run on a windows 2012r2 server we are getting the error
Facter: error while resolving custom fact "install_server__check_service": No such file or directory - pgrep -f "dtserver"
looking at the code
lib\facter\install_server__check_service.rb
it looks like the issue is we do not have pgrep on puppet agents or windows boxes. For now the quick fix i would recommend is adding
confine :kernel => 'Linux'
to line 2 so it would look something like
Facter.add(:install_server__check_service) do
  confine :kernel => 'Linux'
  setcode do

this will allow the fact to work in Linux and stop the error for windows, ( and i am not sure if dtserver even runs on windows)

Hi,
Thanks for your help. install_server__check_service.rb has been updated as suggested
Marcin


Demo server (gitlab) is down; cannot generate the auth sessions

Since last week, the demo server hosted on your AWS instance seems to have some problems, so the minimal script does not run.
$ cd webarena && python minimal_example.py

FYI
os.environ["SHOPPING"] = "http://ec2-3-131-244-37.us-east-2.compute.amazonaws.com:7770"
os.environ["SHOPPING_ADMIN"] = "http://ec2-3-131-244-37.us-east-2.compute.amazonaws.com:7780/admin"
os.environ["REDDIT"] = "http://ec2-3-131-244-37.us-east-2.compute.amazonaws.com:9999"
os.environ["GITLAB"] = "http://ec2-3-131-244-37.us-east-2.compute.amazonaws.com:8023"
os.environ["MAP"] = "http://ec2-3-131-244-37.us-east-2.compute.amazonaws.com:3000"
os.environ["WIKIPEDIA"] = "http://ec2-3-131-244-37.us-east-2.compute.amazonaws.com:8888/wikipedia_en_all_maxi_2022-05/A/User:The_other_Kiwix_guy/Landing"
os.environ["HOMEPAGE"] = "PASS"  # The home page is not currently hosted in the demo site

Error - It fails here:
 p = subprocess.run(
     ["python", "scripts/generate_test_data.py"], capture_output=True,
+    check=True
)

# It will generate individual config file for each test example in config_files
assert os.path.exists("config_files/0.json")

Actually error handling is poorly done, the root cause is .auth generation has failed:
FileNotFoundError: [Errno 2] No such file or directory: './.auth/gitlab_state.json'

Stack trace:
Traceback (most recent call last):
  File ".../webarena/browser_env/auto_login.py", line 128, in <module>
    main()
  File "<@beartype(__main__.main) at 0x7fbea390ce00>", line 11, in main
  File ".../webarena/browser_env/auto_login.py", line 113, in main
    renew_comb(list(sorted(pair)))
  File "<@beartype(__main__.renew_comb) at 0x7fbef4f7afc0>", line 43, in renew_comb
  File ".../webarena/browser_env/auto_login.py", line 83, in renew_comb
    page.get_by_test_id("username-field").click()
  File "$CONDA_PREFIX/lib/python3.11/site-packages/playwright/sync_api/_generated.py", line 15436, in click
    self._sync(
  File "$CONDA_PREFIX/lib/python3.11/site-packages/playwright/_impl/_sync_base.py", line 104, in _sync
    return task.result()
           ^^^^^^^^^^^^^
  File "$CONDA_PREFIX/lib/python3.11/site-packages/playwright/_impl/_locator.py", line 148, in click
    return await self._frame.click(self._selector, strict=True, **params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "$CONDA_PREFIX/lib/python3.11/site-packages/playwright/_impl/_frame.py", line 489, in click
    await self._channel.send("click", locals_to_params(locals()))
  File "$CONDA_PREFIX/lib/python3.11/site-packages/playwright/_impl/_connection.py", line 61, in send
    return await self._connection.wrap_api_call(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "$CONDA_PREFIX/lib/python3.11/site-packages/playwright/_impl/_connection.py", line 461, in wrap_api_call
    return await cb()
           ^^^^^^^^^^
  File "$CONDA_PREFIX/lib/python3.11/site-packages/playwright/_impl/_connection.py", line 96, in inner_send
    result = next(iter(done)).result()
             ^^^^^^^^^^^^^^^^^^^^^^^^^
playwright._impl._api_types.TimeoutError: Timeout 30000ms exceeded.

I also tried to use the .auth cookies I had generated before, and then I consistently get 502 Errors from the gitlab server instance.
Other information:

playwright==1.32.1
~/.cache/ms-playwright: chromium-1055/  ffmpeg-1008/    firefox-1391/   webkit-1811/`

Any help would be appreciated.

Gitlab is back and the authentication on it should work now.

The minimal_example.py demo still doesn't run (I tried it few days ago and right now). Currently it blocks forever on browser_env/auto_login.py. Can you run the demo?
  52 │   context = browser.new_context()
> 53 │   page = context.new_page()


Static IP address allocation per POD

Currently multus-cni doesn't support. It will be good, if multus-cni supports it as some of the VNF requires it.

Hi @subrnath
In containernetworking/plugins of github (i.e. reference implementation of CNCF/CNI), there is 'static IPAM' plugin. So how about to use it?
https://github.com/containernetworking/plugins/tree/master/plugins/ipam/static
Could you please let me know it solves? Then I will close the issue.

Thanks.
We are using ipvlan. Here is the definition -
apiVersion: "kubernetes.com/v1"
kind: Network
metadata:
name: Test
plugin: ipvlan
args: '[
{
"name": "Test",
"type": "ipvlan",
"master": "",
"ipam": {
"type": "host-local",
"subnet": "CIDR/prefix",
"rangeStart": "",
"rangeEnd": ""
}
}
]'
POD deployment file -
apiVersion: extensions/v1beta1^M
kind: Deployment^M
metadata:^M
generation: 1^M
labels:^M
k8s-app: cmm-k8s^M
kubernetes.io/cluster-service: "true"^M
name: multus-internal3^M
namespace: default^M
spec:^M
minReadySeconds: 111^M
replicas: 1^M
selector:^M
matchLabels:^M
k8s-app: cmm-k8s^M
strategy:^M
rollingUpdate:^M
maxSurge: 1^M
maxUnavailable: 0^M
type: RollingUpdate^M
template:^M
metadata:^M
creationTimestamp: null^M
labels:^M
app: cmm^M
k8s-app: cmm-k8s^M
kubernetes.io/cluster-service: "true"
annotations:
networks: '[
{ "name": "Test" },
]'
spec:^M
hostNetwork: true
  containers:^M
  - name: multus-internal3^M
    command: [ "sh", "-c"]^M
    args:^M
       - while true;^M
         do^M
         echo -en '\n';^M
         sleep 10;^M
         done;^M
    image: <IP_ADDRESS>:5000/ubuntu:v1
    imagePullPolicy: IfNotPresent^M
    resources:^M
      limits:^M
        cpu: "1"^M
        memory: 1Gi^M
      requests:^M
        cpu: "1"^M
        memory: 1Gi^M
    securityContext:^M
      capabilities:^M
        add:^M
        - ALL^M
      privileged: true^M
    terminationMessagePath: /dev/termination-log^M
    terminationMessagePolicy: File^M
  dnsPolicy: ClusterFirst^M
  nodeSelector:^M
    nodename: DPDK0^M
    nodetype: DPDK
  restartPolicy: Always^M
  schedulerName: default-scheduler^M
  securityContext: {}^M

Here this POD will get any IP address from the start and end range of the subnet as defined in the network  object file. Our requirement is this POD to get a specific IP from the range. As i can see from this link you sent, we need to build a new plug in? How to link it with the existing ipvlan base network definition file?

'static' plugin will be packaged in next CNI plugin release. If you want to use it right now, you need to build the pluign.
Please see example configuration for static IPAM:
https://github.com/containernetworking/plugins/tree/master/plugins/ipam/static#example-configuration
You can use this IPAM with 'ipvlan'. Please change "ipam" section from 'host-local' to 'static'.

Ok. Thanks.
But i can see we need to create number of network object yaml files as same as the number of PODs.
Currently we are anyway going with rangeStart and rangeEnd having the same IP address so that corresponding POD gets the same IP address.
"type": "host-local",
"subnet": "CIDR/prefix",
"rangeStart": "IP1",
"rangeEnd": "IP1"
Original feature request which i proposing is to have, single network object file with "type" as "host-local", "rangeStart": "IP1" , "rangeEnd":"IP10". Now I would like each POD to get a predefined IP address from this range. Like -
POD1 - IP1
POD2-IP2
POD3-IP3.. so on..
So in deployment file of the POD, we can specify the desired IP address -
annotations:
networks: '[
{ "name": "<name of the network","IP:","" },
]'
Is something similar possible?

Not only multus-cni also other CNI does not support to launch network with some arguments in pod/network resource as far as I know.


Remove version: Peppy.Osu! version 2024.131.0

https://github.com/microsoft/winget-pkgs/issues/152768#issuecomment-2102735187

Resolves #152768

Microsoft Reviewers: Open in CodeFlow

URL: https://github.com/ppy/osu/releases/download/2024.131.0/install.exe
OK
(Automated message - build 882)


Accessioning Review

I ran into this screen when I tried to save the Avars as being no match to any of the other entries. None of them have geometry, so maybe that's the issue.

This happened again. I think it's just every time I try to save a No Match.

Happened with New Rome too. Louise's original didn't have any geometry, so I didn't match it. It seems to consistently go wrong when I save a "no match"

If indexMatch received a pid value of None, it would crash.


StockTradingEnv.init() got an unexpected keyword argument 'config'

I means the FinRL/finrl/agents/elegantrl/models.py has been outdate.
initialize the env using env_config is not right at this moment.

env-stocktrading environment has been updated quickly and only stable-baseline3 agent is working fine right now.

And another issue is there are some dupilicated codes in preprocessor and data_processors (for example proceesor_yahoofinance and yahoodownloader + preprocessor) and only the latter one is working fine in env-stocktrading environment. Hope these code will get an overhaul soon. Thank you!

elegantRL needs this parameters to init, while stocktradingenv cannot provide these arguments, thus resulting some errors.

for the record, rllib has the same issue.

a suggestion is to check the importing function. You should import the environment env_stocktrading_np.py instead of env_stocking.py, since the env class defined in the former file has the 'config' attribute.

Yeah, i have tried that. But I encounter other errors regarding to elegantRL. (haven't tried rllib before :)


Various RPC discrepancies compared to Lotus

Describe the bug
1) StateSectorPreCommitInfo
When sector precommit info is not found/non-existent, the RPC returns an error instead of null (as Lotus does).
Expected (Lotus response):
{"jsonrpc":"2.0","result":null,"id":1}"

Actual (Forest):
{"jsonrpc":"2.0","error":{"code":-32603,"message":"SectorPreCommitOnChainInfo not found"},"id":1}

2) 'Message` deserialization has required fields that are optional in Lotus and default to 0 there
The fields
version
nonce
gas_limit
gas_fee_cap
gas_premium

are all optional when using Lotus (and will default to 0), whereas in Forest they are required and deserialization fails without them.
3) StateWaitMsg missing parameters
In Forest, this RPC only accepts two parameters and two are missing.
Currently accepted parameters:
message_cid
confidence

Missing parameters:
limit: ChainEpoch
allow_replaced: bool

To summarize, Lotus accepts all 4 in that exact order:
message_cid
confidence
limit
allow_replaced

To reproduce

Call the related RPCs
Log output

Log Output
paste log output...

Expected behaviour

Screenshots

Environment (please complete the following information):

OS:
Branch/commit
Hardware

Other information and links

I will look into this

Hey @alt-ivan
Could you post the lotus version (lotus --version) you were testing against? And it will be great to have the payload you were using as well.
Regarding Filecoin.StateSectorPreCommitInfo, I get the below response when the sector number is invalid
# payload
{"jsonrpc":"2.0","id":0,"method":"Filecoin.StateSectorPreCommitInfo","params":
["f04040",18446744073709551615,[{"/":"bafy2bzacednxz2tzqrneijbboehcrcbdt2357musif4dyetzyyjlvu6q6xnpq"},{"/":"bafy2bzaceblbbv22npiwjhqta3dmxxpzyedjrfxugsell3ouprocd3x333bqo"}]]
}
# response
{"error":{"code":1,"message":"precommit info does not exist"},"id":0,"jsonrpc":"2.0"}

Hi @hanabi1224 ,
I am using this lotus image via docker-compose (though I suppose it could be considered very old at this point): ghcr.io/chainsafe/lotus-devnet:2024-04-07-d6d22ca
If there is parity with the newer version, I will update our codebase accordingly, and please ignore the first item :)

Update on 3) StateWaitMsg missing parameters
@alt-ivan could you describe how you tested this RPC method with 3 or 4 parameter against lotus? I tried to supply more than 2 parameter and got below error
# payload
{"jsonrpc":"2.0","id":0,"method":"Filecoin.StateWaitMsg","params":
[{"/":"bafy2bzaceb46hxz3jzwkgrztleojhad2eruk5dhyrz3ejqbgiyw4hzcp2dr4i"},0, 100]
}

# response
{"error":{"code":-32602,"message":"wrong param count (method 'Filecoin.StateWaitMsg'): 3 != 2"},"id":0,"jsonrpc":"2.0"}

According to the doc, it should accept exactly 2 parameter.

[
{
"/": "bafy2bzacea3wsdh6y3a36tb3skempjoxqpuyompjbmfeyf34fi3uy6uue42v4"
},
42
]

For the limit parameter, there is another Filecoin.StateWaitMessageLimited yet to be implemented in forest.

Update on 2) 'Message deserialization has required fields that are optional in Lotus and default to 0 there`
Fixed in https://github.com/ChainSafe/forest/pull/4440

@alt-ivan, thanks for clarifying! Yes, I was testing the V0 API. Let me see what I can do for the V1 PRC methods.

@alt-ivan 1) and 3) have been fixed by #4443 and #4444 respectively.  Thanks for reporting!

@hanabi1224
I have tested and can confirm that the changes are working as expected! Thanks!
I will report tho other discrepancies I've found here. If you want me to open another issue, please let me know. Both pertain to V1 of the Lotus API.

MpoolPending

expected response: empty json array result: [] if no messages found
actual response: result: null

'StateSectorGetInfo`

expected response: result: null if sector commit info not found
actual response:"error": {
"code": -32603,
"message": "Info for sector number 2 not found"
},

Hey @alt-ivan MpoolPending and StateSectorGetInfo should have been fixed by #4481 and #4482 respectively. Thanks for reporting, as always!

@hanabi1224
Thank you! I can confirm the changes are working as expected 🎉

@alt-ivan Great news. Feel free to open a new issue if anything else comes up.


Add tlsVerify bool to SearchImage for varlink

Cockpit wants to be able to search images on systems without
tlsverify turned on.
Signed-off-by: Daniel J Walsh<EMAIL_ADDRESS>

Fixes https://github.com/containers/libpod/issues/2293

this needs a make varlink_generate_api_generate to update the api documentation; after that, LGTM

When the registry container the image we are searching for is listed in insecure registries section in conf file and tlsVerify: true is passed to the API, there is not error but the image is returned.
Is this expected?
According to the cli docs for tlsVerify this should not happen, https://github.com/containers/libpod/blob/master/docs/podman-search.1.md

If you manually set the flsVerify flag to true and the registry is listed as insecure.  Then the containers/image should only use the registry if it can be verified.  IE it will connect with https only not fall back to http://.  At least that is my understanding.  @umohnani8 @vrothberg @mtrmac Please verify.

If you manually set the flsVerify flag to true and the registry is listed as insecure. Then the containers/image should only use the registry if it can be verified. IE it will connect with https only not fall back to http://. At least that is my understanding. @umohnani8 @vrothberg @mtrmac Please verify.

Correct. This behaviour is expected according to the docs and also the podman CLI follows this behaviour.
However, the varlink API does not respect the tlsVerify parameter if the registry is listed in insecure section, but instead it always returns the images from insecure registries.

@KKoukiou You are saying with this patch varlink is working differently then podman search --tlsverify=true?

@rhatdan Correct.
$ sudo varlink call unix:/run/podman/io.podman/io.podman.SearchImage '{"name": "my-ubuntu", "tlsVerify":true}'
{
  "images": [
    {
      "description": "",
      "is_automated": false,
      "is_official": false,
      "name": "my-ubuntu",
      "star_count": 0
    }
  ]
}
$ sudo podman search my-ubuntu --tls-verify=true
ERRO[0000] error searching registry "localhost:5000": couldn't search registry "localhost:5000": pinging docker registry returned: Get https://localhost:5000/v2/: http: server gave HTTP response to HTTPS client

my ubuntu image exists in a local private registry, and the configuration looks like this.
$ sudo cat /etc/containers/registries.conf
[registries.search]
registries = ['localhost:5000']

[registries.insecure]
registries = ['localhost:5000']

Ok then that is a bug.

I think we have an issue with the Varlink API, in that we have three settings.
One Use Default TLSVerify (Whatever is set in registries.conf)
Force on, force off.
In the CLI we can check if the user specified a tlsverify flag and then set it, otherwise we use default.
In the API, how should we indicate this. Perhaps have three contants.  Default, On, Off?

There was talk of an OptionalBool type (unset, true, false) elsewhere

Yes we are using that, but I don't see how we do that in Varlink.

@baude Does varlink have optional parameters?

@baude Does varlink have optional parameters?

not per'se ...

@baude A nullable bool aka ?bool should work, yes?  Use nil for Default and then true or false if set by user.

you saw this in varlink docs as valid ?

@baude Correct. Search https://varlink.org/Interface-Definition for nullable.

@jwhonce @baude I have made tlsVerify optionalBool,  Not sure how you call this from Varlink, but I think this allows the passing of the optionalBool.

@jwhonce @baude PTAL

LGTM, lets give it a try!

@mtrmac It is fine with me to change the semantics, @baude @jwhonce WDYT?

LGTM with respect to the API.  This will break the API tests in python-podman.

Yes @jwhonce, can you give me a /lgtm

/lgtm


Allow any key type for element keys

This fixes a compatibility issue between the old and new reconcilers, and also allows some more flexibility. It also changes the default tree root name from Foo to RoactTree.
Before this patch, the new reconciler would not allow elements to have a numerical key, which caused problems when mapping an array to children, or similar use cases. Now, values of any type are allowed for element keys; they will be converted to a string in the Roblox renderer.
This also allows increased flexibility for components that add (or modify) the children they receive. As an example, a ScrollingFrame component might want to add a list layout object to its children. To do this, it has to alter props[Roact.Children] to add it, and with that comes the possibility that an existing child might be clobbered:
local function ScrollingList(props)
    -- If there is a child already named UIListLayout, it will be clobbered!
    props[Roact.Children].UIListLayout = Roact.createElement("UIListLayout", {
        -- ...
    })

    return Roact.createElement("ScrollingFrame", {
        -- ...
    }, props[Roact.Children])
end

With this patch, we can use a private, unique value as the list layout's key, allowing children to be named whatever they want:
local listKey = Symbol.named("UIListLayout")

local function ScrollingList(props)
    -- Nothing outside of this component has access to listKey, so it's unique
    props[Roact.Children][listKey] = Roact.createElement("UIListLayout", {
        -- ...
    })

    return Roact.createElement("ScrollingFrame", {
        -- ...
    }, props[Roact.Children])
end

Hooray!


My merge branch

modified code refactored reusable code added a few more features

looks good, merge it up!


[Meta] Get Power Perceiver model into production

Here's the doc which describes what still needs to be done
TODO:

[x] https://github.com/openclimatefix/power_perceiver/issues/149
[x] Import power_perceiver
[x] make new config
[ ] https://github.com/openclimatefix/power_perceiver/issues/177
[ ] #118
[ ] add test for fake data (create new tests folder for power_perceiver, similar to the existing structure for cnn)
[ ] https://github.com/openclimatefix/power_perceiver/issues/152
[ ] run with real data from s3, see if values are sensible
[ ] update forecaster for infrastructure
[ ] Check all the normalisation
[ ] Make sure the config is loading Passiv PV data
[ ] Add tag to Power_perceiver, so if the code changes, the forecasts will continue to work for the moment
[ ] Check the PV system IDs (do we need a look-up-table for the "pv system row number"?)

tbh, given the time pressure, and how fiddly the data is, I'm going to prioritise getting the model working on "real" data first, if that's OK?

tbh, given the time pressure, and how fiddly the data is, I'm going to prioritise getting the model working on "real" data first, if that's OK?

As in, get the batch from s3? and run it on there?

Hey @jacobbieker
Im just looking at the PP code - https://github.com/openclimatefix/power_perceiver/blob/main/power_perceiver/experiments/exp_027_longer_forecasts.py
I'm using this blog as useful guidance
Im a bit worried about the code going into production.
I think we've got two options

Copy the code out to this forecast
keep the code in that repo, but move it to a new folder

Either way, we need to

refactor code, would be good to divide it up to dataloader, SatellitePredictor, SatelliteTransformer, PVRNN and FullModel
add tests to the model, this could be as much in testing https://github.com/openclimatefix/power_perceiver/issues/177
Add method to change Batch object to dict going into model (would be good to make this a Pydantic object)

What do you think?

I'd prefer keeping it in power perceiver, so we get more used to being able to more easily swap out models by importing them rather than copying them in every time. I agree on refactoring, but think we can keep the different parts of the full model together in a new folder, and just break it into fullmodel+component parts, the dataloader and training.
I agree on unit tests, I'd just go with one end to end test of loading the batch at the moment just so we can get the model into production a bit quicker, and then add more unit tests for the individual components.

I'd prefer keeping it in power perceiver, so we get more used to being able to more easily swap out models by importing them rather than copying them in every time. I agree on refactoring, but think we can keep the different parts of the full model together in a new folder, and just break it into fullmodel+component parts, the dataloader and training.
I agree on unit tests, I'd just go with one end to end test of loading the batch at the moment just so we can get the model into production a bit quicker, and then add more unit tests for the individual components.

Thats sounds good, so roughly itll be this

move into new folder
refactor into fullmodel+component parts
add e2e test

Optional later

add tests form component parts

What do you think about a pydantic model for the data / dict that goes into the model?
I'm happy to tackle this if you want

Yeah, a pydantic model for the data would be great, if you could tackle that as you have more experience with that, that would be awesome! I've already started the refactor locally for the production model, I'll push it and add the GH actions/basic test tomorrow

Im doing the github actions at the moment, so hopefully tomorrow you just push the test

Current Status is

ocf data pipes works, can get test data out and it looks ok i..e the correct shapes
model weight son HF, and can load things well

memory issue

Currently cant run batch_size=1 on my laptop or on Github actions. This makes it a bit hard to run things.

Potental solution

Set up a EC2 instance with a GPU on it.
See if test passes, which runs batch_size=1.
Run forecast from ec2 manually, see if results are sensible.
Set up Cron job to run forecast every 5mins, and see what happens
later: set up infrastructure so forecast can be run using terraform code.

Would be interested in your thoughts @JackKelly (and @jacobbieker but I know you are on holiday at the moment)

For batch size 2, running on a CPU

Tried a 't3.2xlarge' which has 32 GB of memory.  - Did not work
Tried a 'm4.4xlarge' which has 64 GB of memory.  - Did not work
Tried a 'm5a.8xlarge' which has 128 GB of memory. - Did not work
Tried a 'm5a.16xlarge' which has 256 GB of memory. Did not work

          RuntimeError: [enforce fail at alloc_cpu.cpp:73] . DefaultCPUAllocator: can't allocate memory: you tried to allocate<PHONE_NUMBER>0 bytes. Error code 12 (Cannot allocate memory)

50324972800 bytes I think is 50GB

Idea: Make sure floats are float32 not float64

When reducing to float32,

Tried a 'm4.2xlarge' which has 32 GB of memory. - Did not work
RuntimeError: [enforce fail at alloc_cpu.cpp:73] . DefaultCPUAllocator: can't allocate memory: you tried to allocate<PHONE_NUMBER>0 bytes. Error code 12 (Cannot allocate memory)
Exactly the same error

Tried 'm6a.24xlarge' which has 384 GB of RAM.  - same error

I managed to run it on a GPU, and batchsize 4 ran into a memory issue on a GPU with 16 GB.

Current status,

Can now run it on laptop. Its takes about 1 min to run a batch size of 4. This could be speed up if use multiple CPU cores for loading data, and could use GPU for inference. The fix was to only predict forward 8 PV systems, not 2056. Probably need to do a cost benefit of what this is worth doing
There also seems to be a problem with some nans in the satellite predictions.
Some GSPs which have no PV systems dont run, but these are that important, can also expand PV looking window
Currently Satellite is 1 hour behind, which can be fixed with a hot fix, but the model shold really be re trained for this.
Model should also been retrained to only use 4 channels.


Rework plugin to support instant execution

This is a draft PR, not submitted upstream yet.

At this stage, the plugin seems to be OK with instant execution. Ignoring the problems reported for the Kotlin Gradle Plugin, 3 tests fail: https://scans.gradle.com/s/zwn6scz6tmhpa/tests/failed

Check task should rerun if additional editorconfig content changed looks suspicious, could be a real problem like some missing/wrong input declarations
Should not add install git hook tasks to submodule something is fishy with the build/projects layout
Should find git folder if Gradle project is not located in root git working dir looks very similar to the previous one

I didn't dig much further on those test failures.

After a second look, there are actually more problems lurking in there. Mostly with the handling of custom third party reporters. CustomReporter instances hold a reference to DependencyHandler and Dependency instances, this is not supported. This will need to be reworked.

@eskatos want to open this as a draft PR against the parent?


Unable to get projected score of games of final completed week of season

Sport
Football
Summary
Unable to get the projected score of the final week - it shows the projected scores to equal the final scores.
Logs/Data
>>> from espn_api.football import League
>>> league = League(league_id=xxxxx, year=2023, espn_s2='xxxx', swid='xxxx')
>>> box_scores = league.box_scores(16)
>>> box_scores[0].away_score
134.64
>>> box_scores[0].away_projected
134.64
>>>

But going back to week 15, all is well in the world:
>>> box_scores = league.box_scores(15)
>>> box_scores[0].away_projected
108.38999999999999
>>> box_scores[0].away_score
115.42

Trying another league, week 16 (where it is NOT the final week of the league), it pulls the week 16 projected scores without issue.

Having a similar issue with 2 week playoffs, I expect it is related.

This is still an issue with the 2024 season


fix(component_jobs.groovy) remove extra brace that snuck in with last PR

(This is same change as was approved in https://github.com/deis/charts/pull/190)

cc @sgoings @jchauncey ONE LAST TIME :grin:

How's that for an LGTM? ;-)


Translating code and improving Spanish terminology

The term previously used to refer to arrays was the Spanish word for matrix, which was inaccurate since not all arrays are matrices.
I've also translate the code and its comments to Spansih

[x] I have read freeCodeCamp's contribution guidelines.
[x] My pull request has a descriptive title (not a vague title like Update index.md)
[x] My pull request targets the master branch of freeCodeCamp.
[x] None of my changes are plagiarized from another source without proper attribution.
[x] My article does not contain shortened URLs or affiliate links.

If your pull request closes a GitHub issue, replace the XXXXX below with the issue number.
Closes #XXXXX

@lynxlynxlynx Do you want to fix the issues or close this as stale?

It's fine as-is.

Thank you for your contribution to the page! 👍
We're happy to accept these changes, and look forward to future contributions. 🎉