Reading c64 disks with Greaseweazle v0.27 and Fluxengine Development build 2021-11-04

I'm trying to read c64 disks and I need advice on how to do this.
I have a Greaseweazle v4 with firmware version v0.27, a PC floppy drive, and the most recent Fluxengine version.
I first tried to use the --usb.greaseweazle option directly to the Greaseweazle, but this just hangs, I found out that this is expected when the Greaseweazle firmware level is >0.22.
Then I tried to follow the many online examples where they first use the Greaseweazle software to read the drive with ./gw read c64-disk.scp. But the examples then all use ./fluxengine convert scptoflux c64-disk.scp c64-disk.flux and then ./fluxengine read c64 -s c64-disk.flux -o c64-disk.d64 or some variant of this. But the new restructured versions of Fluxengine no longer need the two steps (which is great), but the c64 type no longer seems to exist.
It seems there are two new types commodore1541 and commodore1581 in stead, but if I use this like this ./fluxengine read commodore1541 -o c64-disk.d64 -s c64-disk.scp but this results in a segmentation fault.
The setup does work correctly with IBM disks, using the two step read scp and then create img file.
So I don't know what to do, I seem stuck. could you please advise how to proceed?

I've got the same setup, the following line works for me:
./fluxengine read commodore1541 -s drive:1 -o test.d64 --usb.greaseweazle=/dev/ttyACM0
no hangs.

Thank you for the reaction. I just tried using what you advised in my case it looks like this:
./fluxengine read commodore1541 -s drive:1 -o c64-testdrive.d64 --usb.greaseweazle=/dev/tty.usbmodem14511
But I'm unfortunately still getting it just sitting there doing nothing, disk not moving. I also tried with -s drive:0 and -s drive:2, not doing anything either.
Thank you for your help.

The GreaseWeazle code should work with either 0.22 or 0.24 and above, so 0.27 should work fine. I've got a 0.27 F7 Lightning and I've just verified that it works for me (at least as far as running 'fluxengine rpm' goes, as I don't have a C64 disk handy). Can you check to make sure you're running the latest development version of the FluxEngine client?
(This may also be causing what's wrong with trying to decode .scp files as I fixed some stuff not long ago.)

I'm running the version of FluxEngine available last week.
It's also on a Mac, I don't know if that makes any difference. (I was reading other issues and in some cases there seemed to be a difference.)
Also is the command I run above correct? Or should I use different flags?
I could run something like Ubuntu inside VirtualBox and pass the USB through but it seems like that might be overcomplicating things...

I pulled the latest version from the GitHub page just now, built it, same result. I also tried a different system.
I ran: ./fluxengine read commodore1541 -s drive:1 -o c64-testdrive.d64 --usb.greaseweazle=/dev/tty.usbmodem14511
It definitely tries to talk with the Greaseweazle, because if I remove it, it complains the device is not found.

Hi David, a small update. After trying different combinations on different systems I installed the Greaseweazle and FluxEngine software on Ubuntu running inside VirtualBox on one of the Macs I was trying it on earlier. This worked! :-)
It took some fiddling with the USB forwarding, if used in the default USB2 mode the output bandwidth was high enough but the input bandwidth fell quite a bit short. Switching VirtualBox to simulate USB3 pass through did the trick.
I guess I'm left with some questions though:

The drive seems to read much more than needed, it shows a 256 by 256 table at the end. Is this normal for the commodore1541 option? Or should I add something else to limit this?
It only reads side 0, is there a way to read the other head with the appropriate offset? (I've seen references to setting the offset but these either don't work or I'm not using them correctly.)
It remains strange that the same hardware seems to work fine with Ubuntu and not on MacOS. Any other advice? Or something you'd want me to try? I'm more than happy to help figure it out.

Thank you for your help and great software!

Re working on Linux and not Mac: that's weird. I mean, it's just a USB serial device. It should work. I do have a Mac I use for testing and it works there, or at least the last time I tried it. I'll give it another go.
If it's reading more than is correct that means it's seeing some corrupted sectors with garbage sector IDs. If the disk has any exotic copy protection then this might be confusing the FluxEngine decoder.
1541 disks are single-sided, which is why it's not reading the other side. Sometimes it's possible to read the other side of a flippy disk using the other head of a specially modified floppy disk drive but FluxEngine doesn't support that (as modifying the drive is hard). 1581 disks are double-sided --- they're just normal IBM disks.

Re working on Linux and not Mac: Weird indeed :-) It seems to be waiting for some handshake or so. On the Linux system it immediately starts to measure the rotational speed and run.
Re reading more: The resulting .d64 file seems fine, it's just weird how many things it scans. I think it's a regular C64 single sided image. It sees no errors until track 27 and then the errors start growing. I've uploaded the output I'm seeing to give you a better idea. test_output.txt
Would it be possible to take a Greaseweazle generated scp image of the second side and feed this into the FluxEngine client as if it's the first side? I'm pretty sure most of my old disks are used on both sides and I flipped them over.

I've just rewritten the decoder and someone's submitted a bugfix for the Greaseweazle backend. Could you get the latest development release and try again? You may get better reads (much better, in some circumstances).
Yes, the FluxEngine client software will decode .scp files no problem. Just use the filename as the input for -s.

...also, there's a section on flippy disks here: https://cowlark.com/fluxengine/doc/faq.html

Hi David, sorry for the delay, I finally had some time to try it out.
So on the Mac I pulled and built the latest source and unfortunately I'm still getting the same result... It just sits there, it doesn't complain about anything (like the USB device not being there or something like that) and with a ^c it breaks back to the prompt.
On the Linux instance (inside VirtualBox on the same system, cables, everything) I did the same and this works no problems.
I didn't update the Greaseweazle firmware or software as I presumed you only made changes to the FluxEngine code.
Thank you for your help.

I think I know what's wrong now, after hooking up my Mac and Greaseweazle. It seems OSX creates two serial port devices, which on my system look like /dev/tty.usbmodem12345 and /dev/cu.usbmodem123451. FluxEngine only works with the second, not the first.
This could certainly be handled more gracefully because right now it's very confusing. I've found http://stuffthingsandjunk.blogspot.com/2009/03/devcu-vs-devtty-osx-serial-ports.html which seems to shed some light on what's going on.


mark all observations with an annotation on Identify

Just like you can mark all as reviewed, several people want to mark all as "Plant Phenology = Flowering."

hmm, i dunno about that. i feel like you can't really tell if something is
flowering based on those tiny screens. And for the plant obs i see there as
many that aren't flowering as are.  Maybe mark all as 'annotation reviewed'
but maybe that's too weird.
On Tue, Jul 25, 2017 at 8:47 PM, Ken-ichi<EMAIL_ADDRESS>wrote:

Just like you can mark all as reviewed, several people want to mark all as
"Plant Phenology = Flowering."
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/inaturalist/inaturalist/issues/1426, or mute the
thread
https://github.com/notifications/unsubscribe-auth/AG1tk8Ym6mrWWaaOV52mIeuaUzfhL6HTks5sRoy9gaJpZM4OjS5Z
.

--
Charlie Hohn
Montpelier, Vermont


Model.create() should return an array when creating multiple documents at once

Currently, Model.create() accepts single objects or an array of objects to create documents. The callback then receives the created documents as individual arguments, one for each document.
var users = [
    {username: 'first'},
    {username: 'second'}
];
User.create(users, function(err, first, second) {
    // notice first and second are individual arguments
});

This feels very unintuitive as I would expect that if I'm supplying an array, I'd receive an array in the callback. It also makes dealing with the results very difficult when there are variable numbers of objects being created. I've had to use the arguments keyword in combination with a helper function just to get it into an array.
I propose the callback should receive an array of documents that corresponds to the supplied array of objects. Like so:
User.create(users, function(err, users) {
    users.forEach ...
});

This would also mean a breaking change so a bump to 4.0 would be in order.

Already in 3.9.x, will be in v4, which is currently waiting on the next version of the MongoDB core server to be released (current ETA early Feb)


support load error detect

Refs https://github.com/ded/script.js/issues/112.

Is this library still maintained? I would think support for error handling would be a high priority

I switched to https://www.npmjs.com/package/loadjs
That has error callback and retries which I wanted.

Yes, Its ok.
Dont worry more. we need time
Thanks


improve scripts/volmount/crypt/crypt to be more robust against corruption

we will move to explicit .init_done marker file that we only place if whole do_crypt_init function has succeeded with changes synched to disk
we only do losetup logic for dm_type caam
we create a .goodbak copy of the image file every time we successfully finish with do_mount_disk; we are extra careful and create a .prebak copy of the image file before the mounting happens.
we make do_mount_disk try the following before giving up: - mount crypt disk as usual - if fails run an fsk.ext4 -y on that /dev/mapper/ device - if that fails we put back the .goodbak file (if such exist) at code we do this through recursive invocation of the do_mount_disk function with a 4th parameter 'specialrun' mode

@parthitce please help review and test corner cases on this one.


Complain louder about invalid syntax in config file

We had "lacement_zone" instead of "placement_zone" in the CS config for who knows how long.
No error was visible, but the VMs were running in an unintended zone.
If there's invalid syntax in the config file, shouldn't CS print a error message to the log file, or even to the console and refuse to start running?

Similarly if "region" is not defined, CS dies without any error message. It should say something about missing a required config value.

Will plan to refactor the parsing for the cloud_resources.conf and add checks for the required fields for each cloud as well as checking sections against the valid list to catch typos


General loading of sample orientation

This issue also pertains to the file writer and maybe simulation framework. But I didn't know where else to put it.
The current workflow loads the sample orientation using the a3, a4 parameters of a McStas simulation: https://github.com/scipp/essspectroscopy/blob/fa79d4fb08c30c696aa077d25a927513f90059c1/src/ess/spectroscopy/indirect/workflow.py#L620-L622
This won't work for 'real' data. As the TODO comment suggests, we should probably use NXsample. See in particular sample_orientation and orientation_matrix. I don't know how those are meant to be used or why there are two different fields.
We already have some related code: https://github.com/scipp/scippneutron/blob/3d6664b861da0b770fed4c06d9162323f48f4d4f/src/scippneutron/conversion/tof.py#L552

Note that NXsample should be loaded correctly by ScippNexus if it contains such matrices.

I talked to Greg about this offline. It seems unlikely that we will use the sample instead of a3, a4. The latter encode the orientation of the sample table not the sample itself. Data reduction only really needs the orientation of the table. And that will be encoded as some kind of log from NICOS. So that might well be in /entry/parameters which has class NXparameters.

I am keeping the issue open until we have a clear decision.


site title no longer has to be an h1 and wraith tests

Solves #159 and adds styles to go with https://github.com/gios-asu/ASU-Web-Standards-Wordpress-Theme/issues/241

ok, lame.. artifacts dont upload on pull requests. since thats the only thing failing I'm going to merge this in to get the initial wraith results from travis on s3.


Process as test (VM -> Native test)

Resolves #32
Run native test
just test

TODO

[x] VM -> Native test

[x] Postgres
[x] Redis
[x] Redis cluster

[ ] Display test logs if non-zero exit code

Could you add a Github Actions workflow for running the new tests?

Another interesting point to consider here would be to only display the test logs, but filtering out all the other processes' logs might cause problems with visibility because if they fail we won't know why. Essentially we need a way to display logs of a given process if the process ended with a non-zero exit code.

Another interesting point to consider here would be to only display the test logs

I’m currently doing this as follows
''
  ${pc-warapper} >/dev/null 2>&1 &
  pc_pid=$!

  ${pc-warapper} process logs test -f &
  test_logs_pid=$!

  wait $pc_pid
  kill $test_logs_pid
''

On first line you could redirect to tmp file and display last X lines if exit code is non-zero.
Note that displaying only the logs of the test process will get easier once this is addressed. Logs for all processes could then be collected using log_location at the top level of the config file.

On first line you could redirect to tmp file and display last X lines if exit code is non-zero.

Thanks! Will give it a shot.

CI works 🥳

On macOS (Nix 2.17), I get:
❯ just test
nix flake check test/ --override-input services-flake . -L
warning: not writing modified lock file of flake 'git+file:///Users/srid/code/services-flake?dir=test':
• Updated input 'services-flake':
    'github:juspay/services-flake/f171932d130c3ce5184b69144012a225b2e28868' (2023-06-19)
  → 'git+file:///Users/srid/code/services-flake?ref=refs/heads/process-as-test&rev=9928f03781cba69263b83239c6c7bd26d6a1ef97' (2023-08-10)
error: builder for '/nix/store/c53s53w0xis4ak8jaw1df8cninh92d47-redis-test.drv' failed with exit code 1
error: build of '/nix/store/c53s53w0xis4ak8jaw1df8cninh92d47-redis-test.drv', '/nix/store/d9ssfys8nawjym0cgm26g6m5czamhzpa-redis-cluster-test.drv' failed
error: Recipe `test` failed on line 14 with exit code 1

This is odd,
❯ nix log /nix/store/c53s53w0xis4ak8jaw1df8cninh92d47-redis-test.drv
warning: The interpretation of store paths arguments ending in `.drv` recently changed. If this command is now failing try again with '/nix/store/c53s53w0xis4ak8jaw1df8cninh92d47-redis-test.drv^*'

Tests run on NixOS though.

On macOS (Nix 2.17), I get:
❯ just test
nix flake check test/ --override-input services-flake . -L
warning: not writing modified lock file of flake 'git+file:///Users/srid/code/services-flake?dir=test':
• Updated input 'services-flake':
    'github:juspay/services-flake/f171932d130c3ce5184b69144012a225b2e28868' (2023-06-19)
  → 'git+file:///Users/srid/code/services-flake?ref=refs/heads/process-as-test&rev=9928f03781cba69263b83239c6c7bd26d6a1ef97' (2023-08-10)
error: builder for '/nix/store/c53s53w0xis4ak8jaw1df8cninh92d47-redis-test.drv' failed with exit code 1
error: build of '/nix/store/c53s53w0xis4ak8jaw1df8cninh92d47-redis-test.drv', '/nix/store/d9ssfys8nawjym0cgm26g6m5czamhzpa-redis-cluster-test.drv' failed
error: Recipe `test` failed on line 14 with exit code 1

This is odd,
❯ nix log /nix/store/c53s53w0xis4ak8jaw1df8cninh92d47-redis-test.drv
warning: The interpretation of store paths arguments ending in `.drv` recently changed. If this command is now failing try again with '/nix/store/c53s53w0xis4ak8jaw1df8cninh92d47-redis-test.drv^*'

Tests run on NixOS though.

This also happens on my macOS with Nix 2.13.3. Additionally I also tried running the example flake with just ex after updating example/flake.lock to use the latest PC and the example app doesn't run at all.
I tried on different macOS (with Nix 2.15.1) where PC was never ran before and it seems to work fine there. This is weird, have to debug this.
By MacOS I mean aarch64-darwin in both the devices mentioned above.

@srid Now the builds are working on mac, the error wasn't conclusive but it looks like having each PC package run on different port makes it work.

A nicer implementation will be to disable swagger endpoint so that we won't have this conflict.

Yea, I was thinking of an upstream contribution to that end. Or we can set port to 0 by default (in process-compose-flake), forcing the user to set it explicitly if they want it.
just test runs fine for me now.

@srid Looks like the CI is running forever when I set disabled = true; for test process

Done!


enhancement: if a batch query is killed because of OOM risks, return a error message with OOM as the reason to the client

Just leave a record here as users bring it up.

In fact, currently we will report similar message, so I'll close it first.


Resolve warnings on undefined Kino.VegaLite's functions

This resolves the warnings below when Mix install the deps in Command
Line page.
warning: Kino.VegaLite.new/1 is undefined (module Kino.VegaLite is not
available or is yet to be defined

warning: Kino.VegaLite.push_many/2 is undefined (module Kino.VegaLite is
not available or is yet to be defined)

:partying_face: :partying_face: :partying_face: :partying_face: :partying_face:


Unit tests of proc-macro crates

Running cargo miri test --lib on a crate that has [lib] proc-macro = true in Cargo.toml prints the following and does not run the unit tests:
Running unit tests of `proc-macro` crates is not currently supported by Miri.

Unit tests inside proc macro crates are allowed and supported by cargo test --lib.
I found this message originated a year ago from #1675 but I did not find a corresponding issue that I could follow, so here is one.

Good point, we should certainly track this somewhere.
Conceptually cargo treats proc-macros as being on the host while Miri is responsible for the target, so I am not sure if and how this will be possible. I am not deeply familiar with proc-macros and how cargo treats them.


Remove files that remain after updateVersions.sh

These files are rendered via the .tmpl files at present and don't need to be in repository.  All other rendered files have been removed.

/lgtm
you confirmed they are generated ?

I did confirm they are generated prior to submitting the PR.  What is in master is not equivalent to what is generated.   The only delta is in PR: https://github.com/istio/istio/pull/3421.


chore: remove bash script for naiv quickstart sync

The script was initially intended to help syncing the two repositories. The process is different now.

New process is discussed here: #180


[Feat.] Added setOwnerReference in Job Builder

Signed-off-by: Rahul M Chheda<EMAIL_ADDRESS>

@rahulchheda let us know if we can merge this/ needs any other changes to be pulled


Expose Read Only and Write Only storage buffers

Enable flags so you can create buffers using the read only or write only flags which translate to BUFFER_USAGE_TRANSFER_FROM_BIT and BUFFER_USAGE_TRANSFER_TO_BIT

Digging though and after looking at the code more and chatting to @clayjohn not even sure this is useful.
The storage buffer stuff is always creating with GPU alloc which doesn't hit the relevant code so this PR is not effective as it used to be. Might come back to this as I'm poking at trying to solve some problems in this area for my own purposes.
Leaving open for dicussion, and if I get a better solution.


Get Email Body decodes URLs with issues

URLs are decoded with issues, for instance URL

is decoded as

Here is Python code of function get_email_body():
        if self._is_walking_multipart(email_index):
            body = self.get_multipart_payload(decode=True)
        else:
            encoded_body = self._imap.uid('fetch', email_index, '(BODY[TEXT])')[1][0][1]
            try:
                body = decode(encoded_body, 'quopri_codec').decode('UTF-8')
            except:
                body = decode(encoded_body, 'quopri_codec').decode('ISO-8859-1')
        return body

Issue is moved to https://github.com/lasselindqvist/robotframework-imaplibrary2/issues/3


[Messages Overlay] Displayed message keeps screen awake

@halemmerich
Affected hardware version
Bangle 2
Your firmware version
2v18.92
The bug
When a message has arrived and is displayed on the watch it will keep the screen backlight on. The backlight will turn off and back on instantly with some interval.
Video demonstration:
https://github.com/espruino/BangleApps/assets/97237430/026f13a7-d5d2-402b-88cd-fcdb3ff348a2
Expected behaviour
The message arrives and turn the screen on.
Reproduce

Install Messages Overlay and it's dependencies.
Send a test message from Gadgetbridge debug page.
Observe the described behaviour.

Furter notes
I uninstalled Fastload Utils and tested the bug prior to this report.
Installed apps
sched (0.22)
messageicons (0.05)
widmsggrid (0.05)
health (0.23)
widminbat (0.03)
widanclk (0.02)
backswipe (0.02)
torch (0.11)
calculator (0.07)
widram (0.03)
widbthide (0.01)
clkinfostopw (0.03)
dtlaunch (0.23)
quicklaunch (0.15)
alarm (0.41)
powermanager (0.10)
mysetup (0.01)
fastreset (0.01)
bwclklite (0.34)
smpltmr (0.08)
hrm (0.11)
lightswitch (0.07)
setting (0.61)
boot (0.58)
messages (0.60)
dragboard (0.07)
runplus (0.21)
agenda (0.14)
clock_info (0.06)
spotrem (0.09)
recorder (0.29)
messagegui (0.73)
podadrem (0.09)
swscroll (0.05)
gipy (0.20)
android (0.29)
messagesoverlay (0.04)

Interesting, had not seen this before but I can reproduce. It seems to be caused by the LCD being switched on every time a lock event happens. So far so easy, but why didn't this happen before? That part of the code wasn't changed in the last PR. I believe I have a fix, will test a bit and do a PR.

Found the answer, this bug was in 0.03 as well. That will have used quite some battery...


reserved identifier violation

I would like to point out that identifiers like "_APP_TEMPLATE_HH" and "_MEMCACHED_HH" do not fit to the expected naming convention of the C++ language standard.
Would you like to adjust your selection for unique names?

On Tue, Apr 28, 2015 at 3:24 PM, Markus Elfring<EMAIL_ADDRESS>wrote:

I would like to point out that identifiers like "_APP_TEMPLATE_HH
https://github.com/cloudius-systems/seastar/blob/5bc8d7cfadb78caf37b47e1bd351852c52de04d1/core/app-template.hh#L21"
and "_MEMCACHED_HH
https://github.com/cloudius-systems/seastar/blob/5bc8d7cfadb78caf37b47e1bd351852c52de04d1/apps/memcached/memcached.hh#L18"
do not fit
https://www.securecoding.cert.org/confluence/display/cplusplus/DCL51-CPP.+Do+not+declare+or+define+a+reserved+identifier#DCL51-CPP.Donotdeclareordefineareservedidentifier-NoncompliantCodeExample(HeaderGuard)
to the expected naming convention of the C++ language standard.
Would you like to adjust your selection for unique names?

My personal preference is just to use "#pragma once", and we do use it in
quite a few header files.
"#pragma once" isn't standard C++, but all the compilers I tried seem to
support it well, and it's (in my opinion) much less ugly and error-prone
than those include guards.

This is fixed as of 61c905cf7fe69ddcb1bbd9fe8caa493eb41b8a85.


Bug 4650, 4651, 4652 and 4653! lots of small fixes

The big crashes from hockeyapp for RC3

okay all done with this. let's merge, build and release RC4!


Vanilla client support (base functions)

Hi!
Can you please add a basic commands/chest gui interface for vanilla users?
I want to put this mod on a small server and play with friends and other people a Vanilla+ modpack and I'm trying to make everything server-side.
I would love to use this mod for a "vanilla" server like a plugin and looking at some videos and at the code, doesn't seem like it's possible yet.

Hi,
sorry, but this mod relies a lot on the client doing things. E.g. all the active powers would have to be rewritten to react to something that vanilla catches, such as sneaking, jumping, or something else, instead of a custom keybind. I could also not make use of the cooldown bars like I do now. Many powers also rely on the client knowing about them - e.g. the Elytrian's elytra flight power is only possible because of the client.
I hope you understand that making the mod completely server-side is just too much work for me, but you're welcome to work on it yourself if you want to.

ok, thanks anyways!


Noise in depth gt

Thank you for your excellent work!
I've downloaded the test set from the website and converted the depth image into point cloud using the following code:
import os
import numpy as np
from numpy.matlib import repmat

os.environ['OPENCV_IO_ENABLE_OPENEXR'] = '1'

import cv2

image = cv2.imread('0000.exr', cv2.IMREAD_ANYCOLOR | cv2.IMREAD_ANYDEPTH)[..., 0]

pixel_size = image.shape[0] * image.shape[1]

depth_image = image.reshape(pixel_size)
u_coord = repmat(np.r_[0:image.shape[1]:1], image.shape[0], 1).reshape(pixel_size)
v_coord = repmat(np.c_[0:image.shape[0]:1], 1, image.shape[1]).reshape(pixel_size)

fx =<PHONE_NUMBER>443055
fy =<PHONE_NUMBER>443055
cx = image.shape[1] / 2
cy = image.shape[0] / 2

k = np.identity(3)
k[0, 2] = cx
k[1, 2] = cy
k[0, 0] = k[1, 1] = fx

p2d = np.array([u_coord, v_coord, np.ones_like(u_coord)])

p3d = np.transpose(np.dot(np.linalg.inv(k), p2d) * depth_image).reshape(-1, 3)

point_cloud = []

for x, y, z in p3d:
	x, y, z = z, x, -y
	point_cloud.append([x, y, z])

point_cloud = np.array(point_cloud)

from plyfile import PlyData, PlyElement

vertices = np.array([(x, y, z) for x, y, z in point_cloud.reshape(-1, 3)], dtype=[('x', 'f4'), ('y', 'f4'), ('z', 'f4')])
plydata = PlyData([PlyElement.describe(vertices, 'vertex')], text=False)
plydata.write('point_cloud.ply')

The shape of the point cloud seemed nice, however there seemed to be a noticeable amount of noise in the point cloud.
Is this normal or am I using a wrong way to convert the exr file?
I'm opening 0000.exr from small_city_road_horizon_test_depth.tar.

Could you please tell me which depth image you use? I will have a try.

Sorry for the late reply. Have you tried depth from other view? Have you observed similar phenomenon?

Sorry for the late reply. Have you tried depth from other view? Did you observe similar phenomenon?

Thank you very much for you question! I think the way you read depth is right. We have checked our depth map and find the reason. For the best rendering quality, we use the anti-alias setting to reduce noise, which will have a smoothing effect on the image. The depths change dramatically at the edge of the object and anti-alias reduces this change, which will cause the noisy point cloud. The depth map without and with anti-alias are shown below, which is a zoom-in pixel-level comparison.

We think the depth maps rendered with anti-lias can better align the rgb images rendered with anti-alias and can support most of the requirements, like depth guidance for reconstruction. But for generating point cloud from depth map, anti-alias will bring many noises like you find. We have tried generating point cloud from the depth rendered without anti-alias and solved the problem.  We will update a new depth map version without anti-alias soon.

Hi Guys
Can I ask why the depth value from the .exr may has thousands? Is there any scaling for it?

The unit is cm.

The unit in our pose file (street/pose and aerial/pose) is 10000cm. If you use the pose file to train nerf, you can scale the depth file with 10000.

I am trying to fuse serval RGB and depth pairs to generate the pointclouds. Here is my core code and please feel free to point out if there is any problems. According to the instruction from YiXuan and previous post, I divided the depth with 10000 and did the same coordinate conversion. But the results seems wired.

`def depth_to_pc(depth_path, intrinsic, transform_matrix):
depth_map = cv2.imread(depth_path, cv2.IMREAD_ANYCOLOR | cv2.IMREAD_ANYDEPTH)[..., 0]
pixel_size = depth_map.shape[0] * depth_map.shape[1]
depth_image = depth_map.reshape(pixel_size)/10000
u_coord = np.tile(np.arange(depth_map.shape[1]), (depth_map.shape[0], 1)).reshape(pixel_size)
v_coord = np.tile(np.arange(depth_map.shape[0]), (depth_map.shape[1], 1)).T.reshape(pixel_size)
k = np.identity(3)
k[0, 2] = intrinsic["cx"]
k[1, 2] = intrinsic["cy"]
k[0, 0] = intrinsic["fl_x"]
k[1, 1] = intrinsic["fl_x"]
p2d = np.array([u_coord, v_coord, np.ones_like(u_coord)])
p3d = np.transpose(np.dot(np.linalg.inv(k), p2d) * depth_image).reshape(-1, 3)

# Left coord to right
# point_cloud = p3d[:, [2, 0, 1]]
point_cloud = p3d
point_cloud[:, 1] = -point_cloud[:, 1]
ones = np.ones((point_cloud.shape[0], 1))
homogenous_coordinate = np.hstack((point_cloud, ones))
transformed_points = (transform_matrix @ homogenous_coordinate.T).T

return transformed_points[:, :3]

frames = data["frames"]
for frame in frames[:10]:
rel_img_path = frame["file_path"]
transform_matrix = frame["transform_matrix"]
transform_matrix = np.array(transform_matrix)
abs_img_path = os.path.abspath(os.path.join(pose_path, rel_img_path)).replace("/pose", "")
abs_depth_path = abs_img_path.replace("small_city", "small_city_depth",1).replace("small_city_road_down", "small_city_road_down_depth").replace("png", "exr")
if not os.path.exists(abs_img_path) or not os.path.exists(abs_depth_path):
    continue
depth_image = cv2.imread(abs_depth_path, cv2.IMREAD_ANYCOLOR | cv2.IMREAD_ANYDEPTH)[..., 0]
color_image = cv2.imread(abs_img_path)
color_image = cv2.cvtColor(color_image, cv2.COLOR_BGR2RGB)
point_cloud = depth_to_pc(abs_depth_path, camera_intrinsics, transform_matrix)

all_points.append(point_cloud)

colors = color_image.reshape(-1, 3)/255.0
all_colors.append(colors)

merged_points = np.vstack(all_points)
merged_colors = np.vstack(all_colors)
final_pcd = o3d.geometry.PointCloud()
final_pcd.points = o3d.utility.Vector3dVector(merged_points)
final_pcd.colors = o3d.utility.Vector3dVector(merged_colors)
o3d.io.write_point_cloud("final_merged_point_cloud_2.pcd", final_pcd)`

Same problem as @ChaoyiZh. Is there any solution?

I am trying to fuse serval RGB and depth pairs to generate the pointclouds. Here is my core code and please feel free to point out if there is any problems. According to the instruction from YiXuan and previous post, I divided the depth with 10000 and did the same coordinate conversion. But the results seems wired.
os.environ['OPENCV_IO_ENABLE_OPENEXR'] = '1'
def depth_to_pc(depth_path, intrinsic, transform_matrix):
    depth_map = cv2.imread(depth_path, cv2.IMREAD_ANYCOLOR | cv2.IMREAD_ANYDEPTH)[..., 0]
    pixel_size = depth_map.shape[0] * depth_map.shape[1]
    depth_image = depth_map.reshape(pixel_size)/10000
    u_coord = np.tile(np.arange(depth_map.shape[1]), (depth_map.shape[0], 1)).reshape(pixel_size)
    v_coord = np.tile(np.arange(depth_map.shape[0]), (depth_map.shape[1], 1)).T.reshape(pixel_size)
    k = np.identity(3)
    k[0, 2] = intrinsic["cx"]
    k[1, 2] = intrinsic["cy"]
    k[0, 0] = intrinsic["fl_x"]
    k[1, 1] = intrinsic["fl_x"]
    p2d = np.array([u_coord, v_coord, np.ones_like(u_coord)])
    p3d = np.transpose(np.dot(np.linalg.inv(k), p2d) * depth_image).reshape(-1, 3)

    # Left coord to right
    point_cloud = p3d
    point_cloud[:, 1] = -point_cloud[:, 1]
    ones = np.ones((point_cloud.shape[0], 1))
    homogenous_coordinate = np.hstack((point_cloud, ones))
    transformed_points = (transform_matrix @ homogenous_coordinate.T).T

    return transformed_points[:, :3]
frames = data["frames"]

for frame in frames[:10]:
    rel_img_path = frame["file_path"]
    transform_matrix = frame["transform_matrix"]
    transform_matrix = np.array(transform_matrix)

    abs_img_path = os.path.abspath(os.path.join(pose_path, rel_img_path)).replace("/pose", "")
    abs_depth_path = abs_img_path.replace("small_city", "small_city_depth",1).replace("small_city_road_down", "small_city_road_down_depth").replace("png", "exr")
    if not os.path.exists(abs_img_path) or not os.path.exists(abs_depth_path):
        continue
    depth_image = cv2.imread(abs_depth_path, cv2.IMREAD_ANYCOLOR | cv2.IMREAD_ANYDEPTH)[..., 0]
    color_image = cv2.imread(abs_img_path)
    color_image = cv2.cvtColor(color_image, cv2.COLOR_BGR2RGB)
    point_cloud = depth_to_pc(abs_depth_path, camera_intrinsics, transform_matrix)

    all_points.append(point_cloud)

    colors = color_image.reshape(-1, 3)/255.0
    all_colors.append(colors)

merged_points = np.vstack(all_points)
merged_colors = np.vstack(all_colors)

final_pcd = o3d.geometry.PointCloud()
final_pcd.points = o3d.utility.Vector3dVector(merged_points)
final_pcd.colors = o3d.utility.Vector3dVector(merged_colors)
o3d.io.write_point_cloud("final_merged_point_cloud_2.pcd", final_pcd)

Have you successfully generated the reasonable point cloud from a single RGB-depth pairs?

Besides, could you tell me the which json file you use?

Yes. I totally understand we are using Nerf(OPENGL), which is a right-hand coordinate system.   And I can generate one pointcloud with some image-depth pair and the corresponding pose. Here are the files I am using.
small_city/street/pose/block_A/transforms_train.json.
The result I posted before was from the fusion of first ten images.

Thx! I will try to fix this bug.

I am trying to fuse serval RGB and depth pairs to generate the pointclouds. Here is my core code and please feel free to point out if there is any problems. According to the instruction from YiXuan and previous post, I divided the depth with 10000 and did the same coordinate conversion. But the results seems wired.
os.environ['OPENCV_IO_ENABLE_OPENEXR'] = '1'
def depth_to_pc(depth_path, intrinsic, transform_matrix):
    depth_map = cv2.imread(depth_path, cv2.IMREAD_ANYCOLOR | cv2.IMREAD_ANYDEPTH)[..., 0]
    pixel_size = depth_map.shape[0] * depth_map.shape[1]
    depth_image = depth_map.reshape(pixel_size)/10000
    u_coord = np.tile(np.arange(depth_map.shape[1]), (depth_map.shape[0], 1)).reshape(pixel_size)
    v_coord = np.tile(np.arange(depth_map.shape[0]), (depth_map.shape[1], 1)).T.reshape(pixel_size)
    k = np.identity(3)
    k[0, 2] = intrinsic["cx"]
    k[1, 2] = intrinsic["cy"]
    k[0, 0] = intrinsic["fl_x"]
    k[1, 1] = intrinsic["fl_x"]
    p2d = np.array([u_coord, v_coord, np.ones_like(u_coord)])
    p3d = np.transpose(np.dot(np.linalg.inv(k), p2d) * depth_image).reshape(-1, 3)

    # Left coord to right
    point_cloud = p3d
    point_cloud[:, 1] = -point_cloud[:, 1]
    ones = np.ones((point_cloud.shape[0], 1))
    homogenous_coordinate = np.hstack((point_cloud, ones))
    transformed_points = (transform_matrix @ homogenous_coordinate.T).T

    return transformed_points[:, :3]
frames = data["frames"]

for frame in frames[:10]:
    rel_img_path = frame["file_path"]
    transform_matrix = frame["transform_matrix"]
    transform_matrix = np.array(transform_matrix)

    abs_img_path = os.path.abspath(os.path.join(pose_path, rel_img_path)).replace("/pose", "")
    abs_depth_path = abs_img_path.replace("small_city", "small_city_depth",1).replace("small_city_road_down", "small_city_road_down_depth").replace("png", "exr")
    if not os.path.exists(abs_img_path) or not os.path.exists(abs_depth_path):
        continue
    depth_image = cv2.imread(abs_depth_path, cv2.IMREAD_ANYCOLOR | cv2.IMREAD_ANYDEPTH)[..., 0]
    color_image = cv2.imread(abs_img_path)
    color_image = cv2.cvtColor(color_image, cv2.COLOR_BGR2RGB)
    point_cloud = depth_to_pc(abs_depth_path, camera_intrinsics, transform_matrix)

    all_points.append(point_cloud)

    colors = color_image.reshape(-1, 3)/255.0
    all_colors.append(colors)

merged_points = np.vstack(all_points)
merged_colors = np.vstack(all_colors)

final_pcd = o3d.geometry.PointCloud()
final_pcd.points = o3d.utility.Vector3dVector(merged_points)
final_pcd.colors = o3d.utility.Vector3dVector(merged_colors)
o3d.io.write_point_cloud("final_merged_point_cloud_2.pcd", final_pcd)

I think you wanna do
point_cloud[:, 2] = -point_cloud[:, 2]

For me, that makes the points align as expected

I am trying to fuse serval RGB and depth pairs to generate the pointclouds. Here is my core code and please feel free to point out if there is any problems. According to the instruction from YiXuan and previous post, I divided the depth with 10000 and did the same coordinate conversion. But the results seems wired.

Hi, have you solve this problem? I am trying to build point cloud too, but face the same problem as you.

I think you wanna do
point_cloud[:, 2] = -point_cloud[:, 2]

For me, that makes the points align as expected

Hi, would you mind sharing more details about your implementation? I tried point_cloud[:, 2] = -point_cloud[:, 2] but it doesn't work.

I think you wanna do
point_cloud[:, 2] = -point_cloud[:, 2]

For me, that makes the points align as expected

Thanks, with below operations, the points align successfully:
point_cloud[:, 1] = -point_cloud[:, 1]
point_cloud[:, 2] = -point_cloud[:, 2]

Thank you very much for you question! I think the way you read depth is right. We have checked our depth map and find the reason. For the best rendering quality, we use the anti-alias setting to reduce noise, which will have a smoothing effect on the image. The depths change dramatically at the edge of the object and anti-alias reduces this change, which will cause the noisy point cloud. The depth map without and with anti-alias are shown below, which is a zoom-in pixel-level comparison.
We think the depth maps rendered with anti-lias can better align the rgb images rendered with anti-alias and can support most of the requirements, like depth guidance for reconstruction. But for generating point cloud from depth map, anti-alias will bring many noises like you find. We have tried generating point cloud from the depth rendered without anti-alias and solved the problem. We will update a new depth map version without anti-alias soon.

Hi, is there any update on the ground-truth depth map? It seems that the update date of  "small_city_depth" is still 3 months ago. Besides, has the anti-alias problem been solved in the "big_city_depth"?

Thank you very much for you question! I think the way you read depth is right. We have checked our depth map and find the reason. For the best rendering quality, we use the anti-alias setting to reduce noise, which will have a smoothing effect on the image. The depths change dramatically at the edge of the object and anti-alias reduces this change, which will cause the noisy point cloud. The depth map without and with anti-alias are shown below, which is a zoom-in pixel-level comparison.
We think the depth maps rendered with anti-lias can better align the rgb images rendered with anti-alias and can support most of the requirements, like depth guidance for reconstruction. But for generating point cloud from depth map, anti-alias will bring many noises like you find. We have tried generating point cloud from the depth rendered without anti-alias and solved the problem. We will update a new depth map version without anti-alias soon.

Hi, is there any update on the ground-truth depth map? It seems that the update date of "small_city_depth" is still 3 months ago. Besides, has the anti-alias problem been solved in the "big_city_depth"?

Currently, we do not provide the depth map without anti-alias. If you want to use the depth map to guide reconstruction, I think the depth map with anti-alias is more suitable because it aligns with rgb images. If you want to use the depth map to obtain geometry, you can refer to https://github.com/city-super/MatrixCity/blob/main/MatrixCityPlugin/docs/Render-Data.md#depth to render the depth map without anti-alias, we have already provided the camera sequences, render config, plugin and detailed documentation.

we have updated the script extracting point clouds from multi rgb-depth pairs at https://github.com/city-super/MatrixCity/blob/main/scripts/rgbd2pc.py  @ChaoyiZh @yzslab @Goochaozheng @Snosixtyboo


feat(bolt-icons): add with/without icons

Jira
Add icons necessary for the With/Without PR: https://github.com/bolt-design-system/bolt/pull/1295
Summary
Add icons necessary for With/Without
Details
Here are the icons added:

chart-bar.svg
close-circled.svg
credit-card.svg
face-happy.svg
face-sad.svg

Closing out — these updates were already merged down and released in one of the earlier v2.8.x beta releases.


Feature request: invert colors in web viewer

Inverting colors is unfortunately a somewhat important feature for me due to an eye condition.
I put together a proof-of-concept for how this could work, though I'm not too familiar with the code so I don't know if I did it in the best way: https://github.com/ellbur/pdf.js/commit/36a7b7c032e188da83f2f51967edaa4158445cde

Duplicate of #2071


[scheduler] Mystery padding/widget above keyboard when TimeDilation > 1.0

I have a mystery padding above the keyboard when TimeDilation > 1.0 and I come to the screen with my keyboard on, from another screen.
The padding seems to be proportional to the time dilation when timeDilation >1.0 and < 3.0. It seems to be constant at the same height for timeDilation values >= 3.0.
If I change the routing so that FirstScreen never gets generated, the bug is avoided. Similarly, the bug can be worked around by forcing a screen redraw.
What is the cause and how do I need to change my code, to fix this bug?
Steps to Reproduce

Execute flutter run on the code sample

Expected results:
15 (blue) logical pixels between the bottom of the green Container and the top of the grey CueKeyboard widget.
Actual results:
Approximately 150 (blue) logical pixels between the bottom of the green Container and the top of the grey CueKeyboard widget.
The extra space doesn't appear to be accoutned for in the Widget inspector.  There isn't a widget listed in the inspector that can account for the extra space and it isn't padding or margin on the CueKeyboard or the Container or any of their ancestors.

Code sample
import 'package:flutter/material.dart';
import 'package:flutter/scheduler.dart' show timeDilation;
import 'package:keyboard_actions/keyboard_actions.dart';

void main() async {
  // set the system top-level property [timeDilation] to control the animation speed
  timeDilation = 3.0;

  WidgetsFlutterBinding.ensureInitialized();

  runApp(
    MyApp(),
  );
}

class MyApp extends StatefulWidget {
  MyApp({Key? key}) : super(key: key);

  @override
  _MyAppState createState() => _MyAppState();
}

class _MyAppState extends State<MyApp> {
  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      initialRoute: '/',
      onGenerateRoute: (RouteSettings settings) {
        final routes = <String, WidgetBuilder?>{
          '/': (_) => FirstScreen(),
          '/screen2': (_) => SecondScreen(),
        };
        final builder = routes[settings.name]!;
        return MaterialPageRoute(
          builder: builder,
          settings: settings,
        );
      },
    );
  }
}

// ----------------------------------------------------------

class FirstScreen extends StatefulWidget {
  FirstScreen({Key? key}) : super(key: key);

  @override
  _FirstScreenState createState() => _FirstScreenState();
}

class _FirstScreenState extends State<FirstScreen> {
  @override
  void initState() {
    WidgetsBinding.instance?.addPostFrameCallback((_) {
      Navigator.of(context).pushNamed('/screen2');
    });
    super.initState();
  }

  @override
  Widget build(BuildContext context) {
    return Scaffold();
  }
}

// ----------------------------------------------------------

class SecondScreen extends StatefulWidget {
  @override
  _SecondScreenState createState() => _SecondScreenState();
}

class _SecondScreenState extends State<SecondScreen>
    with RouteAware, WidgetsBindingObserver {
  FocusNode? focusNode;

  ValueNotifier<String>? _stringValueNotifier;

  @override
  void initState() {
    super.initState();
    WidgetsBinding.instance!.addObserver(this);
    this._stringValueNotifier = ValueNotifier<String>('');
    this._stringValueNotifier!.addListener(() {});
  }

  @override
  void dispose() {
    _stringValueNotifier?.dispose();

    WidgetsBinding.instance!.removeObserver(this);
    super.dispose();
  }

  @override
  Widget build(BuildContext context) {
    this.focusNode = FocusNode();

    return Scaffold(
      body: KeyboardContent(
        focusNode: this.focusNode!,
        stringValueNotifier: this._stringValueNotifier!,
      ),
    );
  }
}

// ----------------------------------------------------------

class KeyboardContent extends StatelessWidget {
  final FocusNode focusNode;
  final ValueNotifier<String> stringValueNotifier;

  /// A container for an on-screen keyboard
  /// The widget automatically gets rebuilt on value change, triggered
  /// when a keyboard button is pressed.
  KeyboardContent({
    required this.focusNode,
    required this.stringValueNotifier,
  });

  KeyboardActionsConfig _buildConfig(BuildContext context) {
    List<Widget Function(FocusNode)> toolbarButtons = [];

    return KeyboardActionsConfig(
        keyboardActionsPlatform: KeyboardActionsPlatform.ALL,
        keyboardBarColor: Colors.grey[200]!,
        //nextFocus: false,
        actions: [
          KeyboardActionsItem(
            focusNode: focusNode,
            displayActionBar: false,
            displayDoneButton: false,
            toolbarButtons: toolbarButtons,
            footerBuilder: (_) => CueKeyboard(),
          )
        ]);
  }

  @override
  Widget build(BuildContext context) {
    if (!focusNode.hasPrimaryFocus) focusNode.requestFocus();
    return Container(
      decoration: BoxDecoration(color: Colors.blue),
      child: KeyboardActions(
        config: _buildConfig(context),
        child: Container(
          padding: EdgeInsets.only(left: 15, right: 15),
          child: Column(
            crossAxisAlignment: CrossAxisAlignment.stretch,
            children: <Widget>[
              KeyboardCustomInput<String>(
                focusNode: focusNode,
                notifier: stringValueNotifier,
                builder: (context, keyboardTapVal, hasFocus) {
                  return Container(
                      margin: EdgeInsets.only(top: 100),
                      height: 800,
                      width: 100,
                      decoration: BoxDecoration(
                          borderRadius: BorderRadius.circular(32.0),
                          color: Colors.green));
                },
              ),
            ],
          ),
        ),
      ),
    );
  }
}

class CueKeyboard extends StatefulWidget implements PreferredSizeWidget {
  static final double _keyboardHeight = 184.0;

  CueKeyboard({
    Key? key,
  }) : super(key: key);

  @override
  _CueKeyboardState createState() => _CueKeyboardState();

  @override
  Size get preferredSize => Size.fromHeight(CueKeyboard._keyboardHeight);
}

class _CueKeyboardState extends State<CueKeyboard> {
  dispose() {
    super.dispose();
  }

  @override
  Widget build(BuildContext context) {
    return Wrap(
      children: [
        Column(
          children: [Text("Keyboard")],
        ),
      ],
    );
  }
}

Logs
$ flutter run --verbose
[  +53 ms] executing: [C:\flutter/] git -c log.showSignature=false log -n 1 --pretty=format:%H
[  +84 ms] Exit code 0 from: git -c log.showSignature=false log -n 1 --pretty=format:%H
[        ] 77d935af4db863f6abd0b9c31c7e6df2a13de57b
[        ] executing: [C:\flutter/] git tag --points-at 77d935af4db863f6abd0b9c31c7e6df2a13de57b
[  +24 ms] Exit code 0 from: git tag --points-at 77d935af4db863f6abd0b9c31c7e6df2a13de57b
[        ] 2.8.1
[   +6 ms] executing: [C:\flutter/] git rev-parse --abbrev-ref --symbolic @{u}
[  +15 ms] Exit code 0 from: git rev-parse --abbrev-ref --symbolic @{u}
[        ] origin/stable
[        ] executing: [C:\flutter/] git ls-remote --get-url origin
[  +11 ms] Exit code 0 from: git ls-remote --get-url origin
[        ] https://github.com/flutter/flutter.git
[  +70 ms] executing: [C:\flutter/] git rev-parse --abbrev-ref HEAD
[  +14 ms] Exit code 0 from: git rev-parse --abbrev-ref HEAD
[        ] stable
[  +47 ms] Artifact Instance of 'AndroidGenSnapshotArtifacts' is not required, skipping update.
[        ] Artifact Instance of 'AndroidInternalBuildArtifacts' is not required, skipping update.
[        ] Artifact Instance of 'IOSEngineArtifacts' is not required, skipping update.
[        ] Artifact Instance of 'FlutterWebSdk' is not required, skipping update.
[   +7 ms] Artifact Instance of 'WindowsEngineArtifacts' is not required, skipping update.
[        ] Artifact Instance of 'WindowsUwpEngineArtifacts' is not required, skipping update.
[        ] Artifact Instance of 'MacOSEngineArtifacts' is not required, skipping update.
[        ] Artifact Instance of 'LinuxEngineArtifacts' is not required, skipping update.
[        ] Artifact Instance of 'LinuxFuchsiaSDKArtifacts' is not required, skipping update.
[        ] Artifact Instance of 'MacOSFuchsiaSDKArtifacts' is not required, skipping update.
[        ] Artifact Instance of 'FlutterRunnerSDKArtifacts' is not required, skipping update.
[        ] Artifact Instance of 'FlutterRunnerDebugSymbols' is not required, skipping update.
[  +48 ms] executing: C:\Users\md844\Android\SDK\platform-tools\adb.exe devices -l
[  +40 ms] List of devices attached
           emulator-5554          device product:sdk_phone_x86 model:Android_SDK_built_for_x86 device:generic_x86 transport_id:11
[   +6 ms] C:\Users\md844\Android\SDK\platform-tools\adb.exe -s emulator-5554 shell getprop
[  +33 ms] Artifact Instance of 'AndroidInternalBuildArtifacts' is not required, skipping update.
[        ] Artifact Instance of 'IOSEngineArtifacts' is not required, skipping update.
[   +2 ms] Artifact Instance of 'WindowsEngineArtifacts' is not required, skipping update.
[        ] Artifact Instance of 'WindowsUwpEngineArtifacts' is not required, skipping update.
[        ] Artifact Instance of 'MacOSEngineArtifacts' is not required, skipping update.
[        ] Artifact Instance of 'LinuxEngineArtifacts' is not required, skipping update.
[        ] Artifact Instance of 'LinuxFuchsiaSDKArtifacts' is not required, skipping update.
[        ] Artifact Instance of 'MacOSFuchsiaSDKArtifacts' is not required, skipping update.
[        ] Artifact Instance of 'FlutterRunnerSDKArtifacts' is not required, skipping update.
[        ] Artifact Instance of 'FlutterRunnerDebugSymbols' is not required, skipping update.
[  +66 ms] Skipping pub get: version match.
[  +40 ms] Found plugin integration_test at C:\flutter\packages\integration_test\
[  +82 ms] Found plugin integration_test at C:\flutter\packages\integration_test\
[   +6 ms] Generating C:\Users\md844\source\repos\ucue_keyboard_bug\android\app\src\main\java\io\flutter\plugins\GeneratedPluginRegistrant.java
[  +41 ms] ro.hardware = ranchu
[  +13 ms] Using hardware rendering with device Android SDK built for x86. If you notice graphics artifacts, consider enabling software rendering with
"--enable-software-rendering".
[  +24 ms] Initializing file store
[   +9 ms] Skipping target: gen_localizations
[   +4 ms] gen_dart_plugin_registrant: Starting due to {InvalidatedReasonKind.inputChanged: The following inputs have updated contents:
C:\Users\md844\source\repos\ucue_keyboard_bug\.dart_tool\package_config_subset}
[  +23 ms] Found plugin integration_test at C:\flutter\packages\integration_test\
[   +2 ms] gen_dart_plugin_registrant: Complete
[   +1 ms] Skipping target: _composite
[   +2 ms] complete
[   +4 ms] Launching lib\main.dart on Android SDK built for x86 in debug mode...
[   +3 ms] C:\flutter\bin\cache\dart-sdk\bin\dart.exe --disable-dart-dev C:\flutter\bin\cache\artifacts\engine\windows-x64\frontend_server.dart.snapshot --sdk-root
C:\flutter\bin\cache\artifacts\engine\common\flutter_patched_sdk/ --incremental --target=flutter --debugger-module-names --experimental-emit-debug-metadata
-DFLUTTER_WEB_AUTO_DETECT=true --output-dill C:\Users\md844\AppData\Local\Temp\flutter_tools.f728f2b7\flutter_tool.9d976d\app.dill --packages
C:\Users\md844\source\repos\ucue_keyboard_bug\.dart_tool\package_config.json -Ddart.vm.profile=false -Ddart.vm.product=false --enable-asserts --track-widget-creation
--filesystem-scheme org-dartlang-root --initialize-from-dill build\c075001b96339384a97db4862b8ab8db.cache.dill.track.dill --enable-experiment=alternative-invalidation-strategy
[   +7 ms] executing: C:\Users\md844\Android\SDK\build-tools\32.0.0\aapt dump xmltree C:\Users\md844\source\repos\ucue_keyboard_bug\build\app\outputs\flutter-apk\app.apk
AndroidManifest.xml
[   +8 ms] Exit code 0 from: C:\Users\md844\Android\SDK\build-tools\32.0.0\aapt dump xmltree C:\Users\md844\source\repos\ucue_keyboard_bug\build\app\outputs\flutter-apk\app.apk
AndroidManifest.xml
[        ] N: android=http://schemas.android.com/apk/res/android
             E: manifest (line=2)
               A: android:versionCode(0x0101021b)=(type 0x10)0x1
               A: android:versionName(0x0101021c)="0.0.1" (Raw: "0.0.1")
               A: android:compileSdkVersion(0x01010572)=(type 0x10)0x1d
               A: android:compileSdkVersionCodename(0x01010573)="10" (Raw: "10")
               A: package="uk.co.cuedspeech.ucue.debug" (Raw: "uk.co.cuedspeech.ucue.debug")
               A: platformBuildVersionCode=(type 0x10)0x1d
               A: platformBuildVersionName=(type 0x10)0xa
               E: uses-sdk (line=7)
                 A: android:minSdkVersion(0x0101020c)=(type 0x10)0x17
                 A: android:targetSdkVersion(0x01010270)=(type 0x10)0x1e
               E: uses-permission (line=14)
                 A: android:name(0x01010003)="android.permission.INTERNET" (Raw: "android.permission.INTERNET")
               E: application (line=22)
                 A: android:label(0x01010001)="uCUE" (Raw: "uCUE")
                 A: android:icon(0x01010002)=@0x7f080000
                 A: android:name(0x01010003)="io.flutter.app.FlutterApplication" (Raw: "io.flutter.app.FlutterApplication")
                 A: android:debuggable(0x0101000f)=(type 0x12)0xffffffff
                 A: android:extractNativeLibs(0x010104ea)=(type 0x12)0x0
                 A: android:appComponentFactory(0x0101057a)="androidx.core.app.CoreComponentFactory" (Raw: "androidx.core.app.CoreComponentFactory")
                 E: activity (line=29)
                   A: android:theme(0x01010000)=@0x7f0a0000
                   A: android:name(0x01010003)="uk.co.cuedspeech.ucue.MainActivity" (Raw: "uk.co.cuedspeech.ucue.MainActivity")
                   A: android:launchMode(0x0101001d)=(type 0x10)0x1
                   A: android:configChanges(0x0101001f)=(type 0x11)0x40003fb4
                   A: android:windowSoftInputMode(0x0101022b)=(type 0x11)0x10
                   A: android:hardwareAccelerated(0x010102d3)=(type 0x12)0xffffffff
                   E: meta-data (line=43)
                     A: android:name(0x01010003)="io.flutter.embedding.android.NormalTheme" (Raw: "io.flutter.embedding.android.NormalTheme")
                     A: android:resource(0x01010025)=@0x7f0a0001
                   E: meta-data (line=53)
                     A: android:name(0x01010003)="io.flutter.embedding.android.SplashScreenDrawable" (Raw: "io.flutter.embedding.android.SplashScreenDrawable")
                     A: android:resource(0x01010025)=@0x7f040002
                   E: intent-filter (line=57)
                     E: action (line=58)
                       A: android:name(0x01010003)="android.intent.action.MAIN" (Raw: "android.intent.action.MAIN")
                     E: category (line=60)
                       A: android:name(0x01010003)="android.intent.category.LAUNCHER" (Raw: "android.intent.category.LAUNCHER")
                 E: meta-data (line=67)
                   A: android:name(0x01010003)="flutterEmbedding" (Raw: "flutterEmbedding")
                   A: android:value(0x01010024)=(type 0x10)0x2
[   +8 ms] executing: C:\Users\md844\Android\SDK\platform-tools\adb.exe -s emulator-5554 shell -x logcat -v time -t 1
[  +13 ms] <- compile package:ucue_keyboard_bug/main.dart
[  +55 ms] --------- beginning of main

[...]

[  +49 ms] --------- beginning of main
           01-30 13:39:27.586 I/cmd     (25596): oneway function results will be dropped but finished with status OK and parcel size 4
[   +6 ms] executing: C:\Users\md844\Android\SDK\platform-tools\adb.exe -s emulator-5554 shell am start -a android.intent.action.RUN -f 0x20000000 --ez
enable-background-compilation true --ez enable-dart-profiling true --ez enable-checked-mode true --ez verify-entry-points true
uk.co.cuedspeech.ucue.debug/uk.co.cuedspeech.ucue.MainActivity
[  +34 ms] Starting: Intent { act=android.intent.action.RUN flg=0x20000000 cmp=uk.co.cuedspeech.ucue.debug/uk.co.cuedspeech.ucue.MainActivity (has extras) }
[   +1 ms] Waiting for observatory port to be available...
[ +360 ms] W/FlutterActivityAndFragmentDelegate(25618): A splash screen was provided to Flutter, but this is deprecated. See flutter.dev/go/android-splash-migration for migration
steps.
[  +21 ms] Observatory URL on device: http://<IP_ADDRESS>:39329/Bg_xE12sSvQ=/
[        ] executing: C:\Users\md844\Android\SDK\platform-tools\adb.exe -s emulator-5554 forward tcp:0 tcp:39329
[  +16 ms] 60159
[        ] Forwarded host port 60159 to device port 39329 for Observatory
[   +4 ms] Caching compiled dill
[  +24 ms] Connecting to service protocol: http://<IP_ADDRESS>:60159/Bg_xE12sSvQ=/
[ +252 ms] Launching a Dart Developer Service (DDS) instance at http://<IP_ADDRESS>:0, connecting to VM service at http://<IP_ADDRESS>:60159/Bg_xE12sSvQ=/.
[ +116 ms] DDS is listening at http://<IP_ADDRESS>:60162/57G22rjDLdI=/.
[  +43 ms] Successfully connected to service protocol: http://<IP_ADDRESS>:60159/Bg_xE12sSvQ=/
[  +56 ms] DevFS: Creating new filesystem on the device (null)
[  +32 ms] DevFS: Created new filesystem on the device (file:///data/user/0/uk.co.cuedspeech.ucue.debug/code_cache/ucue_keyboard_bugULKJZB/ucue_keyboard_bug/)
[   +2 ms] Updating assets
[  +67 ms] Syncing files to device Android SDK built for x86...
[   +1 ms] <- reset
[        ] Compiling dart to kernel with 0 updated files
[   +2 ms] <- recompile package:ucue_keyboard_bug/main.dart 027f4494-050d-4ec8-becc-6f960203b371
[        ] <- 027f4494-050d-4ec8-becc-6f960203b371
[ +114 ms] Updating files.
[        ] DevFS: Sync finished
[        ] Syncing files to device Android SDK built for x86... (completed in 120ms)
[        ] Synced 0.0MB.
[   +1 ms] <- accept
[   +3 ms] Connected to _flutterView/0xe290b860.
[   +2 ms] Flutter run key commands.
[   +1 ms] r Hot reload.
[        ] R Hot restart.
[        ] h List all available interactive commands.
[        ] d Detach (terminate "flutter run" but leave application running).
[        ] c Clear the screen
[        ] q Quit (terminate the application on the device).
[        ]  Running with sound null safety
[        ] An Observatory debugger and profiler on Android SDK built for x86 is available at: http://<IP_ADDRESS>:60162/57G22rjDLdI=/
[ +491 ms] I/eech.ucue.debu(25618): Waiting for a blocking GC ProfileSaver
[ +242 ms] The Flutter DevTools debugger and profiler on Android SDK built for x86 is available at: http://<IP_ADDRESS>:9101?uri=http://<IP_ADDRESS>:60162/57G22rjDLdI=/

pubspec.yaml

name: ucue_keyboard_bug
description: Demo app showing the keyboard bug when time dialation=3

publish_to: 'none' # Remove this line if you wish to publish to pub.dev
version: 0.0.1+1

environment:
  sdk: '>=2.12.0-0 <3.0.0'

dependencies:
  flutter:
    sdk: flutter
  flutter_localizations:
    sdk: flutter
  keyboard_actions: ^3.4.0
  pedantic:
  uuid: ^3.0.1

# Temporary dependencies
dependency_overrides:

dev_dependencies:
  flutter_test:
    sdk: flutter
  integration_test:
    sdk: flutter

analyzer:
  strong-mode: true
  language:
    enableSuperMixins: true

# The following section is specific to Flutter.
flutter:
  uses-material-design: true

$ flutter analyze
Analyzing ucue_keyboard_bug...
No issues found! (ran in 1.4s)

$ flutter analyze
Analyzing ucue_keyboard_bug...
No issues found! (ran in 1.4s)

md844@LAPTOP-FSG05SD7 MINGW64 ~/source/repos/ucue_keyboard_bug
$ flutter doctor -v
[√] Flutter (Channel stable, 2.10.0, on Microsoft Windows [Version 10.0.19044.1466], locale en-GB)
    • Flutter version 2.10.0 at C:\flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 5f105a6ca7 (8 days ago), 2022-02-01 14:15:42 -0800
    • Engine revision 776efd2034
    • Dart version 2.16.0
    • DevTools version 2.9.2

[√] Android toolchain - develop for Android devices (Android SDK version 32.0.0)
    • Android SDK at C:\Users\md844\Android\SDK
    • Platform android-32, build-tools 32.0.0
    • ANDROID_HOME = C:\Users\md844\Android\SDK
    • Java binary at: C:\Program Files\Android\Android Studio\jre\bin\java
    • Java version OpenJDK Runtime Environment (build 11.0.10+0-b96-7249189)
    • All Android licenses accepted.

[√] Chrome - develop for the web
    • Chrome at C:\Program Files\Google\Chrome\Application\chrome.exe

[X] Visual Studio - develop for Windows
    X Visual Studio not installed; this is necessary for Windows development.
      Download at https://visualstudio.microsoft.com/downloads/.
      Please install the "Desktop development with C++" workload, including all of its default components

[√] Android Studio (version 2020.3)
    • Android Studio at C:\Program Files\Android\Android Studio
    • Flutter plugin can be installed from:
       https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
       https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 11.0.10+0-b96-7249189)

[√] VS Code (version 1.64.0)
    • VS Code at C:\Users\md844\AppData\Local\Programs\Microsoft VS Code
    • Flutter extension version 3.34.0

[√] Connected device (5 available)
    • Pixel 4a (mobile)                  • 17131JEC201655 • android-arm64  • Android 12 (API 31)
    • Android SDK built for x86 (mobile) • emulator-5554  • android-x86    • Android 11 (API 30) (emulator)
    • Windows (desktop)                  • windows        • windows-x64    • Microsoft Windows [Version 10.0.19044.1466]
    • Chrome (web)                       • chrome         • web-javascript • Google Chrome 97.0.4692.99
    • Edge (web)                         • edge           • web-javascript • Microsoft Edge 97.0.1072.69

[√] HTTP Host Availability
    • All required HTTP hosts are available

! Doctor found issues in 1 category.

Screenshot of the app output, with mystery padding encircled:
https://i.stack.imgur.com/ClPJo.png
Screenshot of widget inspector:
https://paste.pics/8c925245d4b1734af6c580fabb153da9

Hi @md84419, can you check if this issue occurs without the keyboard_actions package?

@danagbemava-nc it is an interaction between keyboard_actions and timeDilation.  Both are required to reproduce the issue.
I haven't found a way to reproduce the issue without keyboard_actions.

Hi @md84419, in that case, I believe it would be best to file an issue on the keyboard_actions repo so that this may be properly addressed.
Seeing as keyboard_actions is a third-party package and the issue doesn't reproduce without it, I'm going to close this issue.
Thank you


Add API throttling

Although there're separate keys per repo and 5k available requests at a time, so hopefully a daily build will work just fine. Still, would be nice to at least get proper errors.
https://github.com/octokit/plugin-throttling.js/
https://github.com/actions/toolkit/issues/106 and https://github.com/actions/toolkit/blob/fc9958ec6a57d1b08e9a387c49971faecd0f0405/packages/github/src/github.ts#L18 might be useful on how to use plugins.

Timebox 2h

Resolved by #18.


added notice for insufficient space error w/ volume resizing

Added tool tip to CDK init steps with information about insufficient space errors and remediation using resize.sh script
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Updated bootstrap.sh to skip homebrew installation is it already exists. Cloud9 Ubuntu and AmazonLinux AMI are pre-baked with homebrew installed and .profile/.bash_profile updated
fixes #32


[DATA-209] | Backup consumer group offset

Taking backup consumer group offset, total records for topicPartition to S3.
Restore from S3 to kafka.
Consumer group offset transformation to set current position.

Made a few minor changes to the pipeline. Approved.

Shree Thank you  i will merge and verify


[FLINK-12469][table] Clean up catalog API on default/current database

What is the purpose of the change
(For example: This pull request makes task deployment go through the blob server, rather than through RPC. That way we avoid re-transferring them on each deployment (during recovery).)
Brief change log
(for example:)

Removed get/setCurrentDatabase() from Catalog API
Added getDefaultDatabase() in Catalog API
Provided implementations in sub classes

Verifying this change
This change is already covered by existing tests
Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): (no)
The public API, i.e., is any changed class annotated with @Public(Evolving): (no)
The serializers: (no)
The runtime per-record code paths (performance sensitive): (no)
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (no)
The S3 file system connector: (no)

Documentation

Does this pull request introduce a new feature? (yes)
If yes, how is the feature documented? (JavaDocs)

Thanks @xuefuz for the PR.
Currently this PR doesn't seem to verify whether user-set "defaultDatabase" in catalog impls' constructors actually exists or not upon initialization. Do we need to ensure it actually exists? if so, how? I have something in mind but want to listen to your ideas first.

default DB is most likely from configuration and it cannot be verified until a catalog is opened. This shouldn't be a problem as error will be reported later when the db is actually referenced.

default DB is most likely from configuration and it cannot be verified until a catalog is opened. This shouldn't be a problem as error will be reported later when the db is actually referenced.

I wonder if we should verify its existence upon catalog opening and fail early if the db doesn't exist? The reason being that the default db cannot be changed at runtime. If SQL CLI users realize it's nonexistent in the later stage of their workflow, e.g. the default db name they put has a typo, they may have several options: 1) not use any feature related to default db, 2) reconfig and restart their clients and lost all their previous progress in memory, 3) create that db (not a good option for a typo name). None seems very user-friendly.

default DB is most likely from configuration and it cannot be verified until a catalog is opened. This shouldn't be a problem as error will be reported later when the db is actually referenced.

I wonder if we should verify its existence upon catalog opening and fail early if the db doesn't exist? The reason being that the default db cannot be changed at runtime. If SQL CLI users realize it's nonexistent in the later stage of their workflow, e.g. the default db name they put has a typo, they may have several options: 1) not use any feature related to default db, 2) reconfig and restart their clients and lost all their previous progress in memory, 3) create that db (not a good option for a typo name). None seems very user-friendly.

Okay. Sounds good.

+1
@dawidwys  can you take another look? if you don't have other comments, I'll merge this once CI is green

@bowenli86 Can we move this forward? The YAML discussion should have its own thread. Thanks.

merging...


Export all icons in all resolutions

@ykamendrovskiy we need all of the icons in all resolutions: @1x, @2x, @3x.

We also need icons for video material type and other material type (Lecture screen).

"video material" isn't the lecture recording, is it? What sources are supported then? Youtube, Vimeo, anything else? I mean we can add icons for any of them, or some common video icon for all at once.

@ykamendrovskiy any of the sources - it's set up in the admin panel, so it's easier to add a common video icon.

@etolstoy both "video" and "uncpecified" icons added to Zepplin Cut artboard

@ykamendrovskiy New information on icons:

Icon for "create route" on the location screen
New tabbar icon for Search screen
New tabbar icon for Events screen

@etolstoy done and added to "cut" artboard in Zeplin

Thanks!


[stable/atlantis] AWS credentials toYaml error

Signed-off-by: Scott Crooks<EMAIL_ADDRESS>What this PR does / why we need it:
In the previous PR (https://github.com/helm/charts/pull/21305), I used toYaml, which is redundant, and leads to an error in the ~/.aws/credentials file if you're using a YAML multi-line string like so:
aws:
  credentials: |-
    [default]
    aws_access_key_id = <some-access-key>
    aws_secret_access_key = <some-secret-key>

This gets rendered as:
|-
  [default]
  aws_access_key_id = <some-access-key>
  aws_secret_access_key = <some-secret-key>

Which is wrong.
Which issue this PR fixes
N/A
Special notes for your reviewer:
Checklist
[Place an '[x]' (no spaces) in all applicable fields. Please remove unrelated fields.]

[X] DCO signed
[X] Chart Version bumped
[X] Variables are documented in the README.md
[X] Title of the PR starts with chart name (e.g. [stable/mychartname])

/ok-to-test

@zanhsieh Okay to merge this bad boy?

I ran into this as well trying to upgrade to 3.11.1, ended up running 3.11.0 for the time being.
Looking forward to getting this merged!

@sc250024
Sorry, only ids list in the OWNERS file could merge this. My id is not there.
@jkodroff
@callmeradical
@lkysow
@jeff-knurek
@sstarcher
@anubhavmishra
Would you mind to LGTM this PR please?

/assign @sstarcher

lgtm

/approve

I think the original PR before this should have been a minor version bump (https://semver.org/), but as this is a bug fix within a bug in a patch version, it makes sense as it is now

/lgtm


pickle.load() AttributeError: 'module' object has no attribute 'Gene'

Hi,
I ran your tool following the guidelines but when I try to run the Ribocode.py bit, I am getting this error:
Traceback (most recent call last):
File "", line 1, in
AttributeError: 'module' object has no attribute 'Gene'
It seems to stem from pickle.load()
Let me know if you have any suggestions.
Best

Hi,
Currently version of RiboCode only support the ensemble gtf file, for example,
ftp://ftp.sanger.ac.uk/pub/gencode/Gencode_human/release_19/gencode.v19.annotation.gtf.gz
It looks like the  line of "gene" feature is missing in your gtf file.

Hi,
The GTF file was the one you recommend in your tutorial, the Ensemble one.

Hi,
Can you post the complete error message here ？

Hi,
I just removed the package and reinstalled with pip this time and it then worked flawlessly, not sure why though, exactly the same files were used.
Thanks!

RiboCode use pickle to store and load data. The version of variables and classes are pickled. If you update the RiboCode package, the pickle file should be regenerated.


 Steps to reproduce:

Save this small piece code into a cpp file (for example crashtest.cpp):

DECLCALLBACK(void) test() {
    /* Test comment ******************************/
}

Open the file in VS Code

Expected result: VS Code allows me to edit the file.
Actual result: file opens but VS Code instantly freezes for indefinite time.
VS Code version: Code 1.36.0 (0f3794b38477eea13fb47fbe15a42798e6129338, 2019-07-03T13:20:56.241Z)
OS version: Linux x64 5.0.0-20-generic snap

System Info

Item
Value

CPUs
Intel(R) Core(TM) i5-8400 CPU @ 2.80GHz (6 x 3799)

GPU Status
2d_canvas: unavailable_softwareflash_3d: disabled_softwareflash_stage3d: disabled_softwareflash_stage3d_baseline: disabled_softwaregpu_compositing: disabled_softwaremultiple_raster_threads: enabled_onnative_gpu_memory_buffers: disabled_softwareoop_rasterization: disabled_offprotected_video_decode: disabled_offrasterization: disabled_softwareskia_deferred_display_list: disabled_offskia_renderer: disabled_offsurface_synchronization: enabled_onvideo_decode: disabled_softwareviz_display_compositor: disabled_offwebgl: unavailable_softwarewebgl2: unavailable_software

Load (avg)
1, 0, 0

Memory (System)
15.34GB (0.89GB free)

Process Argv

Screen Reader
no

VM
0%

Extensions: none

@where-is-s This should be fixed upstream with version 1.12.15 of marketplace.visualstudio.com/items?itemName=jeff-hykin.better-cpp-syntax.
if you want the changes immediately, install the Better C++ Syntax extension.
if you're okay with waiting a few days, you can use the VS Code Insider's release.
if you're okay with waiting a while, the changes will eventually be pulled into VS Code stable.

@matter123 thank you for jumping on all these issues.
@where-is-s the issue will be fixed in 1.36.1!

@where-is-s the issue will be fixed in 1.36.1!

Thank you @alexr00! I'm looking forward to that :)


SoLiD ontologies

We need to have coherence in the ontologies we use so that our apps can work together . At the minimum we need to list the ontologies which apps are using, with perhaps descriptions of the problems found in those ontologies. This can help us choose interoperable ontologies. We should also document how different apps use them, so that we can start gathering patterns of deployment.

I didn't notice this issue when I created https://github.com/solid/vocab . Same intention.

Given that we have the solid/vocab repo for the anthologies, can we close this issue?

yep fine. Thanks.


When downloading a wallet via hiro.so info/data to verify the download is excluded

A users asked
Why there isn't a hash or code crc256 to check if application is genuine to thwart supply chain attacks?
We do have signed releases on github. Is there something else we should do here. Or are we assuming that a user who wants to run such a check to be able to find our github repository?
I do not think adding the data to the download (a seperate file with the hashes?) would solve the issue because if hiro.so got compromised both could be replaced by the hacker providing false security.
Should we perhaps provide the option to (a) direct to Github for a user to pick the appropriate version for themselves and/or (b) provide the option to verify authenticity?

You can't download the web wallet from hiro.so. Are they referring to the desktop wallet?

Sorry I will close this here and open it on the desktop repo.

see here https://github.com/hirosystems/stacks-wallet/issues/1146


Transfer ownership of the name "xz" and "xz-sys"

Hi @alexcrichton! I'm Hyeon Kim, programmer from South Korea.
5 years ago, I created xz and xz-sys crate. Or more precisely, I just allocated the names and forgot updating. At the time, Rust didn't have LLVM bindings at all. I tried to make my own bindings as needed, but because I ran out of time, I completed only xz-sys part and stopped working on xz to do something more urgent.
And five years passed, completely forgetting that I stopped working on xz. Today I looked up the LLVM bindings again as needed, and found that packages named xz2 and llvm-sys were created.
It's a shame that what I forgot has confused people. There were people who were confused about which of the two libraries to use.
Transfer ownership of the "xz" crate and rename this library to xz seems to be a good way in many ways. What do you think?

Seems reasonable to me! I don't really have much time to handle renaming right now, but it might be possible to make xz depend on xz2, flag itself as deprecated, and guide folks towards xz2 as a name? Perhaps one day I can get around to renaming things?

@alexcrichton I think it's a good idea. For the time being, let's do it that way, and rename it later if we get some time.

@simnalamburt do you  want to add me to the xz/xz-sys names? Or were you going to publish such a crate?

@alexcrichton I were going to publish a such crate but I didn't yet, but I just invited you to the xz and xz-sys crates thought 👍

If you're busy, we can work slowly and I'll help you, so please don't feel any pressure :)

I've just

Invited you to xz and xz-sys crates
Relicensed xz and xz-sys crates as (APACHE-2.0 OR MIT) license
Aliased xz as xz2 and, xz-sys as lzma-sys with pub use xz2::* statement. You can check the results in https://docs.rs/xz and https://docs.rs/xz-sys

I also didn't forget to mention that "xz and xz-sys is currently an alias of xz2 and lzma-sys"

So maybe we can postpone the renaming thing later?

Looks good to me! The only thing I'd add is a #[deprecated] tag to encourage use of xz2 instead of xz (to avoid having two ways to do the same thing)

If we're going to merge xz2 and xz into xz, isn't deprecating xz will confuse the future users more?

Ok, I don't have time to work on this now, so this seems fine.


Add DiskCacheDatasetLoader

This lets me define something like:
    'dataset_loader_train': {
        '__factory__': 'palladium.dataset.DiskCacheDatasetLoader',
        'path': '/tmp/my-dataset-cache-{key}',
        'attrs': ['engine', 'sql', 'target_column'],
        'impl': {
            '__factory__': 'palladium.dataset.SQL',
            'url': 'some-db-url',
            'sql': 'some-sql',
            'target_column': 'some-column',
        },

And have the dataset loader cache the result of a potentially costly SQL query on the filesystem.

Coverage remained the same at 100.0% when pulling fa15a4cbecf0dc14b78ba141127171d8e41c76b9 on samsungaccelerator:feature-diskcachedatasetloader into 7ad551e9042ad1996d1d21872e2ce4d1b1b89454 on ottogroup:develop.

Looks good, I am about to merge. :-) Have you also been thinking of realizing this as a decorator (and to introduce decorator lists for dataset loaders)?

@alattner Oh I had half forgotten about the decorators.  We could probably just wrap the __call__ methods of dataset loaders with decorators then?  Let me try this.

@alattner Using a pluggable decorator turned out to be much more elegant!  See #20


fix: Prevent Errors in Debugger When Deleting List Widget

Description
Problem
When deleting a List widget in the EE environment, the widget is successfully removed, but errors are displayed in the in-app debugger. These errors disappear upon refreshing the page, leaving users confused as to why the errors appeared in the first place, despite the widget being deleted correctly.
Root Cause

The issue arises from the sequence in which widgets and their associated meta widgets are deleted.

When deleting a widget with meta widgets (e.g., a List widget):

The main widget is removed first via the deleteWidget saga.

Meta widgets are then deleted later during the componentWillUnmount lifecycle method.

The lint evaluation runs between these two processes, detecting meta widgets still present in the state without a parent widget.

This mismatch triggers errors in the in-app debugger, which disappear once the page is refreshed, as the meta widgets have already been deleted by then.

Solution

We have modified the deletion process to ensure that both the main widget and its associated meta widgets are removed within the same action.

Instead of handling meta widget deletion in the general deleteSaga:

We now check for widgets with the hasMetaWidgets flag within the deleteWidget saga.

If such widgets are found, their meta widgets are deleted before the main widget is removed from the layout.

This adjustment ensures that the lint evaluation occurs after both the main widget and its meta widgets have been removed, preventing errors in the debugger.

Fixes #35628
Automation
/ok-to-test<EMAIL_ADDRESS>:mag: Cypress test results

[!CAUTION]
🔴 🔴 🔴 Some tests have failed.
Workflow run: https://github.com/appsmithorg/appsmith/actions/runs/10492285178
Commit: 8c75e9cbb1490a54a83fb1c2ba0b6df923b233fa
Cypress dashboard.
Tags: @tag.All
Spec:
The following are new failures, please fix them before merging the PR:
cypress/e2e/Regression/ClientSide/Widgets/ListV2/Listv2_BasicClientSideData_spec.js
cypress/e2e/Regression/ServerSide/ApiTests/API_All_Verb_spec.js
List of identified flaky tests.
Wed, 21 Aug 2024 16:16:50 UTC

Communication
Should the DevRel and Marketing teams inform users about this change?

[ ] Yes
[ ] No

Summary by CodeRabbit

New Features

Enhanced testing for the list widget to include verification of the drag, drop, and delete functionalities.
Improved widget deletion process by ensuring related meta widgets are also addressed during deletion.
Updated test descriptions for better clarity and scope, specifically focusing on deletion functionality.

Bug Fixes

Expanded test coverage for potential issues in widget behavior, particularly concerning deletion.

@jacquesikot there is test case failure. Please check.

/build-deploy-preview skip-tests=true


Update syntax to swift 3

Xcode 8 requires me to make theses type casting changes in order to build successfully.

Hm, weird. This does't show up for me. Can you inspect what the return type of the .URL method of SwiftyJSON is? It should already be a URL.

In my case it returns an NSURL. I'm using this SwiftyJSON https://github.com/IBM-Swift/SwiftyJSON.git. Which version do you use?
`//MARK: - URL
extension JSON {
//Optional URL
public var URL: NSURL? {
    get {
        switch self.type {
        case .string:
            guard let encodedString_ = self.rawString.addingPercentEncoding(withAllowedCharacters: CharacterSet.urlQueryAllowed) else {
                return nil
            }
            return NSURL(string: encodedString_)

        default:
            return nil
        }
    }
    set {

#if os(Linux)
self.object = newValue?.absoluteString._bridgeToObjectiveC()
#else
self.object = newValue?.absoluteString
#endif
}
}
}
`

Oh I see in Cartfile you have github "appsailor/SwiftyJSON" "swift3". Let me try that.

Pointing to appsailor/SwiftyJSON works for me now


Created Documentation for issue number 924 - Pointers and Two Dimensional Array

Created a documentation for issue number #924 for C/C++ domain. Added a '.md' file for documentation and a '.cpp' file as an example for documentation part.

Please review and merge my pull request.


Get address pipeline

For Meine Stadt Transparent we're using exactly the same pipeline as in the example (we just copied it, to be honest), so I thought it'd be good to merge that function into the core package.
As I've already been at it, I also made refactored other parts of the codebase in the second commit, mostly applying pep8 style, but also extracting test data to files and getting rid of some try-catch-block.

Coverage decreased (-2.4%) to 97.619% when pulling 8b823635f73c802e6927be976b27744f14c0d70b on meine-stadt-transparent:get_address_pipeline into b18296b54e79b664388ad71804e64bdd50161b85 on stadt-karlsruhe:master.

Thanks for your PR, @konstin! It's nice to hear that you've found geoextract to be useful.
However, I am not sure whether moving the complete example into the core is a good idea. This would only be a benefit if it is applicable in many use cases exactly as it is configured. Since the example is written as a basic illustration I doubt that it performs well in real-life scenarios. For example, here's the normalization setup that we currently use in production:
def _abbr(s):
    return r'\b{}(?:\.|\b)'.format(re.escape(s))

_SUBSTITUTIONS = [
    (_abbr('ev'), 'evangelisch'),
    (_abbr('evang'), 'evangelisch'),
    (_abbr('kath'), 'katholisch'),
    (_abbr('st'), 'sankt'),
    (_abbr('u'), 'und'),
    (_abbr('sudl'), 'sudlich'),
    (_abbr('nordl'), 'nordliche'),
    (_abbr('ostl'), 'ostliche'),
    (_abbr('westl'), 'westliche'),
    (_abbr('nsg'), 'naturschutzgebiet'),
    (_abbr('lsg'), 'landschaftsschutzgebiet'),
    (_abbr('kjt'), 'Kinder- und Jugendtreff'),
    (_abbr('kjh'), 'Kinder- und Jugendhaus'),
    (_abbr('jva'), 'Justizvollzugsanstalt'),
    (_abbr('stadt'), 'stadtisch'),
    (_abbr('staatl'), 'staatlich'),
    (_abbr('asv'), 'allgemeiner sportverein'),
    (_abbr('tc'), 'tennisclub'),
    (_abbr('e.v.'), ''),

    (r'(\w)str(?:\.|\b)', r'\1strasse'),

    ('kindertagesstatte', 'kita'),
    ('kindertageseinrichtung', 'kita'),
    ('kinderkrippe', 'kita'),
    ('betriebskindertagesstatte', 'kita'),
    ('musikkindertagesstatte', 'kita'),
    ('sportkindertagesstatte', 'kita'),
    ('kindergarten', 'kita'),
    ('schulkindergarten', 'kita'),

    ('werkrealschule', 'realschule'),

    ('ganztagesschule', 'schule'),
    ('ganztagsschule', 'schule'),
    ('ganztagsrealschule', 'realschule'),
    ('ganztagesrealschule', 'realschule'),
    ('ganztagsgrundschule', 'grundschule'),
    ('ganztagesgrundschule', 'grundschule'),
    ('ganztagsgymnasium', 'gymnasium'),
    ('ganztagesgymnasium', 'gymnasium'),

    ('kinder und jugendhaus', 'jugendhaus'),
    ('kinder und jugendtreff', 'jugendhaus'),
    ('kindertreff', 'jugendhaus'),
    ('jugendtreff', 'jugendhaus'),
    ('kinderhaus', 'jugendhaus'),

    ('jugendmusikschule', 'musikschule'),

    ('schulerhort', 'hort'),

    ('stadtteilfriedhof', 'friedhof'),

    ('ortsverwaltung', 'rathaus'),
    ('stadtamt', 'rathaus'),
    ('burgerburo', 'rathaus'),
]

_normalizer = geoextract.BasicNormalizer(subs=_SUBSTITUTIONS, stem='german')

As you can see, that is far more complex than what the example provides (and it's still very limited due to our focus on communal institutions). Other parts of the pipeline also require a careful configuration that is tuned to the particular use case.
I'm open to arguments for moving specific parts of the example into the core (for example, a GermanAddressExtractor that's basically the PatternExtractor as configured in the example might be a good idea).
Regarding your code style  changes: these are in general very welcome, and it's nice to see someone care about code style! However, note that PEP8 makes no recommendation regarding single- vs. double-quotes for normal string literals and that geoextract uses single-quotes by default. For triple-quoted literals, PEP8 recommends to follow PEP257, but we do not follow that recommendation (we also use a line break after the initial triple-quote).
The code style hasn't been documented before, so I took your PR as a motivation to add a linting step to our test setup. See the updated README for instructions.

That substitution example is very educational! We're actually using the default pipeline by just throwing open street map and extracted pdfs in. We haven't done any evaluation as the locations were making a nice map. (Though we definitely should do)
Thanks for adding flake8, that makes a consistent code style much easier. I've changed the code accordingly and rebased.
I think an entrypoint that just needs a set of locations is quite useful for getting started, both when you're new to geodata extraction and for when you're getting started with a dataset. I'll do some further experimentation and make another Pull Requests when I've figured it out. Of this pull request remain mainly refactorings, so I've changed the title accordingly.

I guess I've been too quick about just changing things because they felt right. As there's not real value remaining, I'm going to close this pull request.
For my own code I've written an AddressPipeline class based on the example code. I'll probably do some manual evaluation and build a substitution list.
class AddressPipeline(geoextract.Pipeline):
    def __init__(self, locations, subs=None, stem='german'):
        if subs is None:
            if stem == 'german':
                subs = [(r'str\b', 'strasse')]
            else:
                subs = []
        normalizer = geoextract.BasicNormalizer(subs=subs, stem=stem)

        name_extractor = geoextract.NameExtractor()

        address_pattern = re.compile(r'''
            (?P<street>[^\W\d_](?:[^\W\d_]|\s)*[^\W\d_])
            \s+
            (?P<house_number>([1-9]\d*)[\w-]*)
            (
                \s+
                (
                    (?P<postcode>\d{5})
                    \s+
                )?
                (?P<city>([^\W\d_]|-)+)
            )?
        ''', flags=re.UNICODE | re.VERBOSE)

        pattern_extractor = geoextract.PatternExtractor([address_pattern])

        extractors = [pattern_extractor, name_extractor]

        keys_to_keep = ['name', 'street', 'house_number', 'postcode', 'city']
        postprocessors = [(geoextract.KeyFilterPostprocessor(keys_to_keep))]

        super().__init__(locations,
                         extractors=extractors,
                         normalizer=normalizer,
                         postprocessors=postprocessors)

@konstin Thanks for sharing your AddressPipeline! Feel free to share your substitution list, too, once you're satisfied with it, I'm certain you'll find things that are of interest to us (and others), too.


error compile: no required module provides package

no required module provides package github.com/bouk/babelfish; to add it:
        go get github.com/bouk/babelfish

The module path is bou.ke/babelfish, so please use that


Use in Windows

There are no instructions about windows installation and start. It's impossible to use in windows?

It's possible, however, you need to run it in Windows Command(search that if you don't know), which may be a little bit tricky to ordinary users.


Add initial accordion implementation

I must have been half asleep when I opened the PR for this, cc @afshin


Fix for [issue 116662661](https://www.pivotaltracker.com/projects/122…

…5464/stories/116662661) - email out high level log messages on prod

we want this merged into dev, not master.  updated the project default branch, lets try again...


Add support for bulk actions in extensions list view

We currently support multi-select in the extensions list view but there are no actions a user can take once you have items selected. I would expect the same actions from a single extension be made available (enable/disable/uninstall).
Example
An example where I attempt to disable multiple extensions but only one gets disabled.

Duplicate of #44508


Multiple lines

When I try to send something like this
client.send_sms(number, "hi\n")
or
client.send_sms(number, """hi
hi""")
I get an error
Is this supposed to happen? Or can you just not use multiple lines in a message.

You should be able to use any string as the message. Have your tried assigning the multiline string as a variable then using the variable instead?

You should be able to use any string as the message. Have your tried assigning the multiline string as a variable then using the variable instead? (Just as a test to make sure it's not just the function call)

Yes I have still got the same error

You should be able to use any string as the message. Have your tried assigning the multiline string as a variable then using the variable instead? (Just as a test to make sure it's not just the function call)

Yes I have still got the same error

Alright, what's the traceback? Put the error in dpaste.org and post it here.

You should be able to use any string as the message. Have your tried assigning the multiline string as a variable then using the variable instead? (Just as a test to make sure it's not just the function call)

Yes I have still got the same error

Alright, what's the traceback? Put the error in dpaste.org and post it here.

https://dpaste.org/2L7i

We have actually had this issue before.

You should be able to use any string as the message. Have your tried assigning the multiline string as a variable then using the variable instead? (Just as a test to make sure it's not just the function call)

Yes I have still got the same error

Alright, what's the traceback? Put the error in dpaste.org and post it here.

https://dpaste.org/2L7i
Looks like either your sid or the request was just straight out denied by text now. Try using auth_reset and giving it your sid again

For now you have to use \\n instead of \n but stay tuned its going to be patched in the next update

You should be able to use any string as the message. Have your tried assigning the multiline string as a variable then using the variable instead? (Just as a test to make sure it's not just the function call)

Yes I have still got the same error

Alright, what's the traceback? Put the error in dpaste.org and post it here.

https://dpaste.org/2L7i
Looks like either your sid or the request was just straight out denied by text now. Try using auth_reset and giving it your sid again

I have tried that when I first got the error

For now you have to use \\n instead of \n but stay tuned its going to be patched in the next update

That's not what he's talking about. He thinks multi-line strings won't work.

You should be able to use any string as the message. Have your tried assigning the multiline string as a variable then using the variable instead? (Just as a test to make sure it's not just the function call)

Yes I have still got the same error

Alright, what's the traceback? Put the error in dpaste.org and post it here.

https://dpaste.org/2L7i
Looks like either your sid or the request was just straight out denied by text now. Try using auth_reset and giving it your sid again

I have tried that when I first got the error

I didn't notice this. do what Leo said and escape the forward slash
client.send_sms(text, "test\ntest")

No multiline strings also won't work at the moment either.

No multiline strings also won't work at the moment either.

Seriously?! Why

I actually didn't know they didn't work until a couple weeks ago so I'm including it in the next update. The way the request sends has to have the \n be in the string instead of an actual newline.

I actually didn't know they didn't work until a couple weeks ago so I'm including it in the next update. The way the request sends has to have the \n be in the string instead of an actual newline.

hmmm weird. alright


add cita_getTransactionProof

add new rpc method cita_getTransactionProof
so add request transaction_proof and response cita_getTransactionProof

Codecov Report

Merging #37 into develop will decrease coverage by 0.19%.
The diff coverage is 0%.

@@            Coverage Diff            @@
##           develop     #37     +/-   ##
=========================================
- Coverage    47.39%   47.2%   -0.2%
=========================================
  Files          124     124
  Lines        19845   19925     +80
=========================================
  Hits          9405    9405
- Misses       10440   10520     +80

Impacted Files
Coverage Δ

libproto/src/protos/request.rs
3.55% <0%> (-0.13%)
:arrow_down:

libproto/src/protos/response.rs
4.3% <0%> (-0.19%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b5ec267...c6f494a. Read the comment docs.


docs: number of stars on create.t3.gg is wrong

Provide environment information
NA
Describe the bug
The number of github stars listed on https://create.t3.gg is 42k, which is almost double how many the project actually has
It was broken in this PR https://github.com/t3-oss/create-t3-app/pull/1750. githubStars is never reassigned to anything else and the fetch achieves nothing, whether or not it succeeds.
let githubStars = 42069;

try {
  (await fetchGithub("https://api.github.com/repos/t3-oss/create-t3-app", {
    throwIfNoAuth: false,
    fetchType: "repo",
  }).then((data) => data?.stargazers_count)) ?? 42069;
} catch (e) {
  console.error("unable to fetch from github", e);
}

42069 is a very misleading fallback which should not be possible. Fall back to zero or a conservative number, rather than a massive one.
Reproduction repo
NA
To reproduce
NA
Additional information
No response

I was about to open an issue, I got confused 😄

Closed via https://github.com/t3-oss/create-t3-app/commit/276dde486b0b2d2db375d2607586b3d632e24ab6

@juliusmarminge you haven't fixed it.
Look carefully at the fetch. It does nothing. It has no side effect.

no astro caching is bust so we get ratelimited, we burned through like 3 api keys before github reached out and asked wtf we were doing xd...
but i made the number closer to reality...
we'll fix it properly and get caching working if we update astro version or something in the future...


⚠ PrivateHD has degraded performance

In dfcac1e, PrivateHD ($PHD) experienced degraded performance:

HTTP code: 200
Response time: 1742 ms

Resolved: PrivateHD performance has improved in 7a406f6.


The bot can't start

Please tick all applicable boxes.

[x] I am using Python 3.5.3 or higher (run python --version on the command line)
[x] I have followed the official guides to install the bot for my system
[x] I have updated my dependencies to the latest version using the appropriate update script

Which version are you using?

[x] The latest master version (release-260419)
[ ] The latest review version

What type of issue are you creating?

[ ] Bug
[ ] Feature request
[x] Question

Description of issue
the bot can't start and i have no idea why

Log file
musicbot.log

Same Problem here

If this is still an issue (I know, the reply is a bit delayed...)
Go to %appdata% --> Python --> Python37 --> site-packages.
Delete the folder youtube_dl and run update.bat, that should fix the issue.


refactor(ci): move to server-side bindings

Part of #1318.

#1365  👈
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @krzema12 and the rest of your teammates on  Graphite