A question regarding how to extend functionality

Hi Jerry, thanks for your efforts and thanks for sharing the results. As a beginner with Rust (but not a beginner with RS), I'm going to port an existing python app to Rust. I was able to include and use your existing realsense-rust crate as of 0.4.
My python code makes use of functions like:
device = config.resolve(self.__pipeline).get_device()
device.get_info(rs.camera_info.firmware_version))
device.get_info(rs.camera_info.serial_number

self.__pose_sensor = device.first_pose_sensor()
self.__pose_sensor.set_option(rs.option.enable_map_preservation, 1)
sensor.set_notifications_callback(self.__callback)

Unfortunately I can't figure out, how achieve something similar with your lib. Would you mind to give me a pointer?

I'm especially missing a way to provide the second parameter for this C function in config.rs:
 pub fn rs2_config_resolve(
        config: *mut rs2_config,
        pipe: *mut rs2_pipeline,
        error: *mut *mut rs2_error,
    ) -> *mut rs2_pipeline_profile;

Any help appreciated.

Anybody?

OK, disregard. Found something different.  https://gitlab.com/aivero/public/librealsense-rs. Pretty incomplete, but easy to extend. Seems simpler knitted and more like the original SDK

Sorry I cannot give quick response due to daytime errands.
The set_notifications_callback is not implemented yet. I would try to implement it if I have time. Before that happens, I make from_raw/into_raw methods public in master branch. You can obtain the raw pointers and pass to realsense-sys to call the raw C API.
You can change Cargo.toml use the master branch.
realsense-rust = { git = "https://github.com/jerry73204/realsense-rust.git" }

To generate the document from source,
cargo doc --open

If you have hard time understanding the API, your comment would be appreciated to help me see the shortcomings.

Don't worry, it is me, who is not getting it :). I'm a Rust beginner, and a lot of your code ist pretty much unreadable for me. In contrary the quoted code, which might not be that elegant, but follows more directly the C++ SDK (as it seems to me) is by that way more readable. Seems, there were also not that experienced programmers at work or they didn't want to prove witch every line, how clever they are :) (I don't say you did it, don't get me wrong).
Especially with the config.resolve function. In your code there is (at least for me) no way to have access to the required *mut rs2_pipeline, because the required handle is behind the private ptr in pipeline.  And to provide a "getter" for thsi seemed somehow unclean to me. The different encapsulation might work for most of the functions, but for such breakouts, which need to have access to other struct instances "over" the borders away, it doesn't work.
Also there is something strange with your lib. It happened to me 100% that the "destructor" was not called if terminating the app from e.g. the debugger. So the next attempt failed with "T.265 already running". I always had to unplug/replug USB to get it to work again.
Anyway, it is just me, don't bother.
I will check the from_raw/into_raw and see, if it helps me.
Thanks

Your guess is correct. The API was more like C++ rather than pyrealsense. It is similar to C++ but with more safety guarantees. An illustrating example is the Pipeline API. It is designed to prevent users from calling stop() twice in compile time. However, the C++ cannot stop you from doing so and raises an exception.
I see the the API is a bit "low-level", just like Intel's C++ SDK. I think it's not the cleverness about the design, but the C++ and Python sides have different considerations based on the choice of language. Writing C++ is unlike Python. We have to take care of every manually allocated memory. The lists and dicts are not free in C++, so we cannot provide a method consuming and returning lists without considering memory safety.
The realsense-rust, which uses much unsafe code to port C++ API, also suffers the burden. I recently reflect on this. It could have more convenient things (like pyrealsense's Device), more comprehensive namings (e.g. Frame<marker::Video> -> VideoFrame),  and more documented. Maybe it will happen in the next version.
Besides, please open an issue that it does not terminate the device properly.

OK, I tried your master version, but I'm appearing problems (might again be Rust basic knowledge): After trying to obtain the raw handles I'm no longer allowed to use the pipeline, since it has been moved (use of moved value: pipeline
value used here after moverustc(E0382)
main.rs(43, 26): `pipeline` moved due to this method call
main.rs(39, 9): move occurs because `pipeline` has type)

It's due to ownership rules in Rust. The Pipeline::into_raw_pars consumes the instance into raw pointers, moving the ownership of pipeline away, so you cannot use it later. Ownership is a fundamental concept in Rust and I suggest the reading.
The correct way is done as the following.

Unpack the pipeline to raw pointers by pipeline.into_raw_parts().
Do whatever you want with realsense-sys.
Recover the pipeline back by Pipeline::from_raw_parts().

Note that the function signature of from/into_raw_parts depends on the pipeline state. The pipeline can switch from inactive to active state and vice versa by start() and stop() methods. Look at my generated doc to see the details. (InactivePipeline, ActivePipeline)
This is the minimal example I can think of (not tested).
use realsense_rust::InactivePipeline;
let pipeline = InactivePipeline::new();
let (pipe_ptr, context_ptr) = pipeline.into_raw_parts(); // unpack the pipeline

// do some dirty job here
// before from_raw_parts, accessing pipeline variable is invalid here.

// recover the pipeline. The unsafe {} block is mandatory
let pipeline = unsafe { InactivePipeline::from_raw_parts(pipe_ptr, context_ptr) };

OK, that works. Thanks for the elaboration. But it all looks very complicated and - forgive me - pretty insane to me (even though it might be the safest code on earth). Maybe I'm to old for this sh*t... There is still a riddle, how to call things like that from that point:
pub fn rs2_config_resolve(
        config: *mut rs2_config,
        pipe: *mut rs2_pipeline,
        error: *mut *mut rs2_error,
    ) -> *mut rs2_pipeline_profile;

I appreciate your efforts, but I think I rather stick with the other code, or with Python, which works like a charm for me.
Thanks nevertheless.

You can register realsense-sys to your Cargo.toml.
[dependencies]
realsense-sys = "0.2"

and call the low level C API like this.
let error = std::ptr::null_mut();
let profile = realsense_sys::rs2_config_resolve(config, pipeline, error);

// error is not automatically checked in C API, do it manually
if !error.is_null() {
    panic!("fail");
}

I won't blame on your choice for your design. For me if I want something just work, I go to Python at first. I switch to Rust if I need performance and safety, especially for large projects with million lines of code. I thanks to your discovery.

Hey, Jerry, you're so kind. Thank you for spending so much time with a Rust noob like me. I was intending to try your latest suggestion, but there is something different compared to yesterday. The pose example does not compile anymore. I swear it did work yesterday:
let poseframe = frames.pose_frame()?.unwrap();
no method named `pose_frame` found for enum `std::option::Option<realsense_rust::Frame<realsense_rust::frame_kind::Composite>>` in the current scope

Interesting. It compiles on my laptop (commit d8aa0913638d19bf0eeee091e07147312351b6f5).
I guess you copy and paste the pose.rs to run the code. If it's the case, it won't compile because I changed the function signature. I would git pull again and run by cargo instead of copying the file.
cargo runtig --example pose

I tried to simplify the API and it's now 0.5.0. The doc is online if you're willing to give it a try.

Yepp. Thanks. That works. I guess I retry with your version. The way at which you implement kind of "state machine" by changing the type is super smart... I will definitely learn more about Rust...
But we are not fully through with this damned resolve call:

The complain is now:
let config_ptr = config.into_raw();
*mut {unknown}
mismatched types

expected struct `realsense_sys::rs2_config`, found a different struct `realsense_sys::rs2_config`

note: expected raw pointer `*mut realsense_sys::rs2_config` (struct `realsense_sys::rs2_config`)
         found raw pointer `*mut realsense_sys::rs2_config` (struct `realsense_sys::rs2_config`)
note: perhaps two different versions of crate `realsense_sys` are being used?rustc(E0308)
main.rs(44, 53): expected struct `realsense_sys::rs2_config`, found a different struct `realsense_sys::rs2_config`
mismatched types

I looked into your code in the first post. I'm a bit curious that you use resolve to get the device. Perhaps we can start by pipeline, which is officially suggested (link). We can find the list thru profile() and device(). The The set_option() on sensor is there. I know resolve() and first_pose_sensor is more convenient. I'll make it available if I have time.

Seems you're using realsense-sys 0.2. Please update to 0.3 such that your code is compatible with realsense-rust 0.5.

Problem remains, even with 0.3

  let profile = realsense_sys::rs2_config_resolve(config_ptr, pipe_ptr, error);
   |                                                     ^^^^^^^^^^ expected struct `realsense_sys::rs2_config`, found a different struct `realsense_sys::rs2_config`
   |

I will answer you in a minute regarding the start sequence I'm using in Python and why.

I looked into your code in the first post. I'm a bit curious that you use resolve to get the device. Perhaps we can start by pipeline, which is officially suggested (link). We can find the list thru profile() and device(). The The set_option() on sensor is there. I know resolve() and first_pose_sensor is more convenient. I'll make it available if I have time.

OK, there might be some noise in this code, but I hope you will be able to separate the nuts and bolts:
            if not self.__playing_requested:
                # Do not call cfg.enable_* after cfg.resolve has been called https://github.com/IntelRealSense/librealsense/issues/5479#issuecomment-567448946
                device = config.resolve(self.__pipeline).get_device()
                try:
                    self.__log.info("SDK: %s", rs.__version__)
                    self.__log.info("Firmware: %s", device.get_info(rs.camera_info.firmware_version))
                    self.__log.info("Serial Number: %s", device.get_info(rs.camera_info.serial_number))
                    self.__log.info("USB: %s", device.get_info(rs.camera_info.usb_type_descriptor))
                except:
                    pass    # Info not available in current master branch (2.31.0)

                # Makes only sense if not playing back
                # Get first pyrealsense2.pose_sensor in order to set options, load maps, save maps, set static node, get static node
                self.__pose_sensor = device.first_pose_sensor()
                self.__pose_sensor.set_option(rs.option.enable_map_preservation, 1)

                sensor = device.sensors[0]
                sensor.set_notifications_callback(self.__callback)

                if self.__test_startup_config("loading", lambda x: isinstance(x, dict)):
                    load_map_config = self.__start_config["loading"]
                    if load_map_config["enabled"]:
                        self.__log.debug("Loading map from '%s'", load_map_config["file_name"])
                        with open(load_map_config["file_name"], mode='rb') as map_file:
                            input_map = map_file.read()
                            self.__pose_sensor.import_localization_map(input_map)
            else:
                self.__log.debug("Enabling playback from file '%s", self.__playing_config["file_name"])
                config.enable_device_from_file(self.__playing_config["file_name"], False)

            self.__last_fix = None
            self.__last_pose = None

            # Start the pipeline
            self.__pipeline.start(config)

I even could drop the "try/except" sequence. 2.31.0 is long time history...

For me the important part is to get the "sensor", which allows me to load/save maps, set_/get_static_markers etc.

It compiles with realsense-sys 0.3 on my laptop. I think it's not about the version.
Note that cargo allows multiple versions of one crate co-exist together. In your case, the realsensor-rust depends on -sys 0.3, while your crate depends on -sys 0.2. Cargo will compile both versions.
It's fine in some cases. For example, you're running a large project. You don't want an old dependency of dependencies breaks your code. You can pick any version you want. It gets troubles only when you pass the type from new version to dependency's old API.
Could you show your Cargo.toml and rust version rustc --version such that I can inspect what's going on?
(I would update my toolchain by rustup update sometimes.)

I confirm, your sample works with cargo run --example pose
I separated the code from Pose into a new main and copied your Cargo.toml, so basically I' working from a new project.
Cargo.toml:

anyhow = "1.0"
tokio = { version  = "0.2", features = ["rt-threaded", "macros"] }
realsense-rust = { git = "https://github.com/jerry73204/realsense-rust.git" }
realsense-sys = "0.3"

rustc --version
rustc 1.47.0 (18bf6b4f0 2020-10-07)

See a git dependency there. I thinks it's incompatibility among git repo and -sys. Cargo will not automatically pull new commits until cargo update.

Oh boy, this is a crash course :) You suggested me to do the git dependency in order to benefit from "from_raw/to_raw"
Did that cargo update, but the problem remains, somehow.
error[E0308]: mismatched types
  --> src/main.rs:44:53
   |
44 |     let profile = realsense_sys::rs2_config_resolve(config_ptr, pipe_ptr, error);
   |                                                     ^^^^^^^^^^ expected struct `realsense_sys::rs2_config`, found a different struct `realsense_sys::rs2_config`
   |
   = note: expected raw pointer `*mut realsense_sys::rs2_config` (struct `realsense_sys::rs2_config`)
              found raw pointer `*mut realsense_sys::rs2_config` (struct `realsense_sys::rs2_config`)
   = note: perhaps two different versions of crate `realsense_sys` are being used?

error[E0308]: mismatched types
  --> src/main.rs:44:65
   |
44 |     let profile = realsense_sys::rs2_config_resolve(config_ptr, pipe_ptr, error);
   |                                                                 ^^^^^^^^ expected struct `realsense_sys::rs2_pipeline`, found a different struct `realsense_sys::rs2_pipeline`
   |
   = note: expected raw pointer `*mut realsense_sys::rs2_pipeline` (struct `realsense_sys::rs2_pipeline`)
              found raw pointer `*mut realsense_sys::rs2_pipeline` (struct `realsense_sys::rs2_pipeline`)
   = note: perhaps two different versions of crate `realsense_sys` are being used?

error: aborting due to 2 previous errors

For more information about this error, try `rustc --explain E0308`.
error: could not compile `positions`.

To learn more, run the command again with --verbose.

It must be some dependency mix: If I click the symbol rs2_config_resolve, VSCode opens bindings.rs from the stored github reference, but obviously this should be resolved by the realsense-3.0, I guess...

It's pretty fine to use git repo. It happens to be consistent with crate.io's realsense-rust for now.
Now the master branch has Config::resolve() and Device::first_xxx_sensor() methods. Could you test them for me? The doc can be generated by the command to help you find the signature of the methods.
cargo doc --open

You code somewhat translates to the following.
let pipeline = Pipeline::new()?;
let config = Config::new()?
    .enable_stream(StreamKind::Depth, 0, 640, 0, Format::Z16, 30)?
    .enable_stream(StreamKind::Color, 0, 640, 0, Format::Rgb8, 30)?;
let profile = config.resolve(&pipeline)?;

let device = profile.device()?;
let serial_number = device.serial_number()?;

let sensor = device.first_pose_sensor()?;

The set_notifications_callback is still TBD because I have to wrap my head to pass FFI callbacks to the C side. You can fallback to raw pointers to get around this.

OK, thanks for doing this. I pretty much appreciate your efforts. Great.
I changed you code a bit so that it meets my primary use case (I'm sure the other formats work as well):
   let pipeline = Pipeline::new()?;
   let config = Config::new()?
       .enable_stream(StreamKind::Pose, 0, 0, 0, Format::_6Dof, 200)?;
   let profile = config.resolve(&pipeline)?;

   let device = profile.device()?;
   let serial_number = device.serial_number()?;

   let sensor = device.first_pose_sensor()?;

Indeed, I was able to retrieve the serial number and there is also a "Sensor".
But I wasn't able to start the pipeline anymore:
let mut pipeline = pipeline.start(config)?;
Gives an error:
error[E0277]: the trait bound `std::option::Option<&realsense_rust::Config>: std::convert::From<realsense_rust::Config>` is not satisfied
  --> src/main.rs:44:33
   |
44 |     let mut pipeline = pipeline.start(config)?;
   |                                 ^^^^^ the trait `std::convert::From<realsense_rust::Config>` is not implemented for `std::option::Option<&realsense_rust::Config>`
   |
   = help: the following implementations were found:
             <std::option::Option<&'a T> as std::convert::From<&'a std::option::Option<T>>>
             <std::option::Option<&'a mut T> as std::convert::From<&'a mut std::option::Option<T>>>
             <std::option::Option<T> as std::convert::From<T>>
   = note: required because of the requirements on the impl of `std::convert::Into<std::option::Option<&realsense_rust::Config>>` for `realsense_rust::Config`

error: aborting due to previous error; 1 warning emitted

For more information about this error, try `rustc --explain E0277`.

..and an attempt to get the recommended fw-version gives the usual borrow after moved value nonsense:
let fw_version = device.recommended_firmware_version()?;

I changed the start() to accept the reference start(&config) to prevent taking the ownership.
Plus, by calling config.resolve(&pipeline), the returned profile may live longer than pipeline. I'm not sure if it is proper on the C side. Could you try this code to see what happens?
let config = Config::new();
// configure your config here
let pipeline = Pipeline::new();
let profile = config.resolve(&pipeline)?;
drop(pipeline); // explicitly drop the pipeline

// check if profile works after pipeline is dropped
let device = profile.device()?;
let sensor = device.first_post_sensor()?.unwrap();

Hmm. That doesn't compile at "config.resolve"
no method named `resolve` found for enum `std::result::Result<realsense_rust::Config, realsense_rust::Error>` in the current scope

method not found in `std::result::Result<realsense_rust::Config, realsense_rust::Error>`rustc(E0599)
main.rs(47, 22): method not found in `std::result::Result<realsense_rust::Config, realsense_rust::Error>`
no method named `resolve` found for enum `std::result::Result<realsense_rust::Config, realsense_rust::Error>` in the current scope
method not found in `std::result::Result<realsense_rust::Config, realsense_rust::Error>`rustc(E0599)

What am I missing?

By reading the error message, you have to unpack Result before actually calling resolve. You can see the explanation on std doc. For those errors not completely related to this crate, you can try to google it to find the answer.

Yes, sorry. I should be able to find that by myself. But there is no unpack for config.
Giving up. Thanks

I see your great effort. Actually I think Rust is the wrong language for your project. For one time usage application and quick development, Python or Go should be preferred.
Rust is targeted for system programming capable of writing OS kernels. There are reasons not to adopt try/catch. Rust writes result? sugar to simulate exception-style handling instead of .unwrap(). Rust also has explicit RC, unlike RC in Swift is done implicitly. RC incurs costs and some systems simply does not have heap to implement RC, which are not concerned in web/applications. It's not about lacking of them. Each language adopts proper design to reach their goals.
Regarding to your long journey, hope this helps finding proper choices for your project.

Well, yes. I'm not fanatic on this. I choose whatever is the lowest hanging fruit for a project. Rust is not for this, it turns out. It was an attempt. I failed, having to accept this.
BTW: After having unwrapped whatever sprang into the way, the last line didn't compile, obviously also because of an "easy to google"  error:
let sensor = device.first_post_sensor()?.unwrap();
no method named `first_post_sensor` found for struct `realsense_rust::Device` in the current scope

help: there is an associated function with a similar name: `first_pose_sensor`rustc(E0599)
no method named `first_post_sensor` found for struct `realsense_rust::Device` in the current scoperustc(E0599)
main.rs(52, 21): there is an associated function with a similar name

But maybe also some hidden Rust magic made this not to compile, who knows.
 OK, Friday 13th and lock down and 2020...

(Post deleted). Ah, this time it was just a typo. post instead of pose...
Not sure, if it is again an "easy to google thing", but just to finalize this:
error[E0382]: use of moved value: `pipeline`
   --> src/main.rs:48:6
    |
46  | let pipeline = Pipeline::new();
    |     -------- move occurs because `pipeline` has type `std::result::Result<realsense_rust::Pipeline<realsense_rust::pipeline_kind::Inactive>, realsense_rust::Error>`, which does not implement the `Copy` trait
47  | let profile = config.unwrap().resolve(&pipeline.unwrap())?;
    |                                                 -------- `pipeline` moved due to this method call
48  | drop(pipeline); // explicitly drop the pipeline
    |      ^^^^^^^^ value used here after move
    |
note: this function consumes the receiver `self` by taking ownership of it, which moves `pipeline`
   --> /Users/decades/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/src/rust/library/core/src/result.rs:970:19
    |
970 |     pub fn unwrap(self) -> T {
    |                   ^^^^

error: aborting due to previous error; 3 warnings emitted

For more information about this error, try `rustc --explain E0382`.

Did I ever see a more obtrusive language in my life? I admit not.   This is like hell...

Actually I think Rust is the wrong language for your project.

BTW: The code (python) is ready and running flawlessly in production since months, even on a RPI 3 with 200 fps. I just wanted to give Rust a chance to see the benefits. I just see, it is an overrated, overhyped, weird language. Just of the sake of making old style C programmers happy, which managed to do a print of a text plus a newline plus some underscores (of course depending on the length of the text) in one line of code.
I'm over this for a long time. And I don't have myself to prove I'm such a crack... 👍
But this is all not your bad. For sure.


F# Open Standup:  Monday 11/16/2005 --- 10:00 AM pt 10:30 AM pt

Hi,
So for the longest time we have wanted to do an open stand-up for those of us working on the Visual FSharp Tools, nothing mentioned will be NDA so if you are an MVP as well as a contributor and know NDA stuff please don’t discuss it in an open forum.  Let’s try out Skype see how it works out for us.
It is primarily for those who are contributing to, or are interested in contributing to the Visual F# tools open source project.  If you are in that group please join us on this Skype for business meeting, phone in should work if you don't have access to Skype for business.
Agenda
It’s a standup … no agenda … A few brief words about what you are working on, what interesting issues have cropped up and what you will to work on next.  If the time doesn’t work for you, then post an issue with your stand-up progress, we will review it.
Be kind, be brief, be present
Thanks
Kevin
.........................................................................................................................................
Join Skype Meeting
This is an online meeting for Skype for Business, the professional meetings and communications app formerly known as Lync.
Join by Phone
Toll-free number: +1<PHONE_NUMBER>
Toll number: +1<PHONE_NUMBER>
Find a local number
Conference ID: 630930
Help
Welcome to Skype for Business Online Meetings!  Please note, all participants using the Join by Phone option must dial the full number for access to the call!
[!OC([1033])!]
.........................................................................................................................................

Are there any plans to record this and post it somewhere?

@NatElkins We have no plans to record it, or even to post notes.  The intent is to allow developers actively working on features to share their progress and insights.

Great @KevinRansom !

Will join.

thx guys.
@vasily-kirichenko if you want we can chat later (not today) about vs extension and how to move forward the repo, vb -> f# , i'd like to help
some info url https://github.com/Microsoft/VisualStudio-SharedProject/issues/3

Roslyn is just about to check-in a project system based on CPS.  We should consider whether we want to follow suit.


removed extra zingocli infosudo apt install tree

wow, i guess i shouldnt use !s in a commit message, yikes

but perhaps they should be changed to log::debug?

wow, i guess i shouldnt use !s in a commit message, yikes

It's not git's fault, bash is catching the !!s before they make it to git. If you don't use -m and go to the interactive commit message screen, you can use as many !s as you want!

I merged this to fix a merge conflict, and then didn't format after. @fluidvanadium could you run a cargo format and push?


RHEL7 failed to enable service

Upon installing Kapacitor on RHEL7, doing the following to try to start it on startup comes up with this error...
#systemctl enable kapacitor
Failed to execute operation: Too many levels of symbolic links

I believe this is due to...
# ls -lh /etc/systemd/system
lrwxrwxrwx. 1 root root   41 Apr  7 10:01 kapacitor.service -> /usr/lib/systemd/system/kapacitor.service
-rw-rw-r--. 1 root root  466 Mar 22 22:47 kibana.service
-rw-r--r--. 1 root root  511 Mar 30 13:47 logstash.service

You can see the others are real .service files, but this is a symlink.

@jasonkeller What version did you install? Did you install via the rpms package we provide? What is the value of /usr/lib/systemd/system/kapacitor.service? Is it another symlink or an actual file?

Installed it via the repository with yum...
kapacitor.x86_64                     1.2.0-1
# If you modify this, please also make sure to edit init.sh

[Unit]
Description=Time series data processing engine.
Documentation=https://github.com/influxdb/kapacitor
After=network.target

[Service]
User=kapacitor
Group=kapacitor
LimitNOFILE=65536
EnvironmentFile=-/etc/default/kapacitor
ExecStart=/usr/bin/kapacitord -config /etc/kapacitor/kapacitor.conf $KAPACITOR_OPTS
KillMode=process
Restart=on-failure

[Install]
WantedBy=multi-user.target
Alias=kapacitor.service

@jasonkeller May be due to this https://github.com/systemd/systemd/issues/3010.

@adityacs that's entirely possible, but I couldn't say for certain that the issue lies entirely there.  Logstash, Kibana, Telegraf, and Elasticsearch all work just fine as packaged.

@jasonkeller From your screenshot I don't see symlinks created for elasticsearch.service and kibana.service. If theose services were enabled then there should have been a symlink. Also, for kapacitor there is a symlink created. I believe it's enabled. Have you tried rebooting your system and then check that kapacitor starts on reboot?

I believe it's putting them elsewhere...
# systemctl disable logstash
Removed symlink /etc/systemd/system/multi-user.target.wants/logstash.service.

Kibana wasn't actually enabled for startup...
# systemctl enable kibana
Created symlink from /etc/systemd/system/multi-user.target.wants/kibana.service to /etc/systemd/system/kibana.service.

Please pardon some of the confusion on this box; it's a temporary staging area that we're testing different products in and shaking down our methodology and configurations before moving to production.
# ls -lh /etc/systemd/system/multi-user.target.wants/
lrwxrwxrwx. 1 root root 45 Mar  9 10:18 elasticsearch.service -> /usr/lib/systemd/system/elasticsearch.service
lrwxrwxrwx. 1 root root 46 Mar  9 14:36 grafana-server.service -> /usr/lib/systemd/system/grafana-server.service
lrwxrwxrwx. 1 root root 41 Apr  7 10:01 kapacitor.service -> /usr/lib/systemd/system/kapacitor.service
lrwxrwxrwx. 1 root root 34 Apr 11 09:41 kibana.service -> /etc/systemd/system/kibana.service
lrwxrwxrwx. 1 root root 37 Mar  9 14:05 nginx.service -> /usr/lib/systemd/system/nginx.service
lrwxrwxrwx. 1 root root 40 Mar 17 09:57 telegraf.service -> /usr/lib/systemd/system/telegraf.service

It appears it wants actual service files in both /etc/systemd/system and /usr/lib/systemd/system, not just a symlink from the first to the second.

I am using centOS7 and I don't see any problem. I have a symlink from
/etc/systemd/system to /usr/lib/systemd/system
kapacitor.service -> /usr/lib/systemd/system/kapacitor.service

This error
Failed to execute operation: Too many levels of symbolic links

typically means that you have infinity recursive symlinks somewhere.
My guess is that systemd isn't being picky about symlinks but rather it followed the path of symlinks and never actually ended up at a real file.

Ironically now I can't even start Kapacitor; it fails after mere milliseconds of starting with no log output

In my experience I have seen systemd drop output from STDERR if the process immediately dies. Try running it by hand and see what output you get
sudo -u kapacitor kapacitord -config /etc/kapacitor/kapacitor.conf

I seem to have it sorted by copying the .service file instead of the symlink, and now systemd is happy.  I have the service starting and it should start on restart now.
Before I go too far OT, is there another place to post questions/issues other than influx community (which has been down all day now)?

Before I go too far OT, is there another place to post questions/issues other than influx community (which has been down all day now)?

No, for now using github issues is fine. Hopefully we have it back up soon, and are able to avoid such outages in the future.

Thank @nathanielc ; especially with Kapacitor I've had a very, very difficult time trying to find reliable configuration examples for what I'm trying to accomplish (basically taking stream output from Telegraf (which is gathering SNMP octet counters from network devices), calculating a longer average derivative (such as daily), and feeding it back into Telegraf to put into Elasticsearch in a different index set.
This is of course all in the name of downsampling the data, and I have to keep data bucketed per-interface per-device.  Basically I'll have 1 minute data kept 30 days; daily averages kept for probably 2 years for trending.  Thus far I've been striking out, and it's the last piece of this puzzle I need to put together.  Part of the reason for using Kapacitor to do the calculation is to try to make it resistant to a counter reset/rollover as I'm told my compatriots in the NOC love resetting counters on equipment.  Unfortunately I'm totally lost as to how to get Kapacitor working for this, especially since I have to use Elasticsearch and not InfluxDB (netflow will also be stored in this DB, and I need quick scale-out and redundancy options).

@jasonkeller Do you have a TICKscript you have tried at this point? Sounds like you now have the service running? What step are you on next?
What you describe sounds like pretty straightforward down sampling task so I should be able to help you out.  There is one hurdle you willll have, which is that Kapacitor has no persistence of the data it is processing. So if Kapacitor restarts you will loose all data in RAM, the typical answer to that problem is store the data in InfluxDB and Kapacitor can resume where it left off, but in your case that is going to be difficult. I would consider backing Kapacitor by InfluxDB at least for a small buffer of time so that you can overcome that issue. But we can come back to that problem when we get there.
Right now what have you tried and where are you stuck?

So far I have this in Telegraf...
[[inputs.http_listener]]
  service_address = "localhost:8186"
  read_timeout = "10s"
  write_timeout = "10s"

[[outputs.elasticsearch]]
  urls = [ "http://<IP_ADDRESS>:9200" ]
  timeout = "5s"
  enable_sniffer = false
  health_check_interval = "10s"
  index_name = "telegraf-%Y.%m.%d"
  namedrop = ["aggregate*"]

[[outputs.elasticsearch]]
  urls = [ "http://<IP_ADDRESS>:9200" ]
  timeout = "5s"
  enable_sniffer = false
  health_check_interval = "10s"
  index_name = "telegraf-%Y"
  namepass = ["aggregate*"]

[[outputs.influxdb]]
  urls = ["http://localhost:8086"]
  database = "telegraf"

And here's the TICKscript...
# cat /etc/kapacitor/rollup.tick
stream
    |from()
        .database('telegraf')
        .retentionPolicy('autogen')
        .measurement('interface')
        .groupBy(*)
    |window()
        .period(1h)
        .every(1h)
        .align()
        |derivative('ifHCInOctects')
                .as('in_bandwidth')
                .unit(1s)
                .nonNegative()
    |influxDBOut()
        .database('telegraf')
        .retentionPolicy('autogen')
        .measurement('aggregate')
        .precision('s')

stream
    |from()
        .database('telegraf')
        .retentionPolicy('autogen')
        .measurement('interface')
        .groupBy(*)
    |window()
        .period(1h)
        .every(1h)
        .align()
        |derivative('ifHCOutOctects')
                .as('out_bandwidth')
                .unit(1s)
                .nonNegative()
    |influxDBOut()
        .database('telegraf')
        .retentionPolicy('autogen')
        .measurement('aggregate')
        .precision('s')

Configuration of Kapacitor is just the default, though I ended up having to set this in the conf file to get the service to respond...
[[influxdb]]
  enabled = false

OK looks good so far expect for the influxdb telegraf output.
[[outputs.influxdb]]
  urls = ["http://localhost:9092"]
  database = "telegraf"

Telegraf can write to Kapacitor as if it were an InfluxDB server, just change the port to be the default Kapacitor port.
Once that is setup check that Kapacitor is receiving data from Telegraf via:
kapacitor stats ingress

You should see lots of entries from the telegraf database.

oooh!  that poke did it.  I'd have never guessed to switch an output from telegraf to the api endpoint of kapacitor (I thought that was only for commands to it, not data).
# kapacitor stats ingress
Database   Retention Policy Measurement Points Received
_kapacitor autogen          edges                    15
_kapacitor autogen          ingress                  15
_kapacitor autogen          kapacitor                 3
_kapacitor autogen          nodes                    12
_kapacitor autogen          runtime                   3
telegraf                    dhcp                   2540
telegraf                    interface              9687
telegraf                    system                   12

I'm hoping I'm at least close with this configuration, but I can't be perfectly sure how it's going to turn out (if it's going to be bucketed properly, how it's rolling up the interval, how much memory it needs, etc).
Memory and CPU are big concerns; right now my production flight-list will have Kapacitor co-located on the same node with Telegraf with 4 vCPU and 8GB of memory (I'm pushing for 16GB, but not sure I'm going to get it).  I have 1600 devices I'll be polling probably every minute.

@jasonkeller Community is back up, should we move this conversation over there?

Sure; https://community.influxdata.com/t/using-kapacitor-to-rollup-metrics/522

Great, I'll follow that thread over there. For now I am going to close this issue here.

Same issue with influxd v1.3.

For the record, I hit this issue with influxdb.service. The Debian package was failing in the configure stage:
$ sudo dpkg-reconfigure influxdb --force
Failed to execute operation: Too many levels of symbolic links

The workaround is to recreate the systemd wants symlink:
$ sudo systemctl disable influxdb
Removed symlink /etc/systemd/system/influxdb.service.
Removed symlink /etc/systemd/system/multi-user.target.wants/influxdb.service.
$ sudo systemctl enable influxdb
Created symlink from /etc/systemd/system/influxd.service to /lib/systemd/system/influxdb.service.
Created symlink from /etc/systemd/system/multi-user.target.wants/influxdb.service to /lib/systemd/system/influxdb.service.


Locale

Question
Hi there, I added a new locale translations for Icelandic on uploadcare/uploadcare-widget - is it possible to merge it with this library?

Hi @disaerna, you can simply run npm update uploadcare-widget in your project. As a result, the uploadcare-widget package will be updated to the latest version, and you'll be able to use the Icelandic locale.

Thanks for the answer 👍  I updated my package and all packages are up to date.
But after updateing my translations are still not added to the Locale type in */types/index.d.ts.
Am I missing something?

? 🤔

Great 💯  Thank you


Added --update and --nsescriptdb commands to sicat.py

This commit introduces two new commands, --update and --nsescriptdb, to sicat.py.
These commands enable the updating and registration of the nse_script.db database.
Usage:
python sicat.py --update --nsescriptdb

Sorry too late, I will review this at Sunday
Thanks.

Sorry for waiting so long to review, please check out some of the comments I've made.
thanks

updated branch.
please review:)


monolingual corpora - input

Development of the monolingual corpora input page (also sets up output page, but will keep the main development of that for the next PR to keep PR size down)

Interface screenshots:


Read information under mouse

Dear Devs,
I created a quick demo to read specific layout of my framebuffer based on mouse position.
It's write gl_InstanceIndex inside layout and inject which is under mouse to show high-lighting.
This demo is based on deferred demo.
Currently, demo does all I want but performances are not very perfect and validation layout give some error.
Do you know a mean to improve them ?

install barrier to have better sync ?
use optimal VK_IMAGE_LAYOUT_XXX
avoid to dump all image's data and setup better viewport/scissor (I think GPU->CPU bandwidth is not an issue).

See my Vulkan fork (this commit 399d3051d05cb92cf155291312e5fe289a2d0471).
Thanks
PS: Sorry, I code it quickly and my knowledge about Vulkan are not very good.
Don't hesitate to integrate this demo if you think it's important to community.

I don't plan on adding such a demo to my repo. If you're having performance problem, try to narrow down the area you read from (e.g. to just 1 pixel) and maybe do async readbacks.


Add support for placeholders similar to crowdin.yml GitHub Integration

Is your feature request related to a problem? Please describe.
I would like to switch to this Action to no longer requiring a dedicated Account for up and downloading the files, but from a Conversation with a Support-Member did I find out, that this action doesn't seem to support placeholders like %locale% or %fileextension% within commit_message or translation?
Quote from the Mail:

I'm afraid currently it doesn't work in this way. But you're welcome to add this to the issue and our developers will fix it :
https://github.com/crowdin/github-action/issues

Describe the solution you'd like
Add support for changing the placeholders similar to the CLI or GitHub Integration.
Describe alternatives you've considered
There aren't really any other alternatives I could think of other than staying with the current system of having a dedicated account.
Additional context

Hi @Andre601!
Have you consider options commit_message, pull_request_title or pull_request_body?
You can find all the available options in the Supported Options section of the Readme.

This is not what I was mentioning in my issue.
As I just said there did I had a conversation with the Crowdin support where it was pointed out that placeholders don't seem to be supported (yet) within the GitHub Action.
I in fact even mentioned that "[...] this action doesn't seem to support placeholders like %locale% or %fileextension% within commit_message or translation?"
While commit_message wouldn't be a big problem is translation a required option to support the placeholders, so that translations would be saved within the right directory and as the right file format (%locale%.%fileextension% would become de_CH.json)

Hello @Andre601
Placeholders are specified in the configuration files, kindly check an article on the matter below:
https://support.crowdin.com/configuration-file/#writing-a-simple-configuration-file
The following placeholders are supported in Crowdin:
https://support.crowdin.com/configuration-file/#placeholders

So how would the configuration look like when migrating from an existing github integration?

Configuration file would look just the same as for the integration. Here you may check more details about the migration:
https://github.com/crowdin/github-action/issues/7

Configuration file would look just the same as for the integration. Here you may check more details about the migration:
#7

Doesn't really help me that much tbh.
I have this PR open right now to implement the GitHub Action in the hope to "retire" the dedicated User account for translations.
What exactly would I need to update to make everything work like the old system?

To add to this does #7 not help me at all since the linked project on GitHub is no longer available and therefore can't be looked at to understand the file-structure.
As much as I appreciate the help would I appreciate it more if it was an actual, up-to-date viewable example and not one that has been deleted since.

Hi! Sorry for the delays, I'm passing your comments to our devs and will come back with updates on the matter

I quickly updated my comment with a bit more info (mainly added and updated some links)

Got it, thanks!

Hi @Andre601
Please check this pull request
commit_message is supported only in workflow without placeholders (not in the config file of github actions or cli)
In case commit is needed for each language separately, then steps should be created for each language. Just like here.

Hi @Andre601
Please check this pull request
commit_message is supported only in workflow without placeholders (not in the config file of github actions or cli)
In case commit is needed for each language separately, then steps should be created for each language. Just like here.

That's kind of the point for this issue then.
Not having such placeholders available is frustrating, especially if you switch from the setup using an Account.

Hi @Andre601
Thanks for the prompt reply, passing your comments to the team

Currently, Crowdin Github Action works on Crowdin CLI and it can build all languages or just one

Any updates or similar on this one?
While I honestly don't care about the action anymore (Just seemed to useless for my case to warrant a switch) would I still like to know if there has been any progress about this within the past year.

Hi @Andre601!
I will double-check this with the team and come back to you

Hi @Andre601!
The Crowdin GitHub Action already supports all the placeholder configurations that are available in the CLI or GitHub integration. Actually, GH Action is based on Crowdin CLI.
Could you please check the usage and supported options sections?


Updating list of entities matching a family

A class which will efficiently return an up-to-date list of all entities matching a family of components.

Implemented with queries


Headings: Add missing design specs

Which page is this about?
Headings: https://cfpb.github.io/design-system/foundation/headings
Describe your issue
Need to list the design specs for the variation headings.

Heading with icon
Eyebrow headings
Slug heading

It looks like the eyebrow headings and slug headings have specs that were added, but the "Heading with icon" variation still needs specs to be written and added.

It looks like the eyebrow headings and slug headings have specs that were added, but the "Heading with icon" variation still needs specs to be written and added.

For the "Heading with icon" variation, I added some bare-bones text specs and a link to our list of icons - hope that's sufficient!


Added incidents to dialogue [Incomplete]

Tried to implement #718 however I dont have much knowledge in this area of development, I managed to add the frontend, I attempted some backend. All that needs to be done:

Store the boolean of it in the database
Listen to when a incident is created/updated and send notification
Text to send when incident is created/updated

It would be nice if someone would finish this. It's a nice feature that added more options and benefits to use uptime for someone who wants to inform his community about incidents.


[Question] type sharing when plugin has a newer version of a dependency?

Is type sharing supported when a plugin has a newer version of a dependency than the host process?  I am having issues getting this to work when I deploy a plugin with an interface from an assembly with a slighly higher build version (no breaking interface changes).  I load the plugin assembly as per examples:
using (var loader = McMaster.NETCore.Plugins.PluginLoader.CreateFromAssemblyFile(pluginPath, config => config.PreferSharedTypes = true))
(In this case I'm setting default behavior to prefer all, but I've also tried explicitly listing the interface with no luck).
Then I run this code to verify the interface I loaded is assignable:
               foreach (var pluginType in loader                     .LoadDefaultAssembly()                     .GetTypes()                     .Where(t => typeof(IPlugin).IsAssignableFrom(t) && !t.IsAbstract))
However the 'IsAssignableFrom(t)' is always false when I attempt to load a plugin where the IPlugin interface is contained in a slightly higher build than the host process.  The only difference in the IPlugin interface is that the host loads it from an assembly versioned <IP_ADDRESS> and the plugin includes it from an assembly versioned <IP_ADDRESS> (an intentional change on my part to repro this - no code changes at all between these versions).
I was under the impression this was what type sharing was designed to solve, but perhaps I am using it incorrectly or have wrong expectations?  How would I go about troubleshooting this?

I've hit the same issue recently, though I don't have a definitive test to say when this began happening.  In my case, I was lucky that it was an internal package that was setting the version higher and was able to lower it for the time-being.
That said, I agree and thought this was what preferred types was supposed to fix, by forcing the assembly using the higher version to unify with the host, but that was not working with any of my configurations.

same issue here.

I tracked that down to https://github.com/natemcmaster/DotNetCorePlugins/blob/main/src/Plugins/Loader/ManagedLoadContext.cs#L102 throwing an exception if the version assemblyName is higher than the version in the default load context.

I published a new package BTCPayServer.NETCore.Plugins.Mvc or BTCPayServer.NETCore.Plugins which is a fork of this project with the fix in the PR.


Add possessive quantifiers to avoid catastrophic backtracking

Fixes the crash in https://github.com/openai/tiktoken/issues/245 by prohibiting the regex engine from backtracking catastrophically via possessive quantifiers.

Interestingly these possesives make the encoding a lot faster again in fancy-regex.
Before this change (but with large byte pair merge PR cherry-picked):
num_threads: 1, num_bytes: 98379553
tiktoken 	11,946,036 bytes / s
tiktoken 	11,961,343 bytes / s
tiktoken 	11,995,846 bytes / s
tiktoken 	11,951,263 bytes / s
tiktoken 	11,983,405 bytes / s

Same, with these changes applied:
num_threads: 1, num_bytes: 98379553
tiktoken 	14,511,827 bytes / s
tiktoken 	14,638,134 bytes / s
tiktoken 	14,644,029 bytes / s
tiktoken 	14,729,030 bytes / s
tiktoken 	14,666,903 bytes / s

Updating the regex libs makes it a tiny bit faster still:
num_threads: 1, num_bytes: 98379553
tiktoken 	14,485,590 bytes / s
tiktoken 	14,854,049 bytes / s
tiktoken 	14,891,086 bytes / s
tiktoken 	14,843,007 bytes / s
tiktoken 	14,874,520 bytes / s

This is almost 2x faster than before any of the optimizations.

Opened an issue for increasing the default backtrack limit, see: https://github.com/fancy-regex/fancy-regex/issues/134, but it shouldn't be necessary here anymore.

Was the backtrack limit reverted intentionally in 05e66e8? https://github.com/openai/tiktoken/commit/05e66e8db7ef220d3c0b1aafbee5af289345684b#diff-b1a35a68f14e696205874893c07fd24fdb88882b47c23cc0e0c80a30c7d53759L421-R438
Was there a regression?

Yes, it was reverted intentionally. There are OpenAI internal encodings where setting the limit caused issues.

Thanks for checking @tmm1, @hauntsaninja.


feat(agora.io Video Conference): added Agora demo from agora.io sdk

added .eslintignore to ignore sdk file from agora.io.
Created AgoraDemo.js
Created /api/agora
Updated API index to include Agora example

#891

@YasharF I can add some instructions in the /api/agora.pug file. What should I include? I've made the other changes

I made an attempt to test this.  I may have missed something, but here is what I got:

When loading http://localhost:8080/api/agora I see the following two errors in the Chrome console:

A bad HTTP response code (404) was received when fetching the script.
Failed to load resource: net::ERR_INVALID_RESPONSE

If I grab an AppID from https://dashboard.agora.io/projects and put it in the filed on the page, and then click on Join it doesn't change the view and the following error shows up in the Chrome console:

19:37:17:762 Agora-SDK [ERROR]: Media access: {code: "Access to screen denied"}
A couple of other items:

The request for the AppID needs to be removed from the view.  A better place for that would be in the env.example.  That probably means that env variable needs to be passed from the API example controller to the pug template.
The instructions for obtaining a key needs to go in the readme.md like the other instructions.  The other instructions is a good example how they need to be formatted.


limit resolve and download to image and video

we are resolving non image/video claims - they may not be able to be viewed, but can be downloaded using the direct download button

I think this is fine/good.
If we want file-type based restrictions, this should be config-based.

This could be closed or renamed to "support configurations to limit file serve limits similar to file publish limits.


colums aligning left on mobile

Hi! I've been playing with MJML and find it super useful for my work, where I need to create music artists newsletter announcements.
I've created a first draft with a full background (in a full container wrapper), logo, two columns with text and image and button, then a list of dates.
On mobile though, the two colums with pictures get aligned left no matter what I try to fix it. I got it working somehow, once... but can't find out how (shouldn't have done it in the middle of the night...). I have the email test where it worked but even after trying to compare html I can't find the issue.
The list of dates sections are wonky too, and get broken when on mobile, all aligning left. It's got a title section and dates sections that are 33% each.
Here's the code... I'm hoping someone can help out, it's kinda driving me crazy :)
thanks! ed
somehow the code breaks on github so uploaded on pastebin
https://pastebin.com/nenu7Msf

Hey @edtrash, I just tried and the images look properly centered: https://mjml.io/try-it-live/HkpF0CUoW.
Can you maybe share a version of your template with hosted images? It would help to see the issues with the right assets (I had to guess you had a colored background-image with white texts 🤓).
If you're seeing those issues in a specific email client, please specify which (a screenshot would help too).
About your dates section, I suspect you're seeing this on iOS, you need to minify the output (mjml -m with the MJML CLI) like stated in the documentation.
PS: you have set attributes on closing tags, like line 100 with `</mj-column width="33%">
PPS: I love Petit Biscuit 😄

Hey there! Thanks for your very quick answer.
Is there any way to take it up directly with you through email?
The newsletter I'm building has confidential info on it and i'd like to not spill it out on the web :)
And maybe we could continue in French too ! If that's alright shoot me a note at<EMAIL_ADDRESS>The mobile email client I'm testing with is Android AquaMail.
I'll try minifying the output right now.
glad you like the music :) thanks !

My email is<EMAIL_ADDRESS>but debugging over email won't be easy, can you rather join the community MJML Slack? Then we can chat in private there: https://slack.mjml.io/


support find_first, find_last, position_first, and position_last

We currently have find_any and position_any. These are similar to the find and position methods on Iterator but they don't always find the first match. It'd be nice to support compatible versions. I would want to give them different names so as to discourage people from using them, since find_any will be more efficient.
Implementing this is similar to the existing find_any code but a bit trickier. The way that find_any works is that it makes a consumer with a full() method. full() returns true if execution should stop. We use a single shared atomic and, when a matching item is found, we set this flag to true, signaling others to halt.
To make this work for find_first, we need to halt not when any solution is found, but if a solution has been found in the sections to your left. The easiest way I can think to do this is to have each call to split_off, which breaks the consumer into two pieces, allocate an Arc<AtomicBool> (or something) for those two pieces to share. The left half will signal if it finds something by writing true; the right half will stop when either it finds something, or it sees that true has been written. The reduce method will then act just like it does now, preferring a value from the left half over the right half.
If you've not hacked on Rayon before, also check out the guide to development.

Instead of a tree of AtomicBool, what about a single AtomicUsize at the root that stores the smallest index found so far. Leaves only have one thing to read instead of log(n) things.

@edre hmm, so, I thought about it, and for some reason I thought it wasn't as good, but now that I think about it I suspect I was just wrong.
I was concerned about a tree with multiple levels:

0..100

0..50

0..25
25..50

51..100

51..75
75..100

and now we find a match at index 52 first. The main thing I wanted was that we could cut off the search at 75..100 without having to wait for 0..50. But I guess we can do that, no problem, we just set the current minimum to 52 -- this will cause 75..100 to stop, but not the rest.
OK, yes, seems better! =)

However, that only works for indexed iterators.

You could keep the same approach for unindexed iterators by generating a fake index. Set the next most significant byte on the right half at each split. Assumes splitting is balanced enough to not run out of bits in a usize.

0..100 index 0b0000

0..50 index 0b0000

0..25 index 0b0000
25..50 index 0b0010

51..100 index 0b1000

51..75 index 0b1000
75..100 index 0b1100

One possibility is to fake an index on the splits, something like this on 32-bit:

0x80000000 (0..100)
0x40000000 (0..50)
0x20000000 (0..25)
0x60000000 (25..50)
0xC0000000 (50..100)
0xA0000000 (50..75)
0xE0000000 (75..100)

Great minds think alike ;)

Clever. Use a u64, not a usize -- I guess past a certain point we can just max out anyway, right?

Do 32 bit platforms support AtomicU64?
If splitting is unbalanced, you can just stop changing the index when you run out of bits, and at the worst leaves close to each other cannot share progress.

Do 32 bit platforms support AtomicU64?

No, and that's not stable yet anyway.

Anyone mind if I self-assign this? Looks like a good item to work on for me to learn Rayon, and I have a start of an implementation.
Admittedly, this might be blocked on AtomicU64 becoming stable, if that's definitely what's desired for unindexed iterators.

@schuster go for it!  I think GitHub will only formally assign to project members, but consider it yours.
Just stick with AtomicUsize for now, especially if you can make sure it still works when out of bits anyway, as @edre suggested.

Okay, I have a basic (to-be-cleaned-up) implementation that assumes the iterator is unindexed, but it sounds like we want to use the actual index in the case of indexed iterators. How can I do that kind of specialization within Rust when the method is declared as part of the general ParallelIterator trait?
I've just started to learn about impl specialization - is that the way to go?

We can't use specialization until that's stabilized in Rust.  We do some "fake" specialization in collect to regain performance, but that's a bit fragile, not the kind of thing we should do a lot.
In this case, I think the only advantage of indexed iterators is that it would be easier to split perfectly.  Once you've written the unindexed way, it shouldn't make much difference.  I expect the logic of that unindexed mode to be a bit tricky, but the performance impact is probably not enough to be a concern.

Thanks, that works.
Also, I see three possible ways to write the find_first/find_last code, and I'm curious what might be preferred:

Write two completely separate modules, with some repeated code.
Pass around a "direction" value representing first/last, and branch based on it whenever the first or last code needs to work differently.
Use the polymorphism approach by implementing a trait on the direction value with methods for each bit of specialized code.

Option 1 of course repeats code (mostly boilerplate), while 2 and 3 each introduce different kinds of overhead. My hunch is that option 2 is the best in a Rust-based setting (because option 3 would have to jump to an arbitrary code pointer and probably impede various optimizations), but I'm still getting a good intuition for what overhead is acceptable or not in this kind of program. Thoughts?

Option 2 seems simplest -- I'd see how that goes.
Option 3 shouldn't result in function pointers unless you force it through trait objects.  Plain generic code should still monomorphize to statically-dispatched function calls, if not inlined entirely.
Generally speaking, try not to worry about fine-grained performance until you're sure it's a problem.  Write clean working code first, then see where the performance bottlenecks are.

Good advice, thanks.


[Q] How to clear the run immediately after finishing the experiment?

Problem: I'm working on a machine with limited disk memory. I'm starting hundreds of experiments overnight (due to many cross-validations), and sometimes the memory on the machine gets full because of storing wandb artifacts. When the memory becomes 100% full, all following runs get crashed.
Question: How to set wandb to clean the memory after a run (i.e., automatically call wandb sync --clean)? I don't need to keep the artifacts on the machine.

Hey Andrei, at the moment this is not possible. You will have to periodically delete the runs yourself or maybe use a cronjob. We are planning to add an argument to the wandb.finish function that will automatically do this. I will let you know once this is done.

I'm very much in the same boat as Andrei and would love to see this implemented!


Search polish

Cleans up a few minor issues / visual tweaks in search
Creator

[ ] Build relevant tests if any apply
[ ] Ensure there are no new warnings
[ ] Upload an animated GIF showcasing the solution (if it applies)
[ ] Set a relevant reviewer

Reviewer

[ ] Run test and make sure all tests pass
[ ] Review function and form on iOS
[ ] Review function and form on Android
[ ] Review function and form on Web
[ ] Review new test logic

BodyCopy and BodyText free

Cursor color (blueish on iOS) (Greenish Teal on Android)


[ODS-5077]Duplicate Check for both Core and Community extension to have same TPDM schema name -ODS/API Runtime


Ambient lighting from sky missing on macOS (in editor and in build)

Godot version
v4.0.stable.mono.official [92bee43ad]
System information
macOS Monterey, MBP 2020, Intel Iris Plus Graphics, Forward+ (so Vulkan? but this is macOS, so still OpenGL?)
Issue description
While testing GDQuest's Godot 4 macOS build I notice my lighting is different from their posted screenshot (missing ambient lighting in shaded area, results in complete blackness).

https://github.com/gdquest-demos/godot-4.0-new-features/issues/32
I searched around the closest reported issue appear to be this one, but since it was tagged for 3.x I am unsure if it is the same issue given rendering backend are different:
https://github.com/godotengine/godot/issues/55754
Further inspection at GDQuest source project, suggest the problem lies at ambient lighting from sky, no matter which sky material I use (procedural, physical or panorama), I can never get it to contribute to ambient lighting. Other forms of ambient lighting source appears to work fine, such as background or color mode.

I think this might have something to do with either the radiance map generated by Godot, or just the material shading itself wasn't using the radiance map properly.
I am new to Godot, please correct me if I was wrong about something. (I am guessing based on my experience with Unity and Unreal).
Steps to reproduce
2 ways:

Run build on Intel macOS from here: https://gdquest-demos.itch.io/godot-40-new-features

Download source from https://github.com/gdquest-demos/godot-4.0-new-features/ and open in Godot 4 stable

Not all scenes use sky as ambient lighting, but 3d_animation_tree_audio and 3d_physics_nodes scene noticeably suffer from this issue.
Minimal reproduction project
Sources are listed above.
It can also be trivially verified by adding an environment with sky as ambient lighting source to any new project.
This only appears to break with Forward+ renderer, Mobile renderer are not affected.

FWIW, I'm also seeing this same behavior on a 2020 MBP (Intel) on Ventura (v13.0.1) as well (not just Monterey).

2 additional info:

adding a Reflection Probe to the scene cause the Forward+ to render ambient color correctly again.
I also tried to capture gpu trace via Xcode and see that without Reflection Probe, the ambient_buffer texture is black.

Tested under Godot 4.0.2;

More comment on this, today I tried the same demo on an Apple certified Blackmagic eGPU (with Radeon Pro 580) and the ambient lighting renders correctly, so the problem is definitely with Intel GPU.
https://support.apple.com/en-us/HT208897

Forward+ (so Vulkan? but this is macOS, so still OpenGL?)

MoltenVK is used to translate Vulkan to Metal on macOS. That said, OpenGL is implemented with Metal on recent macOS versions (there is no native OpenGL driver anymore).


Add suffixes to diagnostics and other cleanup

Also a few related clean ups to diagnostics
Old look (from the log_diagnostics_example):

Old look hidden to avoid clutter

New look:

In particular, notice that the width of the diagnostics has been significantly reduced - within vscode the value no longer wraps on my 1920 width monitor. The value is still 105 columns wide, so there is room for improvement however.

Looks good; this is a nice simple QoL improvement.

I dig it!

bors r+


Update train.py

Hi, thanks for the PR, but I don't think the changes are correct. These flags should not be config files, they are strings. The config files are passed via the config_paths flag.


Failed to find assets path for "Frameworks/App.framework/flutter_assets"

Hi, I'm writing here to report a potential bug I've encountered building and running my app on iOS: it doesn't load any asset I provided. Assets are correctly defined, since on Android it builds ok. They get replaced with question marks or other icons as you can see in this screen:

The error in Xcode log is: Failed to find assets path for "flutter_assets".
I've tried both master and stable channel, getting the same error. Then, I switched to beta channel and now everything is ok (correct assets are loaded in the running app), so maybe it's a problem that has been solved in latest commits/releases.
Steps to Reproduce
flutter build ios --release
Logs
Failed to find assets path for "flutter_assets"
Engine run configuration was invalid.
Could not launch engine with configuration.
Flutter doctor and analyze where ok, sorry but now I don't have all the logs anymore since I switched the channel.

I have the same problem too

I have the same problem too

I have the same problem too

I have the same problem too

I think your 'Script' is Missing!
"$FLUTTER_ROOT/packages/flutter_tools/bin/xcode_backend.sh" build
"$FLUTTER_ROOT/packages/flutter_tools/bin/xcode_backend.sh" embed

Hi, could you please confirm if this problem still exists with the latest versions on stable or master?
Thanks a lot

Without additional information, we are unfortunately not sure how to resolve this issue. We are therefore reluctantly going to close this bug for now. Please don't hesitate to comment on the bug if you have any more information for us; we will reopen it right away!
Thanks for your contribution.

The issue still exists, Environment:
Doctor summary (to see all details, run flutter doctor -v):
[!] Flutter (Channel stable, v1.7.8+hotfix.4, on Mac OS X 10.15.1 19B88, locale zh-Hans-CN)
✗ Downloaded executables cannot execute on host.
See https://github.com/flutter/flutter/issues/6207 for more information
[✓] Android toolchain - develop for Android devices (Android SDK version 28.0.3)
[✓] Xcode - develop for iOS and macOS (Xcode 11.2)
[✓] iOS tools - develop for iOS devices
[✓] Android Studio (version 3.5)
[✓] VS Code (version 1.39.2)
[✓] Connected device (1 available)
! Doctor found issues in 1 category.

Flutter page failed to load
Failed to find assets path for "Frameworks/App.framework/flutter_assets"
[VERBOSE-2:engine.cc(118)] Engine run configuration was invalid.
[VERBOSE-2:FlutterEngine.mm(308)] Could not launch engine with configuration.

flutter pages failed to load, logs are below:
Failed to find assets path for "Frameworks/App.framework/flutter_assets"
[VERBOSE-2:engine.cc(118)] Engine run configuration was invalid.
[VERBOSE-2:FlutterEngine.mm(308)] Could not launch engine with configuration.

Hi @aberror
this issue is dated back 9 months,
please open a new issue proving all or some of the information
requested in the template,
thank you

Launching lib/main.dart on iPhone 11 Pro Max in debug mode...
Running Xcode build...
Xcode build done.                                           59.6s
Failed to find assets path for "Frameworks/App.framework/flutter_assets"
[VERBOSE-2:engine.cc(127)] Engine run configuration was invalid.
[VERBOSE-2:shell.cc(437)] Could not launch engine with configuration.
Debug service listening on ws://<IP_ADDRESS>:58867/XOWojV3-_ls=/ws
Syncing files to device iPhone 11 Pro Max...

1- rm -rf ios/Flutter/App.framework
2- Change the Runner target build settings Build Phases > Thin Binary script as follows:
/bin/sh "$FLUTTER_ROOT/packages/flutter_tools/bin/xcode_backend.sh" embed
/bin/sh "$FLUTTER_ROOT/packages/flutter_tools/bin/xcode_backend.sh" thin

I think your 'Script' is Missing!
"$FLUTTER_ROOT/packages/flutter_tools/bin/xcode_backend.sh" build
"$FLUTTER_ROOT/packages/flutter_tools/bin/xcode_backend.sh" embed

When I add that I get an error /bin/sh: /packages/flutter_tools/bin/xcode_backend.sh: No such file or directory this is when running in xcode, and I've cleaned

I think your 'Script' is Missing!
"$FLUTTER_ROOT/packages/flutter_tools/bin/xcode_backend.sh" build
"$FLUTTER_ROOT/packages/flutter_tools/bin/xcode_backend.sh" embed

where I should write that ?

1- rm -rf ios/Flutter/App.framework
2- Change the Runner target build settings Build Phases > Thin Binary script as follows:
/bin/sh "$FLUTTER_ROOT/packages/flutter_tools/bin/xcode_backend.sh" embed
/bin/sh "$FLUTTER_ROOT/packages/flutter_tools/bin/xcode_backend.sh" thin

work for me, thanks

create a new flutter project.
and then copy folder lib and file pubspace.yaml from old project to the new project.
I found many file, are missing on folder ios at an old project

I think your 'Script' is Missing!
"$FLUTTER_ROOT/packages/flutter_tools/bin/xcode_backend.sh" build
"$FLUTTER_ROOT/packages/flutter_tools/bin/xcode_backend.sh" embed

it's worked for me! thx

flutter upgrade

worked for me !

1- rm -rf ios/Flutter/App.framework
2- Change the Runner target build settings Build Phases > Thin Binary script as follows:
/bin/sh "$FLUTTER_ROOT/packages/flutter_tools/bin/xcode_backend.sh" embed
/bin/sh "$FLUTTER_ROOT/packages/flutter_tools/bin/xcode_backend.sh" thin

Thank you so much @dahabit you helped me a lot :)


Add IDR index data

Hello, I still have an issue when executing the following command
 python evodiff/conditional_generation_msa.py --model-type msa_oa_dm_maxsub --cond-task idr --num-seqs 1 --amlt
INDEX FILE LEN 10634
Traceback (most recent call last):
  File "/home/xxxxx/evodiff/evodiff/conditional_generation_msa.py", line 1065, in <module>
    main()
  File "/home/xxxxx/evodiff/evodiff/conditional_generation_msa.py", line 150, in main
    src, start_idx, end_idx, original_msa, num_sequences, b_src, b_start_idx, b_end_idx, oma_id = get_IDR_MSAs(index_file, data_top_dir,
  File "/home/xxxxx/evodiff/evodiff/conditional_generation_msa.py", line 826, in get_IDR_MSAs
    msa_data, new_start_idx, new_end_idx, num_sequences, b_start_idx, b_end_idx, oma_id = subsample_IDR_MSA(index_file, tokenizer, max_seq_len=max_seq_len, n_sequences=n_sequences,
  File "/home/xxxxx/evodiff/evodiff/conditional_generation_msa.py", line 882, in subsample_IDR_MSA
    all_files = os.listdir(data_dir + 'human_protein_alignments')
FileNotFoundError: [Errno 2] No such file or directory: 'data/human_idr_alignments/human_protein_alignments'

I added the --amlt flag so that it reads from the data directory coming with the repo. But the following lines suggest we are still looking for this human_idr_alignments/human_protein_alignments directory. Where can I download this data? Thank you.

You should be able to download the data associated with the paper from here:
https://github.com/alexxijielu/reverse_homology/
https://zenodo.org/records/5146063


Add RSO  (Statistical Rejection Sampling Improves Preference Optimization) Examples/Trainer https://arxiv.org/abs/2309.06657

DeepMind proposed a new method, "Statistical Rejection Sampling Optimization (RSO)" to align and learn from Human Preferences, which looks very interesting. https://arxiv.org/abs/2309.06657.
It seems like we can reuse a lot of components from DPO using the same loss, what do you think about adding the missing utils, e.g. conduct_rejection_sampling and then create an example for HHH.
Requirements are:

[ ]  Collect pairwise human preferences data, with each example containing a prompt, an accepted response, and a rejected response. -> HHH
[ ] Train a pairwise Reward Model on the preference dataset to predict which response is “accepted” for a given prompt. -> could be part of the exmaples
[ ]  Train an SFT Model on a different instruction dataset -> part of the examples
[ ] Apply RSO
- [ ] create new dataset -> new script
- [ ] train SFT again -> reuse script with different loss?

RSO implementation

Sample responses using the SFT Model and rank each response with the Reward Model
Use Statistical rejection sampling to conduct samples favoring higher rewards. Label the selected responses as preference pairs based on the relative reward.
Iteratively train the SFT Model using the new labeled preference pairs with logistic loss.

Iteratively train the SFT Model using the new labeled preference pairs with logistic loss.

This would be the exact same DPO implementation, right?

Thats my understanding as well.

@lvwerra please reopen


no files state for files

@keybase/react-hackers @MarcoPolo this adds the 'no files' state for the files page

:mag_right: :zap: These commits introduced some visual changes:

Added: Files-No_files_Public

Added: Files-No_files_Private

Changed: File-Normal_Public_No_Modified_meta (before) (after)

:mag_right: :zap: These commits introduced some visual changes:

Added: Files-No_files_Public

Added: Files-No_files_Private

👍 % C

replaced by https://github.com/keybase/client/pull/2931


[BEAM-10670] Cherry-pick: Use non-SDF based translation for Read by default on all runners except Dataflow

(cherry picked from commit 99b1a926decaf372e763ea409e1d42703dda59c8)
Please add a meaningful description for your change here

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[ ] Choose reviewer(s) and mention them in a comment (R: @username).
[ ] Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
[ ] Update CHANGES.md with noteworthy changes.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI.

Run Java PreCommit

Run Java PreCommit

Run Java PreCommit

It seems like the test is failing consistently. I'll look into it.

Run Java PreCommit

Run Java_Examples_Dataflow PreCommit

Run Java_Examples_Dataflow_Java11 PreCommit

@ihji Please take another look : )

LGTM. Thanks!


Auto discover Overkiz devices (Somfy TaHoma box, Connexoon)

Is your feature request related to a problem? Please describe.
To improve discoverability of this integration when it will be in core, it would be good to auto discover based on available Overkiz gateways in the network.
Describe the solution you'd like
Implement the autodiscover flow using ssdp, zeroconf or discovery.
Additional context
My OverKiz device uses the gateway id in the hostname and the MAC address for Overkiz starts with [F8811A](https://ouilookup.com/vendor/overkiz).
Hostname: gateway-xxxx-xxxx-xxxx
MAC address: F8:81:1A:xx:xx:xx

Do you need Somfy OUI equipement or help to code ?

I just found out that they broadcast _kixbox._tcp., which we could use perhaps.

Since we will support Rexel and CozyTouch soon under the TaHoma domain, it would be great to have autodiscover whem moving this integration to core.
If we can not do it based on the tcp, we can have a look at this in the future: https://github.com/home-assistant/core/pull/45087

Since we will support Rexel and CozyTouch soon under the TaHoma domain, it would be great to have autodiscover whem moving this integration to core.
If we can not do it based on the tcp, we can have a look at this in the future: https://github.com/home-assistant/core/pull/45087

Thanks! I believe best way would be to utilise the zeroconf and dhcp discovery possibilities of Home Assistant.


Text colorization in Test Steps works only for the first test in a session

When you run a test template, only the steps of the first test get colorized. The test steps of the rest tests do not parse the {spot} tag.
Same happens if the test is data driven - only the first execution parses the colorization correctly

Hi @adrianth, Happy New Year!
Did you have a chance to take a look at this issue?

Hi @TripleG, Happy New Year! I had identified the fix for this shortly after you posted the issue, but didn't get to publish a new release including that fix yet. It should be available in about one week.

Hi @TripleG, this issue is now fixed with beta version 1.3.8.


[Web] multiple lines of multi-byte characters in the textField will be hide

Steps to Reproduce

Enter multiple lines of multi-byte characters in the textField

Expected results:
All text will be shown
Actual results:
The bottom of the text is hidden.

`flutter doctor -v`
```
```
[√] Flutter (Channel beta, 1.23.0-18.1.pre, on Microsoft Windows [Version 10.0.18363.1171], locale ja-JP)
    • Flutter version 1.23.0-18.1.pre at C:\src\flutter
    • Framework revision 198df796aa (4 weeks ago), 2020-10-15 12:04:33 -0700
    • Engine revision 1d12d82d9c
    • Dart version 2.11.0 (build 2.11.0-213.1.beta)
[√] Android toolchain - develop for Android devices (Android SDK version 29.0.1)
• Android SDK at C:\Users\hatadev\AppData\Local\Android\sdk
• Platform android-30, build-tools 29.0.1
• Java binary at: C:\Program Files\Android\Android Studio1\jre\bin\java
• Java version OpenJDK Runtime Environment (build 1.8.0_212-release-1586-b04)
• All Android licenses accepted.
[√] Chrome - develop for the web
• Chrome at C:\Program Files (x86)\Google\Chrome\Application\chrome.exe
[√] Android Studio (version 3.6)
• Android Studio at C:\Program Files\Android\Android Studio1
• Flutter plugin version 45.1.1
• Dart plugin version 192.7761
• Java version OpenJDK Runtime Environment (build 1.8.0_212-release-1586-b04)
[√] VS Code (version 1.51.0)
• VS Code at C:\Users\hatadev\AppData\Local\Programs\Microsoft VS Code
• Flutter extension version 3.16.0
[√] Connected device (3 available)
• Web Server (web) • web-server • web-javascript • Flutter Tools
• Chrome (web)     • chrome     • web-javascript • Google Chrome 86.0.4240.183
• Edge (web)       • edge       • web-javascript • Microsoft Edge 86.0.622.63
</details>

@hatano0x06 Tried a series of emojis in a simple textfield and it shows me below:

If we use maxLines property and set to 4 and use same emojis again, it shows all of them properly, as below:

Which one from above correctly interpretes your issue ?

flutter doctor -v
[✓] Flutter (Channel beta, 1.23.0-18.1.pre, on Mac OS X 10.15.4 19E2269 x86_64,
    locale en-IN)
    • Flutter version 1.23.0-18.1.pre at /Users/dhs/documents/Fluttersdk/flutter
    • Framework revision 198df796aa (4 weeks ago), 2020-10-15 12:04:33 -0700
    • Engine revision 1d12d82d9c
    • Dart version 2.11.0 (build 2.11.0-213.1.beta)

[✓] Android toolchain - develop for Android devices (Android SDK version 30.0.0)
    • Android SDK at /Users/dhs/Library/Android/sdk
    • Platform android-30, build-tools 30.0.0
    • Java binary at: /Applications/Android
      Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build
      1.8.0_242-release-1644-b3-6222593)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 12.0.1)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 12.0.1, Build version 12A7300
    • CocoaPods version 1.9.3

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 4.0)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin version 46.0.2
    • Dart plugin version 193.7361
    • Java version OpenJDK Runtime Environment (build
      1.8.0_242-release-1644-b3-6222593)

[✓] VS Code (version 1.50.1)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.15.1

[✓] Connected device (2 available)
    • Web Server (web) • web-server • web-javascript • Flutter Tools
    • Chrome (web)     • chrome     • web-javascript • Google Chrome
      86.0.4240.193

• No issues found!

@darshankawar
I don't put any restrictions on the input, so I set it to infinite ( maxLines: null )
the problem happens when is maxLines is null

flutter doctor -v
[✓] Flutter (Channel master, 1.24.0-8.0.pre.149, on Mac OS X 10.15.4 19E2269
    darwin-x64, locale en-IN)
    • Flutter version 1.24.0-8.0.pre.149 at
      /Users/dhs/documents/Fluttersdk/flutter
    • Framework revision 8e7748e74c (2 days ago), 2020-11-08 20:32:04 -0800
    • Engine revision 0f7cdca65f
    • Dart version 2.12.0 (build 2.12.0-27.0.dev)

[✓] Android toolchain - develop for Android devices (Android SDK version 30.0.0)
    • Android SDK at /Users/dhs/Library/Android/sdk
    • Platform android-30, build-tools 30.0.0
    • Java binary at: /Applications/Android
      Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build
      1.8.0_242-release-1644-b3-6222593)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 12.0.1)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 12.0.1, Build version 12A7300
    • CocoaPods version 1.9.3

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 4.0)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin version 46.0.2
    • Dart plugin version 193.7361
    • Java version OpenJDK Runtime Environment (build
      1.8.0_242-release-1644-b3-6222593)

[✓] VS Code (version 1.50.1)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.15.1

[✓] Connected device (4 available)
    • SM A260G (mobile) • 5200763ebcfa861f • android-arm    • Android 8.1.0 (API
      27)
    • macOS (desktop)   • macos            • darwin-x64     • Mac OS X 10.15.4
      19E2269 darwin-x64
    • Web Server (web)  • web-server       • web-javascript • Flutter Tools
    • Chrome (web)      • chrome           • web-javascript • Google Chrome
      86.0.4240.193

• No issues found!

Duplicate of https://github.com/flutter/flutter/issues/64792


Fix #2 remove currentContext

remove currentContext
not optimal but it's works with new 0.14 version
if it merged can you publish a new version ?

I'm ok with the $transformEvent i would have done that too.


Implement physics support in the GLTF module

Implements and closes https://github.com/godotengine/godot-proposals/issues/5268
This PR implements physics support in the GLTF module. This allows importing physics objects from a GLTF file using the OMI_collider and OMI_physics_body GLTF extensions. https://github.com/omigroup/gltf-extensions/pull/127 and https://github.com/omigroup/gltf-extensions/pull/125
The code is designed in a modular and carefully abstracted way. There are helper classes GLTFCollider and GLTFPhysicsBody that serve as an intermediary between the imported OMI_* physics data and Godot's nodes.
The *_dictionary methods handle converting to/from the JSON Dictionary data that gets stored in the GLTF files, and the *_node methods handle converting to/from Godot nodes. GLTFDocumentExtensionPhysics is the stateless master class that manages the other two helper classes during the import and export processes.
OMI_collider is designed to be able to work independently of OMI_physics_body to allow for simpler files. This PR handles this case. If a collider is by itself, we create two nodes, a body (StaticBody3D or Area3D, depending on the collider's isTrigger flag), and a CollisionShape3D child.
Both importing and exporting GLTF files with physics is fully supported. GLTF -> Godot -> GLTF round-trip conversions should keep everything functional (even if the data is slightly different, such as Godot adding a physics body parent to GLTF files with standalone colliders). Godot -> GLTF -> Godot round-trip conversions should also keep everything functional for the data present in the standard (GLTF OMI physics won't keep track of gravity scale etc).
One implementation note:

While I was in the process of making this PR, MSFT_physics appeared as a competing standard to OMI's physics extensions. Due to the abstractions I mentioned earlier, if MSFT_physics ends up taking off as a popular standard, it should be fairly simple to modify this code to allow reading (and maybe writing) Microsoft physics by modifying the *_dictionary methods. For now, this PR only implements reading and writing OMI physics data.

Here is a test project that tests most things listed above: Godot_GLTF_OMI_physics.zip
Question: Godot 4.0, or Godot 4.1? On one hand, Godot 4.0 is supposed to be in feature freeze, so delaying to 4.1 makes sense. However, this is a low-risk PR, it should have zero impact on anyone not using GLTF physics. I am also interested in backporting this to Godot 3.x to help with using GLTF as an independent scene format interchangeable between different Godot versions (and also interchangeable between other engines).

Hey :wave: (original proposal author here). Sorry for delay reviewing this.
I'll start with thoughts before digging in the code:

I'm not very familiar with glTF, and took a look at OMI_physics. Looks interesting, however as an end-user, how would this be supported? Currently, we have introp with general modeling tools (like Blender) by using name prefixes (which makes it easy to use, but is kind of a hack). Would tools like Blender need to support colliders, or would a separate tool be required to create them?

Sub-point: I see that Blender does support some extensions (https://docs.blender.org/manual/en/latest/addons/import_export/scene_gltf2.html#extensions), however none are currently for physics
I also see that Blender does have some 3rd-party collision tools such as https://blendermarket.com/products/collider-tools

I can't find any details on MSFT_physics (search engines don't return any results). Could you provide a link? Thanks :smile:
For satisfying my proposal, I think this does a good job! Love seeming progress on this.

I think this is an important step towards user-generated model import, hopefully this goes into the engine! Thanks a lot for working on this.

@Cretezy Thanks! I've updated the OP with a clickable link to MSFT_physics.

Would tools like Blender need to support colliders, or a separate tool be required to create them, or would they possibly need to be made in Godot since it's the only one supported currently?

For now, you can use Godot to create them. It has also been implemented in Third Room. Eventually it would be great to have Blender support it, it does have physics primitives available, but I am a Blender noob so unfortunately I don't have more details of how this will work in Blender.

Code tested :+1:
I was also able to add one of my basic glTF models (a path for a racing game), generate the static body + collider, export it, and re-import it succesfully.

Here's the model for reference: plane.zip
While trying out a more complex scene, figured initial velocity and physics properties (bounce, mass) were missing, but that seems to be missing as part of the OMI_physics_body spec. It does seem that the
MSFT_physics spec has it though. This is more of a side-note again, as I think this is already great progress, and enables 3d user-generated (static) maps in Godot!

@Cretezy Mass is supported, but more complex physics information should probably be left to another extension. At some point in the future we could see OMI_physics_material start to exist.

Some things don't match in your opening post.

(GLTF OMI physics won't keep track of inertia or angular velocity etc).

Can you revise?

What's your take on merging as any changes will be less easy?

I think it would be good to merge sometime soon. If we have to make breaking changes, there is still time to do this before Godot 4.0 is out. But we may want to first merge the PRs on OMI's end for the standards.
I think the main change remaining that was discussed is that I did not implement inertia tensors either in this PR or in the standard. I'm not familiar with this so I'd prefer if somebody else implements this. One option is to merge in the PRs without inertia tensors and then other people can make PRs that add it. A merged PR is not feature frozen.

I'm pretty happy with it. there will continue to be small changes here or there but overall it seems pretty solid.
If you look at browsers for example, common practice for these sorts of spec implementation issues is to make it an opt-in feature flag as long as the spec is in draft.
So we should make a ProjectSettings property (needs to be synced to everyone working on the project) to allow glTF import and export. This would allow us to add it to 4.0 with very low risk.

I don't think it makes sense to have as a feature flag. It's only used if people try to import/export physics with GLTF.

See the argument against disabling by default https://github.com/godotengine/godot/pull/59675#issuecomment-1349970524


Incompatibility with VRHandsFrame

VRHandsFrame crashes when the wrist overlay is active.
VRCX overlay doesn't start when VRHF is active.

working fine here for me interesting.

Might be something else, then? I'm also using Desktop+ but it seems to be working fine with either individually. SRanipal is also running most of the time but I haven't noticed any problem yet.
Platform: SteamVR (Vive Pro Eye)
Games tested: VRChat, SteamVR Home
Steps:
→Start SteamVR with VRCX (VRCX is working correctly), then start VRHandsFrame (crashes automatically)
→Start SteamVR without VRCX, then start VRHandsFrame (works correctly), then start VRCX with VRChat (overlay doesn't seem to work)
I'll try starting it with VRC again instead of SteamVR, just to be sure.

Mmmmm. It seems to be working fine without Desktop+, which is weird.


tbb fitter, adaptive seeds per task, and some miscellany

Benchmarks are in phiphi:/home/dsr/mictest-mt/2016-06-20-tbb-fitter
This converts runFittingTestPlex() to TBB threading; this does actually result in some modest scalability improvements, particularly at high thread counts.  I also discovered that SlurpInTracksAndHits() is only safe for full matriplexes, so this version calls InputTracksAndHits() instead for partial matriplexes, replacing a min() operation.  Also pulled the Pool struct out into a separate header so that the fitter could use it, and moved the initialization of the ExecutionContext that holds fitters and cloners for TBB out of the constructor.  In the constructor, it was being called before the options had been processed, and so was initializing for the wrong thread count, and always creating one CandCloner thread.  Initialization now follows the same pattern as the OMP versions.
Added an adaptive calculation for seeds per task, which calculates a stride based on the number of seeds in the current bin and the thread count, using that value if it is smaller than the configured stride.  This substantially improves the performance of the TBB builder for the CMSSW tests with high thread counts.
The fitter now always calls OutputFittedTracks(), and make_validation_tree() has been modified to print a count of tracks better than 80% and the mean and standard deviation of the pulls, based on code Matthieu wrote for the GPU version.  Also updated make_validation_tree() to be independent of coordinate system.
maxCandsPerEtaBin was being overrun for small track counts (hence larger variance).  A safety factor was added that increases the value the most for small track counts.
hi.zbin was also being overrun, likely due to smearing pushing some hits off the end of the detector.  Added a check for this.  This change resulted in a very small improvement in the hits found, and changed the platform and vector-length differences, so running off the end of that vector was one contributor to those differences.
benchmark-mic-cmssw.sh wasn't found, benchmark-cmssw-mic.sh was, so updated runBenchmark.sh, @cerati should check this is correct.  Also some other miscellaneous bug fixes for Debug and cartesian coords that aren't tested in the benchmarks, and a correction to a mistaken change I made to the time reporting (does not affect benchmarks).

plots copied at:
http://uaf-6.t2.ucsd.edu/~cerati/mictest/2016-06-20-PR56/

@dan131riley yes you are correct about benchmark-cmssw-mic.sh (I ran it by hand, and later added the buggy line to the main script)


Add new project based on HAP-NodeJS to README

Hey there,
From some time we are working on the node-red-contrib-homekit fork which last update is like year ago and a contact to project owner is lost.
We got our fork here:
https://github.com/oliverrahner/node-red-contrib-homekit-bridged
Please add it to README so more people know us :)
Thanks!

+1 I recently (last week) found this and I'm using it with the node-red openhab2 plugin and it's amazing!

@sjorge thanks :)
We are still working to improve it.

@sjorge If you like my work then please rate module on node-red site:
https://flows.nodered.org/node/node-red-contrib-homekit-bridged


Proposal: add option to skip loading children images

I have a situation where I want to load a background image on a parent div - and wait for it to load - before I make the page usable.
I've commented out lines 255-275 and this is solving the specific issue I'm encountering, so I thought I'd propose adding an ignoreChildren or similar option to do the same thing.

Thanks for reporting this issue. Have you tried using a child selector ( background: item ) so that the other img tags aren't being selected?


Norwegian Translation

Translation for Norwegian

Have you tested with the translation and everything works well?

Have not tested it yet. I am not at home for the moment and i dont have a Google assistant to test in on.
Will be back home next weekend. Do you want me to test it before you merge it?

Yes, please, just in case, I wish I could test it myself, but my multi language skills are limited to programming, unfortunately.

If i want to test it on my LG tv. I need to open Plex app on my tv and then ask Google to for example play "Breaking bad" ? Or is it possible to get my tv to also open the Plex app aswell?

"Plex clients must be open in order to be detected or receive commands from this component"
If you are able to open the plex app somehow by using an automation in Home Assistant you could add that as part of the intent script, but Plex Assistant doesn't do that for you. Plex itself can only see clients when they are open.
Have you not setup and used Plex Assistant yet?

I have Alexa in my house. But have just found out that my Sonos speaker have Google Assistant. So i was searching around for Plex things and Home Assistant. Then i found your integration and wanted to test it. Translation is so that my kids also can use it 😊

What is missing here? I had forgotten about this :)

I've added this into the next major version to be released soon.

Great! 👍
tor. 11. feb. 2021 kl. 16:50 skrev Ryan Meek<EMAIL_ADDRESS>
I've added this into the next major version to be released soon.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/maykar/plex_assistant/pull/54#issuecomment-777593812,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AEJHDA4CULJWSCTV3ZGMUT3S6P4EXANCNFSM4SAI3N7Q
.

--
Mvh
Henning Wikene