Solution for dealing with clocks and time in embedded systems

I have just released embedded-time to handle clocks, instants, durations more easily in embedded contexts. It follows a similar idea to the C++ std::chrono library. I'd love to get some feedback, suggestions, PRs, etc.
embedded-time provides a comprehensive library for implementing abstractions over hardware and work with clocks, instants, durations, periods, and frequencies in a more intuitive way.

Clock trait allowing abstraction of hardware timers for timekeeping.
Work with time using milliseconds, seconds, hertz, etc. rather than cycles or ticks.
Includes example for the nRF52_DK board
Conversion to/from core::time::Duration

Example Usage:
struct SomeClock;

impl Clock for SomeClock {
    type Rep = i64;

    fn now() -> Instant<Self> {
        // read the count of the clock
        // ...
        Instant::new(count as Self::Rep)
    }
}

impl Period for SomeClock {
    // this clock is counting at 16 MHz
    const PERIOD: Period = Period::raw(1, 16_000_000);
}

// read from a Clock
let instant1 = SomeClock::now();

// ... some time passes

let instant2 = SomeClock::now();
assert!(instant1 < instant2);    // instant1 is *before* instant2

// duration is the difference between the instances
let duration: Option<Microseconds<i64>> = instant2.duration_since(&instant1);

// add some duration to an instant
let future_instant = instant2 + Milliseconds(23);
// or
let future_instant = instant2 + 23.milliseconds();

assert(future_instant > instant2);

Related:
#122 #207 #24 #46 #201 #129 #103 #59 #186

I see only duration types. What about frequencies and baudrates?

I see only duration types. What about frequencies and baudrates?

@burrbull There is a Period type that is an i32 rational::Ratio. I use it in the durations to specify the "magnitude" of the value. Frequencies or data/baud rates could be specified using this Period type. Period also has a method to get the reciprocal which may be helpful. If you can help me with some use-cases, it may be something that can be added to embedded-time as a "first-class" semantic.

@burrbull Initial frequency-type functionality has been added in v0.5.0

Thank you for your work very much.

Looks excellent. One thing that could be useful is a bridge to core::time, which is only Duration at the moment.

@therealprof While I don't think that core::time is particularly usable in embedded systems, I do recognize the benefit of having that bridge when it's needed. I'll add it to my list, and thank you very much for the suggestion.

@therealprof Added with v0.5.2

I'm curious whether it would be appropriate to using embedded-time in the embedded-hal crate or if that would make it too non-general. If it wouldn't be appropriate, I would love any feedback in ways I can make embedded-time usage with embedded-hal more seamless.

Looks good to me in general, appreciate the effort. I'll need to find some time to check it out in detail though.

This looks really great - are there any plans to slowly integrate this into embedded-hal through PRs? I don't think this will see widespread adoption otherwise since most of the downstream embedded crates are already using their own time primitives etc.

embedded-time is trying to solve a different problem than embedded-hal so I don't think integration here makes too much sense. What we can do is encourage HAL impls to use embedded-time rather than rolling their own types and this is probably best done by someone doing PRs to do the switcheroo. I happen to be the maintainer of a few HAL impls so if anyone is looking for crates which could serve as the first stepping stone to general adoption feel free to use do PRs against https://github.com/stm32-rs/stm32f0xx-hal or https://github.com/stm32-rs/stm32f4xx-hal and I will help driving that to completion.

@therealprof Thanks for clearing up the embedded-hal question, I wasn't at all sure whether it would be a good fit there. I will do whatever I can do to support PRs for individual HAL impls including opening them up myself unless I have updates needed in the embedded-time crate itself.

After some feedback from a HAL implementor, I created the this PR (FluenTech/embedded-time#22) which would change all the "inner" types to unsigned. I would love to get some feedback about this change before I merge it.

@therealprof I am a little confused, actually, about what you said about embedded-hal. I'm still trying to wrap my head around what the objective of embedded-hal is. Device-specific HALs impl embedded-hal so that other code can be written abstracted from the actual device being used. This could be applications, or a device driver for some sensor, external memory, etc. The goal (in my opinion) would be to provide complete, generic access the main functionality. Obviously, there will be very device-specific details that would be impractical to include. I think that for applications, device drivers, etc to work time (clocks, timers, durations, instants, frequency, etc) in a generic way, embedded-hal would be the place it needs to reside. Perhaps, I misunderstand the purpose of embedded-hal?

The purpose of embedded-hal is to facilitate the interfacing between hardware and drivers/applications. As you've said yourself embedded-time is mostly hardware independent.

I think that for device drivers to work with time-related features (clocks, timers, durations, instants, frequency, etc) in a generic way, embedded-hal would be the place it needs to reside.

I don't see it why it needs to be an integral part of embedded-hal instead of a dependency. But requirements and expectations can change...

I don't see it why it needs to be an integral part of embedded-hal instead of a dependency. But requirements and expectations can change...

Sorry, I wasn't commenting on where the code should be, but rather saying that (whether as a dependency or direct integration), it seems to me that embedded-hal is where it should be "used".

The purpose of embedded-hal is to facilitate the interfacing between hardware and drivers/applications. As you've said yourself embedded-time is mostly hardware independent.

Would it make sense to split it up? The only hardware-abstraction part of it is the Clock trait. However, it does depend on the TimeInt and Duration traits as well as Instant and Period types -- so basically everything. Being relatively new to Rust, I can't visualize what that would take.

Would it make sense to split it up? The only hardware-abstraction part of it is the Clock trait. However, it does depend on the TimeInt and Duration traits as well as Instant and Period types -- so basically everything.

I'd wait and see where this goes. I can totally see this being useful for a lot of applications, if it gets picked up by other crates and turns out to be useful I'd have no regrets integrating it more deeply.
At the moment I see most of the usefulness in the HAL impls themselves with a hint of use in the timer related stuff in e-h.

I guess as long as embedded-hal exposes the interfaces necessary to impl the Clock trait, drivers could still utilize embedded-time. However, I'm fairly certain that it currently does not.

We could also do it the other way around and provide all the fun clock, timer and countdown traits via embedded-time and phase it out from embedded-hal. They're only borderline useful as-is and could be much more powerful with an proper time calculations, basically we're only using simple units (plus some copy and paste simple conversion in the HAL impls).
Also it would be great if we could have something like a monotonic system clock, real time clock, alarms, scheduling...

Also it would be great if we could have something like a monotonic system clock, real time clock, alarms, scheduling...

As things sit right now, the Clock trait can be implemented for any monotonic source. That could be a peripheral timer, RTC, etc. I have an implementation for the nrf52 with two of the 32-bit peripheral timers chained together and that is my "system clock". I've also done an example using the RTC.
The next release will probably be with Timers added and I'm hoping to be able to use those to schedule tasks executed by the Clock impl's interrupt.
@therealprof I'm certain you have spent much more time thinking about where embedded rust is right now, where we want to go, and how to get there, so I would greatly appreciate any guidance.

I dont think embedded time is trying to solve a different problem at all - Just look at all the issues noted in the original post mentioning the need for some crucial clock traits like periods and frequencies, and to tie them to the std lib. Could the traits and Period struct at least be moved into embedded-hal to ensure comparability? I don't think it's wise to expect the ecosystem to depend on a 3rd party crate to ensure cross-ecosystem compatibility for timings - that should be something that embedded-hal does.

Hey thanks for working on this! There's lots of useful stuff here, and how / what we integrate is an interesting question.
I think i'd echo @therealprof in that for now I would continue this as a separate project, and to try out integrating it with HALs and other components. This gives us space to experiment a bit (as we require for new hal additions) and means we're not tied to the HAL for releases (you may have noticed there's a lot going on at the moment).
To me it seems that the possible steps from there are:
a) we could review and bless embedded-time, and take over ownership (and maintenance) of the repo / crate (so it's separate but, first party).
b) we could look to integrate the validated traits with the hal directly
c) we could do a bit of both and move the types directly applicable to hardware in but keep the less-general ones separate.
I think the key component delineation to me is whether this is intended to be std compatible and how they can be implemented:

if it were to provide an abstraction over std or no_std then it would be better in the separate library (ie. using features and type aliasing, Instant::now() if we elected to provide this)
if it is directly implementable on hardware (or used in this, for example Timer::start(), Timer::now()) it should be in the hal
and if it's implementable using hardware components, but does not reflect physical hardware (for example, Timer::instant() might use but not reflect a hardware primitive), it's a bit ambiguous but i would lean towards having it in the library

However I can also see the benefits to going in either direction.
In terms of moving traits into the hal, it is reasonably straightforward to add a trait and then import and re-export that from the original crate, so we can move traits into the hal without breaking any external dependencies.

I just release v0.6.0 of embedded-time. With feedback and advice from @eldruin and @TheZoq2. I made a number of changes.
Added

Fallible Clock implementations

Timers

Software timers spawned from a Clock impl object.
One-shot or periodic/continuous
Blocking delay
Poll for expiration
Read elapsed/remaining duration

Changed

Switched to unsigned inner integer types
Changed the Clock trait to allow stateful implementations (added &self) to methods

I think i'd echo @therealprof in that for now I would continue this as a separate project, and to try out integrating it with HALs and other components. This gives us space to experiment a bit (as we require for new hal additions) and means we're not tied to the HAL for releases (you may have noticed there's a lot going on at the moment).

I like that idea. I think embedded-hal (as well as embedded-time) are both still pretty fluid. It's my intention to continue development in a way that serves my purposes, but also keeps an eye toward maybe being suitable for incorporation at some later date.

I think the key component delineation to me is whether this is intended to be std compatible and how they can be implemented:

if it were to provide an abstraction over std or no_std then it would be better in the separate library (ie. using features and type aliasing, Instant::now() if we elected to provide this)
if it is directly implementable on hardware (or used in this, for example Timer::start(), Timer::now()) it should be in the hal
and if it's implementable using hardware components, but does not reflect physical hardware (for example, Timer::instant() might use but not reflect a hardware primitive), it's a bit ambiguous but i would lean towards having it in the library

There is only one part that must implemented in a hardware-specific manner and that's the Clock trait. Everything else is hardware-agnostic.
Here's a rough snapshot of some of the clock and instant interfaces:
struct ClockImpl {
    <hardware-specific state goes here>
}
impl embedded_time::Clock for ClockImpl {
    fn now(&self) -> Instant {
        <reading of hardware goes here>
    }
}

let my_clock = ClockImpl
let instant_1 = my_clock.now()
let instant_2 = instant_1 + 10.seconds()
let some_timer = my_clock.new_timer(2.seconds()).into_oneshot().start()
...
let remaining = some_timer.remaining()
let elapsed = some_timer.elapsed()
some_timer.wait()    // blocks until expiration

let elapsed_time = my_clock.duration_since(instant_1)
let dur_until_instant_2 = my_clock.duration_until(instant_2)

And here is an actual Clock implementation from the example:
pub struct SysClock {
    low: nrf52::pac::TIMER0,
    high: nrf52::pac::TIMER1,
    capture_task: nrf52::pac::EGU0,
}
impl SysClock {
    pub fn take(
        low: nrf52::pac::TIMER0,
        high: nrf52::pac::TIMER1,
        capture_task: nrf52::pac::EGU0,
    ) -> Self {
        Self {
            low,
            high,
            capture_task,
        }
    }
}
impl time::Clock for SysClock {
    type Rep = u64;
    const PERIOD: time::Period = <time::Period>::new(1, 16_000_000);
    type ImplError = Infallible;

    fn now(&self) -> Result<time::Instant<Self>, time::clock::Error<Self::ImplError>> {
        self.capture_task.tasks_trigger[0].write(|write| unsafe { write.bits(1) });

        let ticks =
            self.low.cc[0].read().bits() as u64 | ((self.high.cc[0].read().bits() as u64) << 32);

        Ok(time::Instant::new(ticks as Self::Rep))
    }
}

time::Clock looks super useful hey, are there any soundness issues with reading from the two timers sequentially? (ie. what happens if high rolls over while you're reading the low section)?

@ryankurte, Thanks for taking a look at the crate. There are a lot of changes happening at the moment.
I assume your question refers to the nrf52_dk example. With this chip, I can trigger a signal from software that causes both timers to capture atomically, then I just read the captured values out of each.

ahh nice, it looks like a similar approach is possible on ST cores but, might require checking for overflow.

Some new releases of embedded-time
0.9.1 - 2020-08-07
Changed

Re-export Fraction type in the duration and rate module

Allows for a single import (eg use duration::*) to support Generic usage which uses Fraction

Unify readme's and crate-level documentation

The GitHub README.md and crates.io crates-io.md are now generated using cargo-readme

0.9.0 - 2020-08-05
Added

From implementations for infallible intra-rate and intra-duration conversions
Implementations of common traits to types
More frequency types: mebihertz (MiHz) and kibihertz(KiHz)
More test coverage

Changed

Fallible intra-rate and intra-duration conversions use try_from()/try_into() rather than the try_convert_ methods
Replaced try_into_generic() with to_generic()
Replaced try_from_(duration/rate) with to_duration() and to_rate()
Return Options from checked_ methods rather than Results
Remove impl-specific Clock error type (ImplError)

Removed

trait module

duration- and/or rate-specific functionality can be imported separately by using duration::* and rate::*

I just started with embedded development in rust and I want to say that I am in line with @sourcebox's points above.
I am trying to write a simple embedded-hal driver on the ESP32C3. I need to measure how long a GPIO pin stays high. For that I do not need a sophisticated timer: access to a monotonic timer (tick count + frequency) is sufficient. It would be especially interesting on the ESP32C3 where there are only 2 general purpose timers. With access to the system clock I would not need to consume a timer.
For what it's worth, I think basic time access functionalities should be intergrated in embedded-hal. Having them in a separate crate just makes life more difficult for the implementer of hardware-specific HAL that would have to track and implement traits from multiple crates. And if they don't, the ecosystem will fragment itself with partial HAL implementations that do not deliver the expected hardware abstraction.
I see that this discussion started years ago. Is there still life in it? How does it move forward?

I never wanted to write my own crate for dealing with time, but to have a solution for my own projects until something "official" is released:
https://github.com/sourcebox/emtick-rs
I don't know if the general concept behind it is good enough as solution within e-h, maybe it could be discussed.
And yes: the code for the conversions between ticks/ms/µs looks scary and expensive, but at least in my own testing, the compiler did a decent job of optimizing it.

Hi, @sourcebox , what are the main differences between your implementation and fugit? I saw that yours does not have any dependency, and that is always desirable (I believe). However, it would be great to have a performance comparative. Also, memory footprint should be studied.
I agree with you, we just need a decent toolbox for dealing with time and timers in embedded systems. It would be great if e-h adopts/integrates either fugit or your new crate. The main drawback of your solution (I think) is that several HALs already use fugit, and maybe it is worth it to adopt it in e-h as well. I'll take a look to your code later with more time.

As far as I understand fugit, it tries to eleminate runtime cost completely. This however leads to the fact that you have to deal with types like Duration::<u32, 1, 1_000> and pass them around, which I'm personally not a big fan of.
My solution is a more pragmatic one. It should be easier to use but has some runtime overhead when converting from/to natural time units. So doing this should be kept minimal. The amount of impact on overall performance is hard to decide. In my code, time calculations are typically used rarely compared to the overall coverage. Optimizing this to the max would not have too much effect globally. But this may vary depending on the use case.
I would suggest that you do a real world example using fugit, my crate, maybe embedded-time etc. and do some comparision in terms of ease-of-use and performance.

For what it's worth, I think basic time access functionalities should be intergrated in embedded-hal

@sourcebox 100%. I would add that the more advanced functionalities can be incrementally added as the community asks for them. Starting with a trait with an API that sources a tick-count and mapping to seconds (I say seconds because it's the standard unit of time, but ideally, this would be generic between seconds, us, ms, etc). The more advanced stuff can be added down the line.

Quoting @dirbaio in the chat for a comparison of embedded-time vs. fugit:

there's definitely interest indeed, the problem is it's very unclear how to do it. Decisions that must be done is:

Bit width. Hardcoded, configurable via Cargo features, configurable via generics?
Tick rate: Hardcoded, configurable via Cargo features, configurable via generics? Const generics? typenum? Just a frequency value? or a full NUM/DENOM fraction?
Who gets to choose these settings? the HAL implementing the trait, or the driver using the trait?
Single global clock, or multiple clocks? If multiple clocks, can they have different settings?

Example 1: embassy-time:

bit width: hardcoded to u64, so it never overflows. (prioritizing convenience over efficiency)
tick rate: configurable via cargo features. Some HALs choose it, some let the end user choose.
Single global clock, so you can do Instant::now() from anywhere without having to pass around stuff.

Example 2: fugit

bit width: configurable with generics
tick rate: full NUM/DENOM fraction, configurable with const generics.
Who chooses? the HAL implementation (I think the HAL can choose to be generic so the end user chooses? but either way the driver can't choose)
Multiple clocks, each can have its own settings.

as you can see they're polar opposites.
So, adding some Clock/Instant/Duration traits to embedded-hal means we have to make SOME choice on these questions and if we choose X then use cases that would be better suited by Y would suffer

I've put together an RFC in the wg repo: https://github.com/rust-embedded/wg/pull/762
I believe I've addressed most of the points:

Bit width. That is to be determined by the implementor. I am of the opinion that the value being read should be entirely up to the implementor, so long as it encapsulates a type that is suitable for step-counting. This could be u8, Wrapper(u8), u128, SomeTypeThatCounts, and so on
Tick rate: again, implementor dependant. embedded-hal makes no assumptions about what an InputPin actually is. for all it matters, it could just be a struct that just randomly chooses high or low. What matters is that a) the trait encapsulates the act of reading something that is asumed to be increasing one at a time and b) a specification of what is being counted, be it a measurement of time, distance moved (e.g. with ab-encoders), etc
Who gets to choose settings? Same as with the I2c trait. The implementor.
Single or global clock? Again, implementor.


feat: add function to get Basic Presence

resolves #99

Looks good. I think the only things that are left are:

Be sure checks are passing (Prettier / ESLint).
Update the docs site to include a page for the new endpoint.
Update the README.md file to link to that page.

@all-contributors please add @JenCrawford for code and documentation

@JenCrawford This has been released. Thanks for your contribution!


version upgrade for feathers v4

Do you guys plan to release a version which is compatible with feathers v4?

What exactly broke from v3 to v4?

feathers-hooks-common is using his own feathersjs v3 from internal node_modules. And this version has different code than v4. For example it checks for SKIP in hooks.

softDelete2 is broken due to SKIP


Given JSON input is too large when using "cudf" backend

Describe the bug
I'm trying to load some data from jsonl files using the cudf backend but I get an error of  "ValueError: Metadata inference failed in read_single_partition." and "RuntimeError('CUDF failure at: /__w/cudf/cudf/cpp/src/io/json/nested_json_gpu.cu:86: Given JSON input is too large')".
Steps/Code to reproduce bug
files = get_all_files_paths_under("/path/data") # ['/path/data/file1.jsonl', ... , /path/data/fileN.jsonl']
meta = {
    'field1': np.dtype('int64'),
    'fieldN': np.dtype('O')
}

dataset = DocumentDataset.read_json(input_files=files, backend="cudf", add_filename=True, input_meta=meta)

Expected behavior
I expect the data to load as well as when using pandas backend but "faster" and making use of GPUs parallelization.
Environment overview (please complete the following information)

Environment location: enroot + pyxis on slurm cluster
Method of NeMo-Curator install: NeMo Framework docker image, 24.05llama3.1, NeMo Curator 0.4.0

Additional logs
ValueError: Metadata inference failed in `read_single_partition`.

You have supplied a custom function and Dask is unable to
determine the type of output that that function returns.

To resolve this please provide a meta= keyword.
The docstring of the Dask function you ran should have more information.

Original error is below:
------------------------
RuntimeError('CUDF failure at: /__w/cudf/cudf/cpp/src/io/json/nested_json_gpu.cu:86: Given JSON input is too large')

Traceback:
---------
  File "/usr/local/lib/python3.10/dist-packages/dask/dataframe/utils.py", line 195, in raise_on_meta_error
    yield
  File "/usr/local/lib/python3.10/dist-packages/dask/dataframe/core.py", line 7175, in _emulate
    return func(*_extract_meta(args, True), **_extract_meta(kwargs, True))
  File "/usr/local/lib/python3.10/dist-packages/nemo_curator/utils/distributed_utils.py", line 267, in read_single_partition
    df = read_f(file, **read_kwargs)
  File "/usr/local/lib/python3.10/dist-packages/cudf/io/json.py", line 96, in read_json
    df = libjson.read_json(
  File "json.pyx", line 45, in cudf._lib.json.read_json
  File "json.pyx", line 137, in cudf._lib.json.read_json

Paraphrasing some points after discussing with @ayushdg. cuDF had an issue where it could not read individual jsonl files larger than 2.1 GB . See: https://github.com/rapidsai/cudf/issues/16138.
cuDF fixed this in: https://github.com/rapidsai/cudf/pull/16162 that should be a part of the release next week. Couple of steps:

For the time being we recommend splitting the large file into smaller chunks using NeMo Curator's make_data_shards functionality.
We'll test if the cuDF fix works as expected & suggest using the latest nightly containers once a new version of cuDF is out in a week. We'll update this issue then.

Thank  you @ryantwolf! For the time being I'll try to use make_data_shards and wait for the dev build next week!

Should be working now. Let us know if you run into this issue again.


Move top-level lints to [workspace.lints] manifest table

Describe the feature

In the root /Cargo.toml: add [workspace.lints] with the lints that are in (almost) every crate, and remove said lints from the crates:#![warn(missing_debug_implementations, missing_docs, unreachable_pub, rustdoc::all)]
#![deny(unused_must_use, rust_2018_idioms)]

In all package Cargo.toml: add [lints] workspace = true after [package]
Run cargo clippy --workspace to verify
It's fine if you have to manually allow some of these in individual crates

Additional context
No response

Happy to work on this.

When the warn flag is applied to rustdoc.all as suggested, the compiler emits an unknown_lints warning, as seen below on the blockchain-tree crate:
warning[E0602]: unknown lint: `rustdoc.all`
  |
  = note: requested on the command line with `-W rustdoc.all`
  = note: `#[warn(unknown_lints)]` on by default

For more information about this error, try `rustc --explain E0602`.

I want to solicit feedback on supressing unknown_lints warnings or leaving it as-is before moving foreward.

When the warn flag is applied to rustdoc.all as suggested, the compiler emits an unknown_lints warning (not limited to but shown by) the blockchain-tree crate compilation step:
warning[E0602]: unknown lint: `rustdoc.all`
  |
  = note: requested on the command line with `-W rustdoc.all`
  = note: `#[warn(unknown_lints)]` on by default

For more information about this error, try `rustc --explain E0602`.

I just want to solicit feedback on supressing unknown_lints warnings in the parent Cargo.toml or to leave it as-is.

When the warn flag is applied to rustdoc.all as suggested, the compiler emits an unknown_lints warning (not limited to but shown by) the blockchain-tree crate compilation step:
warning[E0602]: unknown lint: `rustdoc.all`
  |
  = note: requested on the command line with `-W rustdoc.all`
  = note: `#[warn(unknown_lints)]` on by default

For more information about this error, try `rustc --explain E0602`.

Submitted a PR, but soliciting feedback on removing the unknown_lints = "allow" line.

Replied in the PR

Well I don't know what happened here, @tcoratger do still you want take this?

It's fine if you have to manually allow some of these in individual crates

@DaniPopes Let me reopen one. I saw that there were a lot of exceptions and as I had a lot of work I didn't want to stuck the issue and I told myself that someone would go faster but I'm going to reopen.


Error in migration when using postgres.

Cashier Version: v10.0.0
Laravel Version: v5.8.32
PHP Version: 7.3.8
Database Driver & Version: postgres 10

Description:
Then migration 2019_05_03_000001_create_customer_columns gives an error when using the postgres with Laravel. Running php artisan migrate on a fresh installation gives the following error.
Illuminate\Database\QueryException  : SQLSTATE[42704]: Undefined object: 7 ERROR:  collation "utf8mb4_bin" for encoding "UTF8" does not exist at character 57 (SQL: alter table "users" add column "stripe_id" varchar(255) collate "utf8mb4_bin" null, add column "card_brand" varchar(255) null, add column "card_last_four" varchar(4) null, add column "trial_ends_at" timestamp(0) without time zone null)

Steps To Reproduce:

Fresh installation of laravel
Change .env to use postgres as driver
Add cashier as dependency
php artisan migrate

Hmm, doesn't this collation exists on Postgres? I'm unfamiliar with Postgres so I don't know for sure. Stripe actually recommends to use this for MySQL but now I'm not sure if it's universal for every DB type: https://stripe.com/docs/upgrades#what-changes-does-stripe-consider-to-be-backwards-compatible

Can confirm, had this issue too. Had to delete the collation from the migration.

We'll need to find a universal way for this since it's recommended by Stripe to add this.

Just had the same issue.
As https://stripe.com/docs/upgrades#what-changes-does-stripe-consider-to-be-backwards-compatible says: If for example you’re using MySQL, you should store IDs in a VARCHAR(255) COLLATE utf8_bin column (the COLLATE configuration ensures case-sensitivity in lookups).
IMHO, there are two ways to fix that:

Ignore since its specific for MySQL;
Reproduce the same behavior for PgSQL.

Searching a little but, I've found that using ILIKE instead of LIKE for queries, as you can see here, can reproduce the case-sensitive behavior.

Maybe the Laravel Schema should implement an $table->string('stripe_id')->sensitive() flag.

Postgres uses utf8mb4 by default, but it's called "utf8".  This should use the database config values:
        'mysql' => [            ...            'charset'     => 'utf8mb4',
and
        'pgsql' => [              'charset'  => 'utf8',
values.

@rwdim if we'd change it to utf8_bin would that work?

The solution for me was to add Cashier::ignoreMigrations() to
AppServiceProvider::register, and change
``` to ```$table->string('stripe_id')->nullable()->index();``` in
my migrations directory.

R

On Mon, Aug 26, 2019, 7:58 AM Dries Vints<EMAIL_ADDRESS>wrote:

> @rwdim <https://github.com/rwdim> if we'd change it to utf8_bin would
> that work?
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/laravel/cashier/issues/747?email_source=notifications&email_token=AADFWGLWQCHPLYSXKVLGGN3QGPHOHA5CNFSM4IL5CQJ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5EJHMI#issuecomment-524850097>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/AADFWGO73DCVXSG2RKLOZC3QGPHOHANCNFSM4IL5CQJQ>
> .
>

@rwdim okay.
Can anyone try out on postgres by changing the migrations to use utf8_bin instead to see if that works?

No, it does not work., neither does collation('utf8').
Undefined object: 7 ERROR:  collation "utf8_bin" for encoding "UTF8" does

not exist

Based on what I read, utf8mb4_bin is a case-insensitive solution in mysql
that mirrors the default utf8 case-insensitive behavior in postgresql.
Therefore, selecting utf8 as the character set for the database is all you
need to do in PG, and omit the collation.
Based on the PG docs, collation is used to implement logic specific to
locales (en_us vs de_de vs ja_jp) (
https://www.postgresql.org/docs/11/collation.html
https://www.postgresql.org/docs/10/collation.html ).
R

How about adding a collation field to the cashier config settings, and if
it's set use that?
Otherwise don't do any collation.
On Mon, Aug 26, 2019, 8:32 AM Randy Dryburgh<EMAIL_ADDRESS>wrote:

No, it does not work., neither does collation('utf8').
Undefined object: 7 ERROR:  collation "utf8_bin" for encoding "UTF8"

does not exist

Based on what I read, utf8mb4_bin is a case-insensitive solution in mysql
that mirrors the default utf8 case-insensitive behavior in postgresql.
Therefore, selecting utf8 as the character set for the database is all you
need to do in PG, and omit the collation.
Based on the PG docs, collation is used to implement logic specific to
locales (en_us vs de_de vs ja_jp) (
https://www.postgresql.org/docs/11/collation.html
https://www.postgresql.org/docs/10/collation.html ).
R

I think maybe removing the default collation and noting something in the docs could be a good idea.
@staudenmeir do you think if we could add this to the schema builder to provide a solution for both postgres and mysql (and maybe the other two drivers as well)?

One thing... not sure why utf8mb4_bin was used, since the key that comes
back appears to me to be ascii salt+uuid(16) . I know Stripe suggests that
it may be something else in the future, but given that they don't say
what, default collation is as good a guess as a utf8 scope-limited variant.
Just my 0.000002BC :)
On Mon, Aug 26, 2019 at 9:37 AM Dries Vints<EMAIL_ADDRESS>wrote:

I think maybe removing the default collation and noting something in the
docs could be a good idea.
@staudenmeir https://github.com/staudenmeir do you think if we could
add this to the schema builder to provide a solution for both postgres and
mysql (and maybe the other two drivers as well)?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/laravel/cashier/issues/747?email_source=notifications&email_token=AADFWGNOCPYPOS5TAVDKPSTQGPTCNA5CNFSM4IL5CQJ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5ESBLI#issuecomment-524886189,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AADFWGLC7C5FGFOZ3OHXULLQGPTCNANCNFSM4IL5CQJQ
.

This was "caused" by https://github.com/laravel/framework/pull/29213. Before, Laravel ignored the collation on PostgreSQL.
I don't see how we could fix this in the schema builder.
Not the most elegant solution: We could detect the database driver in the migration and only add the collation on MySQL.

I think maybe it's best that we just remove the collation for now and add a note in the docs. I'll send in a PR.


Important "done" tasks appear before not-important "to be done" tasks

Right now, when we do a todo list -a to list every task (including the done ones) they are sorted by priority first, then by status:
 1 [X]              Trivial task I've already done @personal (100%)
 2 [X]   2016-01-19 Another trivial task I took care of @personal (100%)
 3 [ ]              Trivial todo, waiting me to stop procrastinating @personal
 4 [ ]              Another trivial todo task I have to do @personal
 5 [X] ! 2016-04-09 This was important, but I've already done this @personal (100%)
 6 [X] ! 2016-04-05 This was important as well, but I done it yesterday  @personal (100%)
 7 [ ] !            This is important and I have to do this @personal
 8 [ ] !            This is important as well and is waiting for me @personal

IMHO it makes more sense to list them sorted by status, then by priority, so we get to see the tasks that still need action (both not important and important) closer to the bottom of the output, even though some of them are not as important as some done tasks.
 1 [X]              Trivial task I've already done @personal (100%)
 2 [X]   2016-01-19 Another trivial task I took care of @personal (100%)
 3 [X] ! 2016-04-09 This was important, but I've already done this @personal (100%)
 4 [X] ! 2016-04-05 This was important as well, but I done it yesterday  @personal (100%)
 5 [ ]              Trivial todo, waiting me to stop procrastinating @personal
 6 [ ]              Another trivial todo task I have to do @personal
 7 [ ] !            This is important and I have to do this @personal
 8 [ ] !            This is important as well and is waiting for me @personal

The fix for this is to simply swap the main.py#L40-41 lines, but since I believe this is a matter of preference I'm raising a "question" issue in order to see what you think.
If you feel this makes sense ping me and I'll submit the PR.

I don't generally want to alter the default sorting or change sorting orders constantly due to preference (I'd rather keep that stable), but this particular request makes a lot of sense (I don't really use -a so never realized this).
Feel free to submit a PR for this. :+1:

@hobarrera My thoughts exactly. I feel personal preference sorting settings should be left to the config file (or flags), but I'm suggesting this because I think those are better (i.e.: more intuitive) default settings.
I'll do the PR when I'm on my laptop :+1:


Implementation of KineticEnergyDynPyCodeComponent and KineticEnergySymPyCodeComponent classes

Two classes for code reporting are to create. KineticEnergyDynPyCodeComponent and KineticEnergySymPyCodeComponent have to embed an examples of DynPy or/and SymPy classes and methods application which present kinetic energy calculations.

Imports and printing automations required to do. Link to the jupyter notebook below.
https://cocalc.com/projects/cdcf3856-3e8b-4d1c-bcb2-5c42ba749d83/files/UCZESTNICY/Krzysztof Kaczmarzyk/ReportTools.ipynb#id=4c7349

https://github.com/bogumilchilinski/dynpy/blob/dc2c43860a0e45de3fdac0fae53d6e61abbc0d93/utilities/components/mech/en.py#L274 L355

Closed as finished.


Reduce verbosity of p2p logs

Motivation
These were added in #2427 and carried over from testnet but we don't
need them to be so verbose
Changes
Reduces verbosity of some log-level p2p logs
TODO

[x] Explain motivation or link existing issue(s)

DevOps Notes

[x] This PR does not require configuration changes (e.g., environment variables, GitHub secrets, VM resources)
[x] This PR does not affect public APIs
[x] This PR does not rely on a new version of external services (PoET, elasticsearch, etc.)
[x] This PR does not make changes to log messages (which monitoring infrastructure may rely on)

bors merge


Way to check when same tab is selected?

No only check when a different tab is tapped

Issue fixed for iOS, run the following command to resinstall the iOS lib:
rm -rf platforms

Android fix need to be publish, encountering local problems right now.


grpclb_end2end_test failure

https://grpc-testing.appspot.com/job/gRPC_master_tsan_cpp/1457/testReport/junit/(root)/c++_linux_tsan_native/bins_tsan_grpclb_end2end_test___gtest_filter_SingleBalancerTest_Vanilla__GRPC_POLL_STRATEGY_epoll/
Output:
I0511 14:37:59.222790804    9845 ev_epoll_linux.c:95]        epoll engine will be using signal: 40
D0511 14:37:59.223388421    9845 ev_posix.c:107]             Using polling engine: epoll
D0511 14:37:59.224404397    9845 dns_resolver.c:316]         Using native dns resolver
D0511 14:37:59.224500122    9845 test_config.c:403]          test slowdown factor: sanitizer=5, fixture=1, poller=1, total=5
Note: Google Test filter = SingleBalancerTest.Vanilla
[==========] Running 1 test from 1 test case.
[----------] Global test environment set-up.
[----------] 1 test from SingleBalancerTest
[ RUN      ] SingleBalancerTest.Vanilla
I0511 14:37:59.233376428    9845 grpclb_end2end_test.cc:410] starting backend server on port 26859
I0511 14:37:59.235422569    9926 server_builder.cc:258]      Synchronous server. Num CQs: 1, Min pollers: 1, Max Pollers: 2, CQ timeout (msec): 10000
I0511 14:37:59.277781591    9845 grpclb_end2end_test.cc:417] backend server startup complete
I0511 14:37:59.280528179    9845 grpclb_end2end_test.cc:410] starting backend server on port 2116
I0511 14:37:59.281756693    9969 server_builder.cc:258]      Synchronous server. Num CQs: 1, Min pollers: 1, Max Pollers: 2, CQ timeout (msec): 10000
I0511 14:37:59.318836859    9845 grpclb_end2end_test.cc:417] backend server startup complete
I0511 14:37:59.322123232    9845 grpclb_end2end_test.cc:410] starting backend server on port 29440
I0511 14:37:59.323389219    9987 server_builder.cc:258]      Synchronous server. Num CQs: 1, Min pollers: 1, Max Pollers: 2, CQ timeout (msec): 10000
I0511 14:37:59.359263968    9845 grpclb_end2end_test.cc:417] backend server startup complete
I0511 14:37:59.362089020    9845 grpclb_end2end_test.cc:410] starting backend server on port 11626
I0511 14:37:59.363214964   10009 server_builder.cc:258]      Synchronous server. Num CQs: 1, Min pollers: 1, Max Pollers: 2, CQ timeout (msec): 10000
I0511 14:37:59.403775868    9845 grpclb_end2end_test.cc:417] backend server startup complete
I0511 14:37:59.407117428    9845 grpclb_end2end_test.cc:410] starting balancer server on port 10984
I0511 14:37:59.408368901   10040 server_builder.cc:258]      Synchronous server. Num CQs: 1, Min pollers: 1, Max Pollers: 2, CQ timeout (msec): 10000
I0511 14:37:59.441150687    9845 grpclb_end2end_test.cc:417] balancer server startup complete
I0511 14:37:59.460542764   10059 grpclb_end2end_test.cc:187] LB: recv msg 'initial_request {
  name: "servername_not_used"
}
'
I0511 14:37:59.460619895   10059 grpclb_end2end_test.cc:265] LB: sleeping for 0 ms...
I0511 14:37:59.460904429   10059 grpclb_end2end_test.cc:268] LB: Woke up! Sending response 'server_list {
  servers {
    ip_address: "\177\000\000\001"
    port: 26859
  }
  servers {
    ip_address: "\177\000\000\001"
    port: 2116
  }
  servers {
    ip_address: "\177\000\000\001"
    port: 29440
  }
  servers {
    ip_address: "\177\000\000\001"
    port: 11626
  }
}
'
I0511 14:37:59.462197219    9845 grpclb.c:511]               Missing LB token for backend address 'ipv4:<IP_ADDRESS>:26859'. The empty token will be used instead
I0511 14:37:59.462243678    9845 grpclb.c:511]               Missing LB token for backend address 'ipv4:<IP_ADDRESS>:2116'. The empty token will be used instead
I0511 14:37:59.462272807    9845 grpclb.c:511]               Missing LB token for backend address 'ipv4:<IP_ADDRESS>:29440'. The empty token will be used instead
I0511 14:37:59.462298463    9845 grpclb.c:511]               Missing LB token for backend address 'ipv4:<IP_ADDRESS>:11626'. The empty token will be used instead
test/cpp/end2end/grpclb_end2end_test.cc:478: Failure
Value of: status_and_response.first.ok()
  Actual: false
Expected: true
test/cpp/end2end/grpclb_end2end_test.cc:484: Failure
      Expected: 100
To be equal to: backend_servers_[i].service_->request_count()
      Which is: 124
test/cpp/end2end/grpclb_end2end_test.cc:484: Failure
      Expected: 100
To be equal to: backend_servers_[i].service_->request_count()
      Which is: 124
test/cpp/end2end/grpclb_end2end_test.cc:484: Failure
      Expected: 100
To be equal to: backend_servers_[i].service_->request_count()
      Which is: 0
test/cpp/end2end/grpclb_end2end_test.cc:484: Failure
      Expected: 100
To be equal to: backend_servers_[i].service_->request_count()
      Which is: 123
I0511 14:38:00.408971440    9845 grpclb_end2end_test.cc:434] backend about to shutdown
I0511 14:38:00.410799055    9845 grpclb_end2end_test.cc:437] backend shutdown completed
I0511 14:38:00.410831830    9845 grpclb_end2end_test.cc:434] backend about to shutdown
I0511 14:38:00.415370313    9845 grpclb_end2end_test.cc:437] backend shutdown completed
I0511 14:38:00.415418703    9845 grpclb_end2end_test.cc:434] backend about to shutdown
I0511 14:38:00.416912034    9845 grpclb_end2end_test.cc:437] backend shutdown completed
I0511 14:38:00.416944794    9845 grpclb_end2end_test.cc:434] backend about to shutdown
I0511 14:38:00.418563513    9845 grpclb_end2end_test.cc:437] backend shutdown completed
I0511 14:38:00.418603867    9845 grpclb_end2end_test.cc:434] balancer about to shutdown
I0511 14:38:00.420256558    9845 grpclb_end2end_test.cc:437] balancer shutdown completed
[  FAILED  ] SingleBalancerTest.Vanilla (1210 ms)
[----------] 1 test from SingleBalancerTest (1210 ms total)

[----------] Global test environment tear-down
[==========] 1 test from 1 test case ran. (1210 ms total)
[  PASSED  ] 0 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] SingleBalancerTest.Vanilla

 1 FAILED TEST
D0511 14:38:00.435271341    9845 ev_posix.c:107]             Using polling engine: epoll
D0511 14:38:00.435316574    9845 dns_resolver.c:316]         Using native dns resolver
D0511 14:38:00.438284413    9845 ev_posix.c:107]             Using polling engine: epoll
D0511 14:38:00.438349398    9845 dns_resolver.c:316]         Using native dns resolver
D0511 14:38:00.440938902    9845 ev_posix.c:107]             Using polling engine: epoll
D0511 14:38:00.441000112    9845 dns_resolver.c:316]         Using native dns resolver
D0511 14:38:00.447013569    9845 ev_posix.c:107]             Using polling engine: epoll
D0511 14:38:00.447087439    9845 dns_resolver.c:316]         Using native dns resolver
D0511 14:38:00.451094932    9845 ev_posix.c:107]             Using polling engine: epoll
D0511 14:38:00.451169512    9845 dns_resolver.c:316]         Using native dns resolver
Command exited with non-zero status 1
real 1.28
user 0.94
sys 0.21

Seen in gRPC_master_linux #2521.

this is fixed in #11159, to be merged soon

David, I think this is fixed now, isn't it?  If so, please close.


Refactoring of stub.c

@Largo
The implementation of stub.c had security issues, such as the use of strcat with fixed-length buffers, and the code complexity made it difficult to review partial fixes due to the mixture of TCHAR and char types.
I spent a month thoroughly understanding stub.c and its design, and I have been refactoring stub.exe to make it secure and the code more readable. The refactoring follows these guidelines:

Use char type for string operations because the archive path strings are in UTF-8, allowing for paths with multibyte characters.
Use heap memory for file path operations to prevent buffer overflows, enabling support for paths longer than MAX_PATH.
Improve the archive data format to be more secure and compact.

This effort is still ongoing, but the implementation of stub has become safer than before. However, several more changes are needed to complete the secure implementation.
stub.c compiles successfully with Mingw32 and Mingw64 on Windows. Please let me know if there are any compilation issues with Wine.

Thank you so much. I looked into it and it looks good!
I'm preparing a release right now.

The source code of the stub has now been made readable by humans.
The current implementation standardizes file path handling to UTF-8 and uses a variable-length buffer for path concatenation, eliminating the dependency on MAX_PATH and the possibility of buffer overruns. Additionally, security risks associated with script execution have been resolved. The stub cannot execute any scripts located externally.
However, security risks that have existed since before the fork from OCRA have not yet been addressed. In the next step, we will resolve these remaining security risks.

The refactoring of stub.c has been completed in the commit https://github.com/Largo/ocran/commit/a7b8347ba95879b2722b4c4415d404de2f711ed2.
The clear security issues in the implementation have been resolved. However, since security standards in actual operation vary by organization, I believe there are still areas for improvement based on external feedback.
The next step for stub is to handle paths with multibyte character strings. Users outside English-speaking countries use characters from their own languages in their usernames, which appear in the temporary paths. Additionally, I believe there is a need to replace WIN32API calls with standard C functions.


Move addIdentityChangeListener to public api

See forum post https://groups.google.com/forum/#!topic/orient-database/Yh78cHSCVFg
We work with pojo filling via service -> dao layers where transactions are demarcated at the service layer. So when we save/update an entity, the pojo gets a temporary id, which changed after commit.
We are using the identity change listener to fill our Pojo's so the in memory version the correct id.
Something like this in our dao layer:
   public void save(EntityPojo entity){
         //get an attachedObject and fill with entity
      ...
        ORecordInternal.addIdentityChangeListener(db().getRecordByUserObject(attachedObject, false), new OIdentityChangeListener() {
            @Override
            public void onBeforeIdentityChange(ORecord record) {
            }

            @Override
            public void onAfterIdentityChange(ORecord record) {
                entity.setId(record.getIdentity());
                entity.setVersion(record.getVersion());
            }
        });
        db.save(attachedObject);
        //fill entity with fields from attachedObject

As you can see, this is very useful, so my request is to move this functionality to public api, so it doesnt get lost during an internal refactor (as proposed in the forum post).

@wouterv does this always work for you. Below is simple testcase which just fails (now and then):
import com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx;
import com.orientechnologies.orient.core.id.ORID;
import com.orientechnologies.orient.core.record.OIdentityChangeListener;
import com.orientechnologies.orient.core.record.ORecord;
import com.orientechnologies.orient.core.record.ORecordInternal;
import com.orientechnologies.orient.core.record.impl.ODocument;
import com.tinkerpop.blueprints.impls.orient.OrientGraphFactory;

import org.junit.After;
import org.junit.Before;
import org.junit.Test;

public class DetachTestcase {

    private OrientGraphFactory factory;

    @Before
    public void setUp() throws Exception {
        factory = new OrientGraphFactory("memory:graphs");
    }

    @Test
    public void testDetaching() throws Exception {
        for (int i = 0; i < 10000; i++) {

            final MyPojo pojo = createPojo();
            if (pojo.id == null) {
                throw new IllegalStateException("null at : " + i);
            }

        }

    }

    public MyPojo createPojo() throws Exception {
        MyPojo pojo = null;
        ODatabaseDocumentTx database = factory.getDatabase();
        try {

            database.getMetadata().getSchema().getOrCreateClass("Test");
            database.begin();
            final ODocument bank = database.newInstance("Test");
            pojo = new MyPojo();
            database.save(bank);
            detachBank(bank, pojo);
            database.commit();
        } finally {
            database.close();
        }
        return pojo;
    }

    private void detachBank(final ODocument bank, final MyPojo pojo) {

        ORecordInternal.addIdentityChangeListener(bank, new OIdentityChangeListener() {
            @Override
            public void onBeforeIdentityChange(final ORecord record) {

            }

            @Override
            public void onAfterIdentityChange(final ORecord record) {
                final ORID newIdentity = record.getIdentity();
                pojo.id = newIdentity.toString();
            }
        });

    }

    @After
    public void tearDown() throws Exception {
        factory.close();
    }

    public static class MyPojo {
        public String id;

        @Override
        public String toString() {
            return "MyPojo{" + "id='" + id + '\'' +
                    '}';
        }
    }
}

I get something like:
java.lang.IllegalStateException: null at : 2589
where at part is random...mostly tetscase is successful...

@machak We experience the same problems, but we wrote a workaround for it (cut out the uninteresting parts):
        //boolean keeps record if the change listener has been called. no countdownlatch or something is needed, because call happens in the same thread
        final MutableBoolean idHasBeenSet = new MutableBoolean();
        ORecordInternal.addIdentityChangeListener(recordByUserObject, new OIdentityChangeListener() {
            @Override
            public void onBeforeIdentityChange(ORecord record) {
            }

            @Override
            public void onAfterIdentityChange(ORecord record) {
                entity.setId(record.getIdentity());
                entity.setOccVersion(OrientUtils.convertOccVersion(record.getVersion()));
                idHasBeenSet.setValue(true);
            }
        })
        ...
        final PersistentObject save = db().save(attachedObject, getClusterForClass(entity))
        if (!idHasBeenSet.booleanValue()) {
            Serializable id = save.getId();
            entity.setId(id);
            entity.setOccVersion(OrientUtils.convertOccVersion(db().getVersion(save)));
        }

We never have any problems anymore, but maybe you could raise your problem in a seperate issue to orient as you created a reproducable testcase.

No more relevant, closing it.


AngularJS UI Bootstrap NuGet out of date

AngularJS UI Bootstrap 2.1.3 was just released but NuGet is only showing 1.3.3: https://www.nuget.org/packages/Angular.UI.Bootstrap/

All of the current v2 releases have just been pushed to NuGet with the most recent being 2.1.3.
I no longer use NuGet for frontend assets, or this project, which is why I stopped publishing them when v2 came out. I'll continue to release these since there seems to be some demand for them but they'll probably lag behind some.
The NuGet package is maintained at xt0rted/angular-ui-bootstrap-nuget so if there's ever a version missing you can create an issue there about it which I'll see sooner than something posted on this project.

Thanks @xt0rted!
Closing this as we don't maintain the NuGet releases, and due to the information posted here as a location for contacting for updating the releases.


Support Github Discussions for non organization repositories

Is your feature request related to a problem? Please describe.
As it stands now, the REST API from GitHub only supports organization repositories.
Describe the solution you'd like
This is minor but this confused me at first.
For the short term I've proposed a one liner addition to the documentation : https://github.com/jreleaser/jreleaser.github.io/pull/8
In the longer run, it seems like looking into the GraphQL API could be a solution.
Not sure it is worth fixing though.
Describe alternatives you've considered
Nothing, warning the user is probably enough.
Additional context
Add any other context or screenshots about the feature request here.

In case this is not clear. The short term solution is definitely enough for me to close this issue :)

Agreed. I was surprised to find that the REST API only provides access to Discussions through organizations and teams.
It's also possible to create a discussion based on a release, which gets a copy of the release's body, but if you'd want different content then I'm afraid it's not possible, yet. I'll keep looking for options.

Right, I'm guessing you're referring to https://github.com/jreleaser/jreleaser/issues/186 here.

That's correct. One option I'm considering is to stop using hub4j (which provides access to most of GitHub's API) and craft a smaller version of the API as it's done for Gitea/GitLab already.

I'm looking into the issue a bit myself right now on hub4j's side. Having a thinner API layer would indeed have some nice additional goodies (GitHub seems to be pushing GraphQL hard you could get more functionalities that way). It's more work though.

Not really satisfied, but https://github.com/hub4j/github-api/pull/1180

Awesome. Thanks!

https://github.com/jreleaser/jreleaser/issues/186 has been fixed, which links a release with a discussion topic.
The GitHub Discussions announcer was originally conceived to perform this task. Technically speaking it could be removed though it also allows opening a "release" discussion with additional content, not just a copy of the release body.

I think the current features are enough. The discussion announcer works with teams, while the release can post to a discussion category regardless of teams or not.


Update link to GitHub repo

I'm unsure whether it is intend to be so, but it looks like a typo to me.

The change is good! It used to be my repo, but it was shifted over when I went corporate.


patch for issue #177 validates disabled fields

A quick patch for issue #177 "validates disabled fields". validateForm now ignores fields that have the disabled attribute. And it even includes a test to verify the behavior.

Closed in https://github.com/bcardarella/client_side_validations/commit/0576d6890fed12401867731c1d60ce9e9bd68e74

My test still fails, and stepping through in the debugger confirms this patch doesn't work for my use case:

A form is loaded with an enabled field with validation.
That field is disabled.
The form is submitted.

Expected: If all the enabled fields are valid, the form submits successfully.
Actual: The field we disabled is validated, and if it's in an invalid state form submission is cancelled.
(Also your JS test suite seems to be acting up right now: Despite not being listed as an option, jQuery 1.4.4 is still being loaded by default; and test/javascript/public/test/form_builders/validateForm.js has a syntax error on line 3 ("clientSideValidations.forms.['new-user']" instead of "clientSideValidations.forms['new_user']")

@taavo good catch, I was only supporting individual inputs and missed actual form submission. This has now been fixed: https://github.com/bcardarella/client_side_validations/commit/6040bb10de8c6064b7f24d2e7ad6372b44f07c03

That'll do it. Thanks.


Fix initial pick position

For toy 1x1 datasets, the initial pick position was out of bounds. Refs #436
Contributor Checklist:

[ ] I have added or updated my entry in the creators.json file
[ ] I have added a changelog entry for my contribution
[ ] I have added/updated documentation for all user-facing changes
[ ] I have added/updated test cases
[ ] I have included the rebuilt production build of the client (only if
changes were made to the GUI)

Reviewer Checklist:

[ ] /azp run libertem.libertem-data passed

What about using Math.floor() instead of Math.round() - 1?

LGTM, merging!


Examples index page has incorrect link to background-image example page

This seems to have been fixed by an unknown change


Is there any way we can see storybook code ??

I want to create same example feature in my app. How can i see the story code ? Please help here..
https://5fc05e08a4a65d0021ae0bf2-goamnjvxea.chromatic.com/?path=/docs/presets-sortable-vertical--without-drag-overlay

you can find it from https://github.com/clauderic/dnd-kit/tree/master/stories/2 - Presets/Sortable


lastEventId not implemented?

The spec says that server-sent events should have a lastEventId property, which is set to the value of the last id field that was sent through the stream (or the empty string if there was none). It doesn't necessarily have to belong to the current event; it's just the last event ID witnessed within that stream.
This library does something vaguely similar, but with some deviations:

The property is named id, not lastEventId
It's reset with each new event sent, rather than persisting
It's set to null by default, whereas lastEventId defaults to the empty string
lastEventId forbids null bytes; the id property allows them

I want to bring this in line with the spec, but I'm concerned about backwards compatibility with previous versions of the library. There are a few options:

Leave id as-is and document the differences.
Add lastEventId as a separate property, but keep id and its behavior. This would complicate the codebase, but would allow for better standards compatibility and backwards compatibility.
Do a major version bump, and fix this along with some other spec deviations (e.g. firing CustomEvents instead of MessageEvents; not setting the event's origin; lack of reconnection logic).

lastEventId is indeed not currently supported (see https://github.com/mpetazzoni/sse.js?tab=readme-ov-file#todos-and-caveats), but it shouldn't be hard to add without backwards compatibility issues.
I'm not sure I fully understand your analysis here, especially the id vs lastEventId part, since sse.js makes no attempt (so far) to implement lastEventId; the id field on events is something else entirely.
I'll send a PR with a drat proposal implementation.


#33171: Invalid link for readonly fields in django admin with missing…

… permissions
Trac ticket number

ticket-33171
Branch description
Having the permission to see model A but not FK-related model B will display a link to model B. After clicking on this link, you'll end up on a 403 page. This is at least bad UX and probably a bug.
Thx to @sarahboyce in the Django forum for providing this solution.
Checklist

[x] This PR targets the main branch.
[ ] The commit message is written in past tense, mentions the ticket number, and ends with a period.
[ ] I have checked the "Has patch" ticket flag in the Trac system.
[ ] I have added or updated relevant tests.
[ ] I have added or updated relevant docs, including release notes if applicable.
[ ] I have attached screenshots in both light and dark modes for any UI changes.

Hi @sarahboyce
it seems your idea is flying. What are the next steps for me? How can I unit-test this? Are there some ressources I should be reading?
Best and thx so much for the input!
Ronny

Hi @sarahboyce! I have a bit of a hard time to write the tests.
It's kind of hard deciding what to test where, there are SO many tests 😅 Secondly, I don't manage to get the runtests script to work properly, even though reading the docs.
For some reason I can't see the logs from the broken tests here in GitHub... am I missing something?
But I tried to create a test, could you give me a hint if I'm on the right track here? I know that it's not done yet (I have to at least assign some permissions) but before adding more complexity, I'd like to know that it's ok in general.
Sorry to bother you with noobie questions 🙈

@sarahboyce Finally managed to get the test running. What would you say we need to finish this PR?

Hello @GitRon! Thank you for the work done in this PR. :star2:
Sarah and I were discussing this issue, and she made some suggestions and provided a resolution in the ticket-33171. Given that, we are keen to close this PR. If you wish to continue experimenting, let us know!

Hi @nessita! Thx for the heads-up and @sarahboyce for all the work you've put into this!


Update sbt-bloop to 1.5.13

About this PR
📦 Updates ch.epfl.scala:sbt-bloop from 1.4.8-19-4d9f966b to 1.5.13
📜 GitHub Release Notes
Usage
✅ Please merge!
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

⚙ Adjust future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "ch.epfl.scala", artifactId = "sbt-bloop" } ]

Or, add this to slow down future updates of this dependency:
dependencyOverrides = [{
  pullRequests = { frequency = "30 days" },
  dependency = { groupId = "ch.epfl.scala", artifactId = "sbt-bloop" }
}]

labels: sbt-plugin-update, early-semver-minor, semver-spec-minor, commit-count:1

Superseded by #120.


DO NOT MERGE! Fast Forward stable to 073c2705bab28af648f4fa32b64d327943018692.

This PR will be merged automatically once checks are successful.

This change is

Jenkins job mixer/postsubmit passed

Jenkins job mixer/e2e-suite passed


server VAD doesn't seem to pick up voice stopping

I can successfully  connect and and event is received when I start speaking, but it gets hanged here, it never detects when I stopped and it just stays there until the timeout is reached, am I missing something?
Session updated successfully: {'type': 'session.updated', 'event_id': 'event_AF8eSFQtCeDPjLip7kbgJ', 'session': {'id': 'sess_AF8eRPbIlNNeKrx6ghnHj', 'object': 'realtime.session', 'model': 'gpt-4o-realtime-preview-2024-10-01', 'expires_at':<PHONE_NUMBER>, 'modalities': ['text', 'audio'], 'instructions': 'You are a helpful and bubbly AI assistant who loves to chat about anything the user is interested in and is prepared to offer them facts. You have a penchant for dad jokes, owl jokes, and rickrolling – subtly. Always stay positive, but work in a joke when appropriate.', 'voice': 'alloy', 'turn_detection': {'type': 'server_vad', 'threshold': 0.5, 'prefix_padding_ms': 300, 'silence_duration_ms': 500}, 'input_audio_format': 'pcm16', 'output_audio_format': 'g711_ulaw', 'input_audio_transcription': None, 'tool_choice': 'auto', 'temperature': 0.8, 'max_response_output_tokens': 'inf', 'tools': []}}

Received event: input_audio_buffer.speech_started {'type': 'input_audio_buffer.speech_started', 'event_id': 'event_AF8eZJem71dtYCftbFvOP', 'audio_start_ms': 512, 'item_id': 'item_AF8eZKJuxUJmFcjQzCoPN'}

I spent a bit of time looking at this and got interruption working quite well on my branch. Here are a few things:

OpenAI's server VAD actually works quite well. Watch for the input_audio_buffer.speech_started event
OpenAI sends the entire generated audio upfront, in much shorter time than it takes to play the audio. Watch for the response.audio.delta event
The code in this project, as it is today, receives the entire audio from OpenAI and immediately sends it to Twilio where it is queued for play. Therefore, there isn't a way to "cancel" the playback during an interruption because the audio has already been sent to Twilio
The approach I settled on was to manage the audio queue myself: When audio is received from OpenAI, it is added to an audio buffer. The data in the buffer is sent "just in time" to Twilio. If an interruption occurs, then the buffer is cleared and the response truncated.

Hope that helps!

Just want to chime in that we're looking at interruptions/preemption on the node version at the moment: https://github.com/twilio-samples/speech-assistant-openai-realtime-api-node/issues/9 .
@jhmaddox you're using a queue and Mark message to determine when to send the next response.audio.delta to Twilio?

@robcontreras
I had success with doing this:
                    if response['type'] == 'input_audio_buffer.speech_started':
                        await websocket.send_json({ "event": "clear",
                                                    "streamSid": stream_sid })

It flushes the twilio audi stream. Basically clears whatever you sent already.

https://github.com/twilio-samples/speech-assistant-openai-realtime-api-python/pull/13 I have a PR here if anyone would like to test. It should work within a reasonable error around your interruption. We're looking internally as well.

https://github.com/twilio-samples/speech-assistant-openai-realtime-api-python/pull/13 closing with this merged


update samples for the latest sdk release

https://gist.github.com/flutterdevrelgists/e75b493dae1287757c5e1d77a0dc73f1
https://gist.github.com/flutterdevrelgists/5c0e154dd50af4a9ac856908061291bc

Some deprecation warnings and needing to use a super param.

Good catch, fixed!


Fix broken links to data assembly python notebooks

Links to python notebooks on page https://github.com/griffithlab/DeepSVR/wiki/Data-Assembly give 404 file not found errors.
Broken links:

https://github.com/griffithlab/DeepSVR/blob/master/notebooks/Add project specific features.ipynb

https://github.com/griffithlab/DeepSVR/blob/master/notebooks/Duplicate analysis.ipynb

https://github.com/griffithlab/DeepSVR/blob/master/notebooks/parse bam read count.ipynb

Hi @aloraine I apologize for the broken links. Prior to releasing the database we had to remove some data for compliance reasons, therefore, the dataframes and notebooks used to generate the prepared data from raw data were removed. I have updated the wiki page to inform users of this change and removed the links from the wiki. If you would like to learn more about how we performed the analysis I am happy to answer any questions you might have! Sorry for the inconvenience.
Best,
Erica


[React Native] bundleRelease fails on task :app:uploadBugsnagReleaseSourceMaps

Describe the bug
When running ./gradlew bundleRelease, the job fails.
Note: what's weird is that it was working untill yesterday, and without any changes it started failing with:
> Task :app:bugsnagReleaseReleaseTask FAILED
Bugsnag: Uploading to Releases API
{"errors":["API key not recognised: @string/BUGSNAG_ID"],"status":"error"}

So I replaced the corresponding field with the corresponding value and got the next error.
Environment

Bugsnag version: 7.5.3
Device (e.g. iphonex): OnePlus 5T

Example code snippet
./gradlew bundleRelease

Throws the following error.
> Task :app:uploadBugsnagReleaseSourceMaps FAILED

FAILURE: Build failed with an exception.

* What went wrong:
A problem was found with the configuration of task ':app:uploadBugsnagReleaseSourceMaps' (type 'BugsnagUploadJsSourceMapTaskGradle53Plus').
> File '/project/node_modules/@bugsnag/source-maps/bin/cli' specified for property 'bugsnagSourceMaps' does not exist.

Hi @Andarius - this was caused by an alpha release of the plugin (v5.5.0alpha01), which was used in your build due to the use of a wildcard dependency version. Wildcard dependency versions look like this:
classpath "com.bugsnag:bugsnag-android-gradle-plugin:5.+"

We've released v5.6.0 of the plugin which will prevent the issue from happening - you could also pin the dependency to a specific version which would avoid any possibility of similar issues occurring:
classpath "com.bugsnag:bugsnag-android-gradle-plugin:5.6.0"

Thanks a lot for the quick answer ! Is there a way to specify only the latest stable version ? Like 5.+@stable ?

@Andarius Gradle does allow for locking dependency versions which might be able to achieve what you're after. Otherwise I'd recommend just pinning the latest stable version. There are some useful plugins such as this one which can help with checking for updates on pinned dependencies.


[Publication] Right part scroll works incorrectly for the publication request window

EPAM AI DIAL chat version
0.13.0
How to reproduce
Create publiсation request window with enough content for scroll in left and right parts
Actual result
Left part scroll works correctly

Right part scroll works as common scroll

Expected result
Right part scroll should apply only for right part of the window
Confidential information

[X] I confirm that do not share any confidential information

verified successfully on staging


Transition 2-3

copy/paste
The next few sections take advantage of the Chemical Inventory resource. Using this resource is recommended, but optional.
The goal of the chemical inventory is to identify the chemicals used to produce and manufacture a product, including those chemicals that remain in the product when used and any chemicals that are produced after use and/or during waste management treatment.
To do this, you'll have to learn which chemical substances are present in the final product. A chemical inventory also considers the chemicals used to manufacture a product because they may remain in the product and workers and potentially the local community are exposed to those chemicals. It is also important to think about what the chemical may become when it is released to the environment.  Many chemicals degrade into other chemicals or ultimately break down to CO2 and water over time.
Ideally, it would be possible to know all of the reagents, catalysts, auxiliaries, products, by-products, residuals and emissions at each life cycle stage. However, gathering this information is not always a simple task. The more you know about what is in the material, the better you can identify areas for improvement.
Keep in mind that even small amounts of toxic chemical ingredient additives can greatly impact the sustainability attributes of a product. Additives to plastics and residual chemicals from manufacturing can:
Affect recycling, incineration and value-recovery options Be hazardous to workers or the environment Leak into the environment after disposal Affect the quality of recycled materials and how they can be used in future products
Prompt - gives them the option of going ot the chemical inventory (go directly to guiding questions for creating a screening chemical inventory, skipping the intro part since it is written here verbatim) OR going straight to section 3

Closing b/c duplicate.


Postgres Function support added

Postgres Function support added
Why make this change?

Reference associated issue using # syntax. e.g. Closes #XX

Include summary (1-2 sentences) of linked issue to avoid redirecting reviewers to different pages.
Include links to any additional discussion threads related to this change.

What is this change?

Summary of how your changes work to give reviewers context of your intent.

Links to relevant documentation / StackOverflow, if applicable.

How was this tested?

[ ] Integration Tests
[ ] Unit Tests

Sample Request(s)

Example REST and/or GraphQL request to demonstrate modifications
Example of CLI usage to demonstrate modifications

@microsoft-github-policy-service agree

Thank you for your contribution , @sanjeev23oct. Can you fill out the PR description template with contextual info? Please also add tests for validating your changes, which will help us review.


Support for skillshare.com

Please follow the guide below

You will be asked some questions and requested to provide some information, please read them carefully and answer honestly
Put an x into all the boxes [ ] relevant to your issue (like that [x])
Use Preview tab to see how your issue will actually look like

Make sure you are using the latest version: run youtube-dl --version and ensure your version is 2016.06.12. If it's not read this FAQ entry and update. Issues with outdated version will be rejected.

[x ] I've verified and I assure that I'm running youtube-dl 2016.06.12

Before submitting an issue make sure you have:

[x ] At least skimmed through README and most notably FAQ and BUGS sections
[x ] Searched the bugtracker for similar issues including closed ones

What is the purpose of your issue?

[ ] Bug report (encountered problems with youtube-dl)
[x ] Site support request (request for adding support for a new site)
[ ] Feature request (request for a new functionality)
[ ] Question
[ ] Other

The following sections concretize particular purposed issues, you can erase any section (the contents between triple ---) not applicable to your issue

If the purpose of this issue is a bug report, site support request or you are not completely sure provide the full verbose output as follows:
Add -v flag to your command line you run youtube-dl with, copy the whole output and insert it here. It should look similar to one below (replace it with your log inserted between triple ```):
$ youtube-dl -v <your command line>
[debug] System config: []
[debug] User config: []
[debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKcj']
[debug] Encodings: locale cp1251, fs mbcs, out cp866, pref cp1251
[debug] youtube-dl version 2016.06.12
[debug] Python version 2.7.11 - Windows-2003Server-5.2.3790-SP2
[debug] exe versions: ffmpeg N-75573-g1d0487f, ffprobe N-75573-g1d0487f, rtmpdump 2.4
[debug] Proxy map: {}
...
<end of log>

If the purpose of this issue is a site support request please provide all kinds of example URLs support for which should be included (replace following example URLs by yours):

Single video: https://www.youtube.com/watch?v=BaW_jenozKc
Single video: https://youtu.be/BaW_jenozKc
Playlist: https://www.youtube.com/playlist?list=PL4lCao7KL_QFVb7Iudeipvc2BCavECqzc

Hi Sir
i want to request you to add this one website for downloading video
this is a elearning website like (lynda, udemy and pluralsight)
here is course links
https://www.skillshare.com/classes/technology/Design-Build-Your-Own-Personal-Computer/694021855/classroom/discussions?via=my-classes&enrolledRedirect=1
but there is not single video link
every video is playing in single link
may b i am wrong cuz i am not programmer
here is free course link which you can enroll and check
https://www.skillshare.com/classes/technology/Design-Build-Your-Own-Personal-Computer/694021855/classroom/discussions?via=my-classes&enrolledRedirect=1
if you want premium account then tell me i will give you mine

hello is it possible or not ?
plz ans requested sins around 2 months

@yan12125  bro can you plz add this site its realy need

@jaan143 Please wait until someone is going to working on it.

@yan12125   bro i am waiting sins 3 months no one doing on it :(
as you check date of request :(

I know. There are lots of new site requests here, and usually those with the most users are worked first.

I'm trying to add support to SkillShare, but I'm not a Python dev and the hardest bit is that they have reCaptcha on login page. They also enable login with facebook, I'm not sure if that helps.
Besides the login, it's quite easy to implement the download.
All courses come with a data-account and the course ìd. The course ID is the last parameter of the URL, the data-accountwhich is part of the...`
with that said, you can call their open API, like:
https://edge.api.brightcove.com/playback/v1/accounts/{data-account}/videos/{video-id}
If you try the URL https://www.skillshare.com/classes/JavaScript-Complete-Guide-to-learning-JavaScript/1391908967, you'll get the { data-account:<PHONE_NUMBER>001 } and from the URL, you get the { videoid:<PHONE_NUMBER> }.
converts to: https://edge.api.brightcove.com/playback/v1/accounts/3695997568001/videos/1391908967
If the course is free, they'll give you a json with the videos of the course. Otherwise, I believe we need to get a premium account and should be the same.
Eq:
{
  "account_id": "3695997568001",
  "ad_keys": null,
  "created_at": "2016-07-20T02:05:24.811Z",
  "cue_points": [],
  "custom_fields": {},
  "description": null,
  "duration": 103862,
  "economics": "AD_SUPPORTED",
  "id": "5042560812001",
  "link": null,
  "long_description": null,
  "name": "Promo CompleteJavaScript",
  "published_at": "2016-07-20T02:05:24.811Z",
  "reference_id": null,
  "sources": [
    {
      "app_name": "rtmp://cp150446.edgefcs.net/ondemand",
      "asset_id": "5042638008001",
      "avg_bitrate": 2682000,
      "codec": "H264",
      "container": "MP4",
      "duration": 103862,
      "height": 720,
      "size": 34811823,
      "stream_name": "mp4:3695997568001/3695997568001_5042638008001_5042560812001.mp4?__nn__=1497926354001&slist=3695997568001/&auth=daEbnbob7ccdYb0bAaUcvcnaUbucsdxawbX-byTs4a-hca-nmBBArtz_zCys_GEwA_DsC&aifp=bcosuds",
      "width": 1280
    },
    {
      "asset_id": "5042638008001",
      "avg_bitrate": 2682000,
      "codec": "H264",
      "container": "MP4",
      "duration": 103862,
      "height": 720,
      "size": 34811823,
      "src": "http://uds.ak.o.brightcove.com/3695997568001/3695997568001_5042638008001_5042560812001.mp4?pubId=3695997568001&videoId=5042560812001",
      "width": 1280
    },
    {
      "asset_id": "5042638008001",
      "avg_bitrate": 2682000,
      "codec": "H264",
      "container": "MP4",
      "duration": 103862,
      "height": 720,
      "size": 34811823,
      "src": "https://udso-a.akamaihd.net/3695997568001/3695997568001_5042638008001_5042560812001.mp4?pubId=3695997568001&videoId=5042560812001",
      "width": 1280
    },
   ...
}

So you can just download all the source.src videos.

At the mean time, @jaan143, you can download with wget
wget https://udso-a.akamaihd.net/3695997568001/3695997568001_5042638008001_5042560812001.mp4?pubId=3695997568001&videoId=5042560812001 -o test.mp4

Hope that helps

Hi @jaan143, try this Chrome App. It worked fine for me: https://chrome.google.com/webstore/detail/nbkekaeindpfpcoldfckljplboolgkfm

sadly to say 9 months gone and still my request is pending

Does anyone have access to a premium account I can borrow for testing?

Hey @haamedk  I have !!! You can get from me !

They are having a special right now for 1 week. You can get 3 months of premium as a trial for $0.99 USD.
https://www.skillshare.com/membership/checkout?via=site-banner&coupon=SITESCHOOL
I'm going to buy into it, I do hope someone adds support to youtube-dl. It is my fav download tool.

Today, I'm wishing I knew more python.  Skillshare uses BrightcoveExperience, which is already written and existing plugins use it.  If someone wants to do this, I've put up a $20 bounty on bountysource.  I know it's not much, and I wish I could pay a coder what they are worth, but hey, OSS doesn't usually pay that much I suppose.  You have my appreciation in advance.  Thanks guys for all your hard efforts.  I'm off to go watch some python videos.  If you need to test things with my premium account, let me know.

#16582 Should provide support for skillshare!

Is this done yet?

@HAKSOAT It should be but they seem not to want to merge my fork.

That's strange

I'd love to see skillshare support for those times I want to learn something new while I am traveling with less or no internet! Thank you for the work on the pull request @Oshawk — how about adding it in here? Anyone?
Thank you 💪.

The tutorials on skillshare are great, but the youtube-dl support for this site isn't there anyway. And other direct download tool is not developed yet.
I just create a Greasymonkey userscript which can help downloading, but also not that conviniet.

The tutorials on skillshare are great, but the youtube-dl support for this site isn't there anyway. And other direct download tool is not developed yet.
I just create a Greasymonkey userscript which can help downloading, but also not that conviniet.

Could you share Greasymonkey javascript codes?

The tutorials on skillshare are great, but the youtube-dl support for this site isn't there anyway. And other direct download tool is not developed yet.
I just create a Greasymonkey userscript which can help downloading, but also not that conviniet.

Yeah share the greasemonkey script and if also any tips for using it. thanks.

I wished this was now available since udemy playlist can be downloaded easily and skillshare is also using brightcove which is already working in youtube-dl. As far as login is concerned, For Udemy, We need to use a browser plugin to extract the cookie file which then is used by youtube-dl to make requests pretending to be browser. Maybe that can be done here too.
There is a working repo on github which is working on single videos, It doesn't uses youtube-dl in background so it looses a lot of functionality and features. Maybe that repo can be used as reference here.


test_pageserver_http_get_wal_receiver_not_found setup is flaky

A recent failure in main: https://github.com/neondatabase/neon/actions/runs/3564596050/jobs/5989355902
main seems failing:passing about 50:50 at the moment, each failure is unique in its flakiness.
______ ERROR at setup of test_pageserver_http_get_wal_receiver_not_found _______
[gw2] linux -- Python 3.9.2 /github/home/.cache/pypoetry/virtualenvs/neon-_pxWMzVK-py3.9/bin/python
subprocess.CalledProcessError: Command '['/tmp/neon/bin/neon_local', 'pageserver', 'start']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:
/github/home/.cache/pypoetry/virtualenvs/neon-_pxWMzVK-py3.9/lib/python3.9/site-packages/pytest_lazyfixture.py:39: in fill
    _fillfixtures()
test_runner/fixtures/neon_fixtures.py:982: in _shared_simple_env
    env = builder.init_start()
test_runner/fixtures/neon_fixtures.py:623: in init_start
    self.start()
test_runner/fixtures/neon_fixtures.py:619: in start
    self.env.start()
test_runner/fixtures/neon_fixtures.py:916: in start
    self.pageserver.start()
test_runner/fixtures/neon_fixtures.py:1778: in start
    self.env.neon_cli.pageserver_start(overrides=overrides, extra_env_vars=extra_env_vars)
test_runner/fixtures/neon_fixtures.py:1583: in pageserver_start
    return self.raw_cli(start_args, extra_env_vars=extra_env_vars)
test_runner/fixtures/neon_fixtures.py:1356: in raw_cli
    raise Exception(msg) from subprocess.CalledProcessError(
E   Exception:             Run ['/tmp/neon/bin/neon_local', 'pageserver', 'start'] failed:
E                 stdout: Starting pageserver at 'localhost:17094' in '/tmp/test_output/test_pageserver_http_get_wal_receiver_not_found/repo'.....
E   pageserver has not started yet, continuing to wait.....
E
E                 stderr: pageserver start failed: pageserver did not start in 10 seconds
------------------------------ Captured log setup ------------------------------
2022-11-28 12:47:58.977 INFO [neon_fixtures.py:2706] get_test_output_dir is /tmp/test_output/test_pageserver_http_get_wal_receiver_not_found
2022-11-28 12:47:58.978 INFO [neon_fixtures.py:2727] test_output_dir is /tmp/test_output/test_pageserver_http_get_wal_receiver_not_found
2022-11-28 12:47:58.978 INFO [neon_fixtures.py:2706] get_test_output_dir is /tmp/test_output/test_pageserver_http_get_wal_receiver_not_found
2022-11-28 12:47:58.985 INFO [neon_fixtures.py:910] Config:
default_tenant_id = 'e48fa9efc5dba27783c8804a719877f2'

[etcd_broker]
broker_endpoints = ['http://<IP_ADDRESS>:17001/']
etcd_binary_path = '/usr/local/bin/etcd'

[pageserver]
id=1
listen_pg_addr = 'localhost:17094'
listen_http_addr = 'localhost:17095'
auth_type = 'Trust'

[[safekeepers]]
id = 1
pg_port = 17096
http_port = 17097
sync = false
2022-11-28 12:47:58.986 INFO [neon_fixtures.py:1319] Running command "/tmp/neon/bin/neon_local init --config=/tmp/tmpc86zbfk2 --pg-version 14"
2022-11-28 12:47:58.986 INFO [neon_fixtures.py:1320] Running in "/tmp/test_output/test_pageserver_http_get_wal_receiver_not_found/repo"
2022-11-28 12:48:00.758 INFO [neon_fixtures.py:1347] Run success: Starting pageserver at 'localhost:17094' in '/tmp/test_output/test_pageserver_http_get_wal_receiver_not_found/repo'.
pageserver started, pid: 14048
Successfully initialized timeline 2435465a403cfc1de7cc88a8b3b70cc6
Stopped pageserver 1 process with pid 14048

2022-11-28 12:48:00.758 INFO [neon_fixtures.py:1319] Running command "/tmp/neon/bin/neon_local pageserver start"
2022-11-28 12:48:00.758 INFO [neon_fixtures.py:1320] Running in "/tmp/test_output/test_pageserver_http_get_wal_receiver_not_found/repo"
2022-11-28 12:48:10.820 INFO [neon_fixtures.py:1355]             Run ['/tmp/neon/bin/neon_local', 'pageserver', 'start'] failed:
              stdout: Starting pageserver at 'localhost:17094' in '/tmp/test_output/test_pageserver_http_get_wal_receiver_not_found/repo'.....
pageserver has not started yet, continuing to wait.....

              stderr: pageserver start failed: pageserver did not start in 10 seconds

2022-11-28 12:48:10.820 INFO [neon_fixtures.py:777] Cleaning up all storage and compute nodes
2022-11-28 12:48:10.820 INFO [neon_fixtures.py:2487] Stopping safekeeper 1
2022-11-28 12:48:10.820 INFO [neon_fixtures.py:1319] Running command "/tmp/neon/bin/neon_local safekeeper stop 1 -m immediate"
2022-11-28 12:48:10.820 INFO [neon_fixtures.py:1320] Running in "/tmp/test_output/test_pageserver_http_get_wal_receiver_not_found/repo"
2022-11-28 12:48:10.827 INFO [neon_fixtures.py:1347] Run success: safekeeper 1 is already stopped: no pid file "/tmp/test_output/test_pageserver_http_get_wal_receiver_not_found/repo/safekeepers/sk1/safekeeper.pid" is present

2022-11-28 12:48:10.827 INFO [neon_fixtures.py:722] no remote storage was set up, skipping cleanup

https://github.com/neondatabase/neon/pull/2984 fixed this


Drop knative.dev/serving as dependency

Since 7a6abca serving-operator is no longer using serving's
Istio client so there's no need depending on knative.dev/serving anymore.

/cc @houshengbo

/ok-to-test


after i update the react-native. I got this error, "undefined is not an object (evaluating 'React.PropTypes.bool')".

after i update the react-native. I got this error, "undefined is not an object (evaluating 'React.PropTypes.bool')".
OS: MAC
react native: 0.26.1
iOS

please read the changelog before you upgrade your RN, React has been seperated from React-Native since RN0.25
change
import React, {
  PropTypes,
  ...
} from 'react-native';

to
import React, {
  PropTypes
} from 'react';
import {
  ...
} from 'react-native';

@nihgwu is right. Thanks for answering.

I have done changes, but still gives me the error.

@devekoperiOS perhaps the third-party components haven't upgrade to RN0.26

i am also getting the same issue... @nihgwu your solution did'nt helped... any idea ?

@VishalBumpy perhaps you just forget to install 'react'? it's not installed automatically when you init a new project

@nihgwu  I did, still i am getting the same error

@nihgwu i solved the issue... seems my npm ran into some error. :)

Sorry guys, I am struggling with react-native as a beginner and when you suggest to "update the file" i have no idea which file are you referring to... Could you please point me to the file that needs to be changed? @nihgwu @devekoperiOS


[W5.7][W10-4]Yuen Jun Rong

Ready for review

In Parser.java line 92 and 93, I think you might have overlooked that the lines are improperly indented according to the style guide.


Attempting to use AES encryption with no password throws unhelpful  exception

I tried turning off the password in one of the zip output stream unit tests and got
Message:
  System.ArgumentNullException : Buffer cannot be null.

Looks to be because some of the logic goes off entry.AESKeySize being set to a non-zero value (deciding whether to write the > AES extra data etc), but them some of it goes off Entry.IsCrypted being true, which is only done automatically if a password has already been specified when the entry is added, so some of the other crypto bits get skipped.
Gets itself in a bit of a mess basically.
Possibly PutNextEntry should throw if you try to add an entry with AESKeySize set, but no password set?

Originally posted by @Numpsy in https://github.com/icsharpcode/SharpZipLib/discussions/506#discussioncomment-49971

Fixed by #509


How do you pass data between views?

What mechanism do you propose to pass data between views when using Pancakes since the view does not get a Bundle and the view is getting created in xml?

I imagine that I'd create a ViewFactory with whatever values or dependencies the view needs, then pass that stuff onto the view in the createView method. ViewFactories get serialized, so any fields in the ViewFactory should be Serializable. Bundles could work well here, but any Serializable type would be fine. That would look something like this:
public class DogViewFactory implements ViewFactory {

    private final Bundle dogs;

    public DogViewFactory(Bundle dogs) {
        this.dogs = dogs;
    }

    @Override
    public View createView(Context context, ViewGroup container) {
        DogView dogView = LayoutInflater.from(context).inflate(R.layout.view_dog, container, false);
        dogView.initWithDogs(dogs);
        return dogView;
    }
}

viewStack.push(new DogViewFactory(dogs));


pyspark 2.4 support

Does this library currently work with spark 2.4?

We have not tried it with Spark 2.4 yet.
On Wed, Feb 13, 2019 at 3:44 PM mattomatic<EMAIL_ADDRESS>wrote:

Does this library currently work with spark 2.4?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/twosigma/flint/issues/63, or mute the thread
https://github.com/notifications/unsubscribe-auth/AAwbrHasxpH0N4PydNjQOOvfgBQU4YFJks5vNHkagaJpZM4a6PJ7
.

It does not work wit 2.3.2 as well

what issues do you see with 2.3.2? Internally we use flint with 2.3.2
without issues.
On Fri, Feb 15, 2019 at 1:53 AM Sandro Cavallari<EMAIL_ADDRESS>wrote:

It does not work wit 2.3.2 as well
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/twosigma/flint/issues/63#issuecomment-463929000, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAwbrIZKDz7cGE8o1d7di37jFP3jI6yCks5vNlmFgaJpZM4a6PJ7
.

https://github.com/twosigma/flint/pull/64
Managed these changes to get it to build under spark 2.4

Hi I am trying to follow the example from here
https://github.com/twosigma/flint/blob/master/example/Flint Example.ipynb
Here is my example
booking = spark.read.option('header', True).option('inferSchema', True).csv(CSV_FILE)
booking2 = flintContext.read.dataframe(booking)
I see the below error
TypeError                                 Traceback (most recent call last)
~/anaconda/envs/featuretool/lib/python3.7/site-packages/ts/flint/java.py in new_reader(self)
37         try:
---> 38             return utils.jvm(self.sc).com.twosigma.flint.timeseries.io.read.TSReadBuilder()
39         except TypeError:
TypeError: 'JavaPackage' object is not callable
During handling of the above exception, another exception occurred:
TypeError                                 Traceback (most recent call last)
 in
----> 1 booking2 = flintContext.read.dataframe(booking)
~/anaconda/envs/featuretool/lib/python3.7/site-packages/ts/flint/context.py in read(self)
82         '''
83
---> 84         return readwriter.TSDataFrameReader(self)
~/anaconda/envs/featuretool/lib/python3.7/site-packages/ts/flint/readwriter.py in init(self, flintContext)
49         self._sqlContext = self._flintContext._sqlContext
50         self._jpkg = java.Packages(self._sc)
---> 51         self._reader = self._jpkg.new_reader()
52         self._parameters = self._reader.parameters()
53
~/anaconda/envs/featuretool/lib/python3.7/site-packages/ts/flint/java.py in new_reader(self)
38             return utils.jvm(self.sc).com.twosigma.flint.timeseries.io.read.TSReadBuilder()
39         except TypeError:
---> 40             return utils.jvm(self.sc).com.twosigma.flint.timeseries.io.read.ReadBuilder()
41
42     @property
TypeError: 'JavaPackage' object is not callable.
I don't see in the example if flint jar has been passed in notebook but I also tried building and passing  something like below but it did not help. I am running on spark-2.4.0-bin-hadoop2.7
SUBMIT_ARGS = "
--master yarn
--deploy-mode client
--jar /Users/kkum25/Downloads/flint-master/target/flint-assembly-0.6.0-SNAPSHOT.jar
pyspark-shell"
Spark "SPARK_HOME" for local this code and children
#os.environ["SPARK_HOME"] = "/usr/lib/spark"
#os.environ["PYSPARK_PYTHON"] = "./MYENV/py3.6/bin/python"
os.environ["PYSPARK_SUBMIT_ARGS"] = SUBMIT_ARGS
findspark.init()
from pyspark.sql import SparkSession
spark = SparkSession
.builder
.appName("packages test")
.enableHiveSupport()
.getOrCreate()

I am running on spark-2.4.0-bin-hadoop2.7 and seeing the below error. Any idea ?

from ts.flint import windows
sp500_previous_day_return = sp500_return.shiftTime(windows.future_absolute_time('1day')).toDF('time', 'previous_day_return')
Traceback (most recent call last):
File "", line 1, in
File "/Users/kkum25/anaconda/envs/featuretool/lib/python3.7/site-packages/ts/flint/dataframe.py", line 1591, in shiftTime
tsrdd = self.timeSeriesRDD.shift(window._jwindow(self._sc))
File "/Users/kkum25/anaconda/envs/featuretool/lib/python3.7/site-packages/ts/flint/dataframe.py", line 154, in timeSeriesRDD
self._jdf, self._is_sorted, self._junit, self._time_column)
File "/usr/local/Cellar/apache-spark/2.2.1/libexec/python/lib/py4j-0.10.4-src.zip/py4j/java_gateway.py", line 1133, in call
File "/usr/local/Cellar/apache-spark/2.2.1/libexec/python/pyspark/sql/utils.py", line 63, in deco
return f(*a, **kw)
File "/usr/local/Cellar/apache-spark/2.2.1/libexec/python/lib/py4j-0.10.4-src.zip/py4j/protocol.py", line 319, in get_return_value
py4j.protocol.Py4JJavaError: An error occurred while calling o124.fromDF.
: java.lang.NoSuchMethodError: org.apache.spark.sql.catalyst.plans.physical.ClusteredDistribution$.apply$default$2()Lscala/Option;
at com.twosigma.flint.timeseries.TimeSeriesStore$.isClustered(TimeSeriesStore.scala:149)
at com.twosigma.flint.timeseries.TimeSeriesStore$.apply(TimeSeriesStore.scala:64)
at com.twosigma.flint.timeseries.TimeSeriesRDD$.fromDFWithPartInfo(TimeSeriesRDD.scala:509)
at com.twosigma.flint.timeseries.TimeSeriesRDD$.fromDF(TimeSeriesRDD.scala:304)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
at py4j.Gateway.invoke(Gateway.java:280)
at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
at py4j.commands.CallCommand.execute(CallCommand.java:79)
at py4j.GatewayConnection.run(GatewayConnection.java:214)
at java.lang.Thread.run(Thread.java:745)

#64
Managed these changes to get it to build under spark 2.4

before this change what issue you were seeing ?

Is there any blockers to merging this in? I'd like to use Flint on Databricks but I don't see any compatible versions of Spark being offered (2.2.1 or 2.4.[0,1,2] are only versions of Spark currently available).
I think this is a great project and would love to help mature it!

I have been successfully using @mattomatic's changes to run on Spark 2.4.

Does this library work with spark 2.2.x  version?


github asking for authentication sign in

github is asking for sign in and password while SSH into RPI. Please remove this

Please clarify your issue.
Do you mean you are SSH into a RPI and trying to perform the git clone operation to get the installation package?
When I tested that operation it worked fine for me (signed out of github).