Migrations succeed when run from beginning but not when run in parts

Environment
Knex version: 0.20.15
Database + version: Postgres 12.3 Docker Image
OS: Ubuntu 20.04
Bug
The migration is not committed to the database before Knex starts the next migration. We have a long list of migrations:
migrations/
    table/
<PHONE_NUMBER>0052_v1-1-0.js
<PHONE_NUMBER>1135_v1-2-0.js
<PHONE_NUMBER>3605_WEBAPI-3036.js
<PHONE_NUMBER>5400_WEBAPI-3039.js <-- this is the migration that isn't being committed
<PHONE_NUMBER>5845_WEBAPI-3258.js <-- this is the migration that is starting before the prior is committed
<PHONE_NUMBER>1136_WEBAPI-1481.js
<PHONE_NUMBER>2115_WEBAPI-1428.js
<PHONE_NUMBER>1244_WEBAPI-1061.js
<PHONE_NUMBER>3555_WEBAPI-1209.js
<PHONE_NUMBER>0600_WEBAPI-1465.js
<PHONE_NUMBER>4700_WEBAPI-1494.js
<PHONE_NUMBER>5242_WEBAPI-904.js
<PHONE_NUMBER>0625_WEBAPI-1531.js
<PHONE_NUMBER>2406_WEBAPI-1537.js
<PHONE_NUMBER>2424_WEBAPI-1551.js
<PHONE_NUMBER>2634_WEBAPI-1578.js
<PHONE_NUMBER>3958_WEBAPI-1102.js
<PHONE_NUMBER>0402_WEBAPI-1647.js
<PHONE_NUMBER>4436_WEBAPI-1624.js
<PHONE_NUMBER>3735_WEBAPI-1790.js
<PHONE_NUMBER>1033_WEBAPI-1945.js
<PHONE_NUMBER>1634_WEBAPI-1887.js
<PHONE_NUMBER>5254_WEBAPI-1892.js
<PHONE_NUMBER>2634_WEBAPI-2058.js
<PHONE_NUMBER>1012_WEBAPI-2062.js
<PHONE_NUMBER>4423_WEBAPI-2061.js
<PHONE_NUMBER>0245_WEBAPI-2061.js
<PHONE_NUMBER>3545_WEBAPI-2061.js
<PHONE_NUMBER>4545_WEBAPI-1894.js
<PHONE_NUMBER>0515_WEBAPI-1065.js
<PHONE_NUMBER>1933_WEBAPI-2049.js
<PHONE_NUMBER>1911_WEBAPI-2236.js
<PHONE_NUMBER>1435_WEBAPI-2233.js
<PHONE_NUMBER>4911_WEBAPI-2290.js
<PHONE_NUMBER>0510_WEBAPI-2478.js
<PHONE_NUMBER>1430_WEBAPI-2454.js
<PHONE_NUMBER>0830_WEBAPI-1107.js
<PHONE_NUMBER>2030_WEBAPI-2772.js
<PHONE_NUMBER>1930_WEBAPI-2796.js
<PHONE_NUMBER>4250_WEBAPI-2747.js
<PHONE_NUMBER>1511_WEBAPI-2866.js
<PHONE_NUMBER>4930_WEBAPI-1664.js
<PHONE_NUMBER>1722_WEBAPI-2975.js
<PHONE_NUMBER>4711_WEBAPI-2992.js
<PHONE_NUMBER>4122_WEBAPI-2741.js
<PHONE_NUMBER>5330_WEBAPI-2360.js

When this fails, we'
The database had been migrated up through<PHONE_NUMBER>1033 prior to this run using knex.migrate.latest(directory ['./migrations/table', './migrations'], sortDirsSeparately: true}). When we attempt to then migrate to<PHONE_NUMBER>5845 after a software update, the transaction for<PHONE_NUMBER>3605 is never committed prior to<PHONE_NUMBER>5400 executing and this migration fails because it relies on the prior transaction. I have debugged the migration itself and it has run. Here is the relevant log output from running the command with "DEBUG=knex:*":
Rolling the database forward...
  knex:client acquired connection from pool: __knexUid2 +4ms
  knex:query select * from information_schema.tables where table_name = ? and table_schema = current_schema() trx30 +7ms
  knex:bindings [ 'knex_migrations' ] trx30 +7ms
  knex:client releasing connection to pool: __knexUid2 +1ms
  knex:client acquired connection from pool: __knexUid2 +0ms
  knex:query select * from information_schema.tables where table_name = ? and table_schema = current_schema() trx30 +1ms
  knex:bindings [ 'knex_migrations_lock' ] trx30 +1ms
  knex:client releasing connection to pool: __knexUid2 +2ms
  knex:client acquired connection from pool: __knexUid2 +0ms
  knex:query select * from "knex_migrations_lock" trx30 +2ms
  knex:bindings [] trx30 +2ms
  knex:client releasing connection to pool: __knexUid2 +0ms
  knex:client acquired connection from pool: __knexUid2 +0ms
  knex:query select "name" from "knex_migrations" order by "id" asc trx30 +1ms
  knex:bindings [] trx30 +1ms
  knex:client releasing connection to pool: __knexUid2 +1ms
  knex:tx trx33: Starting top level transaction +30ms
  knex:client acquired connection from pool: __knexUid2 +11ms
  knex:query BEGIN; trx33 +11ms
  knex:bindings undefined trx33 +11ms
  knex:query update "knex_migrations_lock" set "is_locked" = ? where "is_locked" = ? trx33 +1ms
  knex:bindings [ 1, 0 ] trx33 +1ms
  knex:query select * from information_schema.tables where table_name = ? and table_schema = current_schema() trx33 +1ms
  knex:bindings [ 'knex_migrations' ] trx33 +1ms
  knex:query select * from information_schema.tables where table_name = ? and table_schema = current_schema() trx33 +1ms
  knex:bindings [ 'knex_migrations_lock' ] trx33 +1ms
  knex:query select * from "knex_migrations_lock" trx33 +2ms
  knex:bindings [] trx33 +2ms
  knex:query select "name" from "knex_migrations" order by "id" asc trx33 +1ms
  knex:bindings [] trx33 +1ms
  knex:query select max("batch") as "max_batch" from "knex_migrations" trx33 +1ms
  knex:bindings [] trx33 +1ms
Migration WEBAPI-3036...
  knex:query create table "data-v2" ("pkey" serial primary key, "iType" varchar(255) not null, "iId" varchar(255) not null, "iVersion" integer not null, "mGroup" integer not null default '0', "mResVer" integer not null, "mDataVer" integer not null default '0', "etag" text not null, "def" text not null) trx33 +1ms
  knex:bindings [] trx33 +1ms
  knex:query alter table "data-v2" add constraint "data_v2_itype_iid_iversion_mgroup_unique" unique ("iType", "iId", "iVersion", "mGroup") trx33 +8ms
  knex:bindings [] trx33 +8ms
	Complete!
  knex:query insert into "knex_migrations" ("batch", "migration_time", "name") values (?, ?, ?) trx33 +10s
  knex:bindings [ 2, 2020-07-15T13:17:56.842Z, '20200304143605_WEBAPI-3036.js' ] trx33 +10s
  knex:client acquired connection from pool: __knexUid8 +10s
  knex:query select * from information_schema.tables where table_name = ? and table_schema = current_schema() trx32 +7ms
  knex:bindings [ 'knex_migrations' ] trx32 +6ms
  knex:client releasing connection to pool: __knexUid8 +6ms
  knex:client acquired connection from pool: __knexUid8 +1ms
  knex:query select * from information_schema.tables where table_name = ? and table_schema = current_schema() trx32 +7ms
  knex:bindings [ 'knex_migrations_lock' ] trx32 +7ms
  knex:client releasing connection to pool: __knexUid8 +6ms
  knex:client acquired connection from pool: __knexUid8 +1ms
  knex:query select * from "knex_migrations_lock" trx32 +7ms
  knex:bindings [] trx32 +7ms
  knex:client releasing connection to pool: __knexUid8 +2ms
  knex:client acquired connection from pool: __knexUid8 +0ms
  knex:query select "name" from "knex_migrations" order by "id" asc trx32 +3ms
  knex:bindings [] trx32 +3ms
  knex:client releasing connection to pool: __knexUid8 +3ms
  knex:tx trx34: Starting top level transaction +10s
  knex:client acquired connection from pool: __knexUid8 +0ms
  knex:query BEGIN; trx34 +3ms
  knex:bindings undefined trx34 +3ms
  knex:query delete from "data-v2" where "iType" = ? trx34 +2ms
  knex:bindings [ 'environment' ] trx34 +2ms
  knex:query ROLLBACK trx34 +4ms
  knex:bindings undefined trx34 +5ms
  knex:tx trx34: releasing connection +9ms
  knex:client releasing connection to pool: __knexUid8 +10ms
migration file "20200518105400_WEBAPI-3039.js" failed
migration failed with error: delete from "data-v2" where "iType" = $1 - relation "data-v2" does not exist
  knex:query update "knex_migrations_lock" set "is_locked" = ? trx33 +4ms
  knex:bindings [ 0 ] trx33 +4ms
  knex:query ROLLBACK trx33 +3ms
  knex:bindings undefined trx33 +2ms
  knex:tx trx33: releasing connection +8ms
  knex:client releasing connection to pool: __knexUid2 +7ms

As you can see, transaction 33, which is<PHONE_NUMBER>3605, is not committed before transaction 34, which is for the NEXT migration, is started. Transaction 34 appears to be a brand new connection and cannot see the in-progress changes from transaction 33.
As a work-around, I am coding a solution that uses "knex.migrate.up" in a loop instead of "knex.migrate.latest". I will update if I find that this lets us work around the issue. Disabling transactions completely would also not work as we need transactions for many of the migrations that deal with data changes instead of table changes.
Note: in our case, the batches do not matter, so replacing latest with up works for us; anything relying on the ability to easily reverse the latest batch of migrations cannot use this work-around.

Using "knex.migrate.up" in a loop fixed this issue in our use case.

@raijinsetsu There is a migration parameter which regulates whether or not all migrations are wrapped in a single transaction. Have you tried modifying it?

Are you sure that you are handling promises within migrations correctly? Knex should not attempt to execute next migration prior to promise returned by previous migration being resolved, but if you are creating any promises within that promise that you are not awaiting yourself, they will be ignored.

@kibertoad
I am 100% certain that the promise is being handled correctly. The code INSIDE the migration has completed before the code inside the later migration is run. I added console.log calls and you can see that "    Complete!" output BEFORE the next migration is started.
Which parameter are you referencing? There is a "disableTransactions" option which does say anything about single vs. multiple transactions. It simply states that transactions will be disabled in all migrations which, as I stated earlier, is not appropriate.
The Knex debug logging CLEARLY shows that this is an issue with Knex: the transaction for the first migration is NOT committed before the transaction from the second migration is started.

PS - I cannot share the code base, but here is the first migration:
import * as Knex from 'knex';

import { commonTableSetup } from '../../util/Migrations';

export async function up(knex: Knex) {
    console.log('Migration WEBAPI-3036...');
    const textType = knex.client.config.client === 'mssql' ? 'nvarchar(max)' : 'longtext';

    await knex.schema.createTable('data-v2', (table) => {
        table.increments('pkey').primary();
        table.string('iType', 255).notNullable();
        table.string('iId', 255).notNullable();
        table.integer('iVersion').notNullable();
        table.integer('mGroup').notNullable().defaultTo(0);
        table.integer('mResVer').notNullable();
        table.integer('mDataVer').notNullable().defaultTo(0);
        table.text('etag', textType).notNullable();
        table.text('def', textType).notNullable();

        table.unique(['iType', 'iId', 'iVersion', 'mGroup']);
    });

    console.log('\tComplete!');
}

export async function down(knex: Knex) {
    await knex.schema.dropTable('data-v2');
}

The issue ended up being that one of the migrations was incorrectly managing a query stream. This lead to a transaction being used after it was committed which caused a set of cascading failures.
Closing this for now. I will re-open or re-submit if I find this to still be an issue.


docs(integrations/cloudfilter): improve docs and examples

What changes are included in this PR?

Add some docs in crate.
Fix doc build
Improve readonly example

cc @Xuanwo and @oowl , PTAL :)


JS-Controller 3.3 - wrong type "number" but has to be "string"

tankerkoenig.0 | warn | Read-only state "tankerkoenig.0.stations.8.e5.feed" has been written without ack-flag with value "0"
tankerkoenig.0 | warn | Read-only state "tankerkoenig.0.stations.8.status" has been written without ack-flag with value ""
tankerkoenig.0 | warn | Read-only state "tankerkoenig.0.stations.7.diesel.3rd" has been written without ack-flag with value "0"
tankerkoenig.0 | warn | State value to set for "tankerkoenig.0.stations.7.diesel.short" has wrong type "number" but has to be "string"
tankerkoenig.0 |warn | Read-only state "tankerkoenig.0.stations.7.diesel.combined" has been written without ack-flag with value ""
ioBroker/ioBroker.js-controller#1299

tankerkoenig.0
2021-05-01 07:20:25.168
warn
(1843) Read-only state "tankerkoenig.0.stations.9.diesel.short" has been written without ack-flag with value "0"

tankerkoenig.0
2021-05-01 07:20:25.168
warn
(1843) Read-only state "tankerkoenig.0.stations.9.diesel.feed" has been written without ack-flag with value "0"

tankerkoenig.0
2021-05-01 07:20:25.167
warn
(1843) Read-only state "tankerkoenig.0.stations.9.e10.combined" has been written without ack-flag with value ""

tankerkoenig.0
2021-05-01 07:20:25.167
warn
(1843) Read-only state "tankerkoenig.0.stations.9.e10.3rd" has been written without ack-flag with value "0"

tankerkoenig.0
2021-05-01 07:20:25.167
warn
(1843) State value to set for "tankerkoenig.0.stations.9.e10.short" has wrong type "number" but has to be "string"

tankerkoenig.0
2021-05-01 07:20:25.167
warn
(1843) Read-only state "tankerkoenig.0.stations.9.e10.short" has been written without ack-flag with value "0"

tankerkoenig.0
2021-05-01 07:20:25.166
warn
(1843) Read-only state "tankerkoenig.0.stations.9.e10.feed" has been written without ack-flag with value "0"

Hier noch ein Link zu einem sehr interessanten Kommentar zu diesen Meldungen: https://forum.iobroker.net/post/622288

i have placed a Pull-request.
If someone need a quick solution, just replace the file:
/opt/iobroker/node_modules/iobroker.tankerkoenig/main.js
with the one of my pull request:
https://github.com/anwa/ioBroker.tankerkoenig/tree/jscontroller

Fixed with the support of @anwa - new version is 2.0.11


[FEATURE] - API Host Variable

Is your feature request related to a problem? Please describe.
As the author of Secureworks Taegis XDR normalizers for Akamai products via ULS, the lack of an API host variable for use in the ULS_TCPUDP_FORMAT option complicates the insertion of a source value to indicate the cloudy origin in which events were sourced. At this time the complication is handled by manual insertion/configuration of the API host in the ULS_TCPUDP_FORMAT option string.
Example:
ULS_TCPUDP_FORMAT='{"gc_hostname": "lab01.guardicore.com", "ulsfeed": "Akamai-GC-NETLOG", "event": %s}'

Describe the solution you'd like
To reduce the configuration complexity, I desire ULS read the EDGERC host value and the value be accessible as a variable in the ULS_TCPUDP_FORMAT option.
Example:
%h = EDGERC host variable
ULS_TCPUDP_FORMAT='{"gc_hostname": %h, "ulsfeed": "Akamai-GC-NETLOG", "event": %s}'

Describe alternatives you've considered
The current alternative is manual insertion of the host value.

Additional context
Add any other context or screenshots about the feature request here.
The above FORMAT schema is being utilized by Secureworks Taegis XDR for all ULS sourced Akamai products for systematic identification and normalization providing the highest user experience.

Hi @scwxsljohnson , thanks for raising this topic.
Just before I have a closer look at this - would it be an option for you to provide the EDGERC credentials for GC via ENV vars ?
The CLI allows it and therefore we could probably fix forward pretty "easily".
Currently, there is some "strict" segmentation between the CLI part and the ULS overlay, so ULS itself doesn't have the "edgerc" data by now.
Anyhow, please correct me, if I am off here:

The data could be set at ULS start - as ULS is pinned to a single .edgerc / section by default - so "hard" setting it in the ULS_TCPUDP_FORMAT should behave identically to what we could with the proposed variable approach
If we implement this request, instead of ingesting 'edgerc' data, personally I'd tend to go down a more variable way like allowing the usage of ENV vars to be set within that FROMAT variable - would this also help on your particular issue ? We could easily provide some "script" that upfront sets any choosable ENV var to the desired value (i.e. data from the EDGERC file)

I feel i am not 100% getting the use case, you're trying to solve but maybe the above is already targeting the right direction.
Please provide me a couple more details so I am able to support on your request.
Best regards
Mike

I should have stated in my original submission. I'm utilizing Docker Compose to collect for multiple products (EAA, ETP, and GC) vs Python.
Your proposal to make the host value from the EDGERC sections available as a variable for use in the ULS_TCPUDP_FORMAT option is acceptable.
I noticed currently the host definition varies per product (gc_hostname, host, eaa_api_host). Would it be possible to utilize a common variable in the ULS_TCPUDP_FORMAT option making it easy for customers to utilize said variable?
ULS_TCPUDP_FORMAT examples utilizing %h as variable:
GC
ULS_TCPUDP_FORMAT='{"gc_hostname": %h, "ulsfeed": "Akamai-GC-NETLOG", "event": %s}'
EAA
ULS_TCPUDP_FORMAT='{"eaa_hostname": %h, "ulsfeed": "Akamai-EAA-ACCESS", "event": %s}'
ETP
ULS_TCPUDP_FORMAT='{"etp_hostname": %h, "ulsfeed": "Akamai-ETP-DNS", "event": %s}'
NOTE: The above format offers precise matching of Akamai events by Taegis XDR when transmitted by ULS.

Hi @scwxsljohnson,
I have implemented something you could try using the "development" branch (and the corresponding docker tag).
The variable representing the "API hostname" is called {api_hostname}.
I've also added this new feature to the docs: https://github.com/akamai/uls/blob/development/docs/ARGUMENTS_ENV_VARS.md#replacement-vars-in-http--tcpudp-format
Please let me know if this is helping you going forward.
Please be aware that for the rest of this week my answers might be delayed. I will pick up things again starting from next week.
Some background information:

for performance reasons, we're doing the the "transition" only once at startup - so ULS will not be able to handle dynamic changes (but I think there would be tons of additional issues when dynamically changing values in edgerc :)  )
The implementation is a bit "clunky" as the different classes handling input / output are not sharing too much of the variables, but this shouldn't bother your at all
I needed to use the input class to find the "correct" hostname value, as you actually could have a couple defined in one config section - so this is why i had to think a little loner on this topic

Let me know if this works for you
Best regards
Mike

I pulled the development branch and utilized the variable. The variable doesn't result in the host value from the EDGERC file.
CONTAINER                      REPOSITORY          TAG                 IMAGE ID            SIZE
uls-guardicore-GC-INCIDENT-1   akamai/uls          development         eafe7587a163        417MB
uls-guardicore-GC-NETLOG-1     akamai/uls          development         eafe7587a163        417MB
ULS_TCPUDP_FORMAT='{"gc_hostname": {api_hostname}, "ulsfeed": "Akamai-GC-NETLOG", "event": %s}'
TCPdump
{"gc_hostname": {api_hostname}, "ulsfeed": "Akamai-GC-NETLOG", "event": {"flow_id":

I'm utilizing containers, but here is my detail.
root@uls01:~/docker-compose/uls-guardicore# docker compose images
CONTAINER                      REPOSITORY          TAG                 IMAGE ID            SIZE
uls-guardicore-GC-INCIDENT-1   akamai/uls          development         eafe7587a163        417MB
uls-guardicore-GC-NETLOG-1     akamai/uls          development         eafe7587a163        417MB
root@uls01:~/docker-compose/uls-guardicore# docker compose logs
GC-NETLOG-1  | {"dt": "2024-03-25T15:31:36.244391", "uls_product": "GC", "uls_feed": "NETLOG", "uls_output": "TCP", "uls_version": "1.7.3", "uls_runtime": 300, "event_count": 17, "event_count_interval": 17, "event_ingested_interval": 17, "event_bytes_interval": 37038, "event_rate": 0.06, "mon_interval": 300}
GC-INCIDENT-1  | {"dt": "2024-03-25T15:31:36.191944", "uls_product": "GC", "uls_feed": "INCIDENT", "uls_output": "TCP", "uls_version": "1.7.3", "uls_runtime": 300, "event_count": 1, "event_count_interval": 1, "event_ingested_interval": 1, "event_bytes_interval": 3032, "event_rate": 0.0, "mon_interval": 300}

Aha! Pulled the image again and I now have the expected outcome.
root@uls01:~/docker-compose/uls-guardicore# docker compose images
CONTAINER                      REPOSITORY          TAG                 IMAGE ID            SIZE
uls-guardicore-GC-INCIDENT-1   akamai/uls          development         2ef41f39fa98        417MB
uls-guardicore-GC-NETLOG-1     akamai/uls          development         2ef41f39fa98        417MB
{"api_host": "labxx-x.xx.guardicore.com", "ulsfeed": "Akamai-GC-NETLOG", "event":

Wohoooo ;)
This is great. Please let me know if this does, what you've expected it to do - Feel free to close the ticket as well, it will find its way into the latest release coming up, soon !

From end-to-end I'm producing the desired outcome.
A side note. The documentation for the enhancement is missing double quotes around the variable. I did not catch this until I compared your latest update with what I had scraped from the docs. - https://github.com/akamai/uls/blob/development/docs/ARGUMENTS_ENV_VARS.md#replacement-vars-in-http--tcpudp-format

Brilliant, I am happy this is working as expected.
You're correct the example generates invalid JSON - thx for the hint. I have fixed it in the testing branch on our repo - will likely jump towards "devel" within the next couple of days.

Thanks @MikeSchiessl ! I took the liberty or reworking the doc section a little bit beyond just the syntax fix.

I have just merged the latest developments into "Devel"
For your interest:
I have added 2 additional variables (ULS FEED and ULS INPUT) - so you can use that also as a dynamic variable in the output generation ;)
Best
Mike

Final format I'll be utilizing:
ULS_TCPUDP_FORMAT='{"api_host": "{api_hostname}", "ulsfeed": "Akamai-{uls_input}-{uls_feed}", "event": %s}'

hi @scwxsljohnson , we just released ULS v1.7.3 which includes your change request + an OS ENV VAR substitution in addition to your request.


Apply echo decay after first repeat.

Allow independent control of the mix level of audiodelays.Echo and the decay amount by applying echo decay only when inserting echo value back into echo_buffer.

After experimenting with BlockInput on delay_ms, I quickly discovered that it causes undesirable glitching in the audio. The only solution I could think of to fix this problem is to change the echo buffer paradigm. What I've done is implemented a buffer position and rate with 8 "sub-bits". When delay_ms is updated, only the rate of iteration over the echo_buffer (with 8 additional bits of precision) is changed. During the write process, all words between the current position and the next position are written to using a basic loop.
NOTE: In the process, I did remove the separate buffer read & write positions because I didn't think they were necessary for this application.
Here is an example script to test this new feature out. You will here pitch shifting as the rate of playback over the echo buffer changes. This is normal and only affects the output when delay_ms is altered.
import board, audiobusio, audiomixer, synthio, audiodelays, time

CHANNELS = 1
SAMPLE_RATE = 44100
MAX_DELAY = 100

i2s_bclk, i2s_lclk, i2s_data = board.GP19, board.GP20, board.GP21
audio = audiobusio.I2SOut(bit_clock=i2s_bclk, word_select=i2s_lclk, data=i2s_data)
mixer = audiomixer.Mixer(voice_count=1, channel_count=CHANNELS, sample_rate=SAMPLE_RATE, buffer_size=2048)
audio.play(mixer)

synth = synthio.Synthesizer(channel_count=CHANNELS, sample_rate=SAMPLE_RATE)
synth.envelope = synthio.Envelope(attack_time=0.02, attack_level=0.75, sustain_level=0.5, release_time=0.1)

lfo = synthio.LFO(scale=MAX_DELAY/4, offset=MAX_DELAY*3/4, rate=0.75)
synth.blocks.append(lfo)

echo1 = audiodelays.Echo(max_delay_ms=MAX_DELAY, delay_ms=lfo, decay=0.5, mix=0.5, sample_rate=SAMPLE_RATE, channel_count=CHANNELS)
echo1.play(synth)
mixer.voice[0].play(echo1)

while True:
    for notenum in (60, 64, 67, 71): # C4, E4, G4, B4
        synth.press(notenum)
        time.sleep(0.1)
        synth.release(notenum)
        time.sleep(0.15)

In addition, you can actually achieve delays far beyond the max_delay_ms parameter at the expensive of audio quality. This could be useful for users who have limited ram or are intentionally looking for a "lo-fi" delay sound. In the example below, we're getting 50x the allocated delay (10ms to 500ms). I do hear some pitch shifting during this process, so there is likely room for improvement.
import board, audiobusio, audiomixer, synthio, audiodelays, time

CHANNELS = 1
SAMPLE_RATE = 44100

i2s_bclk, i2s_lclk, i2s_data = board.GP19, board.GP20, board.GP21
audio = audiobusio.I2SOut(bit_clock=i2s_bclk, word_select=i2s_lclk, data=i2s_data)
mixer = audiomixer.Mixer(voice_count=1, channel_count=CHANNELS, sample_rate=SAMPLE_RATE, buffer_size=2048)
audio.play(mixer)

synth = synthio.Synthesizer(channel_count=CHANNELS, sample_rate=SAMPLE_RATE)
synth.envelope = synthio.Envelope(attack_time=0.02, attack_level=0.75, sustain_level=0.5, release_time=0.1)

echo1 = audiodelays.Echo(max_delay_ms=10, delay_ms=500, decay=0.75, mix=0.75, sample_rate=SAMPLE_RATE, channel_count=CHANNELS)
echo1.play(synth)
mixer.voice[0].play(echo1)

synth.press(60)
time.sleep(0.1)
synth.release(60)

# Example of manual modification of `delay_ms`
time.sleep(2)
echo1.delay_ms = 400

I know this is may be against your initial plan, @gamblor21, but I think these minor changes allow this effect to be much more powerful.

As promised, here's an example of chorus using this effect. I have to admit, it's very tricky to get just right.
import board, audiobusio, audiomixer, synthio, audiodelays, time

CHANNELS = 1
SAMPLE_RATE = 44100
MAX_DELAY = 80
DEPTH = 0.25
RATE = 0.5
MIX = 0.6

i2s_bclk, i2s_lclk, i2s_data = board.GP19, board.GP20, board.GP21
audio = audiobusio.I2SOut(bit_clock=i2s_bclk, word_select=i2s_lclk, data=i2s_data)
mixer = audiomixer.Mixer(voice_count=1, channel_count=CHANNELS, sample_rate=SAMPLE_RATE, buffer_size=2048)
audio.play(mixer)

synth = synthio.Synthesizer(channel_count=CHANNELS, sample_rate=SAMPLE_RATE)
synth.envelope = synthio.Envelope(attack_time=0.02, attack_level=0.5, sustain_level=0.5, release_time=0.1)

lfo = synthio.LFO(scale=MAX_DELAY*DEPTH/2, offset=MAX_DELAY*(1-DEPTH/2), rate=RATE)
synth.blocks.append(lfo)

echo1 = audiodelays.Echo(max_delay_ms=MAX_DELAY, delay_ms=lfo, decay=0.0, mix=MIX, sample_rate=SAMPLE_RATE, channel_count=CHANNELS)
echo1.play(synth)
mixer.voice[0].play(echo1)

for notenum in (60, 64, 67, 71): # C4, E4, G4, B4
    synth.press(notenum)

while True:
    print("on")
    echo1.mix = MIX
    time.sleep(4)
    print("off")
    echo1.mix = 0
    time.sleep(4)


Removed http/https from external google styles to prevent mixed content warning

Simply removed http/https so it will default to the current scheme.

This change is

Thanks!


THRIFT-5341: Fix Old-Style-Cast, Missing override and Possible noexcept

THRIFT-5341
The changes were made over suggestions of GCC while compiling with the flags  -Wsuggest-override and -Wold-style-cast.
The Old-Style-Cast summed down to enumerations and a double, whereas override was added to all the functions that are inherited/extended the associated base.
plus, variables in a class Constructor changed into each in it's own line, as such:
class Schema : public virtual ::apache::thrift::TBase {
 public:

  Schema(const Schema&);
  Schema& operator=(const Schema&);
  Schema() : cid(0),
             col_name(),
             col_seq(static_cast<KeySeq::type>(0)),
             col_type(static_cast<ColumnType::type>(0)),
             cell_versions(0),
             cell_ttl(0),
             blk_encoding(static_cast<EncodingType::type>(0)),
             blk_size(0),
             blk_cells(0),
             cs_replication(0),
             cs_size(0),
             cs_max(0),
             log_rollout_ratio(0),
             log_compact_cointervaling(0),
             log_fragment_preload(0),
             compact_percent(0),
             revision(0) {
  }

The same class Schema by Thrift-0.13.0 was https://github.com/kashirin-alex/swc-db/blob/414326d40f72c9e4f054e475dbd31fffe932df4d/src/thrift/swcdb/thriftgen-0.13.0/gen-cpp/Service_types.h#L352

with the checks being cancelled/errored (null-rsp. on travis)
At this point I know of one build test with success https://ci.appveyor.com/project/ApacheSoftwareFoundation/thrift/builds/37486127
How should the checks to be proceed/queued ? (I don't have any additional commits/changes for this pull-request - to queue a new build check)

If a CI fails sometimes it helps to force push to trigger a new CI run.
With regard to reviewing this I would recommend to the C++ people.around.

Dear @kashirin-alex , the changes you did look valuable, thanks for this contribution! I would like to review them more thoroughly but its not a trivial change and I would need significantly more time for a good review. If somebody else can jump in thats great, otherwise I may find some time in the next months (but not in the short term, sorry).

@emmenlau , Yes it does stand as relaxed-changes, eg. -Wno-error=* https://github.com/kashirin-alex/swc-db/blob/3c910bee2f9c5eaa4a737638827fdc0400debd1c/CMakeLists.txt#L74
Thanks

True cause of the error (type-ptr-N was equal to already checked fields-members type-ptr-N )

https://travis-ci.org/github/apache/thrift/jobs/757915515#L13765

lib/cpp/# make test:
100% tests passed, 0 tests failed out of 25
Total Test time (real) = 191.02 sec

Is this ready for merge or still in progress?

You can have additional insight on the changes there are with this commit:
https://github.com/kashirin-alex/swc-db/commit/8b200d1ce763f865b403999cd51fb8c7cf3913be

@emmenlau could you have a look at this & possibly merge if all is fine?


Unnecessary line breaks

refmting
let equal i1 i2 =>
  /* most unlikely first */
  i1.stamp === i2.stamp && i1.kind === i2.kind && name_equal i1.name i2.name;

breaks the last line unnecessarily (using default width of 100)

Is this okay to close, or is there work to include this change to refmt?

Fixed.


"Skipped X levels in 5s" is showing double the actual number of levels

Level jumps of 46K are showing up as 92K, 10K jumps are shown as 20K, etc.
I tried stepping through the logic, but I couldn't immediately identify the issue. Would it be simpler to store the last 5 level numbers in memory as opposed to the last 5 level_jumps? Then we could calculate the jump simply by subtracting the 5-ticks-ago level from the current level.

That was an approach I was thinking of but I couldn't be arsed figuring out how to program it. Currently however, it shows the last 5 jumps instead of the last 5 seconds. If it is somehow doubled something went wrong.
I'll try to change it around a bit tomorrow to give more useful, detailed info.

I'm not sure how to check what the proposed fix was but this issue is a simple one: the function getLevelsSkipped() alters the array for no reason by moving the values in the 0, 1, 2 positions up one and adds them to a counter. This destroys the last value of the array and puts the first value in both the 0 and 1 positions. Then it adds the zero position again, doubling it in the total, which is why the skipped levels are about twice what they should be. The place where the function is called has no reason to alter the array like this since it's only supposed to return the sum of the last four level skips. It should probably be changed to something like
function getLevelsSkipped() {
var total = 0;
for (var i = 0; i < levelsSkipped.length-1; i++) {
total += levelsSkipped[i];
}
total += levelsSkipped[0];
return total;
}
While we're looking at the levelsSkipped code, in the main loop the for loop starts at i=4, which is two spots beyond where it should. The first iteration of the loop would be:
levelsSkipped[5] = levelsSkipped[4];
but 3 is the largest index of the array.

Another note: calling the total displayed in the breadcrumbs the levels skipped in the last 5 seconds is a bit misleading. It's really the last 4 level jumps, no matter how long ago they occurred.
For the last five seconds there should probably be an array that stores the level number for each of the last 5 seconds and getLevelsSkipped() would just take the difference between the first and last.

I just added a commit to my pull request that makes it a bit less misleading.  Skipped level count would now drop to 0 if you sat in a fight for more than ~5 seconds. I can't guarantee that it updates exactly every second, but it seems fairly close to me.

I think I see what the (old) code is supposed to be doing now. I'm not entirely sure, but I suspect it would be a lot easier just to keep an array of levels. The main loop would push the current level onto the array, n+1 elements to keep track of the last n seconds. Then getSkippedLevels would just take the difference between the first and last elements of the array to get the level skip count.

RIP skipped levels, you were never meant to be put on this world.


clean up intro

@squillace  90 second rewrite.

Hi @Rick-Anderson, I'm your friendly neighborhood Microsoft Pull Request Bot (You can call me MSBOT). Thanks for your contribution!

It looks like you're a Microsoft contributor (Rick Anderson). If you're full-time, we DON'T require a Contribution License Agreement. If you are a vendor, please DO sign the electronic Contribution License Agreement. It will take 2 minutes and there's no faxing! https://cla.microsoft.com.

TTYL, MSBOT;

@Rick-Anderson : Thanks for your contribution to the Azure documentation! The author, @azdocprs, has been notified to review your proposed change.

@squillace Hi Ralph Please review this contribution. Thanks.

@jomolnar LGTM! Thanks @Rick-Anderson. #sign-off


Copy over content profiles during expansion

Travis is failing because it requires different version of shr-models which isn't linked in yet.

Travis CI is experiencing an outage, but I've verified this branch locally.


NotImplementedError: No operator found for memory_efficient_attention_forward with inputs

I'm a beginner to try unsloth. I run the free notebook Llama 3 (8B), and then got the following error:

I also encountered the following error during the first installing step:
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
xformers 0.0.26.post1 requires torch==2.3.0, but you have torch 2.2.1+cu121 which is incompatible.

facing the same issue , I have no idea why.
the cell was running just a couple of days ago.

facing the same issue

@rantianhua @tariksghiouri @jamiehughes5926 Apologies on the issue! I should have wrote it here - please update the first cell's installatation instructions from
%%capture
import torch
major_version, minor_version = torch.cuda.get_device_capability()
# Must install separately since Colab has torch 2.2.1, which breaks packages
!pip install "unsloth[colab-new] @ git+https://github.com/unslothai/unsloth.git"
if major_version >= 8:
    # Use this for new GPUs like Ampere, Hopper GPUs (RTX 30xx, RTX 40xx, A100, H100, L40)
    !pip install --no-deps packaging ninja einops flash-attn xformers trl peft accelerate bitsandbytes
else:
    # Use this for older GPUs (V100, Tesla T4, RTX 20xx)
    !pip install --no-deps xformers trl peft accelerate bitsandbytes
pass

to
%%capture
# Installs Unsloth, Xformers (Flash Attention) and all other packages!
!pip install "unsloth[colab-new] @ git+https://github.com/unslothai/unsloth.git"
!pip install --no-deps "xformers<0.0.26" trl peft accelerate bitsandbytes

thank you Daniel  :)

the following change suggested is not working on my aws notebook instance with 1 A10G GPU.

the following change suggested is not working on my aws notebook instance with 1 A10G GPU. the error persists.

for my case just following daniel's suggestion I was able to solve the issue.

@Kishore-bluekyte Oh A10G hmm :( Ok thats very weird - what's your Pytorch version?

@Kishore-bluekyte Oh A10G hmm :( Ok thats very weird - what's your Pytorch version?

I am also facing the same issue with A10G on aws notebook, my Pytorch version is currently on 2.1.0

I am also facing the same issue on my Kaggle Notebook. My Pytorch version is 2.1.2 and I am following suggestion mentioned by @danielhanchen to install Unsloth.

Hi guys just bumping the thread to see if anyone is still having this issue? Hopefully not but please let us know if you are - and please update Unsloth to the latest version.


Added CourseDailyMetrics

Added the model, serializer, filter, view and tests for each
Added CourseDailyMetrics to the admin interface

@johnbaldwin Would you mind squashing the commits and pushing again?

Codecov Report

:exclamation: No coverage uploaded for pull request base (john/site-daily-metrics@09826d6). Click here to learn what that means.
The diff coverage is 93.78%.

@@                    Coverage Diff                     @@
##             john/site-daily-metrics      #13   +/-   ##
==========================================================
  Coverage                           ?   93.64%
==========================================================
  Files                              ?       25
  Lines                              ?      614
  Branches                           ?        0
==========================================================
  Hits                               ?      575
  Misses                             ?       39
  Partials                           ?        0

Impacted Files
Coverage Δ

edx_figures/admin.py
0% <0%> (ø)

edx_figures/urls.py
0% <0%> (ø)

edx_figures/views.py
96.36% <100%> (ø)

...dx_figures/migrations/0002_course_daily_metrics.py
100% <100%> (ø)

tests/views/test_course_daily_metrics_view.py
100% <100%> (ø)

edx_figures/serializers.py
100% <100%> (ø)

tests/helpers.py
100% <100%> (ø)

edx_figures/filters.py
100% <100%> (ø)

tests/factories.py
95% <100%> (ø)

tests/test_models.py
91.17% <86.95%> (ø)

... and 3 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 09826d6...aa5dc1f. Read the comment docs.


Add spell check with CSpell

Spell check all files with CSpell. Should prevent typos, which can be cumbersome for both parties during code review.

I've integrated the spell check into ESLint as discussed. Please review again.


Update whisper.cpp to 8e46ba8

See full diff at https://github.com/ggerganov/whisper.cpp/compare/a5defbc1b98bea0f070331ce1e8b62d947b0443d...ggerganov:whisper.cpp:8e46ba80d3c1dcf532a0029f9bcdf99ce9ce7d40
Closes #70, #78, #80
Big features:

Adds OpenVINO support
Adds tdrz support

Will not be merged until we get a stable release to fix to.

Looking at recent changes to whisper.cpp, it feels like this should be updated sooner rather than later. IMO the biggest "feature" is actually that it appears there were important fixes to the log_mel_spectrogram generation in whisper.cpp which improves WER for the smaller models significantly (https://github.com/ggerganov/whisper.cpp/pull/1148).

Figured I'd point out something possibly concerning for those who want to use this branch: https://github.com/ggerganov/whisper.cpp/pull/1220#issuecomment-1699944669

These changes seem to be fine when using normal models, but quantized models produce broken output. For example, the jfk speech outputs only exclamation marks from my testing. I haven't changed any parameters.

It may be worth doing some local testing before upgrading whisper-rs, just in case this affects you.

OpenVINO support is broken: see ggerganov/whisper.cpp#1289

I think I'm gonna pull out the OpenVINO support into a separate PR (given it's completely borked right now), merge everything else, and then push a prerelease to crates.io since upstream seems to be taking a bit to merge stuff.

I think I'm gonna pull out the OpenVINO support into a separate PR (given it's completely borked right now), merge everything else, and then push a prerelease to crates.io since upstream seems to be taking a bit to merge stuff.

That would be fantastic :)
As it seems that the performance boost is huge for both inference speed and WER! Esp the smaller models!

0.9.0-rc.1 has been published to crates.io for anyone who would like to use this branch.

Attempted to give 0.9.0-rc.1 a whirl but ran into a cmake error:
CMake Error: File .cargo/registry/src/index.crates.io-6f17d22bba15001f/whisper-rs-sys-0.7.0/whisper.cpp/ggml-metal.metal does not exist. CMake Error at CMakeLists.txt:149 (configure_file):     configure_file Problem configuring file

@cenzovit Do a cargo update, I released -sys version 0.7.1 which adds ggml-metal.metal to the list of included files.

Hmm, tried to update today and running into a new cmake error:
  CMake Error at CMakeLists.txt:462 (add_library):
    Cannot find source file:

      ggml-metal.m

    Tried extensions .c .C .c++ .cc .cpp .cxx .cu .mpp .m .M .mm .ixx .cppm
    .ccm .cxxm .c++m .h .hh .h++ .hm .hpp .hxx .in .txx .f .F .for .f77 .f90
    .f95 .f03 .hip .ispc

  CMake Error at CMakeLists.txt:462 (add_library):
    No SOURCES given to target: whisper

Only macOS machine I have is a late 2012 MBP running 10.15, but should be recent enough to support whisper.cpp's Metal. Will give it a shot on there.

Should be fixed in -sys v0.7.2, compiles on macOS 10.15

Got further this time, now running into a clang error on compile:
 Undefined symbols for architecture x86_64:
            "_MTLCopyAllDevices", referenced from:
                _ggml_metal_init in libwhisper_rs_sys-3c0021a7bf9aa3b1.rlib(ggml-metal.m.o)
            "_MTLCreateSystemDefaultDevice", referenced from:
                _ggml_metal_init in libwhisper_rs_sys-3c0021a7bf9aa3b1.rlib(ggml-metal.m.o)
            "_OBJC_CLASS_$_MTLComputePassDescriptor", referenced from:
                objc-class-ref in libwhisper_rs_sys-3c0021a7bf9aa3b1.rlib(ggml-metal.m.o)
          ld: symbol(s) not found for architecture x86_64
          clang: error: linker command failed with exit code 1 (use -v to see invocation)

This one may be an issue in whisper.cpp though?

That does seem like a whisper.cpp error since all we do is pass it off to upstream. Unless there's another missing file I don't think it's something we can fix.

Maybe related: https://github.com/ggerganov/whisper.cpp/issues/1344

I wonder if other language bindings are seeing similar issues with the latest whisper.cpp

Further notes, if I link the missing frameworks myself (which I think are suppose to be getting linked by ldflags in the whisper.cpp makefile) via config.toml or my build.rs -- for example in build.rs:
    println!("cargo:rustc-link-lib=framework=Foundation");
    println!("cargo:rustc-link-lib=framework=Metal");
    println!("cargo:rustc-link-lib=framework=MetalKit");

I can get my app to build. However, at runtime I run into a different issue as whisper goes to initialize:
ggml_metal_init: allocating
ggml_metal_init: found device: AMD Radeon Pro 5500M
ggml_metal_init: found device: Intel(R) UHD Graphics 630
ggml_metal_init: picking default device: AMD Radeon Pro 5500M
ggml_metal_init: loading '(null)'
ggml_metal_init: error: Error Domain=NSCocoaErrorDomain Code=258 "The file name is invalid."

Which seems to be occurring because ggml can't find the path to metallib: https://github.com/ggerganov/ggml/blob/6549d12f2e3176050040a86334f17c001e170f13/src/ggml-metal.m#L200C8-L200C8
At this point I am trying to figure out how to just build whisper.cpp without metal via the WHISPER_NO_METAL env variable; however, I can't seem to figure out how to have it set properly at the time whisper.cpp is being compiled.

It should just be as simple as setting it for the cargo build command I would think.

That's what I would have thought, but when I do that it still tries to initialize metal during runtime and fails with the same issue above

If you have a local copy you could probably just edit it out of the Makefile as a temporary workaround, but we'd need to find something else for real use.

Oh I was able to hack macOS 14 onto this system so I can give it a shot again and see as well. Will do that now.

Is there a chance that when whisper-rs-sys runs cmake on whisper.cpp that there is a metal file that needs to be copied to out as well (similar to coreml etc)? Separately, would it be possible to make metal configurable via a feature similar to how coreml is configured?

With macOS 14 I can indeed reproduce this issue. However I get some extra missing lib errors that it appears you didn't get

Full error
= note: ld: Undefined symbols:
            _MTLCopyAllDevices, referenced from:
                _ggml_metal_init in libwhisper_rs_sys-5632026198159c2d.rlib[17](ggml-metal.m.o)
            _MTLCreateSystemDefaultDevice, referenced from:
                _ggml_metal_init in libwhisper_rs_sys-5632026198159c2d.rlib[17](ggml-metal.m.o)
            _OBJC_CLASS_$_MTLComputePassDescriptor, referenced from:
                 in libwhisper_rs_sys-5632026198159c2d.rlib[17](ggml-metal.m.o)
            _OBJC_CLASS_$_NSBundle, referenced from:
                 in libwhisper_rs_sys-5632026198159c2d.rlib[17](ggml-metal.m.o)
            _OBJC_CLASS_$_NSObject, referenced from:
                _OBJC_CLASS_$_GGMLMetalClass in libwhisper_rs_sys-5632026198159c2d.rlib[17](ggml-metal.m.o)
            _OBJC_CLASS_$_NSString, referenced from:
                 in libwhisper_rs_sys-5632026198159c2d.rlib[17](ggml-metal.m.o)
            _OBJC_METACLASS_$_NSObject, referenced from:
                _OBJC_METACLASS_$_GGMLMetalClass in libwhisper_rs_sys-5632026198159c2d.rlib[17](ggml-metal.m.o)
                _OBJC_METACLASS_$_GGMLMetalClass in libwhisper_rs_sys-5632026198159c2d.rlib[17](ggml-metal.m.o)
            ___CFConstantStringClassReference, referenced from:
                 in libwhisper_rs_sys-5632026198159c2d.rlib[17](ggml-metal.m.o)
                 in libwhisper_rs_sys-5632026198159c2d.rlib[17](ggml-metal.m.o)
                 in libwhisper_rs_sys-5632026198159c2d.rlib[17](ggml-metal.m.o)
                 in libwhisper_rs_sys-5632026198159c2d.rlib[17](ggml-metal.m.o)
                 in libwhisper_rs_sys-5632026198159c2d.rlib[17](ggml-metal.m.o)
                 in libwhisper_rs_sys-5632026198159c2d.rlib[17](ggml-metal.m.o)
                 in libwhisper_rs_sys-5632026198159c2d.rlib[17](ggml-metal.m.o)
                ...
            __objc_empty_cache, referenced from:
                _OBJC_METACLASS_$_GGMLMetalClass in libwhisper_rs_sys-5632026198159c2d.rlib[17](ggml-metal.m.o)
                _OBJC_CLASS_$_GGMLMetalClass in libwhisper_rs_sys-5632026198159c2d.rlib[17](ggml-metal.m.o)
            _objc_autoreleasePoolPop, referenced from:
                _ggml_metal_graph_compute in libwhisper_rs_sys-5632026198159c2d.rlib[17](ggml-metal.m.o)
            _objc_autoreleasePoolPush, referenced from:
                _ggml_metal_graph_compute in libwhisper_rs_sys-5632026198159c2d.rlib[17](ggml-metal.m.o)
            _objc_enumerationMutation, referenced from:
                _ggml_metal_init in libwhisper_rs_sys-5632026198159c2d.rlib[17](ggml-metal.m.o)
            _objc_msgSend, referenced from:
                _ggml_metal_init in libwhisper_rs_sys-5632026198159c2d.rlib[17](ggml-metal.m.o)
                _ggml_metal_init in libwhisper_rs_sys-5632026198159c2d.rlib[17](ggml-metal.m.o)
                _ggml_metal_init in libwhisper_rs_sys-5632026198159c2d.rlib[17](ggml-metal.m.o)
                _ggml_metal_init in libwhisper_rs_sys-5632026198159c2d.rlib[17](ggml-metal.m.o)
                _ggml_metal_init in libwhisper_rs_sys-5632026198159c2d.rlib[17](ggml-metal.m.o)
                _ggml_metal_add_buffer in libwhisper_rs_sys-5632026198159c2d.rlib[17](ggml-metal.m.o)
                _ggml_metal_add_buffer in libwhisper_rs_sys-5632026198159c2d.rlib[17](ggml-metal.m.o)
                _ggml_metal_add_buffer in libwhisper_rs_sys-5632026198159c2d.rlib[17](ggml-metal.m.o)
                _ggml_metal_add_buffer in libwhisper_rs_sys-5632026198159c2d.rlib[17](ggml-metal.m.o)
                _ggml_metal_add_buffer in libwhisper_rs_sys-5632026198159c2d.rlib[17](ggml-metal.m.o)
                ...
            _objc_opt_class, referenced from:
                _ggml_metal_init in libwhisper_rs_sys-5632026198159c2d.rlib[17](ggml-metal.m.o)
            _objc_release, referenced from:
                _ggml_metal_free in libwhisper_rs_sys-5632026198159c2d.rlib[17](ggml-metal.m.o)
          clang: error: linker command failed with exit code 1 (use -v to see invocation)

Separately, would it be possible to make metal configurable via a feature similar to how coreml is configured?

This is what I want to do in the end to align Metal with all other acceleration layers.

I think I was able to hack together a Metal patch, will make a new branch and PR for it, you can try it out from there.

With macOS 14 I can indeed reproduce this issue. However I get some extra missing lib errors that it appears you didn't get
Full error
= note: ld: Undefined symbols:
            _MTLCopyAllDevices, referenced from:
                _ggml_metal_init in libwhisper_rs_sys-5632026198159c2d.rlib[17](ggml-metal.m.o)
            _MTLCreateSystemDefaultDevice, referenced from:
                _ggml_metal_init in libwhisper_rs_sys-5632026198159c2d.rlib[17](ggml-metal.m.o)
            _OBJC_CLASS_$_MTLComputePassDescriptor, referenced from:
                 in libwhisper_rs_sys-5632026198159c2d.rlib[17](ggml-metal.m.o)
            _OBJC_CLASS_$_NSBundle, referenced from:
                 in libwhisper_rs_sys-5632026198159c2d.rlib[17](ggml-metal.m.o)
            _OBJC_CLASS_$_NSObject, referenced from:
                _OBJC_CLASS_$_GGMLMetalClass in libwhisper_rs_sys-5632026198159c2d.rlib[17](ggml-metal.m.o)
            _OBJC_CLASS_$_NSString, referenced from:
                 in libwhisper_rs_sys-5632026198159c2d.rlib[17](ggml-metal.m.o)
            _OBJC_METACLASS_$_NSObject, referenced from:
                _OBJC_METACLASS_$_GGMLMetalClass in libwhisper_rs_sys-5632026198159c2d.rlib[17](ggml-metal.m.o)
                _OBJC_METACLASS_$_GGMLMetalClass in libwhisper_rs_sys-5632026198159c2d.rlib[17](ggml-metal.m.o)
            ___CFConstantStringClassReference, referenced from:
                 in libwhisper_rs_sys-5632026198159c2d.rlib[17](ggml-metal.m.o)
                 in libwhisper_rs_sys-5632026198159c2d.rlib[17](ggml-metal.m.o)
                 in libwhisper_rs_sys-5632026198159c2d.rlib[17](ggml-metal.m.o)
                 in libwhisper_rs_sys-5632026198159c2d.rlib[17](ggml-metal.m.o)
                 in libwhisper_rs_sys-5632026198159c2d.rlib[17](ggml-metal.m.o)
                 in libwhisper_rs_sys-5632026198159c2d.rlib[17](ggml-metal.m.o)
                 in libwhisper_rs_sys-5632026198159c2d.rlib[17](ggml-metal.m.o)
                ...
            __objc_empty_cache, referenced from:
                _OBJC_METACLASS_$_GGMLMetalClass in libwhisper_rs_sys-5632026198159c2d.rlib[17](ggml-metal.m.o)
                _OBJC_CLASS_$_GGMLMetalClass in libwhisper_rs_sys-5632026198159c2d.rlib[17](ggml-metal.m.o)
            _objc_autoreleasePoolPop, referenced from:
                _ggml_metal_graph_compute in libwhisper_rs_sys-5632026198159c2d.rlib[17](ggml-metal.m.o)
            _objc_autoreleasePoolPush, referenced from:
                _ggml_metal_graph_compute in libwhisper_rs_sys-5632026198159c2d.rlib[17](ggml-metal.m.o)
            _objc_enumerationMutation, referenced from:
                _ggml_metal_init in libwhisper_rs_sys-5632026198159c2d.rlib[17](ggml-metal.m.o)
            _objc_msgSend, referenced from:
                _ggml_metal_init in libwhisper_rs_sys-5632026198159c2d.rlib[17](ggml-metal.m.o)
                _ggml_metal_init in libwhisper_rs_sys-5632026198159c2d.rlib[17](ggml-metal.m.o)
                _ggml_metal_init in libwhisper_rs_sys-5632026198159c2d.rlib[17](ggml-metal.m.o)
                _ggml_metal_init in libwhisper_rs_sys-5632026198159c2d.rlib[17](ggml-metal.m.o)
                _ggml_metal_init in libwhisper_rs_sys-5632026198159c2d.rlib[17](ggml-metal.m.o)
                _ggml_metal_add_buffer in libwhisper_rs_sys-5632026198159c2d.rlib[17](ggml-metal.m.o)
                _ggml_metal_add_buffer in libwhisper_rs_sys-5632026198159c2d.rlib[17](ggml-metal.m.o)
                _ggml_metal_add_buffer in libwhisper_rs_sys-5632026198159c2d.rlib[17](ggml-metal.m.o)
                _ggml_metal_add_buffer in libwhisper_rs_sys-5632026198159c2d.rlib[17](ggml-metal.m.o)
                _ggml_metal_add_buffer in libwhisper_rs_sys-5632026198159c2d.rlib[17](ggml-metal.m.o)
                ...
            _objc_opt_class, referenced from:
                _ggml_metal_init in libwhisper_rs_sys-5632026198159c2d.rlib[17](ggml-metal.m.o)
            _objc_release, referenced from:
                _ggml_metal_free in libwhisper_rs_sys-5632026198159c2d.rlib[17](ggml-metal.m.o)
          clang: error: linker command failed with exit code 1 (use -v to see invocation)

When you add the libs and get past the undefined symbols errors, do you run into the same runtime issue where it can't find the path to the metal file?

Another maybe related issue (even though it is in llama.cpp)? https://github.com/ggerganov/llama.cpp/issues/1912

Interestingly in: https://github.com/ggerganov/whisper.cpp/pull/1270 there is an incomplete todo for "Can't figure out how to make the CMake build look for ggml-metal.metal in the bin folder"

whisper.cpp upstream error
ggml_metal_init: allocating
ggml_metal_init: found device: Intel HD Graphics 4000
ggml_metal_init: picking default device: Intel HD Graphics 4000
ggml_metal_init: loading '/Users/niko/whisper.cpp/ggml-metal.metal'
ggml_metal_init: loaded kernel_add                         0x600002cf0680 | th_max = 1024 | th_width =   32
ggml_metal_init: loaded kernel_add_row                     0x600002cf7600 | th_max = 1024 | th_width =   32
ggml_metal_init: loaded kernel_mul                         0x600002cf7880 | th_max = 1024 | th_width =   32
ggml_metal_init: loaded kernel_mul_row                     0x600002cfc880 | th_max = 1024 | th_width =   32
ggml_metal_init: loaded kernel_scale                       0x600002cf0c00 | th_max = 1024 | th_width =   32
ggml_metal_init: loaded kernel_silu                        0x600002cf1800 | th_max = 1024 | th_width =   32
ggml_metal_init: loaded kernel_relu                        0x600002cf7480 | th_max = 1024 | th_width =   32
ggml_metal_init: loaded kernel_gelu                        0x600002cf6e80 | th_max = 1024 | th_width =   32
ggml_metal_init: loaded kernel_soft_max                               0x0 | th_max =    0 | th_width =    0
ggml_metal_init: load pipeline error: Error Domain=CompilerError Code=2 "Compiler encountered an internal error" UserInfo={NSLocalizedDescription=Compiler encountered an internal error}
whisper_init_state: ggml_metal_init() failed
error: failed to initialize whisper context

However with whisper-rs I get a different error, regardless of features:
whisper_model_load: adding 1608 extra tokens
whisper_model_load: model ctx     =  464.68 MB
fish: Job 1, 'cargo run --example audio_trans…' terminated by signal SIGILL (Illegal instruction)

After thinking about it, chances are my error's related to trying to use Metal on a legacy Metal GPU (i5-3210M). Probably can't help further with this, as it appears to not just be an isolated case (the Photos app crashed a few days ago due to an internal shader compilation error as well).


The MSVC detection artifact is missing from release builds

Vc version / revision
Operating System
Compiler & Version
Compiler Flags
Assembler & Version
CPU

1.4.2
Windows
MSVC 19.29.30038.1
/MD /O1 /Ob1 /DNDEBUG /fp:fast
N/A
Ryzen 7 2700x

Testcase
#include <Vc/Vc>
#include <Vc/support.h>
#include <iostream>

int main() {
   return 0;
}

cmake_minimum_required(VERSION 3.10)

project (test)

set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED ON)

find_package(Vc 1.3.0 REQUIRED)

add_executable(main main.cpp)

target_compile_options(main PRIVATE "/fp:fast")

target_link_libraries(main PUBLIC Vc::Vc)

Actual Results
CMake finds and instantiates Vc::Vc, but log shows that MSVC version detection fails:
[proc] Executing command: "C:\Program Files\CMake\bin\cmake.EXE" --no-warn-unused-cli -DCMAKE_PREFIX_PATH:STRING=E:\VcDevel\install -DCMAKE_EXPORT_COMPILE_COMMANDS:BOOL=TRUE -DCMAKE_INSTALL_PREFIX:STRING=e:/VcDevelTest/install -He:/VcDevelTest/src -Be:/VcDevelTest/build -G "Visual Studio 16 2019" -T host=x64 -A x64
[cmake] Not searching for unused variables given on the command line.
[cmake] -- Selecting Windows SDK version 10.0.19041.0 to target Windows 10.0.19042.
[cmake] msvc_version.c
[cmake] c1: fatal error C1083: No se puede abrir el archivo origen: 'E:/VcDevel/install/lib/cmake/Vc/msvc_version.c': No such file or directory
[cmake] -- Detected Compiler: MSVC
[cmake] -- Detected CPU: zen
[cmake] -- Configuring done
[cmake] -- Generating done
[cmake] -- Build files have been written to: E:/VcDevelTest/build

Expected Results
Compilation and linking successful with no errors.

I had a look into why this file is actually needed and it seems it is only used to detect the last version of Visual Studio 2013 (_MSC_FULL_VER 180021114), which is the first version where Vc apparently works. If we just require Visual Studio 2015, we can eliminate the entire detection logic for _MSC_FULL_VER and just use CMake's builtin MSVC_VERSION. Given VS2015 is around for quite some time, this seems more reasonable to me.

@bernhardmgruber can I work on it?

Seems I was faster: #290. But feel free to add comments and suggestions!


Can't find an object if one of it's 'belongsTo' parents doesn't reference it anymore.

When a relation is broken, it's not even possible to find the child element anymore.
I agree that relations shouldn't get broken, but I'm not sure this is the best behaviour.
Perhaps only returning the object, without it's parent would be better.
describe('broken relation test', function() {

	var testPouch2;

	before(function() {

		var authorModel = {
			singular:'author',
			plural:'authors',
			relations:{
				books:{hasMany:'books'}
			}
		};
		var bookModel = {
			singular:'book',
			plural:'books',
			relations:{
				author:{belongsTo:'author'}
			}
		};
		var librarySchema = [authorModel, bookModel];
		testPouch2 = new PouchDb('https://user:pass@somecouch.com/library');
		testPouch2.setSchema(librarySchema);

		var john = {
			name:'John Doe',
			books: [
// Breaking the relation, 'ISBN12341234',
			'ISBN5759759'
			],
			id:'jdoe'
		};
		var book1 = {
			title:'Great book',
			id:'ISBN5759759',
			author:'jdoe'
		};
		var book2 = {
			title:'Awesome book',
			id:'ISBN12341234',
			author:'jdoe'
		};

		return testPouch2.rel.save('book', book1)
		.then(function() {
			return testPouch2.rel.save('book', book2);
		})
		.then(function() {
			return testPouch2.rel.save('author', john);
		});

	});

	it('should return at least the book that we\'re looking for', function() {
		return testPouch2.rel.find('book', 'ISBN12341234')
		.then(function(books) {
			expect(books['books'][0].id).to.equal('ISBN12341234');
// This fails because only the author and the book it references are returned.
		});
	});
});

Just curious what you're opinion is on this behaviour.
I'll try submitting a PR if I find the time to dive into the rel#find method's code.

Yeah, I believe I designed this thing such that relations have to be mutual - if they are inconsistent between parent and child, then weird stuff starts to happen. IIRC Ember Data also has this design property.
I can't think of a reasonable way to resolve it except to either say that 1) parents always win, 2) children always win, or 3) you have to keep the two in sync, and too bad if you don't. Out of curiosity, is this an issue you ran into in the wild?

Yes I'm building an object class based on PouchDB and relational-pouch. In my Obj#delete method I first removed the reference to the to-be-deleted object from its parents (belongsTo) and then wanted to actually delete it from the pouch using the find/del pattern. That's where the find failed and I got this behavior. I got it to work by doing the reverse. It also worked by deleting using the base PouchDB methods (get/deleted=true/put).
I understand this is an errory case but as I ran into it I just wanted to have your opinion. My opinion is that a find should always at least return what you're trying to find (if it's there), maybe omitting the related docs if the relation is broken.
A side question, do you know a piece of software that does what I'm trying to build, i.e. provide an object that has methods to do CRUD taking relations into account (like ActiveRecord in Rails) but with Couch/Pouch  as the data source?

On 29 Aug 2015, at 03:44, Nolan Lawson<EMAIL_ADDRESS>wrote:
Yeah, I believe I designed this thing such that relations have to be mutual - if they are inconsistent between parent and child, then weird stuff starts to happen. IIRC Ember Data also has this design property.
I can't think of a reasonable way to resolve it except to either say that 1) parents always win, 2) children always win, or 3) you have to keep the two in sync, and too bad if you don't. Out of curiosity, is this an issue you ran into in the wild?
—
Reply to this email directly or view it on GitHub.

No, relational-pouch is the only one I'm aware of. Also, I should note that CouchDB veterans will tell you that, when you're trying to store relational data in CouchDB, you need to resolve any inconsistencies at query time anyway, because e.g. what happens if there's a sync conflict in a child but not in a parent?  Or what if replication occurs, and the parent replicates before the child but a user query occurs between the two?
I admit I did not do a great job of solving these problems in this plugin - e.g. see https://github.com/nolanlawson/relational-pouch/issues/24. If you'd like to contribute the fix, though, I'd be happy to accept it. :) Looks like you've already got a test to reproduce it, which is awesome; as for how to fix it, I would lean towards whatever Ember Data does to solve this problem, because 99% of the users of this plugin are via ember-pouch.

Closing as dup of #24. This is still a serious issue btw.


where you pay only for the compute and not don't worry about licensing

Choose between not and dont't in the sentence.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 3dfc72b7-fd7c-bd67-48ac-685a5559e5d7
Version Independent ID: 61c5bfa5-05a1-1f77-4db8-5b6b86714db3
Content: Governance of Azure DevTest Labs infrastructure - Resource
Content Source: articles/lab-services/devtest-lab-guidance-governance-resources.md
Service: lab-services
GitHub Login: @spelluru
Microsoft Alias: spelluru

Hello @servisan ,
I have assigned content-author for the further review.

@servisan Thank you for bringing this to our attention!  This has been corrected, and should be live shortly.  Thank you again for improving Microsoft Docs!


Create map button stays disabled on certain cases

Steps to reproduce:

Open the dashboard or a builder map
Hit the new map button / add layer button
On the popup, upload a fil
Click on the "back" arrow that's on the dialog
Go to GDrive, or any other type of file source
Go back to a regular File
Upload a file again
The button is not enabled, but it should

Note: This is likely not related to the migration, it might've been there before, but since we're prodding around these files, it's worth fixing.

In production! 🚀


Compression algorithm and error threshold

Hi all,
I was wondering how best to include information about the compression algorithm and error threshold used for different (raster) assets of a STAC Item. We provide various annotation datasets (e.g., data mask, local incident angle, ...) with the main backscatter datasets for a Sentinel-1 product. Overall the same compression algorithm is used for all datasets, but different error thresholds are set depending on the dataset (e.g., threshold for the data mask can be higher than for backscatter datasets).
This information would be provided per asset and not for each band. I was actually not sure if the raster extension intends to include common information, because currently everything is part of a Raster Band Object, right?
Example of what I thought would be nice to have:
    "vh": {
      "href": "./measurement/s1b-iw-nrb-20180828t171422-20180828t171511-012466-016fd2-32tns-vh-g-lin.tif",
      "type": "image/tiff; application=geotiff; profile=cloud-optimized",
      "title": "VH backscatter data",
      "raster:bands": [
        {
          "data_type": "float32",
          "bits_per_sample": 32
        }
      ],
      "raster:compression": "LERC_DEFLATE",
      "raster:max_z_error": 0.001,
      "file:byte_order": "little endian",
      "file:header_size": 482308010,
      "card4l:border_pixels": 0,
      "roles": [
        "backscatter",
        "data"
      ]
    },

Following https://gdal.org/development/rfc/rfc14_imagestructure.html#rfc-14, as per bits_per_sample (NBITS), we could have the compression information in the raster asset. It is a quite common metadata.
On the other hand, do you have a source for the error threshold?

Thanks for your reply!
MAX_Z_ERROR is specific to LERC compression. More information can be found here for example. It is listed as a creation option for the GTiff driver in GDAL. The compression levels of ZSTD (ZSTD_LEVEL), DEFLATE (ZLEVEL) and other common compression methods could also be included as additional information. To my knowledge there is currently no way to retrieve this information after the raster has been created and I think it would be valuable to include somewhere in a machine-readable way.

Discussed in STAC sprint.
It's not clear the use of having compression in the metadata as there is no known use case of needing to know compression before the file is acccessed.
Closing due to inactivity,


Wait for cluster role and service account before deploying oathkeeper

Description
Changes proposed in this pull request:

Wait for resources before deploying oathkeeper

LGTM
I'm not a big fan of this waiting, but we already had to do this. It would be better to get this to self-heal during the next reconciliation. I think the cleanest solution would be that the maester never reports Readiness.
Hopefully it's only a matter of time until we get rid of it.


Removes unused svg file, closes #81

Codecov Report

:exclamation: No coverage uploaded for pull request base (main@e324a35). Click here to learn what that means.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main     #123   +/-   ##
=======================================
  Coverage        ?   70.96%
=======================================
  Files           ?       18
  Lines           ?     1250
  Branches        ?        0
=======================================
  Hits            ?      887
  Misses          ?      363
  Partials        ?        0

Flag
Coverage Δ

unittests
70.96% <0.00%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e324a35...7f22b9a. Read the comment docs.


Second Instance Directory No Permission

When I try to start the second instance which uses another direction I get the error: No Permission
18:22:24: Running /home/ARK01/Ark/ScorchedEarth/ShooterGame/Binaries/Linux/ShooterGameServer $'ScorchedEarth_P?AltSaveDirectoryName=Scorched?MaxPlayers=255\r?Port=7987?QueryPort=27215?RCONEnabled=True?RCONPort=32331?ServerAdminPassword=hulumulu?ServerPassword?SessionName=DinoSandSharks?listen' -NoTransferFromFiltering -log -server $'-clusterid=DinoSharks123\r'
/usr/local/bin/arkmanager: Zeile 667: /home/ARK01/Ark/ScorchedEarth/ShooterGame/Binaries/Linux/ShooterGameServer: Keine Berechtigung
18:22:24: Server PID: 15046
18:22:29: Bad PID ''; expected '15046'
18:22:29: exited with status 0

How can I do an correct install of a second directory? (I just copied the directory)
arkmanager.cfg
#defaultinstance="main"
configfile_TheCenter="/etc/arkmanager/instances/main.cfg"
configfile_TheCenter="/etc/arkmanager/instances/scorched.cfg"

main.cfg
arkserverroot="/home/ARK01/Ark"
serverMap="TheCenter"
ark_RCONEnabled="True"
ark_RCONPort="32330"
ark_SessionName="DinoSharks"
ark_Port="7778"
ark_QueryPort="27015"
ark_ServerPassword=""
ark_ServerAdminPassword="hulumulu"
ark_MaxPlayers="255"
ark_AltSaveDirectoryName="Center"
arkflag_NoTransferFromFiltering=true
arkopt_clusterid=DinoSharks123
arkflag_server=""
arkflag_log=""

scorched.cfg
arkserverroot="/home/ARK01/Ark/ScorchedEarth"
serverMap="ScorchedEarth_P"
ark_RCONEnabled="True"
ark_RCONPort="32331"
ark_SessionName="DinoSandSharks"
ark_Port="7888"
ark_QueryPort="27125"
ark_ServerPassword=""
ark_ServerAdminPassword="hulumulu"
ark_MaxPlayers="255"
ark_AltSaveDirectoryName="Scorched"
arkflag_NoTransferFromFiltering=true
arkopt_clusterid=DinoSharks123
arkflag_server=""
arkflag_log=""

Check the permissions on the second directory, and on the ShooterGameServer executable.
ls -l /home/ARK01/Ark/ScorchedEarth
ls -l /home/ARK01/Ark/ScorchedEarth/ShooterGame/Binaries/Linux/ShooterGameServer

Assuming ARK01 is the user under which ARK runs, you can try running the following as root:
chown -R ARK01 /home/ARK01/Ark/ScorchedEarth
chmod +x /home/ARK01/Ark/ScorchedEarth/ShooterGame/Binaries/Linux/ShooterGameServer

It also looks like you have a couple of stray carriage returns in your config file.

You pointing at 2 different cfg files for TheCenter
#defaultinstance="main"
configfile_TheCenter="/etc/arkmanager/instances/main.cfg"
configfile_TheCenter="/etc/arkmanager/instances/scorched.cfg

Removed the pointing ro configfile. Still the same Problem. I thinknot is because i created the arkserveroot Directory manually. How can I do a clean install of a Second Directory?

That ShooterGameServer does not have the Executable bit set.  You are also missing the Engine and steamapps directories.
Please try running:
arkmanager update @scorched --validate
chmod +x /home/ARK01/Ark/ScorchedEarth/ShooterGame/Binaries/Linux/ShooterGameServer

Thx did it for me


Can't get insecure repo working

Summary
I have a docker registry running in another namespace
apiVersion: v1
kind: Service
metadata:
  name: registry
spec:
  selector:
    app: registry
  ports:
    - name: registry-port
      protocol: TCP
      port: 5000
      targetPort: 5000
      nodePort: 30500
  type: NodePort

It works fine and I have no trouble pushing to it from my local box...
$ docker push <IP_ADDRESS>:30500/block-party:latest
The push refers to repository [<IP_ADDRESS>:30500/block-party]
...
latest: digest: sha256:6923ac9281f4ed9c5c2fdb1e70acf906d014a104f130f04bcc7878793f8a3479 size: 1988

But when I try to use it like
spec:
  replicas: 1
  selector:
    matchLabels:
      app: block-party
  template:
    metadata:
      labels:
        app: block-party
    spec:
      containers:
        - name: block-party
          image: <IP_ADDRESS>:30500/block-party:latest
          ports:
            - containerPort: 80

It can't get the image if I do
$ sudo microk8s.ctr images pull <IP_ADDRESS>:30500/block-party:latest
INFO[0000] trying next host                              error="failed to do request: Head \"https://<IP_ADDRESS>:30500/v2/block-party/manifests/latest\": http: server gave HTTP response to HTTPS client" host="<IP_ADDRESS>:30500"
ctr: failed to resolve reference "<IP_ADDRESS>:30500/block-party:latest": failed to do request: Head "https://<IP_ADDRESS>:30500/v2/block-party/manifests/latest": http: server gave HTTP response to HTTPS client

I have followed the instructions here
https://microk8s.io/docs/registry-private
~$ cat /var/snap/microk8s/current/args/certs.d/<IP_ADDRESS>:30500/hosts.toml
server = "http://<IP_ADDRESS>:30500"

[host."http://<IP_ADDRESS>:30500"]
capabilities = ["pull", "resolve"]

I also tried locally using registry.network.svc.cluster.local:5000 instead but I still can't get it to use the registry as insecure.
What Should Happen Instead?
As seen from the error message it is still using https instead of http
Introspection Report
inspection-report-20240823_215125.tar.gz
Can you suggest a fix?
Better documentation
Are you interested in contributing with a fix?
yes

seems similar to
https://github.com/canonical/microk8s/issues/3288
But my url matches the example

NM after restarting for the 100th time it started working so I think it was a hung process


Procedures: should string and number argument types be combined?

In Scratch 2.0 procedure arguments can be strings (%s), numbers (%n), or booleans (%b).
The current spec for Scratch 3.0 combines strings and numbers.
Advantages:

rendering is the same for both
implementation is slightly simpler

Disadvantages:

Can't indicate to a user what should be attached
Can't force the number pad to show up on mobile for number arguments

I think we are comfortable with this design decision for now, but I'd be happy to revisit this in the future. The second disadvantage you mention: "Can't force the number pad to show up on mobile for number arguments" is particularly potent for me.


Update spring-boot-starter-camunda to 8.5.8

About this PR
📦 Updates io.camunda.spring:spring-boot-starter-camunda from 8.5.7 to 8.5.8
📜 GitHub Release Notes - Version Diff
Usage
✅ Please merge!
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

🔍 Files still referring to the old version number
The following files still refer to the old version number (8.5.7).
You might want to review and update them manually.
05-examples/demos/04-c7-spring/src/main/resources/META-INF/resources/webjars/camunda/app/cockpit/scripts/instance-historic-activities.js
05-examples/demos/04-c7-spring/src/main/resources/META-INF/resources/webjars/camunda/app/cockpit/scripts/instance-route-history.js
05-examples/invoice/04-c7-spring/src/main/resources/META-INF/resources/webjars/camunda/app/cockpit/scripts/instance-historic-activities.js
05-examples/invoice/04-c7-spring/src/main/resources/META-INF/resources/webjars/camunda/app/cockpit/scripts/instance-route-history.js
05-examples/invoice/04-c8-spring/src/main/resources/META-INF/resources/webjars/camunda/app/cockpit/scripts/instance-historic-activities.js
05-examples/invoice/04-c8-spring/src/main/resources/META-INF/resources/webjars/camunda/app/cockpit/scripts/instance-route-history.js
05-examples/twitter/04-c7-spring/src/main/resources/META-INF/resources/webjars/camunda/app/cockpit/scripts/instance-historic-activities.js
05-examples/twitter/04-c7-spring/src/main/resources/META-INF/resources/webjars/camunda/app/cockpit/scripts/instance-route-history.js

⚙ Adjust future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "io.camunda.spring", artifactId = "spring-boot-starter-camunda" } ]

Or, add this to slow down future updates of this dependency:
dependencyOverrides = [{
  pullRequests = { frequency = "30 days" },
  dependency = { groupId = "io.camunda.spring", artifactId = "spring-boot-starter-camunda" }
}]

labels: library-update, early-semver-patch, semver-spec-patch, old-version-remains, commit-count:1

Superseded by #53.


Show tags for search results

Describe the feature you'd like
I'd like to be able to see all tags of a given search result. This could look something like this:

One might also consider adding either a global or a search setting for this in order to make it optional, but personally I consider this useful and unintrusive enough to have around all the time.
Describe the benefits this feature would bring to BookStack users
We use BookStack as a internal knowledge base for our organization and have important metadata stored in our tags (such as datestamps if pages are related to an event). Showing this information would make it easier to pinpoint which search results are relevant. Probably that's the case for other users too.
Additional context
This feature request should be really simple to implement, but I wanted to double-check whether it has any chance of being added before starting a PR. Also, I have no experience with Laravel/Blade, so my implementation would probably be very basic (read: no extra settings, just adding the tags). If that sounds good to you, please let me know and I'll go ahead.
On a similar note, it would be great to use this tag metadata in other contexts too such as for sorting the search results (I already mentioned that in https://github.com/BookStackApp/BookStack/issues/577).

Thanks @burnoutberni! Closing as per #2487.


Add jupyterlab-kernel-spy to the demo site

On that end, we should probably consider adding kernelspy to the demo site, and ensuring we generate content there, as losing the browser websocket playback, which just got good enough to use, is pretty crippling when you're building something and it goes wrong.
Originally posted by @bollwyvl in https://github.com/jtpio/jupyterlite/issues/195#issuecomment-871411572

Conda forge PR: https://github.com/conda-forge/staged-recipes/pull/15456


openvpn Chef-13

@flaccid does this actually just pass without errors? If so we can remove it

I'm not seeing anything atm as well, not in the openvpn cookbook itself; closing for now.


ModuleNotFoundError: No module named 'transformers_cfg.cli' When Running CLI

Description:
After installing the transformers-cfg package using pip install transformers-cfg, I'm encountering an error when attempting to run the CLI tool (transformers-cfg-cli). This behavior does not occur when the package is installed directly via the setup.py script. The error traceback is as follows:
Traceback (most recent call last):
  File "/home/gfesatidis/.miniconda3/envs/transformers-cfg/bin/transformers-cfg-cli", line 5, in <module>
    from transformers_cfg.cli.cli_main import main
ModuleNotFoundError: No module named 'transformers_cfg.cli'

Steps to Reproduce:

Install the package: pip install transformers-cfg

Attempt to run the CLI tool: transformers-cfg-cli -h

Expected Behavior:
The CLI tool should execute without errors, displaying the help message or performing the intended function.
Actual Behavior:
The CLI tool fails to run, raising a ModuleNotFoundError because the cli module is not found in the package.
Additional Information:
Upon inspecting the installed package directory (site-packages/transformers_cfg/), I noticed that the cli directory is missing. This seems to be the cause of the error.

Environment:
Python version: 3.11.9
Environment: Conda
OS: WSL Ubuntu 24.04.1 LTS

The problem was caused by a missing init.py file, which led to the cli module not being included in the distribution package. This issue has been resolved in the latest release and in commit #90
Thank you for raising this issue! :)


Copyright and Licenses for the code

Hello,
Quick question. The getting started tutorial is great, but I am just wondering what copyright and licenses are applied for the code (and materials used in the code). I would like to modify and reuse it, but I am not sure what I can do about it.
Can I use it for any purposes, or are there any restrictions? It would be highly appreciated if the code on Github shows the license information. Thank you!

All out sample code is MIT licensed. Feel free to use it.

Ok, thank you for your information.
Just one little request.
Could you quickly choose MIT license logo for this repository (to display it explicitly)?
That encourages people (like me) more to use the code (and feel much safer).
Many thanks!


minor code cleanups in layer_builder

Reading through layer_builder.rs implementation I found few minor improvements that can be made.

Might as well fix the other clippy issues in other files.

Might as well fix the other clippy issues in other files.


Task/layout setup

This gets us to a semi-functioning build. Lots of work still to be done.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 61.327%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
536

Relevant Lines:
874

💛  - Coveralls


version: bump to 4.15

now that we branched off 4.14 we can move the main branch to 4.15

/lgtm
Thanks for the PR, the CI failures look like infra issues, but leaving it for you to have a look

/lgtm Thanks for the PR, the CI failures look like infra issues, but leaving it for you to have a look

we surely need 4.15 scheduler plugin builds, which are not available because the mirroring job is broken atm: https://prow.ci.openshift.org/job-history/gs/origin-ci-test/logs/periodic-image-mirroring-openshift-kni

/retest
we have scheduler-plugin builds now

/retest

/retest


Implement fit/predict for custom models

Both fit and predict should be supported for custom models.
Separate function can be implemented to achieve it.

Done


Negative parameter values displayed as "999"

Describe the bug
Negative parameter values for both percent and typeless values displayed as "999" at leas at NTS-1
To Reproduce
Steps to reproduce the behavior:

Make custom oscillator with parameter of percent type from -100 to +100
Set parameter knob to anything less than 12 o'clock
observe '999' displayed while listen that sound actually reflects parameter changes.

Expected behavior
Should correctly display -100...100 at least for percent type values according to SDK manual.
Screenshots
If applicable, add screenshots to help explain your problem.

I also repro'd this with both a typeless and a "%" type parameter with range -12, 12.
In the case of a typeless parameter, value "-999" is shown from minimum knob position to 12 o'clock. After that, values from 0 to 12 are shown. The -999 range seems to behave the same as value 0.
If the type is "%", the behavior is similar except the value shown in the negative range is "999" (no minus sign).


Use SignService v2

This PR is to move to the new sign service. Please contact me on how to share the secrets with you.
These changes should be cherry-picked & applied to all branches you plan to build from as we want to deprecate the old service.
Thanks

Thanks @onovotny! I'll pm you to get the secrets so we can get this merged in.

I'm going to merge this without the appveyor build (can't get it to respond), but these changes will only be seen on a branch we have secrets enabled anyway.

We'll be decommissioning the old service at some point in the next month or two, so if there's other branches you plan on building from, these changes should be put there too. LMK if you run into any issues. Thx!

Sounds good. There are a couple others, so we'll get those merged into those as well


Update service-fabric-concepts-scalability.md

minor edits and updates

@masnider,
Thanks for your contribution as a Microsoft full-time employee or intern. You do not need to sign a CLA.
Thanks,
Microsoft Pull Request Bot

@masnider : Thanks for your contribution to the Azure documentation! The author, @masnider, has been notified to review your proposed change.

Microsoft authors of articles need to make updates through the private repository so the updates can be staged and validated by the current validation rules. After you review staging and fix all validation issues, you can add the sign-off comment to let the PR reviewers know the updates are ready to be merged.
It�s easy to access the private repo version of your articles in GitHub. Just add �pr to the URL, as follows:
Public repo URL: https://github.com/Azure/azure-content/blob/master/articles/azure-portal-overview.md
Private repo URL: https://github.com/Azure/azure-content-pr/blob/master/articles/azure-portal-overview.md
We've closed this pull request. Please resubmit the updates through the private repo. If you need perms to the private repo, see the contributor guide instructions.


Fix for delisted subscription data being sent

Description

Moving the if (_delisted) { break; } above the while (_subscriptionFactoryEnumerator.MoveNext()) loop. Why? The _subscriptionFactoryEnumerator isn't useful, when created, we detect that the subscription is delisted through CheckForDelisting() cause date > delistingDate. But we still need to loop once more to send the new Delisting event, but after we send it we are done and we can break (not going through _subscriptionFactoryEnumerator.MoveNext(), since it will start reading a New zip file that's after the delisting date). We still need to create the _subscriptionFactoryEnumerator since that's the way we detect that the subscription has to be delisted.
Replacing a couple of lines with an already defined method with the same behavior. Method: https://github.com/QuantConnect/Lean/blob/master/Engine/DataFeeds/SubscriptionDataReader.cs#L740

Related Issue

Closes https://github.com/QuantConnect/Lean/issues/2197
Motivation and Context

Delisted symbol should not be sent through OnData(Slice) after it has been delisted and user informed.
Requires Documentation Change

N/A
How Has This Been Tested?

Unit and regression tests. Test algorithm at https://github.com/QuantConnect/Lean/issues/2197
Types of changes

[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] Non-functional change (xml comments/documentation/etc)

Checklist:

[x] My code follows the code style of this project.
[x] I have read the CONTRIBUTING document.
[ ] I have added tests to cover my changes.
[x] All new and existing tests passed.
[x] My branch follows the naming convention bug-<issue#>-<description or feature-<issue#>-<description>

NOTE: Adding regression test for this

NOTE: This caused other regression failures to be noticed. @Martin-Molinero is creating issues for the newly discovered failing regression tests.


Store entry point arguments in fibre's own stack (s390x)

This is a backport of changes from https://github.com/boostorg/context/pull/224 and https://github.com/boostorg/context/pull/226. The issue they aim to resolve only exists on IBM Z / s390x.
In the current implementation fibres spawned by other fibres or threads would have the argument of their entry point (void fiber_entry( transfer_t t)) be allocated on the stack of their parent, which would then cause segmentation fault if the former outlived the latter.
The x86-64 implementation (x86_64_sysv_elf is the combination I tested) supports this use-case, so this change means to achieve feature parity for the s390x implementation in this regard by storing the aforementioned argument in the fibre's own stack.

@alexey-milovidov please kindly review.


NodeMCU not waking up from deep sleep normally

Hello
I've got a NodeMCU v2 doit.am with ESP12E onboard.
When I put my NodeMCU to deep sleep with command "sleep 10" (in telnet), well, it just goes to deep sleep. That's normal. But when it's time to wake up, it just sets the GPIO16 (low/high idk) which doesn't wake my ESP up, but only turns on the second onboard LED. I even tried to set the LED GPIOs manually. Set to 2, the blue LED on the ESP8266 lights up. I then set it to pin 16 and the blue LED on the NodeMCU board lighted up and stays on until I manually reset the ESP. I previously had a NodeMCU v3 board that did not had this issue.
What should I do?
Here's the serial output at baudrate 115200:

Config found and loaded
Starting Console TCP Server on port 23
mode : sta(60:??:??:??:??:??) + softAP(62:??:??:??:??:??)
add if0
add if1
dhcp server start:(ip:<IP_ADDRESS>,mask:<IP_ADDRESS>,gw:<IP_ADDRESS>)
bcn 100
scandone
state: 0 -> 2 (b0)
state: 2 -> 3 (0)
state: 3 -> 5 (10)
add 0
aid 3
cnt
~
connected with e???????????a, channel 11
dhcp client start...
connect to ssid e???????????a, bssid 50:??:??:??:??:??, channel 11
ip:192.168..**,mask:<IP_ADDRESS>,gw:192.168.?.?
ip:192.168..**,mask:<IP_ADDRESS>,gw:192.168.?.?,dns:<IP_ADDRESS>
add 1
aid 1
station: a4:??:??:??:??:?? join, AID = 1
station: a4:??:??:??:??:?? join, AID = 1
tcp_client_connected_cb(): Client connected
state: 5 -> 0 (0)
rm 0
del if0
station: a4:??:??:??:??:?? leave, AID = 1
rm 1
bcn 0
del if1
usl
disconnect from ssid e???????????a, reason 8
station: a4:71:74:3d:75:62 leave, AID = 1
enter deep sleep`

And here is serial output at 74880 baud when the ESP wakes up:

ets Jan  8 2013,rst cause:5, boot mode:(3,7)
~
ets_main.c

(these "~" are to fill up the lines)
What? Now we see!! The ESP wakes up but doesn't execute the code!

Did you connect GPIO16 and reset? This is mandatory, If you want to use
deep sleep.
Am 11. März 2018 5:25:26 nachm. schrieb Marek<EMAIL_ADDRESS>Hello
I've got a NodeMCU v2 doit.am with ESP12E onboard.
When I put my NodeMCU to deep sleep with command "sleep 10" (in telnet),
well, it just goes to deep sleep. That's normal. But when it's time to wake
up, it just sets the GPIO16 (low/high idk) which doesn't wake my ESP up,
but only turns on the second onboard LED. I even tried to set the LED GPIOs
manually. Set to 2, the blue LED on the ESP8266 lights up. I then set it to
pin 16 and the blue LED on the NodeMCU board lighted up. I previously had a
NodeMCU v3 board that did not had this issue.
What should I do?Here's the serial output at baudrate 115200:
Config found and loaded Starting Console TCP Server on port 23 mode : sta(60:::::) + softAP(62:::::) add if0 add if1 dhcp server start:(ip:<IP_ADDRESS>,mask:<IP_ADDRESS>,gw:<IP_ADDRESS>) bcn 100 scandone state: 0 -> 2 (b0) state: 2 -> 3 (0) state: 3 -> 5 (10) add 0 aid 3 cntconnected with ea, channel 11 dhcp client start... connect to ssid ea, bssid 50:::::, channel 11 ip:192.168.*.,mask:<IP_ADDRESS>,gw:192.168.. ip:192.168..**,mask:<IP_ADDRESS>,gw:192.168..*,dns:<IP_ADDRESS> add 1 aid 1 station: a4:::::** join, AID = 1 station: a4:::::** join, AID = 1 tcp_client_connected_cb(): Client connected state: 5 -> 0 (0) rm 0 del if0 station: a4:::::** leave, AID = 1 rm 1 bcn 0 del if1 usl disconnect from ssid e***********a, reason 8 station: a4:71:74:3d:75:62 leave, AID = 1 enter deep sleepAnd here is serial output at 74880 baud when the ESP wakes up:
ets Jan 8 2013,rst cause:5, boot mode:(3,7)ets_main.c
What? Now we see!! The ESP wakes up but doesn't execute the code!
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.

Looks like on my old NodeMCU the GPIO16 was connected to RST. My new one doesn't have that connection. Okay then, closing issue now.


#[source] when specific type is not known

My problem came up from trying to transform errors from .parse(). I tried something along the lines of
use thiserror::Error;
#[derive(Debug, Error)]
enum MyError {
    #[error("Failed to parse")]
    ParseError,
}

fn my_parse_function<F>(s: &str) -> Result<F, MyError>
where
    F: std::str::FromStr,
{
    s.parse().map_err(|_| MyError::ParseError)
}
fn main() {
    let i: Result<u64, MyError> = my_parse_function("1");
    let s: Result<String, MyError> = my_parse_function("something");
}

But I'd be happy to modify my variant ParseError so it can have the #[source] and therefore, I could modify my function my_parse_function like this
fn my_parse_function<F>(s: &str) -> Result<F, MyError>
where
    F: std::str::FromStr,
{
    s.parse().map_err(|e| MyError::ParseError(e))
}

But I can't figure out how to write that (because the error type is actually an associated type of FromStr, name Err, so the actual type is not specify). Is there some obvious solution I missed? It might be that what I'm trying to do is actually a flawed design (but it's not obvious to me at the moment). If it's not flawed and there is no obvious solution I missed, is this a use case likely to be supported in some way?

Those two FromStr impls in your code have different error types (u64 as FromStr vs String as FromStr) so you would need a trait object if you want to represent both as one type. Maybe like this:
#[derive(Debug, Error)]
enum MyError {
    #[error("Failed to parse")]
    ParseError(#[source] anyhow::Error),
}

fn my_parse_function<F>(s: &str) -> Result<F, MyError>
where
    F: std::str::FromStr,
    F::Err: std::error::Error + Send + Sync + 'static,
{
    s.parse().map_err(|e: F::Err| MyError::ParseError(e.into()))
}

If you don't want a trait object there, the alternative would be something like this:
use self::MyError::*;
use std::str::FromStr;
use thiserror::Error;

#[derive(Debug, Error)]
enum MyError {
    #[error("Failed to parse")]
    ParseIntError(#[source] <u64 as FromStr>::Err),
    #[error("Failed to parse")]
    ParseStringError(#[source] <String as FromStr>::Err),
}

fn my_parse_function<F>(s: &str, err: impl Fn(F::Err) -> MyError) -> Result<F, MyError>
where
    F: FromStr,
{
    s.parse().map_err(err)
}

fn main() {
    let i: Result<u64, MyError> = my_parse_function("1", ParseIntError);
    let s: Result<String, MyError> = my_parse_function("something", ParseStringError);
}

or like this:
use std::str::FromStr;
use thiserror::Error;

#[derive(Debug, Error)]
enum MyError {
    #[error("Failed to parse")]
    ParseIntError(#[from] std::num::ParseIntError),
    #[error("Failed to parse")]
    ParseStringError(#[from] std::string::ParseError),
}

fn my_parse_function<F>(s: &str) -> Result<F, MyError>
where
    F: FromStr,
    MyError: From<F::Err>
{
    s.parse().map_err(MyError::from)
}

fn main() {
    let i: Result<u64, MyError> = my_parse_function("1");
    let s: Result<String, MyError> = my_parse_function("something");
}

I see. I've heard of anyhow but never had to get into it. Now I understand a bit more what it could be used for. Also, I didn't understand the kind of use cases supported with anyhow::Error and not supported with std::error::Error. Thank you very much for this thorough answer. I hope it'll be useful not only for me :wink: I'm closing the ticket since it answer perfectly my use case.


How to handle DEPRECATION WARNING. The staging repository ID is not provided?

Hi,
I used Gradle task closeAndReleaseRepository and got the warning:
DEPRECATION WARNING. The staging repository ID is not provided. The fallback mode may impact release reliability and is deprecated. Please consult the project FAQ how it can be fixed.

What should I do? Where should I provide repository ID and what does it really mean?

The the point in FAQ to add nexus-publish-plugin to better handle a staging repository resolution.
I will keep the issue open as I plan to rework the FAQ or even apply the nexus-publish-plugin automatically in a default configuration to make it more natural in reception.

Do I really need nexus-publish-plugin? I publish artifacts with custom groovy/gradle script and it works well and use gradle-nexus-staging-plugin for closing and releasing the artifacts. I don't want to add another dependency to the project (which I'm not going to use) just to remove the warning :). Maybe I'll wait for updated FAQ.

If you are not using Travis (or anything else that sends subsequent requests via different IP addresses) and release on project at the time (+ you manually clean up old staging repositories if anything goes wrong) then GNSP should enough for you (the feature is deprecated and I don't have any intermediate plans to remove it).
However, as described in the FAQ, in [this issue[(https://github.com/marcphilipp/nexus-publish-plugin/issues/11) and in my blog post much more reliable is using GNPS together with NPP.
Btw, I rather planned to update FAQ how to start using NPP, not how to suppress the warning :).

Ok, I'll check it. Thanks!

I am using nexus-publish-plugin but I still get this whenever I execute :closeRepository. I also noticed that the nexus-publish-plugin does not pick up my nexusUsername and nexusPassword credentials. It seems as if this seamless integration does not work anymore.

@Fleshgrinder It should work. Please provide a link to your project. Or, even better, please consider migration to our (my and Marc's) new plugin which - to avoid confusion like your - integrates GSNP and NPP into one plugin - GNPP.

Here you go.
Last time I checked the plugin wasn't hosted anywhere but I see that 1.0.0 is out now. Will switch to it. 😊

1.0.0 has been released last weekend.
The configuration looks ok. Having a CI job would make it easier to analyze the output.

I'm in the process of moving from bintray to Central so all this Nexus stuff is new to me so I first want to build up a solid understanding of the workflow before moving to automation. 😉

TL;TD. This issue does not occur in gradle-nexus-publish-plugin.
A longer version. To make releasing to Maven Central even easier, I and Marc Phillip (the author of nexus-publish-plugin) combined forces to create a next generation, unified, 2-in-1 plugin - gradle-nexus-publish-plugin. It is a recommended solution, as our development effort will be put in that new plugin. See my blog post and the official migration guide.


Tomcat Activation Priority Too High

We have encountered an issue, where the tomcat services are starting too early in the init.d boot order. It appears that the instances have an activation priority of 50, and we would like to define that attribute.

We're also seeing this. Some of our Tomcat applications load data into caches at startup which can take up to 10 minutes. This is resulting in ssh being unresponsive for a long time after reboot since process starting is blocked on Tomcat and sshd starts after it.

I found a manual solution via https://serverfault.com/questions/754676/how-to-change-the-startup-order-of-linux-init-scripts-in-redhat-centos. Adding a chkconfig line to the init script w/desired start/stop priorities, then running chkconfig  reset, updates the priority. Still evaluating this cookbook to see if there's a way to set this via attribute.


Pagination works in explorer nodes when there is no data

Screencast from 12-06-2023 11:20:57 AM.webm

verified on devnet
version : f530d72
https://github.com/threefoldtech/tfgrid-sdk-ts/assets/110984055/49d0ed2c-099d-4de2-bfcc-0adc6cbed60f
https://app.testlodge.com/a/26076/projects/40893/suites/234374?expand_section=426661#case_4250318