Proposal: only enforce blank line separation between functions and classes at the same level of nesting

Black insists that function and class definitions should be separated by blank lines. Good rule! However, I think it's slightly overenthusiastic.
For functions at the same level of nesting, this is a good idea:
 def foo():
     pass
+
 def bar():
     pass

And it's not just between two functions, but between any start/end of a function and surrounding code:
 FOO = 1
+
 def foo():
     pass
+
 BAR = 2

But! This seems unnecessary – and dare I say, ugly and less readable – when the adjacent line of code is on a different nesting level:
    if clogged_stream_maker is not None:

        async def flipped_clogged_stream_maker():
            return reversed(await clogged_stream_maker())

    else:
        flipped_clogged_stream_maker = None

I claim that the above snippet (source would be better without any blank lines.
The same applies to classes, like, how does this blank line help anything? (source
 def test_wrap_non_iobase():
+
     class FakeFile:

         def close(self):  # pragma: no cover
             pass

The intuition here is that blank lines are separators, not headers/footers. And you only need separation between items at the same level of nesting. We don't insist on using blank lines to separate a class or def from the beginning or end of the file; similarly, we shouldn't insist on using them to separate a class or def from the beginning or end of a block.
This would also make black's handling of def and class more consistent with its handling of import. Currently black does use the rule I suggest for blank lines after import statements:
# Black insists on a blank line after the import here:
import foo

BAR = 1

# But not here:
try:
    import foo
except ImportError:
    pass

This annoys me because I have a codebase with lots of functions whose first order of business is to define a nested function:
async def foo():  # black wants to insert a blank line after this line
    def bar():
        ...

    return await run_sync_in_worker_thread(bar)

Note that making this change would not introduce churn into existing codebases, because black already preserves blank lines. It would just introduce fewer unnecessary blank lines in the future.

I support this, @ambv what do you think?

I'm not convinced.

The same applies to classes, like, how does this blank line help anything?
Well, I remember a certain gentleman fight really hard back when Black was removing unnecessary vertical whitespace within function bodies ;-)

An inner class definitely fits the description of "logical sections within a function" that PEP 8 talks about:

Use blank lines in functions, sparingly, to indicate logical sections.

I like that Black is doing now because it makes it painfully obvious where scope boundaries are. I agree it's aesthetically unpleasant but it's consistent with module scope which is also code that runs from top to bottom, just like a function body.
I don't care much about the case when an inner def or class happens right at the beginning of a function (although it stops looking weird if you'd have a docstring!). But what I think we need to keep for readability is the blank line after an inner function or class. Again, scope boundaries.
Well, maybe except for the case when the following line has a lesser indentation level than the inner function/class. In other words, yes:
def outer():
    def inner():
        return False

    some = other * code

No:
def outer():
    try:
        def inner():
            return False
    except E as e:
        ...

I don't care much about the case when an inner def or class happens right at the beginning of a function (although it stops looking weird if you'd have a docstring!). But what I think we need to keep for readability is the blank line after an inner function or class. Again, scope boundaries.

Well, maybe except for the case when the following line has a lesser indentation level than the inner function/class.

... Did you just argue yourself around to agreeing with me? :-)
The proposal is exactly to keep the blank line, except in cases where the adjacent code has a lesser indentation.
Oh, I guess there is one case where that's different than what I originally said:
if ...:
    ...
def ...

According to how I originally phrased this ("different level of indentation") this would not need a blank line; according to how I phrased it now ("lesser level of indentation"), this does need a blank line. I actually meant "lesser" all along, I just forgot that this case existed so didn't realize it mattered which one I said :-). (At the bottom of a function/class this ambiguity can't happen, because only a lesser indentation can close the function/class.)

(Similarly to #449, I want the rule that we'll implement to be trivially explicable to beginners.)
Opening blank line vs. closing blank line
I think it's uncontroversial to say that the opening blank line before an inner class or def could be omitted in all cases where the indentation level of the previous line was lower.
Similarly, with closing blank lines, we could intuit that the same rule applies: if the following line's indentation level is lower than the one of the entire inner class or def, skip the blank line. However, there's more to it.
Inner definition size
My intuition is that omitting blank lines for inner classes and defs is only beneficial if those inner definitions are short. If you have an inner definition (like in a decorator, say) which doesn't fit on one screen, you'd want an additional empty line to make it obvious some block of code ended (and the following return belongs to a different function!).
So this is optimizing for short definitions. But how short?
Inner definition blank lines
This is somewhat related to the previous point. If the inner class or def itself contains blank lines, then skipping the closing blank line will just look weird. In fact, in this case having the symmetrical opening blank line is likely also desirable for clarity.

I don't have a strong intuition for how to format long nested definitions, and I don't feel the intuition that embedding a blank line inside a function somehow forces the function to need surrounding blank lines. I suspect this is getting far enough into weird-edge-case-of-weird-edge-case-land that probably any formatting is going to be somewhat unsatisfying.
In most cases where you can have a nested definition, doesn't black already allow users to insert optional blank lines? Maybe the rule would be: when you have a def or class adjacent to less-indented code, then a blank line is optional; otherwise it's mandatory.

A full example of current behavior which we'd like to standardize:

show on Black Playground

I've done a cursory reading of this issue and related issues. I'm asking because this has been in discussion for almost 4 years:
How does a decision like this get made? Are there any actionable items that are required by the team? Will there be a consensus meeting? Etc.
Looking forward to Black having an opinion on this! (I also don't care which opinion, as long as there is a clear and deterministic one). 👍 Black is an awesome program, keep up the great work!

Thanks! I think this issue still needs some discussion, but there's been related work on blank lines recently in #3035 and #2996.
Personally, I'd like to see them removed. And wrt. ambv's example (updated link that works), directly nested functions and methods have the newlines already removed. To me, the only thing I'd change (in addition to #3035 removing leading space in the def -> space -> some code -> space -> def case) would be the if cases. But, ambv might have a point with this being way better for short definitions.

I think the original request does make a lot of sense to think about empty lines mainly as a means of separation inside one level of indentation. This is clear, unambiguous, and easy to understand.
Sometimes it makes sense to use empty lines regardless of the indentation, as outlined e.g. in https://github.com/psf/black/issues/902#issuecomment-1425713025
Size of function/code
I'm not sure whether the size of an inner function definition should play a role. Long definitions hinder readability in general - regardless whether there are inner function definitions - and can be dealt with refactoring and extracting more smaller functions.
A bad example of "long code" but without inner function definitions:
class A:
    def method1(self):
        if abc:
            while True:
                if not dfe:
                    try:
                        if ghi is None:
                            for jkl in mno:
                                if pqr > 0:
                                    break
                    except:
                        continue
                x = 3
            y = 4
        z = 5 + self.method2(y)
        print("z calculated")

Such code is quite hard to grasp immediately, and I want to suggest that it does not have to be Black's goal to make such code better readable, but that the readability of edge cases and bad code is better left to the developer.
There is one thing the developer can improve easily, namely adding an empty line before the z = ... assignment. The whole long-indented if ... chain thing is then still unreadable, but at least the last two statements z = ... and the print(..) stand apart of the and seem to be a little bit better readable as above ("ok I don't see what this unreadable block is, but after it there are these two statements").
Similarly, Black can't do something about such structures that are even long and exceed screen pages, leaving indentation levels unclear at first grasp. Solution through refactoring, not auto-formatting.
A short example
# with and without empty line after the 'if' block:

def function():
    if abcd > 0:
        efgh = 1
    something = 2

def function():
    if abcd > 0:
        efgh = 1

    something = 2

Personally, I prefer the empty line version, and I mostly use that. It's opinionated, and situation-dependent, and Black leaves both versions as-is, and Black should keep doing so.
Let's think of the empty line as a separator between the if block and the something assignment, which are both on the same level of indentation. Makes sense (to me).
Why are function definitions special?
One might find it elegant to leave the separation with empty lines also in the case of (inner) function definitions to the developer, because after all, def statements and blocks are just statements and blocks like other statement and blocks.
Though, there are widely used guidelines like PEP 8 to separate/surround class and def statements on the global level by two empty lines. And: Method definitions are usually separated by one empty line. This exceptional, "mandatory" use of empty lines is historically grown, practically globally consistent and accepted, and makes sense given that most of these definitions in the real world are longer than just one- or two-liners.
So, it makes sense that Black enforces this, that's pragmatic and agreeable.
The exceptional treatment of class and def statements (over other statements/expressions) when it comes to adding empty lines makes full sense when taking only the vertical spacing into account - adding context about horizontal alignment (indentation) is not needed for the exceptional treatment to make sense.
I want to argue that in case of (horizontal) alignment differences, I don't see a clear or consistent approach, rule set, or community-wide usage that can be followed here. It's opinionated and controversial. Empty lines at the beginning of the block can now be added optionally, see #4043. I don't think they should be enforced (for def or class statements).
As well, I don't see good rationale for enforcing an empty line at the end of an indentation block only for class and def statements. I think Black should treat these cases the same way:
def function():
    try:
        if something:
            do()
    finally:
        pass

def function():
    try:
        def inner():
            do()
    finally:
        pass

A lot of behaviour in OP's example no longer repros, so I think it's time to close this issue.
If someone here wants changes, I'd recommend opening a new issue with a summary of what behaviour changes from Black preview style on main in 2024 are desired.
If you do so, some requests:

Please link to this issue
Consider reading ambv's first two comments on this issue and the issue #4043 first
Just be aware it'll likely be an uphill battle (an invitation to open an issue is not a promise it will be accepted)
Arguments I find most persuasive are links to real world code where people feel Black's style makes things less readable

e-gebes, thanks for the writeup and concrete suggestion. I don't agree that those cases should be treated the same way. As ambv mentions, there's a different scope involved and Black's current style intentionally accentuates this.


Patch notes

Enhancements

Add a "Patch Notes" link to the header throughout the app, and quick links card on the home page
Show a red badge indicator to the patch notes link in the header if a newer version is detected, which can be dismissed by hovering over the link
Add latestVersion field to dbMeta to track latest version a DB was opened in

Not sure if we should consider just storing the latestVersion in localStorage but outside of the exported database?


Try to fix Loc issue

Per User Story https://mseng.visualstudio.com/TechnicalContent/_queries/edit/1537022/?triage=true.
The `$`` coding is causing rendering issues in some languages.

@rpetrusha - Per Loc bug https://mseng.visualstudio.com/TechnicalContent/_queries/edit/1537022/?triage=true, the code formatting around the $' entry in the table worked ok in the en-us and some other languages, but caused rendering issues in MT languages. The only fix seems to be to remove the code formatting, so to make it look better I removed the code formatting from all the entries. Please approve/merge if you agree. Thanks.


anyone can provide the pretrain model for mxnet ?

I've provided the weights, refer to README. The gluon weights should also work for pure MXNet api.


Make SerialisationError a quick_error

This adds impls for fmt::Display and error::Error while keeping the code succinct.

r? @ustulation
(maidsafe_highfive has picked a reviewer for you, use r? to override)

r? @dirvine

Reviewed 3 of 3 files at r1.
Review status: all files reviewed at latest revision, all discussions resolved.

Comments from the review on Reviewable.io


Tags are displayed in source code only for Japanese sites

@TomiakiNiwa commented on Mon Mar 04 2019
Describe the bug
In the Japanese page, the tag is displayed in the last sample source.
To Reproduce
Steps to reproduce the behavior:

Go to 'https://docs.microsoft.com/ja-jp/sql/relational-databases/backup-restore/enable-sql-server-managed-backup-to-microsoft-azure?view=sql-server-2017'
Look at the last source.

Expected behavior
In English pages, it has not occurred.
There is a possibility that CSS error occurred during translation.
Screenshots

Desktop (please complete the following information):
all
Smartphone (please complete the following information):
all
Additional context
none

@welcome[bot] commented on Mon Mar 04 2019
Thank you for creating the issue! One of our team members will get back to you shortly with additional information.

Hi team,
I'm moving this here because I believe is a source issue.
I saw a similar issue some days ago and the problem was with the tags in the code were misplaced causing translatable content being shown as code.
Kind regards

@CeciAc - the English page also has incorrect formatting at exactly the same spot. It doesn't show the <span.. command, but there is a block of code that is incorrectly formatted in markdown.

I will submit a fix for this today and after that, we should see if the problem reproduces in Japanese. Will that work?
Thank you for raising this issue to our attention.

@CeciAc  - I have submitted a fix to the english version. I think this will result in an update to the localized version that should resolve the issue which you so appropriately raised. Thank you.


Improve table display and summation

Hey @tnicola,
I continue testing and improving your plugin. I'd really like to use it for our CI as Cypress only offers parallelization over their expensive subscription plans.
I made a few more improvements:

log message at the beginning to know how many test suites are going to be executed
update again summation for tests
improve colors in the table
improve log message on failures
Fix typo in constant name

Hi @devsalsa, thanks a lot for your contributions, I really appreciate that!I'm going to merge your PR. In the next steps I'll need to add some unit/integration tests and to proper setup a CI process for this repo

In the next steps I'll need to add some unit/integration tests and to proper setup a CI process for this repo

That would be great.


http(...) An unexpected error occurred trying to execute that command

running this: /script run http('GET', 'https://api.mojang.com/users/profiles/minecraft/gunnywaffle', 1000, 1000)
yields this: {name: GunnyWaffle, id: 2a98f2fc5be94ed5adf81b1790361a7e}
Which works as expected.
However, providing a nonexistent username yields an error instead of a well formed error response.
running this: /script run http('GET', 'https://api.mojang.com/users/profiles/minecraft/gunywaffle', 1000, 1000)
yields this: An unexpected error occurred trying to execute that command in red with no further script execution.
A try(...) does not catch anything, with exception specified and there is no callstack in the server console.
I suspect that something in the mod is failing during the handling of an error response.
Here is what it looks like, with an attempt at gunywaffle first, then gunnywaffle.

Fixed in 9d6905e2f6b3ceded1881d10f5e606777561bc0b


Wind vectors

Add wind vectors diagnostic to e3sm_diags.

Note to self: do NOT squash later commits into the first one until this pull request is ready to merge. That way, it will be easy to tell what changes I've made to the original version (i.e., the first commit).

@forsyth2 Thank you for working on reviving this branch!
Under test_model_data_for_acme_diags/climatology, you should be able to find netcdf data having names with the case name (20160520.A_WCYCL1850.ne30_oEC.edison.alpha6) as the first part of the filename. U,V,TAUX,TAUYshould be all included within those files.
I don't remember I sew those Errors you mentioned here.
For the questions:

The super long arrows, it's not an error, but we do need an algorithm to scale them so that they can be presented nicely. The algorithm should be flexible for data at different resolutions (i.e, high resolution and low resolution). This is where my progress stopped during my attempt :(

2a, Climate models often report the u and v components (zonal and meridional wind, respectively) of the wind vector. U and V are output at each grid point. But for visualizing the vectors, it would be too dense to show one vector per grid, therefore regridding is desired.
2b, TAUX and TAUY is different that the TAU we used for cloud optical depth. The Greek letter tau is used quite often as in our case here, its just a coincident.
2c, Surface wind stress is a dragging force applied to the wind blow at surface. it has a unit N/m2. A detailed explanation can be found here: https://marine.rutgers.edu/dmcs/ms501/2004/Notes/Wilkin20041014.htm

we don't need more to add for this set. But same plot with a polar projection zooming in over N. and S. Polar areas will be desired. That would be another PR I suppose.

@forsyth2 The PR looks good. I will take it from here and try finalizing the vectors. Thank you.

@wlin7 As we discussed yesterday, we should make a push to revive and complete this PR. The current block is that the visualization of arrows for wind vectors are not optimized. If you could review the current example and make change suggestions that would be great. https://portal.nersc.gov/project/e3sm/forsyth/wind_vectors_2020_06_15/wind_vectors_results/viewer/

Consider streamlines (https://matplotlib.org/stable/gallery/images_contours_and_fields/plot_streamplot.html) rather than wind vectors

Sounds great, @forsyth2 . It is a better and flexible alternative in many ways. Let's try a few options with our typical global array. The views may look different than the examples as our global grids are much denser.


improved commenting in test_data.py

Review for data tests

Your test_data.py code looks really good! Curious if you want to add "current" in the parametrization tests for vintage? Is there anywhere you test what the current vintage is correct? Otherwise looks good!

I have added the test for the current vintage and will merge. Thanks!


Toggle on/off similar songs to my queued songs

I want to be able to toggle on/off similar songs from playing after I must have been done listening to all my queued songs

Not standard user story format


Show information/progress about saving survey results/json on a server

Add ability to show information on the complete page:

about sending survey the results to the server/ saving
about successful saving the results
about an error and ability to try save results again

Here is the demo: http://plnkr.co/edit/aMcSnl?p=preview


data type in augmentation

Hello,
I'm curious whether the data type is limited to uint8, ranging from 0-255. In certain situations, such as medical imaging, the data type might be float32. Converting it to uint8 could result in a loss of information.
When I use import torchvision.transforms, it also only support uint8.

I think it's because transforms came from 2D images, where 0-255 range is typically used for RGB images. May be volumentations can be updated to work with float32.


How can we get in touch?

Hello,
I recently build Arguflow - https://docs.arguflow.gg/ - and would love to talk to you about your experience with Flexcel. Please ping me through<EMAIL_ADDRESS>if you're interested!

Meeting links if it makes it easier:

https://cal.com/nick-arguflow/30min
https://cal.com/nick-arguflow/15min


main.ts is missing from the TypeScript compilation

Hi guys!
So I'm trying to implement your filemanager in a project but I'm not quite sure exactly how to proceed.
I ran npm i @rign/angular2-filemanager
I have a CloudComponent where I want to use <filemanager>Loading...</filemanager>
Here is my actual code:

cloud.component.ts:
import { Component, OnInit } from '@angular/core';

@Component({
  selector: 'app-cloud',
  templateUrl: './cloud.component.html',
  styleUrls: ['./cloud.component.css']
})
export class CloudComponent implements OnInit {

  constructor() {
   }

  ngOnInit() {
  }

}

cloud.component.html:
<filemanager>Loading...</filemanager>

app.module.ts:
...
import { FileManagerModule } from '@rign/angular2-filemanager/main';
import { CloudComponent, fileManagerConfiguration } from './cloud/cloud.component';

const fileManagerConfiguration: IFileManagerConfiguration = {
  urls: {
    foldersUrl: '/api/folder',
    filesUrl: '/api/files',
    folderMoveUrl: '/api/folder/move'
  },
  isMultiSelection: true,
  mimeTypes: ['image/jpg', 'image/jpeg', 'image/png'],
  maxFileSize: 50 * 1024
};

@NgModule({
  declarations: [
    ...
  ],
  imports: [
    ...
    FileManagerModule.forRoot(fileManagerConfiguration)
  ],
  providers: [
    ...
  ],
  bootstrap: [AppComponent]
})
export class AppModule { }

When I run ng serve, i've got an error: main.ts is missing from the TypeScript compilation

Hi,
Could you give me more info? This main.ts is my file (from filemanager) or from your project?

The one imported by import { FileManagerModule } from '@rign/angular2-filemanager/main';

It seems like a configuration problem. I'll check it.

Fixed in #18 .


Missing warning about extensions

We used to have a warning about extensions at the start of the CSS challenges but with the new curriculum layout, there is no intro page which is where the warning was located.
Not sure if this is intentionally missing now or if the warning should be added back to the block description?

Closing this as fixed - banner is present on the superblock page. Feel free to request reopening if further review is required. :)


Use dev container for integration tests

addresses #600 .
Use simtools-dev:latest for integration tests.
This avoids the compilation of corsika/sim_telarray and the installation of all python packages for each tests.
Reduced run time for tests from 25-28 min to 15-18 min.
Notes:

only one python version is tested (the one installed the container). This should be fine, as unit tests should pick up any differences between the versions
future pull requests which updates packages require to manually call the docker-dev deployment workflow (as these packages are no in simtools-dev:latest). This should be fine, as we very seldomly add new packages.

There is some success.
It seems now that there is no database access. Not sure if we have to open a port or something else.

future pull requests which updates packages require to manually call the docker-dev deployment workflow (as these packages are no in simtools-dev:latest). This should be fine, as we very seldomly add new packages.

What do you mean by "new packages" here?

future pull requests which updates packages require to manually call the docker-dev deployment workflow (as these packages are no in simtools-dev:latest). This should be fine, as we very seldomly add new packages.

What do you mean by "new packages" here?

I mean if we add a dependency to environment.yml in a PR, this new dependency will not be available for the integration tests.

Ahh I see, thanks!


Regex Quantifier {} not working in String Function node

Expected behavior: Appending {4} should match the previous token exactly 4 times.
Actual behavior: Curly braces are just ignored completely :(
P.S. I love your addon! <3
P.P.S. r"(........................................................................................................................................................................................................[\s,])(.*)" is my workaround for now in case anyone else runs into this :P

This should be fixed


some noise at every start output of the i2s_write (IDFGH-7021)

Hi,I am using the demo here
I modify this demo to make the .wav file output to a speaker over the i2s driver(a hardware max9735a I used)
mostly,every time at the starting of the .wav file ,the sound always some noise like "de"
How to deal with it ,Thanks

This example uses i2s_write to play raw music that store in flash or header file. Raw music means only audio data in it, but a .wav format music has a protocol header at the beginning, which indicates the sample_rate, data_bit_width and so on, these information are used to decode the following music and shouldn't be played. You can try to parse the header of .wav file and play the music without the header.
Hope it can help~

FYI, you can refer to how a .wav header is generated in i2s_audio_recorder_sdcard

This example uses i2s_write to play raw music that store in flash or header file. Raw music means only audio data in it, but a .wav format music has a protocol header at the beginning, which indicates the sample_rate, data_bit_width and so on, these information are used to decode the following music and shouldn't be played. You can try to parse the header of .wav file and play the music without the header.
Hope it can help~

think you for your reply
below it my code to play a .wav file stora at  spiffs,  I had read the header of the file ,but the noise still exit
static esp_err_t  play_wav(const char *filepath)
{
    FILE *fd = NULL;
    fd = fopen(filepath, "r");

    if (NULL == fd) {
        ESP_LOGE(TAG, "Failed to read existing file : %s", filepath);
        return ESP_FAIL;
    }
    const size_t chunk_size = 2048;
    uint8_t *buffer = malloc(chunk_size);

    if (NULL == buffer) {
        ESP_LOGE(TAG, "audio data buffer malloc failed");
        fclose(fd);
        return ESP_FAIL;
    }
    int len;
    /**
     * read head of WAV file
     */
    wav_header_t wav_head;
    len = fread(&wav_head, 1, sizeof(wav_header_t), fd);
    if (len <= 0) {
        ESP_LOGE(TAG, "Read wav header failed");
        fclose(fd);
        return ESP_FAIL;
    }
    if (NULL == strstr((char *)wav_head.Subchunk1ID, "fmt") &&
            NULL == strstr((char *)wav_head.Subchunk2ID, "data")) {
        ESP_LOGE(TAG, "Header of wav format error");
        fclose(fd);
        return ESP_FAIL;
    }

    // ESP_LOGI(TAG, "frame_rate=%d, ch=%d, width=%d", wav_head.SampleRate, wav_head.NumChannels, wav_head.BitsPerSample);

    /**
     * read wave data of WAV file
     */
    size_t write_num = 0;
    size_t cnt;
    //i2s_start(I2S_NUM_0);
	do {
        /* Read file in chunks into the scratch buffer */
        len = fread(buffer, 1, chunk_size, fd);
        if (len <= 0) {
            ESP_LOGI(TAG, "break out");
            //i2s_stop(I2S_NUM_0);
            break;
        }

        //pwm_audio_write(buffer, len, &cnt, 1000 / portTICK_PERIOD_MS);
		i2s_write(I2S_NUM_0, buffer, len, &cnt, 100);
        ESP_LOGI(TAG, "%d,%d",cnt,len);
        write_num += len;
    } while (1);

    free(buffer);
	fclose(fd);
	return ESP_OK;
}

afasdfasfasf

Hello，there is video，There is a "de" sound at the beginning of most tones，and below is my main code
void app_play_num(int index)
{
    char file_buf[16];

    sprintf(file_buf,"/audio/%d.wav",index);

    play_wav(file_buf);
    ESP_LOGI(TAG,"going to player %s",file_buf);
}

void speaker_setTemp2_say(void)
{
    char file_buf[32];

    sprintf(file_buf,"/audio/setTime.wav");
    ESP_LOGI(TAG,"going to player %s",file_buf);
    play_wav(file_buf);
}
void speaker_du_say(void)
{
    char file_buf[16];

    sprintf(file_buf,"/audio/min.wav");
    ESP_LOGI(TAG,"going to player %s",file_buf);
    play_wav(file_buf);
}
void app_voice_play(int num)
{
    speaker_setTemp2_say();

    if(num/10)
    {
        app_play_num(num/10); //noise exit at the begin of this code
        app_play_num(0); //noise exit at the begin of this code
    }

    app_play_num(num%10); //noise exit at the begin of this code

    speaker_du_say(); //noise exit at the begin of this code

}
void app_main(void)
{
    spiffs_driver_init();

    hal_i2s_driver_init();
    //play_wav(MOUNT_POINT"/sample.wav");
    app_voice_play(19);
    vTaskDelay(100);
    app_voice_play(13);
    vTaskDelay(100);
    app_voice_play(31);
    vTaskDelay(100);
    app_voice_play(39);
    vTaskDelay(100);
    app_voice_play(91);
    vTaskDelay(100);
    app_voice_play(93);
}

Firstly, for the current I2S driver, due to the driver will start automatically after installation, the uninitialized data may be transported, you can try to stop the i2s after installation and calling i2s_zero_dma_buffer to clear the DMA buffer before start and write data.
Secondly, just in case the audio itself contains the noise, try to print the beginning data of the sending buffer.

Another notice, the DAC only support 8 bit-width, but most audios are at least 16 bit-width, the audio data need to be right shifted 8 bits

The clipped sound mainly caused by the rapid change of data. I guess the polling time between two i2s_write may be too long which lead to the audio not consecutive, try to remove other codes in the i2s_write loop. and also clear the dma buffer every time after finished sending the data.

Firstly, for the current I2S driver, due to the driver will start automatically after installation, the uninitialized data may be transported, you can try to stop the i2s after installation and calling i2s_zero_dma_buffer to clear the DMA buffer before start and write data. Secondly, just in case the audio itself contains the noise, try to print the beginning data of the sending buffer.

here is the wav file
spiffs_image.zip

how to clear the dma buffer

Call i2s_zero_dma_buffer

Call i2s_zero_dma_buffer

Thanks for your reply,would you tell me which file the "i2s_zero_dma_buffer" function is. I can't find the call in the git project

It's not a file, it's a function declared in components/driver/include/driver/i2s.h line 379.

It's not a file, it's a function declared in components/driver/include/driver/i2s.h line 379.

Thank you!
I found event I write the code like this:
to make the num output data consecutly,but the 'de' noise is still exit in num
Is it a problem of wav format, is it better for me to change to pcm format?
#include <stdio.h>
#include "driver/i2s.h"
#include "esp_log.h"
#include "esp_system.h"
#include "nvs_flash.h"
#include "sdkconfig.h"
#include "esp_spiffs.h"
#include <string.h>
static const char *TAG = "speaker";

#define I2S_SAMPLE_RATE     (16000)
#define SAMPLE_BITS     (16)

#define MASTER_BCK_IO 4
#define MASTER_WS_IO 5
#define DATA_OUT_IO 18
#define DATA_IN_IO 21

#define MOUNT_POINT  "/audio"

static const char *partition_label = "audio";

typedef struct {
    // The "RIFF" chunk descriptor
    uint8_t ChunkID[4];
    int32_t ChunkSize;
    uint8_t Format[4];
    // The "fmt" sub-chunk
    uint8_t Subchunk1ID[4];
    int32_t Subchunk1Size;
    int16_t AudioFormat;
    int16_t NumChannels;
    int32_t SampleRate;
    int32_t ByteRate;
    int16_t BlockAlign;
    int16_t BitsPerSample;
    // The "data" sub-chunk
    uint8_t Subchunk2ID[4];
    int32_t Subchunk2Size;
} wav_header_t;

#define MAX_OP_FILE 10
typedef struct
{
	FILE * fds[MAX_OP_FILE];
	int num_use;
}file_op_t;
file_op_t file_op;
char file_name_buf[MAX_OP_FILE][16];

static esp_err_t  play_wav(const char *filepath)
{
    FILE *fd = NULL;
    //struct stat file_stat;

    // if (stat(filepath, &file_stat) == -1) {
    //     ESP_LOGE(TAG, "Failed to stat file : %s", filepath);
    //     return ESP_FAIL;
    // }

    // ESP_LOGI(TAG, "file stat info: %s (%ld bytes)...", filepath, file_stat.st_size);
    fd = fopen(filepath, "r");

    if (NULL == fd) {
        ESP_LOGE(TAG, "Failed to read existing file : %s", filepath);
        return ESP_FAIL;
    }
    const size_t chunk_size = 4096;
    uint8_t *buffer = malloc(chunk_size);

    if (NULL == buffer) {
        ESP_LOGE(TAG, "audio data buffer malloc failed");
        fclose(fd);
        return ESP_FAIL;
    }
    int len;
    /**
     * read head of WAV file
     */
    wav_header_t wav_head;
    len = fread(&wav_head, 1, sizeof(wav_header_t), fd);
    if (len <= 0) {
        ESP_LOGE(TAG, "Read wav header failed");
        fclose(fd);
        return ESP_FAIL;
    }
    if (NULL == strstr((char *)wav_head.Subchunk1ID, "fmt") &&
            NULL == strstr((char *)wav_head.Subchunk2ID, "data")) {
        ESP_LOGE(TAG, "Header of wav format error");
        fclose(fd);
        return ESP_FAIL;
    }

    // ESP_LOGI(TAG, "frame_rate=%d, ch=%d, width=%d", wav_head.SampleRate, wav_head.NumChannels, wav_head.BitsPerSample);

    /**
     * read wave data of WAV file
     */
    size_t write_num = 0;
    size_t cnt;
    //i2s_start(I2S_NUM_0);
	i2s_zero_dma_buffer(I2S_NUM_0);
	do {
        /* Read file in chunks into the scratch buffer */
        len = fread(buffer, 1, chunk_size, fd);
        if (len <= 0) {
            ESP_LOGI(TAG, "break out");
            //i2s_stop(I2S_NUM_0);

            break;
        }

        //pwm_audio_write(buffer, len, &cnt, 1000 / portTICK_PERIOD_MS);
		i2s_write(I2S_NUM_0, buffer, len, &cnt, 100);
        ESP_LOGI(TAG, "%d,%d",cnt,len);
        write_num += len;
    } while (1);

    free(buffer);
	fclose(fd);
	return ESP_OK;
}
void spiffs_driver_init(void)
{
    ESP_LOGI(TAG, "Initializing SPIFFS");

    esp_vfs_spiffs_conf_t conf = {
      .base_path = MOUNT_POINT,
      .partition_label = partition_label,
      .max_files = 5,
      .format_if_mount_failed = false
    };

    // Use settings defined above to initialize and mount SPIFFS filesystem.
    // Note: esp_vfs_spiffs_register is an all-in-one convenience function.
    esp_err_t ret = esp_vfs_spiffs_register(&conf);

    if (ret != ESP_OK) {
        if (ret == ESP_FAIL) {
            ESP_LOGE(TAG, "Failed to mount or format filesystem");
        } else if (ret == ESP_ERR_NOT_FOUND) {
            ESP_LOGE(TAG, "Failed to find SPIFFS partition");
        } else {
            ESP_LOGE(TAG, "Failed to initialize SPIFFS (%s)", esp_err_to_name(ret));
        }
        return;
    }

    size_t total = 0, used = 0;
    ret = esp_spiffs_info(conf.partition_label, &total, &used);
    if (ret != ESP_OK) {
        ESP_LOGE(TAG, "Failed to get SPIFFS partition information (%s)", esp_err_to_name(ret));
    } else {
        ESP_LOGI(TAG, "Partition size: total: %d, used: %d", total, used);
    }
}

void hal_i2s_driver_init(void)
{
    esp_err_t err;
    // master driver installed and send data
    i2s_config_t master_i2s_config = {
        .mode = I2S_MODE_MASTER | I2S_MODE_TX | I2S_MODE_RX,
        .sample_rate = I2S_SAMPLE_RATE,
        .bits_per_sample = SAMPLE_BITS,
        .channel_format = I2S_CHANNEL_FMT_ONLY_RIGHT,
        .communication_format = I2S_COMM_FORMAT_STAND_I2S,
        .dma_buf_count = 6,
        .dma_buf_len = 200,
        .use_apll = 0,
        .intr_alloc_flags = ESP_INTR_FLAG_LEVEL1 ,
    };
    i2s_pin_config_t master_pin_config = {
        .bck_io_num = MASTER_BCK_IO,
        .ws_io_num = MASTER_WS_IO,
        .data_out_num = DATA_OUT_IO,
        .data_in_num = DATA_IN_IO
    };

    err = i2s_driver_install(I2S_NUM_0, &master_i2s_config, 0, NULL);
    ESP_LOGI(TAG,"i2s_driver_install status %d",err);
    err = i2s_set_pin(I2S_NUM_0, &master_pin_config);
    ESP_LOGI(TAG,"i2s_set_pin status %d",err);
    i2s_set_clk(I2S_NUM_0, I2S_SAMPLE_RATE, (i2s_bits_per_sample_t)16, (i2s_channel_t)1);
}

void wav_num_set(int num)
{
	file_op.num_use = 0;
    if(num/10)
    {
        //app_play_num(num/10); //noise exit at the begin of this code
		sprintf(file_name_buf[file_op.num_use++],"/audio/%d.wav",num/10);
        //app_play_num(0); //noise exit at the begin of this code
		sprintf(file_name_buf[file_op.num_use++],"/audio/0.wav");
    }

    //app_play_num(num%10); //noise exit at the begin of this code
	sprintf(file_name_buf[file_op.num_use++],"/audio/%d.wav",num%10);
}

static esp_err_t wav_num_player(void)
{
	int i,j=0;
	wav_header_t wav_head;
	int len;
	for(i=0;i<file_op.num_use;i++)
	{
		file_op.fds[i] = NULL;
		file_op.fds[i] = fopen(file_name_buf[i], "r");
		if (NULL == file_op.fds[i]) {
			ESP_LOGE(TAG, "Failed to read existing file : %s", file_name_buf[i]);
			return ESP_FAIL;
		}

		len = fread(&wav_head, 1, sizeof(wav_header_t), file_op.fds[i]);
		if (len <= 0) {
			ESP_LOGE(TAG, "Read wav header failed");
			fclose(file_op.fds[i]);
			return ESP_FAIL;
		}
		if (NULL == strstr((char *)wav_head.Subchunk1ID, "fmt") &&
				NULL == strstr((char *)wav_head.Subchunk2ID, "data")) {
			ESP_LOGE(TAG, "Header of wav format error");
			fclose(file_op.fds[i]);
			return ESP_FAIL;
		}
		ESP_LOGI(TAG, "frame_rate=%d, ch=%d, width=%d", wav_head.SampleRate, wav_head.NumChannels, wav_head.BitsPerSample);
	}

    const size_t chunk_size = 4096;
    uint8_t *buffer = malloc(chunk_size);

    size_t write_num = 0;
    size_t cnt;

	do {
        /* Read file in chunks into the scratch buffer */
        len = fread(buffer, 1, chunk_size, file_op.fds[j]);
		if(j != file_op.num_use-1)
		{
			if (len < chunk_size) {
				j++;
				len+=fread(&buffer[len], 1, chunk_size-len, file_op.fds[j]);

			}
		}
		else
		{
			if (len <= 0) {

				ESP_LOGI(TAG, "break out");

				break;
			}
		}

        //pwm_audio_write(buffer, len, &cnt, 1000 / portTICK_PERIOD_MS);
		i2s_write(I2S_NUM_0, buffer, len, &cnt, 100);
        ESP_LOGI(TAG, "%d,%d",cnt,len);
        write_num += len;
    } while (1);

    free(buffer);
	for(i=0;i<file_op.num_use;i++)
	{
		fclose(file_op.fds[i]);
	}
	return ESP_OK;
}

size_t sounder_speaker(char * p_data,unsigned int data_len)
{
    size_t i2s_bytes_write = 0;
    i2s_write(I2S_NUM_0, p_data, data_len, &i2s_bytes_write, 1000 / portTICK_PERIOD_MS);
    return i2s_bytes_write;
}

//char *file_arr[5] = {"shi","001","002","ktw","shd"};

void app_play_num(int index)
{
    char file_buf[16];

    sprintf(file_buf,"/audio/%d.wav",index);

    play_wav(file_buf);
    ESP_LOGI(TAG,"going to player %s",file_buf);
}

void speaker_setTemp2_say(void)
{
    char file_buf[32];

    sprintf(file_buf,"/audio/setTime.wav");
    ESP_LOGI(TAG,"going to player %s",file_buf);
    play_wav(file_buf);
}
void speaker_du_say(void)
{
    char file_buf[16];

    sprintf(file_buf,"/audio/min.wav");
    ESP_LOGI(TAG,"going to player %s",file_buf);
    play_wav(file_buf);
}
void app_voice_play(int num)
{
	wav_num_set(num);
    speaker_setTemp2_say();

	wav_num_player();

    speaker_du_say(); //noise exit at the begin of this code

}
void app_main(void)
{
    spiffs_driver_init();

    hal_i2s_driver_init();
    //play_wav(MOUNT_POINT"/sample.wav");
    app_voice_play(19);
    vTaskDelay(100);
    app_voice_play(13);
    vTaskDelay(100);
    app_voice_play(31);
    vTaskDelay(100);
    app_voice_play(39);
    vTaskDelay(100);
    app_voice_play(91);
    vTaskDelay(100);
    app_voice_play(93);

}

You can put your wav file into a software named audacity, it can provide the wave shape and also able to play the audio.

Thanks for reporting, feel free to reopen.


Revert "⬆ Bump nuxt from 2.12.2 to 2.13.0"

Reverts liyasthomas/postwoman#942

Hey @liyasthomas,
Your changes look good to me!
View build log
TravisBuddy Request Identifier: 1dc982e0-b1f9-11ea-b4e4-a33918451e6b


drupal: fix probes to properly query pod health

@sameersbn looks good. You probably did it because / gets a redirect when authentication is enabled instead of a 200? What's the impact of changing all those timeouts and delaySeconds? Is that safe enough for a patch version bump?

@linki The bitnami images render  a loading page while the application is being initialized (at container startup). The health probes consider this as the pod being healthy/ready, which is not really true. The pod should indicate readiness only when the target application has been initialized and ready to accept queries.
This PR changes the http endpoint of the health checks to query the state of the application. Consequently the initial delay has been bumped because we want to give enough time for the application to initialize and start.


Not finding Xunit tests when adding reference to Microsoft.Extensions.DependencyModel

From @fubar-coder on February 2, 2017 9:39
Steps to reproduce:
mkdir test
cd test
dotnet new -t Xunittest
dotnet add test.csproj package -v 1.1.0 Microsoft.Extensions.DependencyModel
dotnet restore
dotnet test

Output:
Build started, please wait...
Build completed.

Test run for C:\temp\test\bin\Debug\netcoreapp1.0\test.dll(.NETCoreApp,Version=v1.0)
Microsoft (R) Test Execution Command Line Tool Version <IP_ADDRESS>
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
Warning: No test is available in C:\temp\test\bin\Debug\netcoreapp1.0\test2.dll. Make sure that installed test discoverers & executors, platform & framework version settings are appropriate and try again.

Expected behaviour:
Tests are found and executed
dotnet --info:
.NET Command Line Tools (1.0.0-rc3-004530)

Product Information:
 Version:            1.0.0-rc3-004530
 Commit SHA-1 hash:  0de3338607

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.14393
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\1.0.0-rc3-004530

Copied from original issue: dotnet/sdk#787

This issue is related to xunit/xunit#1080 and #444.

Move xunit.runner.visualstudio to >=2.2.0-rc1-build1247 solves issue.


Updating package-lock.json file after npm-update -u

Hi,
I noticed that the package-lock.json file isn't affected by the npm-check -u command, after installation is completed.
Is there a way to get npm-check update this file?
Thanks :)

npm-check -u does update package-lock.json for me, check if you're using npm 5 or higher.

Yeah, npm-check just executes npm install ... so the lockfile should be updated if you are using npm 5 or higher

It's not updating for me.
I am using npm v6.8.0

Same thing happened for me. I upgraded wdio/cli from 8.39.1 to 9.1.0 but afterwards the transitive dependency import-meta-resolve was still pinned to 4.0.0 in package-lock.json even though 4.1.0 has been releated and the package.json for wdio/cli points to ^<IP_ADDRESS>  I also tried to delete node_modules and re-run npm install, and the lock file still pointed to import-meta-resolve 4.0.0. After that I deleted the lock file and re-ran "npm install" and even then I still had import-meta-resolve 4.0.0 in the newly generated package-lock.json. Finally, after deleting both node_modules dir and package-lock.json simultaneously and re-running "npm install" I finally got import-meta-resolve 4.1.0 pinned in the lock file. I'm using node 22 and npm 10.8.3.


PhoneGap deferred deep linking doesn't work on iOS.

Deferred deep linking on Phone gap doesn't seem to work.
How to reproduce:

Integrate Cordova PhoneGap SDK into an iOS app
Click on a link for the Branch key that matches the app from the first step, using an iOS device (ideally iOS 11/12)
Install the application onto the device from Step one and open it
Verify that no deep link data was passed into the app.
Close the app.
Click on the same link again, which opens the app
Verify that Link data is returned.

Code modified only by replacing the name of the app with myApp:
Main
document.addEventListener('deviceready', () => {
      $('#loading-app-indicator').css({ display: 'none' });
      Meteor.setTimeout(() => navigator.splashscreen.hide(), 200);

      if (global.facebookConnectPlugin)
        global.facebookConnectPlugin.activateApp();

      document.addEventListener('resume', () => Meteor.reconnect(), false);
      document.addEventListener('resume', () => handleBranch(), false);
      document.addEventListener('resume', () => Push.setBadge(0), false);

      handleBranch();
    }, false);

Supporting Module
export default function handleBranch() {
  if (typeof Branch === 'undefined') {
    throw new Meteor.Error('branch-sdk-not-found', 'Branch SDK not installed');
  }

  Branch.setDebug(true);
  if (Meteor.isDevelopment || false) Branch.setAppleSearchAdsDebugMode();

  Branch.delayInitToCheckForSearchAds();
  Branch.initSession().then(function(data) {
    // for better Android matching
    Branch.setCookieBasedMatching('myApp.app.link');

    // Pass Mixpanel distinct ID to Branch
    if (window.mixpanel) {
      Branch.setRequestMetadata('$mixpanel_distinct_id', window.mixpanel.get_distinct_id());
    }

    let location;

    if (data['+non_branch_link']) {
      location = parse(data['+non_branch_link'], true).pathname;
      FlowRouter.go(location);
    }
  });
}

Branch-related logs upon startup:
2018-11-06 15:00:46.672823-0800 myApp[643:68256] Push.Configure: [object Object]
2018-11-06 15:00:46.672884-0800 myApp[643:68256] Push.setBadge: 0
2018-11-06 15:00:46.705334-0800 myApp[643:68349] [branch.io] BNCServerInterface.m(758) Debug: URL: https://api.branch.io/v1/install.
2018-11-06 15:00:46.705468-0800 myApp[643:68349] [branch.io] BNCServerInterface.m(762) Debug: Body: {
    "ad_tracking_enabled" = 1;
    "app_version" = "0.8.0";
    "apple_ad_attribution_checked" = 0;
    "branch_key" = "**********MY_BRANCH_KEY********";
    brand = Apple;
    country = US;
    debug = 1;
    "facebook_app_link_checked" = 0;
    "first_install_time" =<PHONE_NUMBER>976;
    "hardware_id" = "A580888B-DA60-4954-91CE-063D6376FBB6";
    "hardware_id_type" = random;
    "ios_bundle_id" = "com.myApp.myApp";
    "ios_vendor_id" = "4CE22EAF-8CDF-4538-8217-63BAABAE6D6E";
    "is_hardware_id_real" = 0;
    language = en;
    "lastest_update_time" =<PHONE_NUMBER>000;
    "latest_install_time" =<PHONE_NUMBER>177;
    "local_ip" = "<IP_ADDRESS>";
    model = "iPhone10,2";
    os = iOS;
    "os_version" = "12.1";
    retryNumber = 0;
    "screen_height" = 2208;
    "screen_width" = 1242;
    sdk = "ios0.25.5";
    update = 2;
    "uri_scheme" = myApp;
    "user_agent" = "Mozilla/5.0 (iPhone; CPU iPhone OS 12_1 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/16B92";
}
JSON: {"sdk":"ios0.25.5","facebook_app_link_checked":false,"language":"en","user_agent":"Mozilla/5.0 (iPhone; CPU iPhone OS 12_1 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/16B92","screen_height":2208,"country":"US","app_version":"0.8.0","update":2,"ad_tracking_enabled":true,"brand":"Apple","retryNumber":0,"uri_scheme":"myApp","os":"iOS","local_ip":"<IP_ADDRESS>","hardware_id":"A580888B-DA60-4954-91CE-063D6376FBB6","screen_width":1242,"ios_vendor_id":"4CE22EAF-8CDF-4538-8217-63BAABAE6D6E","ios_bundle_id":"com.myApp.myApp","first_install_time":1535396766976,"hardware_id_type":"random","latest_install_time":1541545240177,"debug":true,"is_hardware_id_real":false,"model":"iPhone10,2","os_version":"12.1","branch_key":"key_live_mpC21mILnkhMX7L07pa1ohhgyrfjRRHB","lastest_update_time":1541545229000,"apple_ad_attribution_checked":false}.
2018-11-06 15:00:46.706429-0800 myApp[643:68349] [branch.io] BNCNetworkService.m(232) Debug: Network start operation https://api.branch.io/v1/install.

2018-11-06 15:00:47.027537-0800 myApp[643:68365] [branch.io] BNCNetworkService.m(227) Debug: Network finish operation https://api.branch.io/v1/install 0.322s. Status 200 error (null).
{"session_id":"588490296934334341","identity_id":"588490296929833031","link":"https://myApp.app.link?%24identity_id=588490296929833031","data":"{\"+clicked_branch_link\":false,\"+is_first_session\":true}","device_fingerprint_id":"588490296904838895"}.
2018-11-06 15:00:47.027666-0800 myApp[643:68458] [branch.io] BNCServerInterface.m(818) Debug: Server returned: Status: 200; Data: {
    data = "{\"+clicked_branch_link\":false,\"+is_first_session\":true}";
    "device_fingerprint_id" =<PHONE_NUMBER>04838895;
    "identity_id" =<PHONE_NUMBER>29833031;
    link = "https://myApp.app.link?%24identity_id=588490296929833031";
    "session_id" =<PHONE_NUMBER>34334341;
}.
2018-11-06 15:00:47.029811-0800 myApp[643:68459] [branch.io] BNCNetworkService.m(232) Debug: Network start operation https://cdn.branch.io/sdk/uriskiplist_v0.json.
2018-11-06 15:00:47.113394-0800 myApp[643:68351] [branch.io] BNCNetworkService.m(227) Debug: Network finish operation https://cdn.branch.io/sdk/uriskiplist_v0.json 0.084s. Status 200 error (null).
{"version":1, "uri_skip_list":["^fb\\d+:","^li\\d+:","^pdk\\d+:","^twitterkit-.*:","^com\\.googleusercontent\\.apps\\.\\d+-.*:\\\/oauth", "^(?i).+:.*[?].*\\b(password|o?auth|o?auth.?token|access|access.?token)\\b"]}.

@GeneShay  We think its related to already reported issue. https://github.com/BranchMetrics/cordova-ionic-phonegap-branch-deep-linking/issues/478
We noticed that you that you were calling facebook plugin so if you could try after the PR 521 is merged it may solve the issue else please let us know and we will work on a fix for the problem.

@GeneShay  - Does the latest version  solve your issue ?


Add E2E test to python-base

Instead of mocked API. To really test if things work. There is now kind which can help setup a Kubernetes cluster inside a CI against which then to test.

It's finally arrived for k8s 1.11+...
Yet we've set up the environment for the main repo. It's using minikube inside the CI to run the tests under e2e_test. If you want to make sure it works, you could potentially put some tests there.

we have e2e env set up in the main repo and we do run all tests from both repos in python-base CI (e.g. search for "e2e" here)
I think we could add e2e tests in base repo without extra env change. Question is how we manage dependency and packaging for the tests

on the other hand we can improve https://github.com/kubernetes-client/python/blob/master/kubernetes/e2e_test/test_client.py to test watch/follow behavior after https://github.com/kubernetes-client/python-base/pull/93 is included in the main repo

If we add e2e tests to the main repo, we'll need to potentially merge a feature / bug fix PR to the base repo first, then check in the base repo commit and the e2e test into the main repo at the same time. It's kind of a pain but it's not impossible.
On the other hand, if we set up e2e in the base repo, we should be fine with the dependencies since the CI are run from the main repo but we do need to tune CI testing. If somebody takes #118 some of the obstacles should be cleared out of the way.

/remove-lifecycle stale
/lifecycle frozen
/area test

/cc @roycaihw

I think https://github.com/kubernetes-client/python-base/issues/117#issuecomment-473100144 and https://github.com/kubernetes-client/python-base/issues/117#issuecomment-476335504 still hold. The problem is in python-base developer UX. Developers should be able to add E2E tests in the same PR as their add new features / fix bugs.
Two naive ways of fixing this:

just port some e2e tests from the main repo to the base repo and verify if they work. If so, it can be the starting point of adding more e2e tests to python-base
merge python and python-base into one repo

This repo has been merged into the main python repo. We are archiving this repo. Please open a new issue in the main repo. Thanks!
/close


Use @fluidframework/azure-client can not create container and get Error: 0x883 at app.js:125

User Azure Fluid quick start docs, and try to start it in my loca.
quickstart-fluid-relay

Logs
The browser console error:
[webpack-dev-server] Server started: Hot Module Replacement enabled, Live Reloading enabled, Progress disabled, Overlay enabled.
bootstrap:24 [HMR] Waiting for update signal from WDS...
app.js:125  Error: 0x883
    at assert (assert.mjs:18:1)
    at new SharedTree (sharedTree.js:28:15)
    at TreeFactory.create (treeFactory.js:27:1)
    at new LocalChannelContext (localChannelContext.js:186:1)
    at RuntimeWithRequestHandler.createChannel (dataStoreRuntime.js:206:1)
    at RootDataObject.createSharedObject (rootDataObject.js:95:1)
    at RootDataObject.create (rootDataObject.js:83:1)
    at createObject (rootDataObject.js:39:1)
    at rootDataObject.js:42:1
    at Array.forEach (<anonymous>)
(anonymous) @ app.js:125
Promise.catch (async)
./src/app.js @ app.js:125
__webpack_require__ @ bootstrap:24
(anonymous) @ startup:6
(anonymous) @ startup:6
index.js:485 [webpack-dev-server] Disconnected!
index.js:485 [webpack-dev-server] Trying to reconnect...

The client code I used:
const { container } = await client.createContainer(containerSchema);

Alternatively, if you start from a clean clone of the repo and use Node 14, it should work.

Thanks @nmsimons ,  You are right.
when i downgrade nodejs version from ~16 to ~14.  the FluidFramework for azure works.

and it works in my local. and connect to azure fluid relay server in Mooncake.

Thanks very much.


⚠ ClamAV has degraded performance

In 4c4b643, ClamAV (https://spamassassin.apache.org/) experienced degraded performance:

HTTP code: 200
Response time: 716 ms

Resolved: ClamAV performance has improved in c899cd0.


RTL Layout

Hi.
I'm using your component.
First of all, thank you for this helpful component.
I have a problem. When I force the app to RTL Layout everything go well but just swipe left and right do Not vice versa.
How can I fix this problem?
thanks.

You still does not support RTL layout

+1

+1

Hi,
I have same problem.
can you help me?

+1

help me ...
I have same problem ...


added new project

Added new proejct "Pic Prism" in project.html file

You are not following my account.

You are not following my account.

@Gyanthakur i have followed you.


config/environment.js semicolon to comma update.

It return build error when I added to environment.js because of semicolon.

Thanks for the fix @yusufsagdic!


archive/zip does not set proper CRC, size and modification date/time

What version of Go are you using (go version)?

$ go version
go version go1.11 linux/amd64

Does this issue reproduce with the latest release?
Yes
What operating system and processor architecture are you using (go env)?
go env Output
$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/fale/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/fale/go"
GOPROXY=""
GORACE=""
GOROOT="/opt/go1.11"
GOTMPDIR=""
GOTOOLDIR="/opt/go1.11/pkg/tool/linux_amd64"
GCCGO="/bin/gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build558580817=/tmp/go-build -gno-record-gcc-switches"

What did you do?

https://play.golang.org/p/4_L3bNL6b9n
What did you expect to see?
those should be set properly:

modification time (2 bytes offset 10)
modification date (2 bytes offset 12)
CRC (4 bytes offset 14)
compressed size (4 bytes offset 18)
uncompressed size is (4 bytes offset 22)

What did you see instead?
All those values are set to 0

I don't believe this is an issue.
Please refer to the Data descriptor section, quoted here:

If the bit at offset 3 (0x08) of the general-purpose flags field is set, then the CRC-32 and file sizes are not known when the header is written. The fields in the local header are filled with zero, and the CRC-32 and size are appended in a 12-byte structure (optionally preceded by a 4-byte signature) immediately after the compressed data:

Offset
Bytes
Description[24]

0
0/4
Optional data descriptor signature = 0x08074b50

0/4
4
CRC-32

4/8
4
Compressed size

8/12
4
Uncompressed size

In the example given, The general-purpose flags field is 0x08, which indicates that the data descriptor is used instead. This makes sense because the file header is created before any information about the data in the file is given.
In the output from the above example, the data descriptor signature 0x08074b50 is present at offset 58.

Closing as "working as intended". @DemonWav is correct that zip uses the data descriptor header. This enables the implementation to output zip files in a streaming manner without the need to perform a seek operation back to the front to populate the fields that you are interested in.
Go's usage data descriptors still outputs a valid zip file. The zip format unfortunately gives the illusion that you can parse an archive in a streaming read manner, when that is not a guarantee provided by the format.

sadly there are systems that consider the file as invalid, but I do understand the point and I'll look for a different solution for those cases.
thanks

@Fale if you can think of a good way to edit the standard library to do what you want, and it doesn't depend on anything special, you could maybe "fork" it and pull it in as a dependency. That may be more work than trying to find another solution.

sadly there are systems that consider the file as invalid, but I do understand the point and I'll look for a different solution for those cases.

Yea, I understand. The situation is unfortunate. Had it been more apparent earlier on that such incorrect parsers were so prevalent, perhaps the Go implementation would have taken a different approach.
However, it can't be changed without unless we buffer the entire file (which is non-performant), or change the API to accept an io.WriteSeeker. An API change is unlikely to accepted to support this use-case.

could it be possible to implement a method to be call at close time that re-reads the buffer and calculate those? I was thinking about implementing something like that. and if this would be a sensible path, this method could be an addition to the api that would not break it (and be callable at programmer will, so no performance hit if not needed)

I recommend giving #23301 a read. There was some discussion about APIs and pros/cons for this feature.

tanks @dsnet  :)


Sprint 2 Issue

Assignment
Points
Grade
Evidence

Pull Request (Integration)
1
0.8
(The pull didnt work 4 times but it worked the last time after changing scrum masters. Pull Request worked, and we got our page onto nighthawk pages

Relevancy Checklist (Peer)
2
1.6
Our checklist was do able for our peers and I followed the checklists for others pretty well, as well.

Lesson Style (Period)
1
0.85
We had our lesson with a lot of the right content and code for people to do but we didn't explain the best and we also didn't have the best organization.

Homework, Popcorn Hacks
1 x 5
4.45
We had good popcorn hacks and they were  do able.

Individual Contribution
1
0.9
I contributed a lot to my group, I stayed on call with the scrum master, stayed during office hours to work on our stuff or if we had mistakes.

Personal Notebooks / Blogs
1
0.9
I submitted all my blogs and I did all the hacks to the best of my ability.

Total
11

Skill
Points
Grade
Evidence

Work Habits (Analytics)
1
0.9
I always ask questions, always am willing to improve and I have a lot of commits

Team Planning (Issue)
1
0.9
I presented really will with my team and  we planned out a few days in advance who was presenting whose part.

Presentation Memories
1
0.9
We all matched in attire and we gave out candy to everyone and wanted everyone to be interactive in our presentation

Grading and Feedback
1
0.9
I graded my peers to how they did and I believe I was very fair with it

Beyond Perfunctory
1
0.8
I definitely could have presented better and made our team teach better, I honestly believe we go a higher score than we deserved.

I thought your table was cool, and you discussed how it would relate to your AP exam topics.
You also put you popcorn hacks and such, as well as descriptions of how you did it.
I thought that your biggest achievements page was cool, you put milestones of you coding journey which was something innovative.

Lalita had a table similar to Manas, but her summary was different. She had screenshots of every popcorn hack and explained how she did it, which I really liked. It would be really helpful when reviewing. She also had an accomplishments, and it was clear that she made a lot of progress and improvement over Sprint 1 and 2.

Made a submenu and summarized the lessons and put the team teaches in the submenu as well. made a public page and was one of her best accomplishments and mentioned how she cam in class late. talked about checkpoints. redid her team lessons from feedback.

Hi Lalita, I thought the descriptions you added to your codes were very helpful, and also liked how your achievement page included your checkpoints showing much progress throughout your journey.

Hi Lalita, I like how you talked about your checkpoints and the organization of your page. I like the submenu and how you talked about how this will benefit you in the future.

It's pretty cool how you had an overview for all the lesson, which would be beneficial when revising these topics for the AP exams. Your blogs is really clean, and mentioned major checkpoints and achievements. It's also cool how

Table summary with link to lessons.  Capture hacks in personal space.  Talked about capturing things through the trimester.  Talked about fixing U2 for learning.   Interest on bio discussed.

Hi Lalita,
I liked that you had a submenu with all the lessons and popcorn hacks. I also liked that you went over your achievements and talked about how you had issues with 3.2, but learned how to code overtime.

I like the achievements page (looks simlar to mine🤔) but I really like the minimalism of your page and how well you were prepared for this and I was able to understand the purpose behind each thing you did on the page. Awesome job Lalita!

Used python knowledge to learn javascript skills, also had a useful table to refer for future study. Also had popcorn hacks that focused on the skills you learned. I also liked how you show your progress throughout CSP.

Hi Lalita, the tables with the summary on each one is very descriptive, and you did a good job presenting. I also think your accomplishments page is very detailed, which helps us understand how you are going through this class.

The table with the custom summary was very organized and easy to access. Your accomplishments were very detailed.

I really liked how you went over each aspect of your learning. Although you came to CSP late, it seems like you learned a lot. I like how you noted your main accomplishments. Maybe next time you can show a method of application.

Great job presenting your page. You showed how you really applied what you learned. I thought it was really interesting that you went back changed up your hacks and really understood went wrong.

I loved how made the table and talked about how it will help with you for future assignments and the AP exam. You had all the popcorn and homework hacks done correctly and explained about what you liked and what you needed to improve on.

your table summary ws really good andyou talked about how it would help you, you also talked about your issues and problems you went through which was really cool

I liked how even though u came to CSP late and had obstacles you quickly adapted and have been improving with code.


Error 500 after privileged user session expiration

When a session expires, especially when the user is an Administrator, a URL protected by Check* middleware will return a 500 instead of a 401. This is because the code ( CheckIsParticipantOrAdmin.php@19 ) doesn't take into account that Auth::user() may be null.
All the other middlewares that checks roles should be affected.

Debuged in  000150f0a741afd0132083f43926cdccccae51e8


Disque can report a QLEN higher than the number of available jobs (race condition)

Hi,
This is another bug, it is highly non-deterministic and requires you to be fairly "lucky" to hit it. One disque server is needed. Same test as in #22, running the same commands, but following the insert of the 8 jobs, run a QLEN. In rare circumstances, disque will report 9 rather than 8. It may be related to deletions not "falling through" or it may be related to other data entirely. When I first provoked this error, the data inserted was random data, so it may be due to an earlier state.
It looks like some kind of race internally, though all my data access is sequential on a single connection toward disque.

This can definitely be closed. It relates to #22, and I think it would be fairly easy to get the model out of sync with reality once you know the model can't clean up in the right way (i.e., the what-does-it-mean-to-flush-a-queue question).


[BACKPORT] [1.11] DCOS-50823 - Mute frequently flakey tests.

High-level description
This mutes tests that are known to be flakey.
Corresponding DC/OS tickets (obligatory)
These DC/OS JIRA ticket(s) must be updated (ideally closed) in the moment this PR lands:

DCOS-50823 Mute flakey tests.

Related tickets (optional)
Other tickets related to this change:

DCOS-19790 Flaky Test For DCOS Release: test_networking.test_if_overlay_ok
DCOS-47438 test_networking.test_dcos_net_cluster_identity[true] non-zero exit status 1
DCOS_OSS-4922 test_rexray.test_move_external_volume_to_new_agent fails on master.
DCOS-50156 flaky test_clusterstate.test_ee_systemd_units_are_healthy
DCOS-46578 DC/OS Enterprise - test_metronome.test_metronome flake - Exception: Job run failed - operation was not completed in 300 seconds.

Checklist for all PRs

[x] Added a comprehensible changelog entry to CHANGES.md or explain why this is not a user-facing change: muting tests
[x] Included a test which will fail if code is reverted but test is not. If there is no test please explain here: N/A muting tests
[x] Read the DC/OS contributing guidelines
[x] Followed relevant code rules Rules for Packages and Systemd

@mesosphere-mergebot label Ready For Review

@orsenthil changed!

Please update the description too. Otherwise mergebot updates on those JIRAs might cause alarm.


nixos/tests/ft2-clone: cleanup

Description of changes

Remove unused config argument
Don't enable the xserver again, since the import ./common/x11.nix already takes care of that
Remove an empty line at the end

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/prs-ready-for-review/3032/3154


Items disappear when closing world

JoeThePanda:
hey guys, anyone having the issue of inventory in wooden hoppers disappearing when you close a world? i have them in my single player world and whenever i close it, whatever stack was in the wooden hoppers disappear.
Latest version for 1.18.1

fixed in 1.18.1-<IP_ADDRESS>


Update for redis v2.4.0 which supports TLS and formatting fixes

The<EMAIL_ADDRESS>NPM introduced support for TLS so you no longer need to (nor should) disable SSL to access Azure Redis Cache from Node. This PR also fixes some minor formatting issues.

Hi @willbar, I'm your friendly neighborhood Azure Pull Request Bot (You can call me AZPRBOT). Thanks for your contribution!

It looks like you're working at Microsoft (willbar). If you're full-time, we DON'T require a contribution license agreement.

If you are a vendor, DO please sign the electronic contribution license agreement. It will take 2 minutes and there's no faxing! https://cla.azure.com.

TTYL, AZPRBOT;

Hey @steved0x, can you review this PR?
Thanks,
Dave

Reviewing with the Cache team. Thank you Will for making this submission.

@steved0x any progress on this PR?

@ggailey777 Hi Glenn, let's merge it so that @willbar is credited as a contributor in the article (per Tyson's recent guidance) and then I will go back in and add the note that Pranav suggests. I'll review the article at the same time for any other content freshness updates needed. Thank you again Will for the contribution!


Document the cipher descriptor and the parameters to each function?

Would it be possible to more thoroughly document the cipher descriptor? Right now it defines the functions but I have no idea what each of the arguments mean or when they are important. I can make some inferences, like the cipher pointer pointing to the actually allocated cipher struct, but a lot of the other arguments (like rekey for example) are non-obvious.  As another example, the encrypt/decrypt functions take a len and reserved argument, but is this the length of the page and the number of bytes reserved that I should subtract from len? Where are the reserved bytes at? Things like that. I know that the SQLite documentation can somewhat fill in here but not entirely.

Would it be possible to more thoroughly document the cipher descriptor?

Possible of course. However, up to now this has not happened, because presumably more than 99.9 % of the SQLite3 Multiple Ciphers users are not interested in this feature.
The option to dynamically register cipher schemes was added about 2 years ago, after discussions with a developer in need of bringing his own encryption scheme. Since then no one else asked any questions about this feature.
The documentation section Dynamic Cipher Schemes holds some basic information, but it certainly could be more elaborate.

Right now it defines the functions but I have no idea what each of the arguments mean or when they are important. I can make some inferences, like the cipher pointer pointing to the actually allocated cipher struct,

Correct. And the size and layout of this structure is defined by the cipher implementation.

but a lot of the other arguments (like rekey for example) are non-obvious.

Agreed.

The latter is usually not the case. A cipher implementation has to provide those functions, which are called by a higher level of the encryption scheme implementation. Typically, you don't call them yourself.

As another example, the encrypt/decrypt functions take a len and reserved argument, but is this the length of the page and the number of bytes reserved that I should subtract from len?

Using len as the parameter name was certainly not my brightest idea. And yes, it is the page size.

Where are the reserved bytes at? Things like that.

Reserved bytes are always located at the end of the page buffer. Usually, SQLite "knows" the number of reserved bytes from information in the database header. However, some legacy cipher schemes encrypt the database header in an incompatible way.
I can try to fill the gaps, but this will take some time, because my time resources are limited.

I know that the SQLite documentation can somewhat fill in here but not entirely.

To a certain degree the publicly available documentation about [SEE}(https://sqlite.org/see/doc/release/www/readme.wiki) provides information, but because SEE is a commercial product, there are not many details about the inner workings.

@utelle I mean, I really only need some things documented I think. Like I have the following questions about the functions:

What is the purpose of the GetSalt_t function? Right now I just return nullptr.
In the GenerateKey_t function, what do the pBt, rekey, and cipherSalt arguments do/what are they for? Do I read or write to them (or both)?
In DecryptPage_t, what is the hmacCheck parameter? What do I do if it's true? Or am I supposed to ignore it?

Right now I just ignore all of these parameters that I'm not certain about. I don't plan on supporting legacy ciphers or anything, so that isn't an issue.
My last two  questions are:

If I want to omit all the built-in ciphers, how do I prevent it from printing a warning (since I'm going to register my own), and is the CODEC_TYPE define actually important? How does it affect the encryption/decryption operations?
Is it possible for me to support in-memory decryption only? As in, can I attach an (encrypted) database onto the end of an executable, maybe use a VFS to tell it where the DB is at, and then make SQLite think that it's a file when it really isn't? Could I still get decryption at least?

This is really all I need to know I think, other than what you've already provided. Though I might have more questions in the future. I understand your time is limited but any assistance is appreciated.

I mean, I really only need some things documented I think.

You can always ask questions here (in GitHub issues) or via mail. Usually I try to respond in a timely manner.

Like I have the following questions about the functions:

What is the purpose of the GetSalt_t function? Right now I just return nullptr.

A cipher scheme can decide to use salt for the key derivation. The salt is an array of 16 bytes, which is stored in the first 16 bytes of the SQLite database header.
Typically, random bytes are used. Sometimes it is necessary to make the salt available to the application. For example, if you want to use salt, but can't encrypt at least part of the header - on some platforms the first 16 bytes must contain the SQLite identifier "SQLite format 3\0". Function GetSalt_t returns a pointer to the salt bytes. If you don't use salt, you can and should return nullptr. The salt can be queried with a SQL function, of which the implementation calls GetSalt_t.

In the GenerateKey_t function, what do the pBt, rekey, and cipherSalt arguments do/what are they for? Do I read or write to them (or both)?

pBt: this is a pointer to the BTree of the current SQLite pager. In the current cipher implementations it is required to be able to read raw bytes from the database header. This is something I plan to change in a future version by providing a utility function for this purpose.
rekey: this is a boolean (taking values 0 or 1). If it is true (1) the key salt should be renewed (generate random bytes). Otherwise new key salt is only generated, if the database is new and empty.
cipherSalt: this is an array of 16 bytes. For an existing non-empty database this can be used to overwrite the salt from the database header.

In DecryptPage_t, what is the hmacCheck parameter? What do I do if it's true? Or am I supposed to ignore it?

hmacCheck: this is a boolean (taking values 0 or 1). For ciphers with a HMAC (stored in the reserved bytes of each page) the value will be usually 1. But it can be temporarily set to 0 to turn off HMAC checks on reading database pages. This is useful if you have a corrupted database and want to try to recover it at least partially.

Right now I just ignore all of these parameters that I'm not certain about.

Whether you need to take these parameters into account, depends on your cipher scheme.

I don't plan on supporting legacy ciphers or anything, so that isn't an issue.

For new cipher schemes it is strongly recommended to leave bytes 16 to bytes 23 of the database header unencrypted - as is the case for the official SEE implementation. SQlite reads these bytes before the encryption scheme is initialized to determine the page size and the number of reserved bytes.

My last two questions are:

If I want to omit all the built-in ciphers, how do I prevent it from printing a warning (since I'm going to register my own),

I guess you mean the compiler warning that no built-in cipher is enabled. Well, currently there are no means to disable it.
However, you are free to remove it from the code. To register your own cipher you will have to modify the code of function sqlite3mc_initialize.

and is the CODEC_TYPE define actually important?

Yes, it holds the default codec type. If it is not set, it will be set to CODEC_TYPE_CHACHA20.
It could be that adjustments to the source code need to be done. Unfortunately, I never got feedback from the developer who wanted to implement his own cipher scheme.

How does it affect the encryption/decryption operations?

It will determine the default cipher scheme, if it was not selected via PRAGMA cipher or via URI parameter.

Is it possible for me to support in-memory decryption only?

No. SQLite3 Multiple Ciphers has the same restrictions as SEE: in-memory databases and temporary databases will not be encrypted.

As in, can I attach an (encrypted) database onto the end of an executable, maybe use a VFS to tell it where the DB is at, and then make SQLite think that it's a file when it really isn't? Could I still get decryption at least?

In principle, you could use the VFS shim apndvfs (implemented in ext/misc/appendvfs.c of the SQLite source distribution). The initialization function of that extension registers the VFS shim for the default VFS. And thereafter you could register the encryption extension on top of the VFS apndvfs. (I have to admit that I haven't tested this, but it should work.) However, if you append a database to an executable, you must open the database in read-only mode.

This is really all I need to know I think, other than what you've already provided. Though I might have more questions in the future. I understand your time is limited but any assistance is appreciated.

Usually, I try to answer questions timely.

@utelle

rekey: this is a boolean (taking values 0 or 1). If it is true (1) the key salt should be renewed (generate random bytes). Otherwise new key salt is only generated, if the database is new and empty.

So if I understand this right, I should generate a new salt if rekey is true, otherwise leave it alone? And if I do generate a new salt, I store it in cipherSalt, and use cipherSalt as the salt for key generation, regardless of the value of rekey?

cipherSalt: this is an array of 16 bytes. For an existing non-empty database this can be used to overwrite the salt from the database header.

Waaaait, so I can use a different salt every time I open a database, and that will trigger a call to generate_key? (I feel like that might make the call recursive but...)

So if I understand this right, I should generate a new salt if rekey is true, otherwise leave it alone?

Yes, if your cipher scheme uses a key salt, that is. If your cipher does not make use of salt, then you may ignore the parameters rekey and cipherSalt.

And if I do generate a new salt, I store it in cipherSalt,

No. You store it in the contect struct of your cipher implementation. And then you have to put it into the first 16 bytes of the database header located on database page 1. Please look at the code of the cipher implementations.

and use cipherSalt as the salt for key generation, regardless of the value of rekey?

Certainly not. Only if rekey equals 0.

cipherSalt: this is an array of 16 bytes. For an existing non-empty database this can be used to overwrite the salt from the database header.

Waaaait, so I can use a different salt every time I open a database,

Of course not. There are 2 situations where you need to specify the salt explicitly:

The database header was corrupted.
The database header is unencrypted.

In both cases you have to provide the salt with which the database was created.

and that will trigger a call to generate_key? (I feel like that might make the call recursive but...)

No. GenerateKey is called after the database was opened and the passphrase was provided, either via PRAGMA key or via URI parameter. If the URI parameters include the parameter cipher_salt, then its value is passed on to GenerateKey.

Okay, last set of questions I think, thank you so much for yoru help:

In principle, you could use the VFS shim apndvfs (implemented in ext/misc/appendvfs.c of the SQLite source distribution). The initialization function of that extension registers the VFS shim for the default VFS. And thereafter you could register the encryption extension on top of the VFS apndvfs. (I have to admit that I haven't tested this, but it should work.) However, if you append a database to an executable, you must open the database in read-only mode.

If I do use a VFS for reading the appended database, how would I enable the decryption support? I would never allow read-write access for such a thing since it could corrupt the executable, but if I did append the database onto the end, I would need some way of reading it back. Do I automatically get decryption with VFSs, or do I need to do something special to enable that, if I were to try this?

If I do use a VFS for reading the appended database, how would I enable the decryption support?

On opening the database you will have to specify the VFS to be used. Assuming that you already registered the "appendvfs" then you specify the VFS as multipleciphers-apndvfs. This will create the VFS if it doesn't already exist.

I would never allow read-write access for such a thing since it could corrupt the executable, but if I did append the database onto the end, I would need some way of reading it back. Do I automatically get decryption with VFSs, or do I need to do something special to enable that, if I were to try this?

You have to enable the "appendvfs" extension and then to specify the VFS to be used as described above.

@utelle Oh okay, thank you! I appreciate all the help!


Record the licenses to which an exception may apply

I'm approaching this from the perspective of trying to make a best guess determination of the SPDX License Identifier from the results of scancode-toolkit. In the absence of an explicit tag, scancode does a fantastic job of identifying the various licenses, but doesn't necessary know how they combine.
In the case where a license and an exception are identified by the scanner, it's relatively easy to stitch the two together using with. It's less easy to guess whether two licenses identified by the scanner are combined via conjunction or disjunction, but we can at least automate creating a consistent SPDX identifier.
In the case where content is multiply licensed and an exception is involved, we can guess to which license the exception belongs, but the guess could be made better if we have a means of identifying to which license an exception likely belongs.
That is, if scancode detects Apache-2.0, GPL-2.0-only, and Classpath-exception-2.0 in content that does not include an SPDX-License-Identifer tag that might otherwise tell us how these are combined, we could reasonably guess that the license might be Apache-2.0 and GPL-2.0-only with Classpath-exception-2.0 and not Apache-2.0 with Classpath-exception-2.0 and GPL-2.0-only  (whether or not and is right is a toss up, but we can make a reasonable assumption that the classpath exception applies to the GPL).
This specific example feels pretty obvious, but I don't know enough about most of the exceptions to understand whether or not this is something that can be reasonably done consistently by adding a bit of extra metadata to the JSON files for the exceptions that lists the licenses to which the exception might apply.
This is part of a class of problem that I assume others are thinking about, e.g., how to combine and collapse licenses (GPL-2.0-only and GPL-2.0-or-later can collapse to GPL-2.0-only, or GPL-2.0-only+ is basically the same as GPL-2.0-or-later). Is that discussion happening somewhere?

Moved to https://github.com/spdx/license-list-XML/issues/1457


fix(k8s): correct cluster detection

The addition of the --no-headers option to the kubectl config get-contexts command broke the if logic for the first case of the exit code 45 as the command headers were originally taken into account.
The --no-headers option was removed; could not see a reason for which it would be better to keep this option and instead update the if logic counts.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Possibly update Kafka-related doc about enable.idempotence default change by Kafka

Environment Details

Helidon Version: 2.x, 3.x, 4.x
Helidon SE or Helidon MP
JDK version:
OS:
Docker version (if applicable):

Problem Description
Kafka has changed the default value of enable.idempotence from false to true between (seemingly) releases 3.0 and 3.0.1. This can cause the Kafka error
org.apache.kafka.common.errors.UnsupportedVersionException: The broker does not support INIT_PRODUCER_ID

Explicitly setting enable.idempotence to false avoids this problem.
We might consider updating our messaging doc to alert users to this.
But, that can lead to another related error:
INFO  o.a.k.c.c.i.ConsumerCoordinator - [Consumer clientId=consumer-workspace-manager-async-publish-1, groupId=workspace-manager-async-publish] Request joining group due to: rebalance failed due to 'The session timeout is not within the range allowed by the broker (as configured by group.min.session.timeout.ms and group.max.session.timeout.ms).' (InvalidSessionTimeoutException)

Because the default timeout value has also changed, apparently from 10000 to 45000.
Explicitly configuring the value to 10000 works around this second issue.
The enable.idempotence issue was reported on our internal Slack channel and as well by other platforms: https://github.com/spring-projects/spring-boot/wiki/Spring-Boot-2.6-Release-Notes#kafka-30

https://github.com/helidon-io/helidon/issues/6718


Don't Panic in getComposeStatus and skip invalid jobs in fsjobqueue New

This handles corrupt job json files by skipping them. They still exist, and errors are logged, but the system keeps working.
If one or more of the json files in /var/lib/osbuild-composer/jobs/ becomes corrupt they can stop the osbuild-composer service from starting, or stop commands like 'composer-cli compose status' from working because they quit on the first error and miss any job that aren't broken.
This pull request includes:

[ ] adequate testing for the new functionality or fixed issue
[ ] adequate documentation informing people about the change such as

[ ] submit a PR for the guides repository if this PR changed any behavior described there: https://www.osbuild.org/guides/

This 'fixes' #3779 by ignoring broken job json files. The big question is what to do with them. Should they be removed? Moved to the side for later examination (and somehow expired before the filesystem fills up)? Or leave them in place (and fill up the filesystem eventually).

Rebased to prevent OCI private key leaking. Apologies :(


Error

"An internal error has occured in the compiler"
Project OpenWindow
File gfxdeviced3d12.cpp
VS2015

Weird. Main branch compiles fine on my machine (win10 and VS2015 Community update 1). I tested both Debug and Release configurations.


Listening to trigger

Is there a way that I can recieve automaticy the triggers notifications?
trigger

Can you elaborate on what you are asking for? What kind of code do you want to be able to write ?
A trigger runs inside the database. A trigger might block execution of a query, and the driver would just see the error.


feat: Add LFortran as linter

see LFortran PR https://github.com/lfortran/lfortran/pull/647
Fixes #589

@certik This adds support for LFortran in VS Code. I will be adding a "Run" button soon, to compile and run code.


Suggestions pop up of the Japanese MS IME got pushed down after multiple commands

Windows Terminal version
1.15.2282.0
Windows build number
10.0.19044.1889
Other Software
No response
Steps to reproduce

Entering multiple commands or just press Enter multiple times.
Japanese MS IME's suggestions pop up got pushed down gradually.

Expected Behavior
this doesn't happen on v1.13.10983.0

what's being typed is shown in the input area

Actual Behavior
This happens on v1.14.1432.0 and above so it might be related to https://github.com/microsoft/terminal/pull/13028
v1.15.2282.0

v1.14.1432.0

also, what's being typed isn't shown in the input area

oh no, we must have lost a DIP<->Pixel conversion somewhere in #13677 or #13678.

We may also (or instead) have mixed up a viewport -> buffer conversion.

I'm bisecting. :)

I may not be able to finish bisecting just yet. Here's the current bisect log:
# bad: [39c71d642179403d52b04a03de7014820ab85657] Debounce window state changes caused by the PTY (#13147)
# good: [29e97b61823297b414c41fc70725569a2acf32cf] Update readme to account for minversion bump (#12332)
git bisect start 'v1.14.1432.0' '29e97b618' 'src/cascadia'
# good: [73f02024d29cf558e42633f93e707631c306bf6f] AtlasEngine: Fix grayscale blending shader (#12734)
git bisect good 73f02024d29cf558e42633f93e707631c306bf6f
# good: [30383ccc2210e3e7d62323421748c6eedf4b5fdd] Revert "Add support for rich code navigation in GitHub (#12855)" (#12909)
git bisect good 30383ccc2210e3e7d62323421748c6eedf4b5fdd

This is weird, but bisect pointed to f4e0d9f2bd1f5da3a6c378453b90320fb6b1c542 (#13024)
It did remove a copy of Terminal::GetCursorPosition that returned the position adjusted by the viewport offset from TerminalApi.cpp.
We may need to introduce something like that again -- or propagate knowledge of the viewport offset up to ControlCore?
(/cc @j4james)

Oh my god, I think we were just getting unimaginably lucky the whole time?
src\cascadia\TerminalCore\TerminalApi.cpp
53:til::point Terminal::GetCursorPosition()

src\cascadia\TerminalCore\terminalrenderdata.cpp
41:COORD Terminal::GetCursorPosition() const noexcept

There were two, and one of them was pulled when we were const-qualified.
James' PR removed the first one.

Sorry about that. I'll try and have a look on the weekend if someone else hasn't fixed it by then.

@j4james no worries, I've submitted PR #13785 for it.


doc(Algebra/Category/MonCat/Colimits): label #print remarks as only working in Lean 3

This also fixes some markdown issues.

Thanks!
bors merge


Bump mypy from 0.931 to 0.940

Codecov Report

Merging #211 (330e304) into main (2d5b5d3) will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##              main      #211   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files           14        14
  Lines          821       821
=========================================
  Hits           821       821

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2d5b5d3...330e304. Read the comment docs.


Deposits into Idle Vaults aren't showing in Earn page Balance

Overview
After depositing into some Idle Vaults, the balance still shows as $0.00 in the Earn page.
References and additional details

Log into app.shapeshift and connect to wallet of choice
Navigate to Earn page and select 'By Provider' tab
Scroll to Find the idle provider. Note the value states $0.00
Select the Idle provider and note the drop down list of positions all have zero dollars
Select on position that has a balance and note in the overview screen there is a balance
However, note that the Underlying Tokens has a zero balance as well

Acceptance Criteria
When there is a balance in the idle position this should be shown in the earn page and provider position
Need By Date
No response
Screenshots/Mockups
No response
Ownership

[X] If my bounty needs engineering or needs product I have added the respective labels on the right
[X] As the sponsor of this bounty I will review the changes in a preview environment (ops/product) or review the PR (engineering)

Estimated effort
No response
Sponsor / Stakeholder
No response
Bounty Hunters

Join our discord
Include an expected timeline for you to complete work in the work plan when you apply for this bounty!
Please refer to this link for some basic info
Please do not start work on this issue until you are approved in Gitcoin.

Just noticed that MM is showing Idle balances, but native is not. Not sure if that helps narrow anything down for you @gomesalexandre  :)

@MBMaria can you please retest? there have been a lot of changes in this part of the codebase since this ticket was written

i am still experiencing this

so looks like the dai vault showing $0.00 is correct but the usdc vault is not showing a balance either and on etherscan it shows there should be.

IdleUSDC Yield


Do you need to search/adjust any hyper-parameters for different dataset?

Do you need to search/adjust any hyper-parameters for different dataset? Since most previous siamese-like trackers need to do this for achieving their best performance.

@sallymmx Hi, for simplicity, we only tune the "update interval" of STARK-ST. There are not any extra hyper-parameters during inference. This makes our tracking performance more stable and makes the framework cleaner.


DMT-15 Set Up a UX Prototype

Related issue

DMT-15

Proposed changes

Add the UX prototype and documented its use.

Additional information

Shows and describes the planned MVP for V.1.0.

Looks good!


node and tests and docs

This pull request adds node support along with tests, examples, and more docs.
To run the tests in node:
$ npm test

to run the tests in the browser:
$ npm install -g browser-run browserify
$ browserify test/*.js | browser-run

hah, thank you! Any objections about becoming a contributor?

Sure! I plan on using this module all over the place.

added you both here an on npm


ng serve not reloading browser tab when has changes, SSR app on mac

Command
serve
Is this a regression?

[ ] Yes, this behavior used to work in the previous version

The previous version in which this bug was not present was
No response
Description
My angular app does not detect changes in one of my apps. I tried to change any files but none of them browser did not update. My other app which is Universal not installed works fine. Both apps running the same environment. When I run npm run dev:ssr browser detects changes and tries to reload but the loading stuck and it needs manually stop loading and refresh to get changes.
Minimal Reproduction
Angular Universal installed.
My device is macOS Monterey 12.0 Beta.

No response

Also i have webpack issue at ng serve, this is a browser issue and when i try to add new style file source to angular.json it not get changes, if i use npm run dev:ssr it get the style file
index.js:43 Uncaught TypeError: __webpack_require__.h is not a function
    at 1211 (index.js:43:58)
    at __webpack_require__ (bootstrap:19:1)
    at __webpack_exec__ (styles.scss?24a6:1:1)
    at styles.scss?24a6:1:1
    at webpackJsonpCallback (jsonp chunk loading:29:1)
    at styles.js:1:87

It's probably related some npm package version, when i need to move apps prod i need change my local env, it needs to be same with server, then i start using nvm, it starts after that changes


ui: multilang date

fyi no need to update translations for changing line numbers any more, but you're free to keep doing it if you want. thanks for the PR!


feat: Replace codebase with Snke source and support CodePipeline deployment

Issue #, if available:
Description of changes:
Checklist:

[ ] Have you successfully deployed to an AWS account with your changes?
[ ] Have you written new tests for your core changes, as applicable?
[ ] Did you add new integration tests?
[ ] Did you run integration tests with your changes?

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

This PR is not reviewable. I will break it into several parts addressing each components separately.


Fix: Null Access error when requesting a non-existant route.

Create a default 404 response to requests for routes that do not exist.
Implementation provides a setter within the Weblink class that allows an individual developer to define an application specific response following the Func type definition if desired.
As well added a test to Path for this... as well as demonstrate the 404 override by simply taking the implementation and replacing the default one, this could probably be subtracted.

@poklj Nice work!


x86_64-unknown-linux-musl rustup-init links to dead page

Problem
Downloading rustup-init for x86_64-unknown-linux-musl results in a dead link.
Steps

Go to rustup.rs
click on other installation options
Try to download x86_64-unknown-linux-musl

x86_64-unknown-linux-musl build file is merged recently but it is not released yet.
But @kinnison will cut a release soon, I believe.

@TimDiekmann the MUSL release is not yet official I think.  That link is, irritatingly, a little over-eager :-D
As @lzutao says, I hope to cut a release soon, so it ought to be fixed then.

Assigned to myself so I remember to update after the release

Seems to work now with the new release, thanks!