Refactor Build.Stage

To support incremental computation, we want to either refactor Build.Stage or add a new concept of Build.IncrementalStage or similar.  Roughly speaking this would look something like this:
interface Build.Stage<T extends SyntacticItem,S extends Build.State> {
  public S apply(T item, S state);
}

The ideal is that the stage is being requested to rework a given section of a WyilFile (or similar).  That is everything under the enclosing item.  The State used by the stage is managed by the build system.  The stage returns the updated State after the given item.  The build system will compare this with the old state and store it.  If no changes, then it will stop.  If there are changes, then it will propagate upwards (somehow).
To begin with a given stage can just default to recompiling the whole file (somehow).  For example, it will force an application to the root item.

This should somehow connect with Java's ForkJoinTask.


Update block info table text at top of block pages on sciviz

Currently the table at the top of each page has the following useful info:

Parameter
Value

experiment_name
social0.2-aeon4

block_start
Wed, 14 Feb 2024 17:37:58.1984 GMT

block_end
Wed, 14 Feb 2024 19:29:19.41334 GMT

block_duration_hr
1.856

block_duration
1.82797

patch_count
3

subject_count
2

It would additionally be nice to have rows for

subject_names
total_pellets
patch_rates (patch_name : rate)
experiment_phase

Done in #451


CupertinoListSection examples not working

Also, CupertinoListTile ain't working.
https://api.flutter.dev/flutter/cupertino/CupertinoListSection-class.html
Tried with Safari and Chrome.

duplicate of https://github.com/flutter/flutter/issues/118161


add feedback to the send proof

This PR introduces feedback for proof verification
To test it send a proof, for example:
make batcher_send_plonk_bls12_381_task

remember to send two tasks
or
aligned --proving_system GnarkPlonkBls12_381 \
--proof batcher/aligned/test_files/plonk_bls12_381/plonk.proof \
--public_input batcher/aligned/test_files/plonk_bls12_381/plonk_pub_input.pub \
--vk batcher/aligned/test_files/plonk_bls12_381/plonk.vk \
--repetitions 2

You will see a Waiting for batch signatures verification in Ethereum... message  every 3 seconds until the proof is verified

Stale


spring-cloud-starter should not depend on Spring Cloud Config

As pointed out in https://github.com/spring-cloud-samples/eureka/issues/11 spring-cloud-starter depends on spring-cloud-config-client  and therefore on Spring Cloud Config. This should be changed IMHO. Spring has always emphasized choice. There are ways to configure a distributed system of Microservices with other approaches i.e. Ansible or Puppet. The Immutable Servers Pattern even argues that a change to a server should alway lead to a new server being start. In those scenarios Spring Cloud Config is not needed. Therefore I would make it optional and remove the dependency.

But Spring Boot (and starters) are about being opinionated and providing an exit route if you have a different opinion. The opinion being expressed here is that the config server is a good thing, and so we make it easy to use it out of the box. So I would argue that if you don't want the full Monty you can just include the dependencies you need (or exclude the ones you don't).

Spring Boot Actuator is also a good thing but still optional.

It's not optional for a Spring Cloud app (using the starters anyway). That kind of feature has obvious immediate applicability in a distributed system. And so does the config client.


Support for image upload

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
Describe the solution you'd like
A clear and concise description of what you want to happen.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.

@humble-barnacle001 Add more info the issue.
Issue template has clear cut def of what is needed to create issue. Pls follow it. I am closing this issue for now. Re-open or create one more with more desp.


fixes for #13767

Fixes #13767:

Implements TLS1.3 ToDos (group name retrieval) and fixes incorrect EC output reference in apps/lib/s_cb.c
Eliminates incorrect OPENSSL_NO_EC and OPENSSL_NO_DH code guards for group-accessor functions and data structures

Checklist

[x] documentation is added or updated
[x] tests are added or updated

There is no real commit message. The Fixes #... should be on a separate line in commit message bod, the commit header line should summarize the change with commit message body giving more details.
Also please unless absolutely necessary you should use a real e-mail address for commits - and it needs to be the address that you submitted with the ICLA. Otherwise the cla-check will fail.

There is no real commit message. The Fixes #... should be on a separate line in commit message bod, the commit header line should summarize the change with commit message body giving more details.
Also please unless absolutely necessary you should use a real e-mail address for commits - and it needs to be the address that you submitted with the ICLA. Otherwise the cla-check will fail.

Close/reopen to kick CLA bot

Close/reopen to kick CLA bot

I added the anonymous github email address as an alias so that the CLA check passes.

I added the anonymous github email address as an alias so that the CLA check passes.

There is no real commit message. The Fixes #... should be on a separate line in commit message body, the commit header line should summarize the change with commit message body giving more details.
Also please unless absolutely necessary you should use a real e-mail address for commits - and it needs to be the address that you submitted with the ICLA. Otherwise the cla-check will fail.

Thanks for the guidance. Looks like I'm already relying too much on the git UI and less on its command line features. Will remember the next time. I also changed user.email to a less anonymous one now.

There is no real commit message. The Fixes #... should be on a separate line in commit message body, the commit header line should summarize the change with commit message body giving more details.
Also please unless absolutely necessary you should use a real e-mail address for commits - and it needs to be the address that you submitted with the ICLA. Otherwise the cla-check will fail.

Thanks for the guidance. Looks like I'm already relying too much on the git UI and less on its command line features. Will remember the next time. I also changed user.email to a less anonymous one now.

To fix the commit message you can use git commit --amend command. Then you just have to do git push --force.

To fix the commit message you can use git commit --amend command. Then you just have to do git push --force.

To fix the commit message you can use git commit --amend command. Then you just have to do git push --force.

Thanks. So done.

To fix the commit message you can use git commit --amend command. Then you just have to do git push --force.

Thanks. So done.

I agree with @kaduk comment about separating the different (related) concerns in different commits: you could edit your git history with an interactive rebase and then force-push to this same branch if you are willing to do so.
This way the history of feedback given in this PR is not lost!
Also, thanks again for working on this issue!

I agree with @kaduk comment about separating the different (related) concerns in different commits: you could edit your git history with an interactive rebase and then force-push to this same branch if you are willing to do so.
This way the history of feedback given in this PR is not lost!
Also, thanks again for working on this issue!

@romen Your comment seems misplaced from another PR? I do not even see @kaduk commenting here.

@romen Your comment seems misplaced from another PR? I do not even see @kaduk commenting here.

There was discussion in the related issue: https://github.com/openssl/openssl/issues/13767#issuecomment-755114640

There was discussion in the related issue: https://github.com/openssl/openssl/issues/13767#issuecomment-755114640

you could edit your git history with an interactive rebase and then force-push to this same branch if you are willing to do so.

Well, then I must admit I'm apparently not sufficiently educated in rebasing: I thought one can use rebasing only to "fiddle with" separate commits if they already exist, but not retroactively create new "piece-part" commits after having done everything in one go. Or is the idea to git reset --hard the branch, add piece-part commits until we arrive at the same state again (heeding the feedback above), and then push --force again? Or do I misunderstand you @romen?

you could edit your git history with an interactive rebase and then force-push to this same branch if you are willing to do so.

Well, then I must admit I'm apparently not sufficiently educated in rebasing: I thought one can use rebasing only to "fiddle with" separate commits if they already exist, but not retroactively create new "piece-part" commits after having done everything in one go. Or is the idea to git reset --hard the branch, add piece-part commits until we arrive at the same state again (heeding the feedback above), and then push --force again? Or do I misunderstand you @romen?

Yes, it is basically the same thing with just one commit, I guess I am mostly using the interactive rebase as that way it is easier to retain commit messages and to abort and go back to original state if I make some mistake.
You can use whatever strategy you are more confident with!

Yes, it is basically the same thing with just one commit, I guess I am mostly using the interactive rebase as that way it is easier to retain commit messages and to abort and go back to original state if I make some mistake.
You can use whatever strategy you are more confident with!

Also, whoever merges this, please remember to squash the commits together!
@baentsch one nice way I learned about while working on openssl, to keep the "addressing feedback" commits separated but still have the commits squashed when we will merge is: git commit --fixup=<commit_ref> (you can also add -e if you still want to edit the commit message to include some notes, but they are going o be discarded when rebasing with git rebase --autosquash upon merge).

Also, whoever merges this, please remember to squash the commits together!
@baentsch one nice way I learned about while working on openssl, to keep the "addressing feedback" commits separated but still have the commits squashed when we will merge is: git commit --fixup=<commit_ref> (you can also add -e if you still want to edit the commit message to include some notes, but they are going o be discarded when rebasing with git rebase --autosquash upon merge).

LGTM now. I'll mark as approval done and leave it up to @mattcaswell to clear it if he thinks the indentation/line split needs to be still improved.

My approval stands

LGTM now. I'll mark as approval done and leave it up to @mattcaswell to clear it if he thinks the indentation/line split needs to be still improved.

My approval stands

24 hours has passed since 'approval: done' was set, but as this PR has been updated in that time the label 'approval: ready to merge' is not being automatically set.  Please review the updates and set the label manually.

24 hours has passed since 'approval: done' was set, but as this PR has been updated in that time the label 'approval: ready to merge' is not being automatically set.  Please review the updates and set the label manually.

Pushed! Thanks for your contribution.

Pushed! Thanks for your contribution.

Also, whoever merges this, please remember to squash the commits together!
@baentsch one nice way I learned about while working on openssl, to keep the "addressing feedback" commits separated but still have the commits squashed when we will merge is: git commit --fixup=<commit_ref> (you can also add -e if you still want to edit the commit message to include some notes, but they are going o be discarded when rebasing with git rebase --autosquash upon merge).

@romen  There's also git commit --squash=<commit_ref> that lets you write commit messages that will not be combined, not discarded, with git rebase --autosquash.

Also, whoever merges this, please remember to squash the commits together!
@baentsch one nice way I learned about while working on openssl, to keep the "addressing feedback" commits separated but still have the commits squashed when we will merge is: git commit --fixup=<commit_ref> (you can also add -e if you still want to edit the commit message to include some notes, but they are going o be discarded when rebasing with git rebase --autosquash upon merge).

@romen  There's also git commit --squash=<commit_ref> that lets you write commit messages that will not be combined, not discarded, with git rebase --autosquash.


[Workspace] Eliminate suffix from checkouts dirname

We don't expect to ever have two checkouts with same name at a time so
there is no need of a suffix for them.
rdar://problem/46423708

@swift-ci smoke test


Back out "chore(dev-middleware): add localhost as default host in start command config"

Summary:
This is a revert of https://github.com/facebook/react-native/pull/44244, as we've observed breaking behaviour where Android emulators could not connect to the dev server with default settings.
The team doesn't have bandwidth/prio to figure this out with the default host value just now, so we are reverting.
Changelog: [Internal] (Nullifies c402dcf)
Differential Revision: D58192651

cc @szymonrybczak — apologies!


AssignmentRules issue for Case

This is the readMetadata request and response XMLs got from the code.
Request:
<readMetadata xmlns="http://soap.sforce.com/2006/04/metadata"> 			<type>AssignmentRules</type> 			<fullNames>Case</fullNames> 		</readMetadata>
Response(truncated):
<records xsi:type="AssignmentRules">                <fullName>Case</fullName>                <assignmentRule>                   <fullName>Standard</fullName>
But when I just amended the criteria and update the metadata, I received Unknown Exception error. Below is the sample UpdateMetadata request and response
Request:
<metadata xsi:type="AssignmentRules"> 				<fullName>Case</fullName> 				<assignmentRule xsi:type="AssignmentRule"> 					<fullName>Standard</fullName>
Response:
<result>             <errors>                <message>unexpected metadata</message>                <statusCode>UNKNOWN_EXCEPTION</statusCode>             </errors>             <fullName>Case</fullName>             <success>false</success>          </result>
Apex example attached
ApexEx.txt

This is a GACK by the looks of it, a Salesforce internal error, it should have some kind of numbers associated with it though? Since you have the XML request sample this is ideal for passing to Salesforce Support (passing Apex MD code will raise to many unneccessary questions as to how your calling the API). My recommendation is to raise a case and given them a way to reproduce this via something like Developer Workbench or CURL with your sample XML and of course support org login granted. Another option to try retrieving one you created in the UI and pull that down, maybe its missing some fields and there is a platform bug in the error handling? Experiment with different assignment rule setups as well. Ultimatly the time you spend on this can server to have Salesforce support help you more quickly, of course you could go both routes and get a case underway. Hope this helps! 👍


Resume vimeo video not working

Question
Hi all,
I am trying to enable resuming a video on page refresh/connection lost. Right now, when I refresh a page, it restarts from the beginning instead of resuming.
Here is the code snippet:
 this.upload = new Upload(this.videoFile, {
      endpoint: undefined,
      uploadUrl: this.model.video.uploadUrl,
      resume: true,
      retryDelays: [0, threeSeconds, fiveSeconds, tenSeconds, twentySeconds],
      onError: (error: Error): void => {
        this.onError(error.message);
      },
      onProgress: (bytesUploaded: number, bytesTotal: number): void => {
        this.progress = Math.round((bytesUploaded / bytesTotal) * percent);
      },
      onSuccess: (): void => {
        console.log('success');
      }
    });

The videoFile here is saved in localStorage so that on refresh, we can retrieve it.
The uploadUrl is always the same. It is saved in the database.
When I do not provide an uploadUrl, I get a 405 error saying that POST is not an allowed method for https://us-files.tus.vimeo.com/files/...

Please let me know what I am not seeing. Thank you again in advance!
Setup details
Please provide following details, if applicable to your situation:

Runtime environment: Browser, Angular 9
Used tus-js-client version: 1.8.0

Are you sure that you are using tus-js-client 1.8.0 and not accidentally installed the 2.1 version from NPM? That version introduced breaking changes regarding resuming.
If that does not help, please include a network recording: https://support.zendesk.com/hc/en-us/articles/204410413-Generating-a-HAR-file-for-troubleshooting

I'm facing the same issue in 2.1. I did some quick testing, and it seems that when uploadUrl is provided manually (for Vimeo uploads), the fingerprint is never stored in localstorage, thus uploads won't resume after page refresh. I also tried setting the fingerprint manually, to see if tus-js-client would pick it up and resume the upload based on that, but it didn't work either (likely the fingerprint isn't even read when uploadUrl is provided).
It seems this could be a quite an easy fix, provided that the fingerprinting would work similarly in both cases (when using endpoint or uploadUrl).
Related: the uploadUrl parameter could also accept a function that could be used to dynamically create a new uploadUrl with Vimeo if there is no fingerprint stored, or use the one that is stored otherwise.
I could try to pull of a pull-request with this kind of functionality, but maybe someone more knowledgeable with the library can first let me know if I'm missing something, and if this would be a bad idea for some reason 😬

I did some quick testing, and it seems that when uploadUrl is provided manually (for Vimeo uploads), the fingerprint is never stored in localstorage, thus uploads won't resume after page refresh.

Yes, you are right and this is intended. If you supply an uploadUrl, tus-js-client doesn't store the URL with the file's fingerprint. Using uploadUrl basically bypasses all fingerprinting and localStorage lookups, so the behavior is not a bug.
If you want to resume the Vimeo upload after the page reloads, you should store the uploadUrl on your own for now. However, we can discuss whether using uploadUrl should store the URL in localStorage.
Does that help?

I tried to reuse the uploadUrl I get from Vimeo, but that does not resume the upload due to something related to the fingerprinting logic (the original topic of this issue).

Ok, in that case I am not sure what the problem is. To be honest, this sounds like a problem on Vimeo's servers and I don't use their services so I cannot debug it on my own. Can you try to contact them?

Hey guys I'm stuck here. Anyone please help.
How do we get the uploadUrl here?
I don't know this.

@nitingupta220 You get it from the Vimeo API. I don't know the details, so please consult their documentation.

Hi, i am also facing kind of same problem.when i upload video on vimeo using tus protocol video uploading is working fine, But the uploading start again if we pause and resume. Using tus-js-clientd

$(document).on("click", "button", function (e) {
var file = $(this).prop("files")[0];
$.ajax({
'url': 'https://api.vimeo.com/me/videos',
'type': 'POST',
'headers': {
'Accept': 'application/vnd.vimeo.*+json;version=3.4',
'Content-Type': 'application/json',
'Authorization': 'bearer ' + 'token'
},
"data": JSON.stringify({
"upload": {
"approach": "tus",
"size": file.size
}
}),
'success': function (result) {
var upload = new tus.Upload(file, {
uploadUrl: result.upload.upload_link,
onError: function (error) {
console.log("Failed because: " + error)
},
                onProgress: function (bytesUploaded, bytesTotal) {
                    var percentage = (bytesUploaded / bytesTotal * 100).toFixed(2)
                    console.log(bytesUploaded, bytesTotal, percentage + "%")
                },
                onSuccess: function () {
                    console.log("Download %s from %s", upload.file.name, upload.url)
                }
            })

            // Add listeners for the pause and unpause button
            var pauseButton = document.querySelector("#pauseButton")
            var unpauseButton = document.querySelector("#unpauseButton")
            pauseButton.addEventListener("click", function () {
                upload.abort()
            })
            unpauseButton.addEventListener("click", function () {
                upload.start()
            })

    // Start the upload
    upload.start()

        },
        'error': function (result) {
        }
    });

@sebyshalom Please contact Vimeo support for this. I don't have much experience with their API.

Ran into this issue myself (well, just being unable to have a working pause/resume button without it restarting from 0) and thought I'd share how I fixed it for anyone else who stumbles across this page in future.
I had to set the chunkSize variable within my 'new tus.Upload()'. By default, chunkSize is infinite, and it seems Vimeo's 'upload-offset' response is only for completed chunks, and so there's no completed chunks until the upload has completed.
By setting it to 5000000 (5mb), I'm able to pause and resume the upload successfully. Upon resuming, it restarts at the last completed chunk, so for a small file (e.g. 30mb), there may be a noticeable drop of a few percentage after resuming. This would be less noticeable for a larger file where 5mb accounts for a smaller percentage, or you could set the chunkSize to 1mb, though the smaller the chunkSize, the slower the upload due to the increased number of http requests.
Hope this helps someone.

Ran into this issue myself (well, just being unable to have a working pause/resume button without it restarting from 0) and thought I'd share how I fixed it for anyone else who stumbles across this page in future.
I had to set the chunkSize variable within my 'new tus.Upload()'. By default, chunkSize is infinite, and it seems Vimeo's 'upload-offset' response is only for completed chunks, and so there's no completed chunks until the upload has completed.
By setting it to 5000000 (5mb), I'm able to pause and resume the upload successfully. Upon resuming, it restarts at the last completed chunk, so for a small file (e.g. 30mb), there may be a noticeable drop of a few percentage after resuming. This would be less noticeable for a larger file where 5mb accounts for a smaller percentage, or you could set the chunkSize to 1mb, though the smaller the chunkSize, the slower the upload due to the increased number of http requests.
Hope this helps someone.

Thanks for sharing, @cdelforce!

It was necessary for me to set the chunkSize to 1mb in order to get it to work for me. The issue seems to be limited to Chrome. I had success in FireFox and Safari without setting a chunkSize.

@websitedesignby Interesting, thanks for sharing. However, the error (ERR_HTTP2_PING_FAILED) looks like something that Vimeo should be looking into on their end.

I had to set the chunkSize variable within my new tus.Upload(). By default, chunkSize is infinite, and it seems Vimeo's 'upload-offset' response is only for completed chunks, and so there's no completed chunks until the upload has completed.
By setting it to 5000000 (5mb), I'm able to pause and resume the upload successfully. Upon resuming, it restarts at the last completed chunk, so for a small file (e.g. 30mb), there may be a noticeable drop of a few percentage after resuming.

I can confirm what @cdelforce said: chunkSize is required for Vimeo and it determines also the resumable "units" of the upload; e.g. with 5MB chunks, if the uploads get interrupted at say 8MB it then resumes at 5MB.
@Acconut how does this fit with the tus protocol?
One thing I can't figure out is why the onProgress log is so frequent (resulting in a progress bar updating smoothly :+1: ) like @sebyshalom showed here, while the HEAD calls (and so the UrlStorage updates) when uploading to Vimeo are only every 5MB uploaded? Isn't the onProgress function executed only at every such call? If not why?

@Acconut how does this fit with the tus protocol?

Not sure what you mean by this question. The tus protocol allows to partition the upload into PATCH requests with a variable size. It is recommended to do the upload in one request, but not required. It seems as Vimeo has an implementation which is not capable of handling requests with arbitrary size. That's quite unfortunate and can only be fixed by themselves, not tus-js-client.

Isn't the onProgress function executed only at every such call?

onProgress is not only executed for HEAD requests, but also while PATCH requests upload data.

Hi all,
I am also struggling with resumable uploads to Vimeo.
To test it I start an upload to Vimeo, store the url in the database and wait till it is past 50%.
Than I refresh the page and start uploading the same file to the same url.
But the upload always starts completely again.
My code:
var upload = new tus.Upload(file, {
		uploadUrl: upload_url,
		chunk_size: 5000000,
		onError: function(error) {
			console.error("Failed because: " + error);
			return;
		},
		onProgress: function(bytesUploaded, bytesTotal) {
			//calculate percentage
			var percentage = (bytesUploaded / bytesTotal * 100).toFixed(2)

			//show percentage in progressbar
			document.querySelectorAll('.media-progress-bar > div').forEach(div=>{
				div.style.width	= percentage+'%';
				div.innerHTML	= ''+percentage+'%';
			});
		},
		onSuccess: function() {
			document.querySelector('[data-id="'+post_id+'"] .filename>div').textContent='Uploaded to Vimeo';
		},
        onShouldRetry: function(err, retryAttempt, options) {
          var status = err.originalResponse ? err.originalResponse.getStatus() : 0
          // If the status is a 403, we do not want to retry.
          if (status === 403) {
            return false
          }

          // For any other status code, tus-js-client should retry.
          return true
        }
})
// Start the upload
upload.start()

@Tsjippy the TUS client for some reason does not handle the resume part of the upload if the upload URL is not created by the client itself. This is how I handled it using the library's internal components (namely the urlStorage which holds the reference to the upload URL until completion):
import * as tus from 'tus-js-client';

// 0. get a reference to the urlStorage
const { urlStorage, fingerprint: getFingerprint } = tus.defaultOptions;
let storedEntry = {};
let upload = null;

// 1. get the upload url first and save it into urlStorage
async function getUploadUrl () {
    hideErrorMessage();

    try {
        const file = input.prop('files')[0];

        if (!file) return;

        let fingerprint = await getFingerprint(file, { endpoint: 'https://www.yourserver.com' });
        let storedEntries = await urlStorage.findUploadsByFingerprint(fingerprint);

        if (storedEntries.length) {
            storedEntry = storedEntries[0];
            if (storedEntry.uploadUrl) {
                console.debug('previous URL found: ' + storedEntry.uploadUrl);
                return startUpload(file);
            }
            // cleanup
            urlStorage.removeUpload(storedEntry.urlStorageKey);
        }

        resetProgress();

        // use Vimeo API to create the upload URL
        // https://stackoverflow.com/a/65582420/4530144
        const response = await fetch('url-returning-vimeo-url');
        const data = await response.json();
        const {videoUri, iframe, uploadUrl} = data;
        console.debug('new upload URL: ' + uploadUrl);

        storedEntry = {
            size: file.size,
            metadata: {
                filename: file.name,
                filetype: file.type,
            },
            creationTime: new Date().toString(),
            uploadUrl,
            videoUri,
            iframe,
        };

        storedEntry.urlStorageKey = await urlStorage.addUpload(fingerprint, storedEntry);

        startUpload(file);

    } catch (error) {
        console.error('Error:', error);
        setErrorMessage('Sorry, an error occurred.');
    }
}

// 2. then upload, and remove the upload reference from urlStorage when done
function startUpload (file) {
    upload = new tus.Upload(file, {
        uploadUrl: storedEntry.uploadUrl,
        headers: {
            // https://developer.vimeo.com/api/upload/videos#resumable-approach-step-2
            Accept: 'application/vnd.vimeo.*+json;version=3.4' // required
        },
        chunkSize: 5 * MB, // required
        onProgress (bytesUploaded, bytesTotal) {
            const percentage = (bytesUploaded / bytesTotal * 100).toFixed(2);
            console.debug(bytesUploaded, `${percentage}%`);
            updateProgress(percentage);
        },
        async onSuccess () {
            urlStorage.removeUpload(storedEntry.urlStorageKey);
            showResults(file);
        },
        onError (error) {
            setErrorMessage('Sorry, upload failed. Please try again.');
            console.error(`Upload failed: ${error}`);
        },
    });

    upload.start();
}

// 2a. clear urlStorage on cancel
async function onCancel (e) {
    upload.abort(true);
    if (storedEntry.urlStorageKey) {
        urlStorage.removeUpload(storedEntry.urlStorageKey);
    }
    storedEntry = {};
    console.debug('aborted');
}

@Acconut can you confirm I'm not missing anything or doing something wrong?
Assuming it's correct (I know it does work), I would suggest to report this in the documentation to help people interacting with Vimeo (it would also solve a few opened issues). I wonder anyway if this could be simplified by allowing the client to accept an externally generated uploadUrl and let it handle the upload/resume automatically also in this case. Or is there any specific reason not to?
I'd be happy to submit a PR for the above if you'd be willing to review/accept.

Thank you, I will try this out

Closing this issue due to inactivity. Feel free to leave a comment if you want to continue the discussion :)

Thanks to all the info in this thread, particularly by cdelforce about setting the chunkSize parameter. This addressed the resume problem for me.
Just wanted to post what Vimeo recommends for chunk size:

NOTE: This high-end pro tip might not apply to general-purpose resumable uploads. But if you're using a tus library that breaks the video file into separate chunks based on a maximum size value, we recommend setting the chunk size in the range of 128–512 MB. Smaller chunk sizes might result in slower upload speeds, especially when there are many chunks.

I think 128MB might be too high for smaller sites uploading videos with smaller resolutions and/or shorter durations, but for those uploading 4k videos, their recommendations sound about right.
It would also be good to update the tus-js-client documentation about Vimeo here about using chunkSize for resuming. I'm going to create a PR for this in a bit.

It would also be good to update the tus-js-client documentation for Vimeo here about using chunkSize for resuming. I'm going to create a PR for this in a bit.

That would be amazing! We personally do not use Vimeo's API and thus mostly rely on community feedback like yours to provide a good documentation for interacting with their services.

Seems to work here by specifying any chunkSize. When leaving out the chunkSize, it does not resume properly indeed.


aCRF audio field handling

Closes #
Why is this the best possible solution? Were any other approaches considered?
Previously working solution.
What are the regression risks?
None.
Does this change require updates to documentation? If so, please file an issue here and include the link below.
No.
Before submitting this PR, please make sure you have:

[x] included test cases for core behavior and edge cases in tests
[x] run nosetests and verified all tests pass
[x] run black pyxform tests to format code
[x] verified that any code or assets from external sources are properly credited in comments

@gushil please fix merge conflicts.

@gushil please fix merge conflicts.

Done.


tox>4 in our current setup breaks codespell for many (all?) charm linting

After tox 4 was released, linting in some(all?) charms has started to fail.  An example is here for kfp-api, which had logs:
Run tox -e kfp-api-lint
kfp-api-lint: commands[0]> tox -c charms/kfp-api -e lint --
lint: install_deps /home/runner/work/kfp-operators/kfp-operators/charms/kfp-api> python -I -m pip install black codespell flake8-builtins flake8-copyright flake8-docstrings flake8==[4](https://github.com/canonical/kfp-operators/actions/runs/3652790155/jobs/6171568089#step:4:5).0.1 isort pep8-naming pyproject-flake8
lint: commands[0] /home/runner/work/kfp-operators/kfp-operators/charms/kfp-api> codespell /home/runner/work/kfp-operators/kfp-operators/charms/kfp-api/. --skip /home/runner/work/kfp-operators/kfp-operators/charms/kfp-api/.git --skip /home/runner/work/kfp-operators/kfp-operators/charms/kfp-api/.tox --skip /home/runner/work/kfp-operators/kfp-operators/charms/kfp-api/build --skip /home/runner/work/kfp-operators/kfp-operators/charms/kfp-api/lib --skip /home/runner/work/kfp-operators/kfp-operators/charms/kfp-api/venv --skip /home/runner/work/kfp-operators/kfp-operators/charms/kfp-api/.mypy_cache
/home/runner/work/kfp-operators/kfp-operators/charms/kfp-api/./lib/charms/observability_libs/v0/juju_topology.py:131: harcoding ==> hardcoding
lint: exit 6[5](https://github.com/canonical/kfp-operators/actions/runs/3652790155/jobs/6171568089#step:4:6) (0.18 seconds) /home/runner/work/kfp-operators/kfp-operators/charms/kfp-api> codespell /home/runner/work/kfp-operators/kfp-operators/charms/kfp-api/. --skip /home/runner/work/kfp-operators/kfp-operators/charms/kfp-api/.git --skip /home/runner/work/kfp-operators/kfp-operators/charms/kfp-api/.tox --skip /home/runner/work/kfp-operators/kfp-operators/charms/kfp-api/build --skip /home/runner/work/kfp-operators/kfp-operators/charms/kfp-api/lib --skip /home/runner/work/kfp-operators/kfp-operators/charms/kfp-api/venv --skip /home/runner/work/kfp-operators/kfp-operators/charms/kfp-api/.mypy_cache pid=1[6](https://github.com/canonical/kfp-operators/actions/runs/3652790155/jobs/6171568089#step:4:7)11
  lint: FAIL code 65 (3.36=setup[3.19]+cmd[0.18] seconds)
  evaluation failed :( (3.43 seconds)
kfp-api-lint: exit 65 (3.5[7](https://github.com/canonical/kfp-operators/actions/runs/3652790155/jobs/6171568089#step:4:8) seconds) /home/runner/work/kfp-operators/kfp-operators> tox -c charms/kfp-api -e lint -- pid=1576
  kfp-api-lint: FAIL code 65 (3.[8](https://github.com/canonical/kfp-operators/actions/runs/3652790155/jobs/6171568089#step:4:9)4=setup[0.26]+cmd[3.57] seconds)
  evaluation failed :( (3.[9](https://github.com/canonical/kfp-operators/actions/runs/3652790155/jobs/6171568089#step:4:10)1 seconds)
Error: Process completed with exit code 65.

Testing quickly locally, it might be that somehow an ignorelist for codespell has changed, and we're now checking some out-of-scope items.  Not sure how this would have happened.
Until we figure out a real fix, a quick patch would be to pin tox<4 for any linting CI we need to pass

Fix is in my branch https://github.com/canonical/kubeflow-ci/pull/39

specifically, the fix is here, where the file specifications for codespell have changed a little

I consider this solved.


Configure style lint to accept BEM classes

Configure style lint rules to accept BEM classes
Check by running npm run lint and accepts also kebab case without breaking existing rules

convered with general regression testing as discussed


Add NuGet repository metadata, fixes #8378

Adds NuGet RepositoryUrl and RepositoryType metadata fields in .NET library guidance.
Fixes #8378

Docs people: Can you take a look at this PR to review and merge it please

Thanks @rpetrusha and @JamesNK 👍


node: One page generator

Combine inner pages into one

Technical page: test-preview.html

https://github.com/andreymatin/html-pages-preview


Remove Python 2.7 from python_requires

https://github.com/Wanderu/pytest-bandit/blob/42af2fd72e385502bac3258f31fb2f868d20b640/setup.py#L26
The trove classifiers do not include 2.7, though, and if you try to run with it, a TypeError occurs on super:
https://github.com/Wanderu/pytest-bandit/blob/42af2fd72e385502bac3258f31fb2f868d20b640/pytest_bandit/controller.py#L20

Hm, good catch. I will make an update for this today. It's easier to drop support for python<3 than support it for 7 months.

😅 Thanks for the PR. I rabbit holed on solving a different issue. This is available in v0.5.0 now on pypi!

All good 👍 I prefer opening PRs anyways when I have the bandwidth.


Complete porting of documentation to Sphinx

The description in the note seems to be in Return.

Berry.h must be completed. A lot of missing functions documentation.
The problema in api documentatiion Is about theme of sphinx, not separate note from descriptiion, Is not a problem of doxygen. Will be solved in the next work, when i'll add new theme.

 I'm trying to sync between the wiji and readthedocs, but I don't find any simple way. Nor how to automate the generation of documentation.


Sample tests for Magnet project

Hi @premnirmal,
This pull request is sample for the unit tests that we plan to write in order to raise the overall coverage of the project to around 80%.
Please let me know if you have any questions.
Furkan Yavuz
DevFactory - Code Quality Team

Thanks! I see the test cases don't subclass junits test case. Do the tests still run?

You'r welcome! :)
About the extending from test case, let me double check. Also we can extend test classes from test case at coming commits.

Hi @premnirmal,
Could you able to review my comments? If you think it is okay, please accept the pull request, then we can start for working on full test coverage.
Thanks,

Hi @furknyavuz . Yes it looks good. Do you mind making a new PR that merges your branch into the branch testing? I don't want to merge this into master until I set up circleCI to run tests

Sure @premnirmal,
I'll open new PR to testing branch.
Thank you,


fixes #1508 and partially #1458 by adding configuration to fail when a field is wrong

Changes
Fixes #the-issue-number: #1508 and partially #1458
(#1458 needs a correction in the documentation, which I leave up to @fzyzcjy)
Checklist

[x] An issue to be fixed by this PR is listed above.
[ ] New tests are added to ensure new features are working. Please refer to this page to see how to add a test.
IMHO to minor change for a test & the result is fast to see.
[x] ./frb_internal precommit --mode slow (or fast) is run (it internal runs code generator, does auto formatting, etc).
[x] If this PR adds/changes features, documentations (in the ./website folder) are updated.
[x] CI is passing. Please refer to this page to see how to solve a failed CI.

Remark for PR creator

./frb_internal --help shows utilities for development.
If fzyzcjy does not reply for a few days, maybe he just did not see it, so please ping him.

Good job! I personally suggest adding a simple unit test or integration test to ensure that we never regress in the future:
Just mimic:
https://github.com/fzyzcjy/flutter_rust_bridge/blob/2cf7d1774f2557876d530bfbfed1fdd01b732b94/frb_codegen/src/binary/commands_parser.rs#L68
(it uses the following dir as fixture as well)
https://github.com/fzyzcjy/flutter_rust_bridge/tree/master/frb_codegen/test_fixtures/binary/commands_parser/flutter_rust_bridge_yaml
For example, add a new test mimicking it, with a broken .yaml file, and we assert! it should error.

Good job! I personally suggest adding a simple unit test or integration test to ensure that we never regress in the future:
Just mimic:
https://github.com/fzyzcjy/flutter_rust_bridge/blob/2cf7d1774f2557876d530bfbfed1fdd01b732b94/frb_codegen/src/binary/commands_parser.rs#L68
(it uses the following dir as fixture as well) https://github.com/fzyzcjy/flutter_rust_bridge/tree/master/frb_codegen/test_fixtures/binary/commands_parser/flutter_rust_bridge_yaml
For example, add a new test mimicking it, with a broken .yaml file, and we assert! it should error.

Thanks :) And thanks for adding me as a contributor!
I implemented the unit tests - for the config file, but as well for the case in which the configuration is part of pub spec.yaml.
That I did not test before (!), but it works all as expected :)

You are welcome!

The code coverage tool is currently sometimes weird...
https://app.codecov.io/gh/fzyzcjy/flutter_rust_bridge/pull/1509/indirect-changes

The code coverage tool is currently sometimes weird... I commited a ignore and let's see whether it works.
https://app.codecov.io/gh/fzyzcjy/flutter_rust_bridge/pull/1509/indirect-changes

@fzyzcjy
CodeCoverage still fails ...
Do we need to derive the Macro I introduced?
in frb_codegen/src/library/codegen/config/config.rs
  /// Configuration for code generation
  /// Refer to `GenerateCommandArgs` for documentations
  #[derive(Debug, Serialize, Deserialize)]
1
+ #[serde(deny_unknown_fields)]
  pub struct Config {
      pub base_dir: Option<String>,
      pub rust_input: String,

change to
// frb-coverage:ignore-start
#[derive(Debug, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
// frb-coverage:ignore-end+

?

Hmm, it is really weird... Let's just ignore it...
https://app.codecov.io/gh/fzyzcjy/flutter_rust_bridge/pull/1509?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=checks&utm_campaign=pr+comments&utm_term=fzyzcjy

Just make Lint::Rust CI happy and I will merge :)

Just make Lint::Rust CI happy and I will merge :)

Done :) Reverted the change, now it is fine to be merged.
This closes #1508, but #1458 as well ... I assume the slight name difference between the cli argument or configuration file ("no-") is ok, as the error messages will help enough to understand that fast.
Otherwise we can change the attributes to be the same, should not be much work.

Good job!

I assume the slight name difference between the cli argument or configuration file ("no-") is ok, as the error messages will help enough to understand that fast.

I will update the doc about this in the next batch (possibly in a few days)

Otherwise we can change the attributes to be the same, should not be much work.

I deliberately make --no only in command line, because it seems a bit counter-best-practice to have negations except for command line.


Add option to use Scheduled Huber Loss in all training pipelines to improve resilience to data corruption

As heavily discussed in https://github.com/kohya-ss/sd-scripts/discussions/294, the presence even of a small number of outliers can heavily distort the image output.
While there have been proposed alternative losses, such as Huber loss, the problem was the loss of fine details when training the pictures. After researching the mathematical formulations of diffusion models, we came to the conclusion that it's possible to schedule the Huber loss, making it smoothly transition from Huber loss with L1 asymptotics on the first reverse-diffusion timesteps (when the image only begins to form and is most vulnerable to concept outliers) to the standard L2 MSE loss, for which diffusion models are originally formulated, on the last reverse-diffusion timesteps, when the fine-details of the image are forming.
Our method shows greater stability and resilience (similarity to clean pictures on corrupted runs - similarity to clean pictures on clean runs), than both pure Huber and L2 losses.

The experiments confirm that indeed this schedule improves the resilience greatly

Our paper: https://arxiv.org/abs/2403.16728
Diffusers discussion https://github.com/huggingface/diffusers/issues/7488
Most importantly, this approach has virtually no computational costs over the standard L2 computation. (and minimal code changes to the training scripts)
cc @cheald @kohya-ss

These are quite different approaches. @cheald makes adjustments to latents, then feeds them into the standard loss.
And as cheald is using mse_loss in the end, I believe these approaches can exist in synergy!

This is very exciting! I've been experimenting with manipulating the mean/stddev of slices of the noise prior to forward noising, and have found that I can directly manipulate the level of detail trained with certain permutations of noise. However, it still suffers from outliers early in training having too large an impact, which I've had to manage through very careful tuning of noise. I'm very excited to try adding this into my experiments - if it performs how I imagine, then it could solve a number of problems which could lead to faster and more controlled training.

Tried a training, the results were impressive.

@gesen2egee Thank you for giving a test run! Would you mind sharing some of them?

Here's my quick training experiment. I just ran each for 9 epochs. Overall, results look very promising.
Ground truth:

General settings: adamw8bit, CosineAnnealingWarmRestarts w/ restart every 2 epochs, unet_lr 1e-4, and I am using masks and masked loss here.
Each image pair is l2 on the left, huber_scheduled on the right.

Marginal improvement, I think!

Here's where it gets fun:
I'm experimenting with recentering and rescaling each noise channel individually, and then also experimenting with shifting and rescaling all noise channels together, as well as independently. This is conceptually similar to an expansion of the idea encoded in the noise_offset routine. It's yielding some very interesting results.
Dependent scale, dependent shift:

Independent channel scaling, dependent channel shift:

Dependent scaling, independent shift:

and finally, independent scaling, independent shift:

My general observation here is that the huber_scheduled loss definitely does improve detail retention, but isn't quite learning as well. However, I suspect that is likely due to nothing more than the lower loss values damping the rate of change in the learned weights. It might be that using huber_loss would permit a larger learning rate relative to l2 loss, which would be great if it can retain the same improvement in details.

Here's another example. I'm working on extending the dynamic range of my trained samples, which I'm doing by pushing the mean of the first channel in the noise. I also cranked my LR up to 4e-4 to see if this would give the huber noise an edge, but it appears to actually not be working that way.
 Upper is l2, lower is huber.
The huber examples have more detail, and it seems like a better dynamic range, but definitely feel more overtrained, which is interesting, since the nominal loss is higher for the l2 variants.

Night L2
Day L2

Night Huber
Day Huber

Loss curves for each:

Excited to try this out. Any idea how this interacts with Min-SNR-gamma, which weights the loss based on timestep?

I'm looking forward to trying this out.  I do have a concern about part of the paper though, namely the function defining the schedule:

I don't think it makes much sense to define the schedule in terms of the indices of timesteps since they will vary between models, and even within SD1.5 there's zero terminal SNR which does modify the noise schedule.  If the reasoning is that loss should behave more like MAE at early timesteps/high noise levels and more like MSE at late timesteps/low noise levels, wouldn't it make more sense to define the schedule in terms of something like SNR directly?  From playing around in a notebook quickly, it seems that 1 - sqrt(1.0 - alphas_cumprod) is fairly close (though I can't say with any certainty that it is the value that makes the most sense to use):

@cheald @drhead we chose the exponential because of its simplicity, to test the claim that it should decrease with the (forward) diffusion timestep. It indeed quite likely may be suboptimal, and the idea about the snr-scheduling sounds very reasonable!
Adding some selectable huber_c schedules, I think, would be a great next addition

There is another alternative for huber function: pseudo-Huber loss divided by delta:
$$\sqrt{\delta^2 + a^2} - \delta$$
whereas the "math" version is:
$$\delta^2*(\sqrt{1 + \frac{a^2}{\delta^2}} - 1)$$
The difference is math version converging to zero when the delta goes to zero, while the modified version transitions between L1 and L2
While the divided version gave us worse results for resilience, it may be better perceptually. (didn't analyze the perceptual part much, unfortunately) The former version, suggested by OpenAI, is used in Diffusers for LCM training.
Here's the gif comparison:
https://github.com/kohya-ss/sd-scripts/assets/14872007/38b95308-7765-42e8-87a9-40e1c3ab00ab
I think it may be worth adding it here too and making some more experiments

@drhead the second main guy agreed that 1 / (1 + sigmas)**2 makes sense, so I think delta = (1-delta_0) / (1 + sigmas)**2 + delta_0, accounting for the zero-snr padding, may do a good gob. Will make some tests once I'll get home

Good suggestion! Btw I'm making the experiments with different losses/schedules right now. Will post it here soon

@drhead (on David Revoy dataset)

Well, I've completed a small-ish ablation test of my own on a larger finetune of data that is in theory clean, and the results look pretty promising!
Here's regular MSE loss with my adaptive loss weights:

And here's with scheduled pseudo-huber loss:

Backgrounds are much sharper, and most significantly the character's form is more consistent.  Very happy with how this turned out and I think I will continue to use this.
Interestingly, my loss weight model chose a different weighting (ignore the fact that it looks weird on the tails, I don't think this is a true reflection of the ideal timestep weighting and is instead an artifact of the model used to train timestep weights which I am still trying to nail down the correct hyperparameters for).

This effectively reflects the relative difficulty of each timestep (in form of the weighting required to equalize the model's capability at each timestep).  So, these changes indicate that scheduled pseudo huber loss made the high-noise timesteps relatively easier, and later timesteps relatively harder.  Again, my model is v-prediction, so the high noise timesteps are closer to x0 prediction and lower noise timesteps are closer to epsilon prediction -- you would probably see different results on epsilon prediction.   But I do find this to be an interesting result.

The default is exponential

I think it will be better to use 'snr' by default as it have been shown to have better quality

Is there any information about when different scheduling method and loss function type should be applied? For example, for a large dataset one, for a small dataset another. For training on photos one, for training on anime artwork another

@kohya-ss It's still "exponential" in the code itself
https://github.com/kohya-ss/sd-scripts/blob/dfa30790a99754f2f20e4980e91a000cce51d8a4/library/train_util.py#L3249

I did want to follow up on this since some further experiments led me to notice some tradeoffs with the outcomes of training on this loss function.
While the benefits to image coherency still seem very clear, it seems that on my training setup, one of the side effects of this loss function is a loss of control over image contrast -- manifesting as things like "dark" in a prompt being less powerful on zero terminal SNR models.  Incidentally, I have a very good idea of what the root cause might be, and it makes perfect sense given what this loss function does.
TL;DR of the above post is that the VAE used by SD 1.5 (and SD 2.1 and Midjourney and the Pixart series of models and DALL-E 3 and EDM2 and probably many others), tends to produce a high magnitude artifact, especially on large images and desaturated images (noting that desaturated in latent space is closer to sepia).  My theory is that the purpose of this artifact is to abuse the normalization layers of the model by placing an artifact that serves to desaturate the image by having a high magnitude that will cause other values to decrease significantly as they pass through the model normalization layers.  @madebyollin made some very helpful graphics demonstrating some of these effects: https://gist.github.com/madebyollin/ff6aeadf27b2edbc51d05d5f97a595d9
With this in mind, it does make perfect sense why this would happen, especially on my training setup.  I'm training a high resolution model (actually with several different resolution groups) with v-prediction.  With v-prediction, higher noise timesteps are closer to x_0 prediction, and the terminal timestep is outright x_0 prediction.  This means that the prediction target has something that is both high magnitude and extremely important for image reconstruction.  Errors are generally likely to be proportional to the magnitude of the target, which is not good when our loss function is more relaxed on large errors at this point.  The loss function can't take into account the fact that this will cause a massive error in pixel space -- for MSE loss, this would be less of a problem, since larger errors would be pulled in much harder.  While the loss objective does effectively ignore large outliers from the data in this way, the VAE artifact is sadly not separable from this.  It is possible that it could be mitigated by finding some metric for a latent that would indicate consistency in its saturation level, but the solution that makes more sense is to just move to a model that doesn't use the SD1.5 VAE.
As a final note, I have found the effects of this to be very similar to the effects of FreeU, both in its improvements of image coherence and in having a similar issue with weakening of saturation (and they do combine together well, I was using it with my samples above).  FreeU's underlying theory is that high-frequency information in the skip connections converges too fast in the decoder and therefore the authors choose to modulate backbone features and skip connections to place more emphasis on low frequency feature maps.  That might be related to why this improves coherency -- mitigating the impact of outliers allowing for low frequency features to be better represented.  I'm sure there's a lot more research to be done on optimizing hierarchical denoising with things like this.
All of that being said, I do think that these issues are not likely to be a problem for epsilon prediction since the VAE artifact is not directly part of the prediction target, and it is not likely to be much of an issue on SDXL or for several other models using a different VAE or for pixel-space models.  I'll be trying it out on SD3 when I can, hopefully their VAE doesn't have the same issues.

Can i ask which function im supposed to use to actually use scheduled huber loss in kohya? there is no explanation.. Do i need to set smooth_l1? or huber? What are the settings im supposed to use?

Can i ask which function im supposed to use to actually use scheduled huber loss in kohya? there is no explanation.. Do i need to set smooth_l1? or huber? What are the settings im supposed to use?

There is some details explained on the main README about-scheduled-huber-loss

Hey, I just tried out this new Huber loss parameter... and it's amazing.
Thanks for all the hard work by the people here!

What’s the practical effect of increasing or decreasing the value of huber_c?

What’s the practical effect of increasing or decreasing the value of huber_c?

My experience is total destruction of the training almost instantly.  Even a 0.01 in either direction resulted in total devastation/noise.


How to integrate MobFox with this plugin

Hi Dear floatinghotspot!
Nice work developing such plugin. I need to integrate MobFox into my application how do I accomplish this using AdMob Pro plugin. Please it possible? if yes then how it is possible does it needs any extra adapter to be installed to let it work or how?
I tried using my MobFox publisher id as followings but it is not working.
admobid = {
//banner: 'ca-app-pub-xxxxxxxxxxxxxxxxxxxx', // instead using admob id
banner: '<< MobFox publisher ID HERE>>',  // inserted my publisher ID here
interstitial: 'ca-app-pub-xxxxxxxxxxxxxxxxxxxxxxxxxx' // here AdMob id for interstitial
};
thanks.

No, you cannot do it like this, you must specify AdMob unit id there.
Instead, try the following:
After install the AdMob plugin, also install the plugin for MobFox mediation adapter:
cordova plugin add cordova-plugin-admob-mobfox

Specify the admob ad unit Id when create banner or interstitial.
Then, in your web portal, find your ad unit id, edit mediation, then add your mobfox id there.


ZestGuidance does not correctly udpate total Coverage when test fails under every inputs

What happened:
When fuzzing a test that will fail under every input, ZestGuidance does not correctly compute the Total Coverage. To be more specific, the total coverage always shows as 0. Another issue caused by this is that valid coverage and total coverage are all the same value at any time. Because these two metrics are only updated under valid inputs.
How to reproduce:
Try to fuzz the following example test with Zest:
    @Fuzz
    public void exampleTest(String str) {
        int a = 0;
        a = a + 1;
        assertTrue(str.length() == -1);
    }

This test always fails because str.length() will not be -1. But the coverage should not be 0.
Example output:
Elapsed time:         3s (no time limit)
Number of executions: 44 (no trial limit)
Valid inputs:         0 (0.00%)
Cycles completed:     0
Unique failures:      2
Queue size:           0 (0 favored last cycle)
Current parent input: <seed>
Execution speed:      14/sec now | 11/sec overall
Total coverage:       0 branches (0.00% of map)
Valid coverage:       0 branches (0.00% of map)

How to improve
In ZestGuidance.java, there should have some totalCoverage update in this else if branch of handleResult() method:
for example:
  } else if (result == Result.FAILURE || result == Result.TIMEOUT) {
      String msg = error.getMessage();

      // We should also increase the totalCoverage but not validCoverage
      totalCoverage.updateBits(runCoverage);
      int nonZeroAfter = totalCoverage.getNonZeroCount();
      if (nonZeroAfter > maxCoverage) {
          maxCoverage = nonZeroAfter;
      }
   ...
  }

Also because there is a coverage responsibility check in completeCycle(), so we need to check the saved input responsibility is the same as valid coverage there instead of the total coverage (because Zest does not save invalid input)
        if (sumResponsibilities != validCoverageCount) {
            if (multiThreaded) {
                infoLog("Warning: other threads are adding coverage between test executions");
            } else {
                throw new AssertionError("Responsibilty mismatch");
            }
        }

Hi @rohanpadhye. I think there could be an elegant solution to this while make the reporting not that weird. The current coverage report can be very misleading when, e.g., all branch of the test in which all path would ultimately result in failures. In my understandings for this case, the total coverage should be at least non-zero if all the failures are triggered. However, the currently implementation of JQF would result in a very small coverage, which is counterintuitive.
As for the input selections, I assume that we could deal with the coverages triggered by failure inputs differently. For example, we could mark the coverages as "failure-induced" and have valid inputs "steal" them later if they also cover these coverages. Fixing this could give a more accurate coverage number and also prevent improper comparisons if we want to, e.g., compare between different tools & guidances.

Thanks for the input @hwang-pku. Marking specific parts of the coverage as "failure-induced" sounds a lot like keeping track of "coverage achieved by failure" and "coverage achieved by normal" separately, which I think is acceptable for reporting purposes. Feel free to open a PR for this, otherwise I will try to work on this in my free time.


Suggestion - make double-clicking more lenient

I often find myself triple clicking to copy a task's details because double clicks are quite inconsistent for me some of the time.

It seems to be a problem with the underlying framework we use to build the GUI. It is annoying but it is unfortunately not feasible for me to change it.


Open files to be uploaded

MultipartEncoder requires file pointers so it can stream them.
Close files after POST.

Rockin


syntax errors to fix

I ran flake8 to find any possible problems with AMUSE (as part of testing to automate making/publishing packages).
Found several errors - most are easy fixes like missing imports, some others I may need help with (those I'll post here).
@ipelupessy could you look at these?

src/amuse/rfi/async_request.py:267:52: F821 undefined name 'index'
            self._result_index=first._result_index[index]
                                                   ^

https://github.com/amusecode/amuse/blob/4a5d227e8b3e6c964a26377805994e434e19ef45/src/amuse/rfi/async_request.py#L267

src/amuse/io/store_v2.py:999:38: F821 undefined name 'Grids'
            class_of_the_container = Grids
                                     ^

src/amuse/datamodel/base.py:1738:29: F821 undefined name 'x'
            new_container = x.copy(memento, keep_structure = True, filter_attributes = filter_attributes)
                            ^

this looks like an error, but it needs to be tested, it looks like the try except block should go away... (at the moment the index is not used anyway, so it does not generate an error in the results

hmm, so I wonder how one can test it?

I fixed most of the problems flake8 found, what remains is a number of F821's: undefined variables.
These are all of them (in src/amuse):
(AmuseDev)  ~/Code/amuse-clean   main ±  flake8 src --count --select=E9,F63,F7,F82 --show-source --statistics
src/amuse/rfi/async_request.py:267:52: F821 undefined name 'index'
            self._result_index=first._result_index[index]
                                                   ^
src/amuse/rfi/tools/create_cython.py:1323:21: F821 undefined name 'todo'
                    todo     #self.out + 'char *'
                    ^
src/amuse/io/store_v2.py:999:38: F821 undefined name 'Grids'
            class_of_the_container = Grids
                                     ^
src/amuse/ic/_limepy/limepy.py:704:41: F821 undefined name 'nl'
        phi_l = numpy.zeros(len(self.r)*nl).reshape(len(self.r), nl)
                                        ^
src/amuse/ic/_limepy/limepy.py:704:66: F821 undefined name 'nl'
        phi_l = numpy.zeros(len(self.r)*nl).reshape(len(self.r), nl)
                                                                 ^
src/amuse/community/ph4/util/plot_smalln_data.py:210:32: F821 undefined name 'tscale'
    if t[0] < 0: dtscale = max(tscale, -t[0])
                               ^
src/amuse/community/ph4/util/plot_smalln_data.py:375:14: F821 undefined name 'filename'
    f = open(filename)
             ^
src/amuse/community/evtwin2sse/interface.py:237:13: F821 undefined name 'is_remnant_stellar_type'
        if (is_remnant_stellar_type( star.stellar_type )):
            ^
src/amuse/community/etics/format-convert.py:74:13: F821 undefined name 'OutputFile'
            OutputFile.close()
            ^
src/amuse/datamodel/base.py:1738:29: F821 undefined name 'x'
            new_container = x.copy(memento, keep_structure = True, filter_attributes = filter_attributes)
                            ^
src/amuse/couple/encounters.py:1072:22: F821 undefined name 'particles'
            for p in particles:
                     ^
src/amuse/couple/encounters.py:1109:69: F821 undefined name 'particles'
                    resolve_collision_code.particles.synchronize_to(particles)
                                                                    ^
src/amuse/couple/encounters.py:1110:21: F821 undefined name 'channel'
                    channel.copy()
                    ^
src/amuse/couple/encounters.py:1111:30: F821 undefined name 'particles'
                    for p in particles:
                             ^
14    F821 undefined name 'OutputFile'
14

fixed


false-positive rust-analyzer diagnostic: mismatched-arg-count

The problem is that we're picking the wrong get function.
Does that code involve proc-macro generated code? Can you extract a test case that shows the same problem (by moving frame_system::Config and DbWeight to a new project)?

I also encountered the same problem when using nalgebra.
Here is an example code for reproduction.
use nalgebra::DVector;

fn main() {
    let z: DVector<f32> = nalgebra::DVector::zeros(3);
    println!("{}", z);
}

Although cargo check does not emit any error, rust-analyzer reports that the number of argments is wrong.

@sho275 that's https://github.com/rust-analyzer/rust-analyzer/issues/8654.
Every case where the type inference doesn't work can cause these false positive diagnostics, because we pick the wrong function. But the root cause can be different: maybe the trait solving got a wrong answer, maybe we didn't expand a macro correctly, or something else. In this case it's because we don't support const generics.

The problem is that we're picking the wrong get function.
Does that code involve proc-macro generated code? Can you extract a test case that shows the same problem (by moving frame_system::Config and DbWeight to a new project)?

it doesn't involve proc-macro generated code

Can you link to the get function that it's resolving to right now, and the get function that you'd expect to be used?

I've got the same false positive error and here is a minimal example on how to reproduce it.
The first new will work correctly, the second new will produce expected 1 argument, found 2 rust-analyzer (mismatched-arg-count) as a lint.
struct MyStruct<const S: usize> { vec: [usize; S] }

impl MyStruct<1> {
    fn new(a: usize) -> Self {
        MyStruct{ vec: [a] }
    }
}

impl MyStruct<2> {
    fn new(a: usize, b: usize) -> Self {
        MyStruct{ vec: [a, b] }
    }
}

pub fn example_error() {
    let _first = MyStruct::<1>::new(0);
    let _second = MyStruct::<2>::new(0, 0);
}

@Chartas const generics are not supported.

That's just #8655, which is not the same root cause as this one.


Ensure asset directory exists before creating an asset in test

This test was failing intermittently on Jenkins. It would fail if the
Whitehall.clean_uploads_root didn't exist. I replicated this locally
by deleting the tmp/test directory and running the test. I saw the
following error:
PublicUploadsControllerTest#test_does_not_redirect_asset_requests_that_are_made_via_the_asset_host:
Errno::ENOENT: No such file or directory @ rb_sysopen - /Users/chrisroos/Code/alphagov/whitehall/tmp/test/env_1/clean-uploads/asset.txt

I believe the test would have passed if it was run after a call to
VirusScanHelpers#simulate_virus_scan as that method will create the
upload directory.

Thanks @thomasleese :-) I'm going to get this merged.


FHI-aims example is broken

By running the FHI-aims example (example/diamond-FHI-aims) I realized that it is not working anymore. (I am using the current development branch.) It throws: "BORN" was not found., although neither nac or Born was requested. I think it is due to line 826 and 850 in scripts/phonopy. Changing or to and in line 826 fixes it for the FHI-aims example, but not sure if this breaks other parts.

Thanks, @Kokookster. You are correct. This was due to my recent change of the code. I fixed this. But this fix may have side effect. The test of the user interface is uneasy (i.e., phonopy is not written well considering test). Currently test is only possible by reports from users, like this issue.

I close this issue because this is fixed.

Many thanks for fixing it!


Use Mocha and mocks to allow for testing without hardware.

I needed this because i was doing development on a pc for a beaglebone

Thank's for the PR :). I'll take a look at it after upgrading a few native add-ons to nan v2.3.2 for Node.js v6 compatibility.

Using v6 also causes issues with my test. mock-fs does not work if v6. But I was thinking can that be mock by using a real directory in the test/ i.e.
test/fs/export
test/fs/gpio7/{value,edge,direction}
then epoll would not have to be mocked. I think I can use rewrite to change GPIO_ROOT_PATH, might work.

I removed the need for mock-fs. but Epoll seems to still need to be mocked. Epoll does not work on regular files.

@thisboyiscrazy I feel bad about closing this pull request. I'm sorry for not investing more time in the work you did here.


Support add and delete model dynamicly

Closes #141
Signed-off-by: wangzhenyu32<EMAIL_ADDRESS>

bors r+


Refactor: Split node_3d_editor_plugin in multiple files

I would like to make some improvements this week to the 3D viewport.
However, the node_3d_editor_plugin contains a lot of "unrelated" things, which led to 8780 lines, way too much to work properly.
This file containing 7 classes with an header of 950 lines, this PR split in 7 files, as advised in this issue: https://github.com/godotengine/godot/issues/33027#issuecomment-1114032060 (3.).
Feel free to point out things to improve.
I have not edited any methods for this PR to simplify the review, except for the constants that I had to move to Node3DEditor.

Thanks for your help! With this I found that JetBrains uses UTF-8 with what internet call "BOM" (UTF-8 BOM). I'll try to push the same code im 5min with UTF-8 classic to see if it fixes this.

Thanks! Static checks pass. Now let's face some fun compiling issues 😑

Ok, Seems ready for review now.


Look into using official chart for Istio

Istio is published as a helm chart already. It would be nice if we could consume that chart and provide meaningful default values from the riff chart to configure Istio in a meaningful way. This would also give users more flexibility in configuring the Istio install.

Done in #8


Get with someone from cyverse about setting up a "master email."

Meet with Chris Martin and see about how to make an "admin" email account to be in charge of repos and the travis ci deployment for now as well as future proofing.

Created with the assistance of Chris Martin and Andy Edmonds. Cyverse is hosting and maintaining this account. The link below leads to the OFS page which should be updated as more are created and as those accounts are bound to services.
https://osf.io/tzmhp/wiki/Administrative Accounts/


Use PaymentProvider & context

Use PaymentProvider and paymentContext to reduce prop drilling in the checkout flows.

Great improvements! 🔥 Everything was working well when purchasing a pack with a purchase type, and errors were being displayed as expected.
Whenever I try to make a bid on an auction, I get the following error from the /add-to-pack endpoint:
"Invalid hook call. Hooks can only be called inside of the body of a function component. This could happen for one of the following reasons:\n1. You might have mismatching versions of React and the renderer (such as React DOM)\n2. You might be breaking the Rules of Hooks\n3. You might have more than one copy of React in the same app\nSee [https://reactjs.org/link/invalid-hook-call]() for tips about how to debug and fix this problem."

Are you seeing that? Also, it never shows crypto and wires as payment options. When console logging inside the checkout-template, the same environment variables (console logged at the top of the component) show up true for the console logs in the terminal but false for the console logs in the developer tools (shown below). Are you seeing crypto/wire as options?

@dallashuggins Thanks so much for taking a look!
I haven't see the invalid hook call yet. I'll try and reproduce and fix that now.
re: never showing crypto and wires, I noticed this but I'm actually seeing this happen in main too. I thought it was just my environment. It's super weird because if I log isEnabled and isEnabled.toLowerCase() === 'true' in get isWireEnabled() , both return as true. It hurts my brain. Yet if I manually return true from this I can test the flows. ¯_(ツ)_/¯

re: never showing crypto and wires, I noticed this but I'm actually seeing this happen in main too. I thought it was just my environment. It's super weird because if I log isEnabled and isEnabled.toLowerCase() === 'true' in get isWireEnabled() , both return as true. It hurts my brain. Yet if I manually return true from this I can test the flows. ¯_(ツ)_/¯

Oh weird! Yeah I realized my environmental variables were set to true and haven't been seeing those are options, so it must've been an issue from a while back. Super confusing - going to look at that more.

@dallashuggins ok so I've been trying to figure out the hook error (SUPER weird btw), and just noticed that I'm seeing that happen on main as well 😬.

@dallashuggins ok so I've been trying to figure out the hook error (SUPER weird btw), and just noticed that I'm seeing that happen on main as well 😬.

Aw gotcha - thanks for looking into it! Makes sense to handle in a future PR.


Format sources with swift-format

Expected to fail because there are still two formatting issues.
https://github.com/apple/swift/pull/69516
@swift-ci Please test macOS

https://github.com/apple/swift/pull/69516
@swift-ci Please test macOS

https://github.com/apple/swift/pull/69516
@swift-ci Please test macOS

https://github.com/apple/swift/pull/69516
@swift-ci Please test macOS

apple/swift#69516
@swift-ci Please test macOS

https://github.com/apple/swift/pull/69516
@swift-ci Please test

https://github.com/apple/swift/pull/69516
@swift-ci Please test

https://github.com/apple/swift/pull/69516
@swift-ci Please test Windows

https://github.com/apple/swift/pull/69516
@swift-ci Please test Windows


Fix for incorrect key to lookup :reconnect_on_error option.

I put the incorrect key in for the transport name. This setting will allow the value to be included with the rest of the configuration for the transport (i.e. within rabbitmq's settings). Found when trying to expand Puppet configuration to support.

This setting is intended to be within the transport context, as it is not specific to the Sensu transport in use.

May I ask why this is help up please?

As of Sensu 0.18 https://github.com/sensu/sensu/blob/master/CHANGELOG.md#0180---2015-05-05 Sensu transport reconnect_on_error defaults to true, which applies to the loaded Sensu transport (default is RabbitMQ).


Make sure heartbeat working always.

Make sure heartbeat working always. FYI @liuhaoyang make sure .net agent hasn't this issue.

Coverage decreased (-0.006%) to 12.289% when pulling 442c2393b9faf4546eaaaa0f57c4f953a2ede545 on fix-heartbeat into 13c417c06e50864acff9ab56552e5ed7c9af056c on master.


netcdf forcings

the current forcing output files are csv which are slow to create and large on disk, ngen supports netcdf input so update to use that

#53


Fix opengl

Checklist

[x] Used a fork of the feedstock to propose changes
[x] Bumped the build number (if the version is unchanged)
[x] Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
[x] Ensured the license file is being packaged.

Changes:

Fixes #25 by adding -Dopengl=ON
Uses Anaconda Inc's CMake recommendations
Add support for ROOT_JENKINS_GIT_URL, ROOT_JENKINS_GIT_URL and ROOT_RUN_GTESTS environment variables so ROOT's nightlies can use this recipe directly
Use GNU sed from conda-forge for patching the sources instead of relying on the host
Copy the upstream FindX11.cmake file and add minor patches as I couldn't figure out how to configure CMake correctly

Hi! This is the friendly automated conda-forge-linting service.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

I've build this and uploaded the binary to my own channel. Trying it on lxplus gives some kind of error from OpenGL's dynamic linking to LLVM:
$ conda create --name test-root-opengl -c chrisburr/label/opengl root python=3.7
$ conda activate test-root-opengl
# It doesn't happen when `root -b` is used
$ root -l -q -x -e 'gStyle->SetCanvasPreferGL(kTRUE); c = new TCanvas(); if (!c->UseGL()) { throw std::runtime_error("OpenGL does not appear to be working");}'

: CommandLine Error: Option 'help-list' registered more than once!
terminate called after throwing an instance of 'std::runtime_error'
  what():  >>> Interpreter compilation error:
inconsistency in registered CommandLine options

I've ran out of time to do any more debugging today but this looks relevant: https://github.com/root-project/cling/issues/201#issuecomment-409573736

I've marked this as ready to review as I'm not sure how much of a problem this is. So far as I can tell it only affects using X11 forwarding from CentOS 7 and it can be worked around by setting export LIBGL_ALWAYS_INDIRECT=1 before launching root.

I've marked this as ready to review as I'm not sure how much of a problem this is. So far as I can tell it only affects using X11 forwarding from CentOS 7 and it can be worked around by setting export LIBGL_ALWAYS_INDIRECT=1 before launching root.

@chrisburr The docker image used for the ci builds is centos 6.10. Maybe this is partly to blame for the X11 error.

I am getting same error running directly on my SL7 desktop.

LIBGL_ALWAYS_INDIRECT=1  root -x "${ROOTSYS}/tutorials/eve/geom_lhcb.C"
   ------------------------------------------------------------
  | Welcome to ROOT 6.16/00                  https://root.cern |
  |                               (c) 1995-2018, The ROOT Team |
  | Built for linuxx8664gcc on Apr 13 2019, 23:33:00           |
  | From tag , 23 January 2019                                 |
  | Try '.help', '.demo', '.license', '.credits', '.quit'/'.q' |
   ------------------------------------------------------------

root [0]
Processing /data/anaconda2/envs/test-root-opengl/tutorials/eve/geom_lhcb.C...
TEveManager::GetGeometry  loading: 'http://root.cern.ch/files/lhcbfull.root' -> 'http://root.cern.ch/files/lhcbfull.root'.
Info in <TGeoManager::Import>: Reading geometry from file: http://root.cern.ch/files/lhcbfull.root
[TFile::Cp] Total 0.32 MB	|====================| 100.00 % [0.6 MB/s]
Info in <TFile::OpenFromCache>: using local cache copy of http://root.cern.ch/files/lhcbfull.root [./files/lhcbfull.root]
Info in <TGeoManager::CloseGeometry>: Geometry loaded from file...
Info in <TGeoManager::SetTopVolume>: Top volume is World. Master volume is World
Info in <TGeoNavigator::BuildCache>: --- Maximum geometry depth set to 100
Info in <TGeoManager::CloseGeometry>: Voxelization retrieved from file
Info in <TGeoManager::CountLevels>: max level = 8, max placements = 277
Info in <TGeoManager::CloseGeometry>: 18491756 nodes/ 709 volume UID's in default geometry
Info in <TGeoManager::CloseGeometry>: ----------------modeler ready----------------
Info in <TGeoManager::SetVisLevel>: Automatic visible depth disabled
Error in <TGLViewerBase::RenderOpaque - pre exit check>: GL Error invalid operation
Error in <RootX11ErrorHandler>: 255 (TGLWidget XID: 56623712, XREQ: 152)
TGLWidget:	56623712
	TGVerticalFrame:	56623707
	TGLSAFrame:	56623696
	TGCompositeFrame:	56623695
	TEveCompositeFrameInTab:	56623688
	TGCompositeFrame:	56623687
	TGTab:	56623237
	TGHorizontalFrame:	56623228
	TGVerticalFrame:	56623227
	TGHorizontalFrame:	56623225
	TGVerticalFrame:	56623214
	TEveBrowser:	56623213

How can I install the conda package created by the docker build?

To answer my own question
conda create -c file://$PWD/build_artifacts root -n myrootenv
seems to work.

I used this PR and built with docker on my SL7 desktop using the linux-python2.7 config. Creating an environment using the build_artifacts directory as a channel and running from that environment I can run without error
env LIBGL_ALWAYS_INDIRECT=1 root -e 'gStyle->SetCanvasPreferGL(kTRUE); c = new TCanvas(); if (!c->UseGL()) { throw std::runtime_error("OpenGL does not appear to be working"); }'

env LIBGL_ALWAYS_INDIRECT=1 root -l -x "${ROOTSYS}/tutorials/eve/geom_lhcb.C"

The rendering is slow over X11 forwarding. I will try in the morning to run directly on my SL7 desktop to see if I can run without the extra env setting.

I checked this morning on my ScientificLinux 7 desktop and if I run without
env LIBGL_ALWAYS_INDIRECT=1
The window manager either restarts or the window hangs.
Because the X11 libraries included in the tarball are linked against the Centos6 version of libGL and libGLdispatch they can only be used directly if the host is also Centos6 or SLC6.
Does conda-forge offer a Centos7 docker image to build in?

@chrisburr Are you able to do a macOS build and put it on your channel?

I can try to make a local build tomorrow on my mac.

I am trying a local build on my mac. I am trying to reproduce the steps manually from azure-pipelines-osx.yml

Thanks for testing. My mac is slowly dying and kernel panics whenever I try to build ROOT but if you've set up the SDK you should be able to replicate conda's build using:
conda build ./recipe -m ./.ci_support/osx_python3.7.yaml --clobber-file ./.ci_support/clobber_osx_python3.7.yaml

Does conda-forge offer a Centos7 docker image to build in?

Building inside the CentOS 6 container shouldn't be an issue. The build links against Anaconda's OpenGL binaries that are repackaged from CentOS 6 (known as CDTs) with the intention of being forwards compatible with newer Linux distributions.
I think the issue with ROOT is that RHEL7's build of OpenGL dynamically links to LLVM (see here, here and here). I haven't figured out how to avoid this problem and probably won't have time to devote it this week.

Yes I have the SDK installed.
I am primarily interested in getting a macOS build so I can build the Fireworks application against that.
Thanks for the command line.

My macOS build works with the test commands for OpenGL. Now on to testing the build of Fireworks!

More good news: after doing a fresh install of ScientificLinux 7.6 on my desktop I can run the GL test commands without
env LIBGL_ALWAYS_INDIRECT=1

More good news: after doing a fresh install of ScientificLinux 7.6 on my desktop I can run the GL test commands without
env LIBGL_ALWAYS_INDIRECT=1

I should also note that I blacklisted the nouveau x11 driver so I could use the proprietary Nvidia driver.

Will this / is this needed for the LLVM/Clang recipe?

I should also note that I blacklisted the nouveau x11 driver so I could use the proprietary Nvidia driver.

That's reassuring, I was confused about how reinstalling could fix this problem.

Will this / is this needed for the LLVM/Clang recipe?

No. If I understand this correctly, ROOT should be patched to use --exclude-libs (or something similar) instead of relying on the fact that LLVM and clang have been built with -fvisibility=hidden.
I checked using nm and the symbols are still global on macOS. It might cause a problem one day but I don't know a nice way of hiding them.
Assuming this PR passes the tests it should be fixed and ready to merge.


link to bert example is broken

link to bert example is broken from
https://huggingface.co/mosaicml/mosaic-bert-base
and
https://www.mosaicml.com/blog/mosaicbert
https://github.com/mosaicml/examples/tree/main/examples/bert#single-task-fine-tuning
https://github.com/mosaicml/examples/tree/main/examples/bert

and

Thanks for flagging this! We're restructuring our examples repo, and the path changed. We will update the docs soon. Here is the link to unblock you for now: https://github.com/mosaicml/examples/tree/main/examples/benchmarks/bert

The HF README for mosaic-bert-base has been updated, and we will update the blogpost. Thank you for flagging this


don't use i18n just with attribute to lookup translation if it is a model

closes #364

Thanks mates


How do you use the fine tuned models (not load, but USE)

I can train in the webui but i cant use the models!
Both the api and the webui use the .wav file for speakers not the finetuned model even when its loaded.
EDIT:
As I say I can load the model in the webui once trained but I have to select a speaker wav file, it wont let me use the fine tuned model.
Same issue with the api, you can load it fully but when you send a json packet you have to include the speaker wav file and thus bypass the fine tuning.
Can you give some instructions on how to use the fine tuned model please.

I sussed it!
You have to copy the reference.wav from the models folder and put that in the Speakers folder then use that as the speakers wav file (with the fine tuned model loaded)
This worked a treat.
Ofcourse I then went to make a new fine tuned model and that's not working with the latest update , the downside of bleeding edge software lol.
Can't do a full bug report on the issue at mo as I must sleep but it's easy to replicate. Give the model a name, copy a wav file in and hit train... it errors about reference.wav not found. Make a reference.wav re run it and another error pops up. I'll get the logs etc after work tomorrow if their needed.
Just happy I have figured out how to use the fine tuned models (can I urge adding how to use them onto the readme?)


Add cmake support

This pull request was opened against the master branch, please open pull requests with respect to the feature branches.


Improve cyclomatic complexity of utility_meter creation

Pull Request Test Coverage Report for Build<PHONE_NUMBER>8
Details

35 of 35 (100.0%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 99.978%

Totals

Change from base Build<PHONE_NUMBER>3:
0.0%

Covered Lines:
4513

Relevant Lines:
4514

💛  - Coveralls


Microsoft.Data.SqlClient in Core Web Application Not Working .net 5

Hi Team,
Microsoft.Data.SqlClient is not working in .net 5 web application. If the target framework is changed to 3.1 it is working fine. Help me out here please

Could you perhaps be a bit more specific? What does "not work" mean?

Hi @MithunAmalraj
Please provide specific details with a repro if possible.

Hi, Sorry, I am new to this forum. Thank you for your time. Below are the details,
Project Type: C# ASP NET Core Web Application
Template: ASP.NET Core Web App (Model-View-Controller)
Framework: .NET 5
In this sample code below, SQL server is called to fetch some data. When debugging is done, data is not loaded in the datatable. While checked with SQL server profiler, request is received at SQL Server. But data is not getting filled.
DataTable objresutl = new DataTable();
SqlDataReader myReader;
using (SqlConnection myCon = new SqlConnection(constring))
{
   myCon.Open();
   using (SqlCommand myCommand = new SqlCommand("GetPopularAirport", myCon))
   {
     myCommand.CommandType = CommandType.StoredProcedure;
     myReader = myCommand.ExecuteReader();
     objresutl.Load(myReader);
     myReader.Close();
     myCon.Close();
   }
}

Error in .NET 5 Image

Working Fine when Target Framework is changed to .NET 3.1

@MithunAmalraj
I tested below code with .NET 5 with Microsoft.Data.SqlClient v2.1.2 and I don't see any issue in retrieving records.
        DataTable objresutl = new();
        using SqlConnection myCon = new(connString);
        myCon.Open();
        using SqlCommand myCommand = new("sp_who2", myCon);
        myCommand.CommandType = CommandType.StoredProcedure;
        using SqlDataReader myReader = myCommand.ExecuteReader();
        objresutl.Load(myReader);

Please validate your solution setup.

@cheenamalhotra one last try here I have uploaded the sample repo, https://github.com/MithunAmalraj/SampleAppNet5.
If its still working for you with this app, I will check more at my end.

Thanks for repro @MithunAmalraj
The behavior is strange. It's mainly with the DataTable Visualizer component it seems.

In Console App (targeting .NET 5.0) there's no issue.
In ASP.NET App (.NET 5.0) DataTable Visualizer does not load data, but the "objresult" contains data if you go to "Rows > Results View":

So I think this is an issue with Visualizer and for that you'd have to reach to Visual Studio team.
I found this discussion thread DataTable-Visualizer-crashes-again which points to an unsolved issue. Please provide them with your repro and that may help them solve it.

@cheenamalhotra Thanks for your time. I will take this forward with the visual studio team.


How to install Keras on arm architecture with only pytorch installed(without tensorflow or jax)

Hi,
I'd like to install Keras 3.0 on ARM architecture machine with only pytorch installed. I use comand of Local installation with some small changes in requirements.txt  and set export KERAS_BACKEND="torch" but I get the following issue:
[root@ares keras]# python pip_build.py --install
Generating files for package 'keras' from sources found in 'keras/src'.
Traceback (most recent call last):
  File "pip_build.py", line 196, in build
    run_namex_conversion()
  File "pip_build.py", line 58, in run_namex_conversion
    namex.generate_api_files(package, code_directory="src", verbose=True)
  File "/root/archiconda3/envs/python38_torch201/lib/python3.8/site-packages/namex/generate.py", line 65, in generate_api_files
    mod = importlib.import_module(entry_point, package=".")
  File "/root/archiconda3/envs/python38_torch201/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 843, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/workspace/workplace/keras/tmp_build_dir/keras/src/backend/numpy/__init__.py", line 2, in <module>
    from keras.src.backend.numpy import image
  File "/root/archiconda3/envs/python38_torch201/lib/python3.8/site-packages/xpytorch_import_hook.py", line 45, in _custom_import
    module = builtins.__origin__import__(
  File "/workspace/workplace/keras/tmp_build_dir/keras/src/backend/numpy/image.py", line 1, in <module>
    import jax
  File "/root/archiconda3/envs/python38_torch201/lib/python3.8/site-packages/xpytorch_import_hook.py", line 45, in _custom_import
    module = builtins.__origin__import__(
ModuleNotFoundError: No module named 'jax'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "pip_build.py", line 231, in <module>
    whl_path = build(root_path, args.nightly, args.rc)
  File "pip_build.py", line 204, in build
    shutil.rmtree(build_directory)
  File "/root/archiconda3/envs/python38_torch201/lib/python3.8/shutil.py", line 709, in rmtree
    onerror(os.lstat, path, sys.exc_info())
  File "/root/archiconda3/envs/python38_torch201/lib/python3.8/shutil.py", line 707, in rmtree
    orig_st = os.lstat(path)
FileNotFoundError: [Errno 2] No such file or directory: 'tmp_build_dir'

Seems that the issue is caused by no jax installed. If I only use pytorch, how could I skip tensorflow and jax with Keras installed? Any help will be appreciated!

To build Keras from sources, you need all dependencies. But for running Keras, you only need the backend you will use. The solution is use an existing build, e.g. from pip. Keras is a pure Python package so the "build" is really just a set of Python file.
Basically: just use pip install keras or conda.

Thanks for your response!!
I know basically I should use pip or conda, but I can't get Keras 3.0 from pip or conda because my python version is 3.8.
I can't upgrade python version, because my AI accelerator only support pytorch on python 3.8, not on python3.9, if I change python version, pytorch is not supported after the change.
So I'd like to change Keras source code and build from source code to make keras work in my machine, do you think it's possible?
If it's possible, which part of source code should I change generallty?

You can try clonign Keras Repo from GitHub.  Update the setup.py to say >=3.8 - https://github.com/keras-team/keras/blob/master/setup.py#L51
Then run, python pip_build.py --install.  Make sure you have virtualenv installed in the python env.

Please reopen the issue if this doesn't resolve the issue.

Thanks, I think I still need other change in Keras Repo, because I can't install tensorflow nor jax in my machine, I run python pip_build.py --install will encounter tensorflow or jax not installed issue. Please refer the error log in my first comment.
Do you have any idea about where I still need to change? Just general idea is enough!

Please help reopen this issue, I can't reopen it myself.

Would you be able to install Python 3.9 on your ARM device? Or use miniconda to create a Python 3.9 env? These issues would go away if you use Python 3.9 and you could just use pip install for everything without having to build.
Python 3.8 EoL is coming soon - https://devguide.python.org/versions/ and many packages incl, Numpy and others have dropped it a while back.


Roots of unity tests porting from c-kzg

Added 4 tests from c-kzg;
Added Readme

Good tests. All passing.

Relocated readme to our folder, since it was on root folder.
Other stuff as said is good. 👍


Added Weather App

Hi @surajondev , i have my project which shows the weather details of the given city. please review the pr .
Thanks.

Thanks for the PR. The website looks dope. It has been merged.
I will be glad if you can leave a star on this repository.


Rust Snippets for Lambda-SQS, Lambda-Kinesis, and Lambda-Kinesis with Batch Failures

Issue #, if available:
N/A
Description of changes:
Rust Snippets for Lambda-SQS, Lambda-Kinesis, and Lambda-Kinesis with Batch Failures
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Thanks!


Upgrade OpenNLP to 1.9.1 [LUCENE-8659]

Since Apache OpenNLP 1.9.1 has been released it would be nice to upgrade Lucene/Solr to use that.

Migrated from LUCENE-8659 by Tommaso Teofili (@tteofili), updated Dec 08 2021

Thanks @hurutoriya for re-surfacing this issue!
Closing as superseded by #11277 upgrading to 1.9.4 version.


test: update cypress tests as per added test data in TED

Description
Updated gitSyncedApp_spec and CommunityIssues_pec as per modified data
How Has This Been Tested?
locally
Checklist:

[X] My code follows the style guidelines of this project
[ ] I have performed a self-review of my own code
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[ ] My changes generate no new warnings
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] New and existing unit tests pass locally with my changes

Test coverage results :test_tube:
:white_circle: Total coverage has not changed
No changes to code coverage between the base branch and the head branch

/ok-to-test sha=f556dc9