Cast boolean to number in MySQL generates error

Hello,
I used to work with JPAQuery in QueryDSL.
Now, I need to make a query with some aggregations using a boolean field (tinyint(1) in MySQL)
Here's my code (I've shortened the request) :
JPAQuery query = new JPAQuery(em);
QMyEntity myEntity = QMyEntity.myEntity;

NumberExpression<Integer> nExprRefused = myEntity.isRefused.castToNum(Integer.class);
NumberExpression<Integer> nExprAccepted = myEntity.isAccepted.castToNum(Integer.class);

return query.from(myEntity)
        .groupBy(myEntity.otherField)
        .list(Projections.constructor(
                       MyProjection.class
                       , nExprAccepted.sum()
                       , nExprRefused.sum()
                       , myEntity.otherField)
                       )
             );

The problem is that the cast is made like this :
CAST (t0.isRefused as INTEGER)
So MySQL throws an error.
I'd like to have :
CAST (t0.isRefused as SIGNED INTEGER)
... but I don't know how to do this.
So if someone here could help me to find a solution it'll be very pleasant :)
Thanks for your help !
Julien

Could you post the exception or error message?

Also which JPA provider do you use?

Hello Timo,
We use eclipslink 2.5.
We don't have error in the code, it's mysql which doesn't accept the request because the cast is made as INTEGER and not as SIGNED INTEGER
(When I change it and execute manually my sql request, it works)
We use MySQL 5.5
Regards,
Julien

Will this be fixed at some point?  I just ran into the same issue since I wanted to use Integers but countDistinct uses Long and my counts won't be big enough that I'll run into an overflow.


All contributors

@allcontributors add @kdubb for code and maintenance

@all-contributors add @iocanel for code and maintenance


2.0.0

Release notes - Planorama - Version 2.0.0
Bug
PLAN-703 Need description update for duration field in Daily Grid
PLAN-702 In Conclar format, remove the venue ("Hyatt Regency Chicago or Airmeet") leaving only the room then comma then duration
PLAN-701 Diff report between a publish and current state not including some added sessions, participant drops
PLAN-700 REPORT - daily rid needs durations
PLAN-699 Back of Badge report needs times in 12 hour format
PLAN-697 DIFF between 2 publishes incorrectly includes dropped session title and description in change tabs
PLAN-696 After adding a person to a session, the full list of people in that session is getting accidentally paged.
PLAN-695 Max # of signups (when signup required) is not being saved after refresh
PLAN-693 Change heading in Daily Grid from Time to Start Time
PLAN-691 In Conclar, would like commas between labels above description field for easier reading.
PLAN-690 Change Conclar label from "Require Signup" to "Requires Signup"
PLAN-689 Change environment labels in Conclar to be more humanly readable
PLAN-688 Revise Conclar dropdowns: Tags-> Area, add Environment
PLAN-687 Conclar is not using the supplied session duration to calculate the end time. It appears to default to 60 minutes.
PLAN-686 Include sessions with status draft in the report Participants Moderating at least One Session
PLAN-685 Include sessions with status draft in the report Scheduled Sessions Streamed and/or Recorded
PLAN-684 sessions saving to wrong database field for max signups
PLAN-683 Need descriptions for Badge Back & Sessions Streamed/Recorded reports
Story
PLAN-682 create a way to reset publish for testing
PLAN-681 create the refresh button for publish table
PLAN-680 REPORT - new reports need placement and descriptions
PLAN-676 REPORT - Sessions that are live streamed and recorded
PLAN-673 tag usage in conclar
PLAN-672 Add participant attendance field to Conclar
PLAN-670 REPORT - Daily Grid Report
PLAN-668 Schedule Diff - from published version N to current Publish
PLAN-667 Schedule Diff - from last publish to live
PLAN-663 REPORT - program ops - Participant schedule for mail merge (back of badge, participant schedule sheets, etc)
PLAN-662 staging needs to more clearly say it is staging
PLAN-659 REPORT - Participants moderating at least one session
PLAN-649 Add day name to ConClar tags field
PLAN-608 add a publish control etc to the admin tab
Task
PLAN-678 Finish up XML for Pubs
PLAN-669 Create a version 2.0
What's Changed

Apostrophes and virtual rooms. by @MattArnold in https://github.com/ChicagoWorldcon/planorama/pull/554
bring staging fixes into dev by @balen in https://github.com/ChicagoWorldcon/planorama/pull/556
tweaks for logs to files by @balen in https://github.com/ChicagoWorldcon/planorama/pull/558
put staging updates into dev by @balen in https://github.com/ChicagoWorldcon/planorama/pull/560
1.7.0 to dev by @Gailbear in https://github.com/ChicagoWorldcon/planorama/pull/562
1.7.0 to dev take 2 by @Gailbear in https://github.com/ChicagoWorldcon/planorama/pull/564
PLAN-672 Add participant attendance type as tag in conclar output by @LVerhulst4321 in https://github.com/ChicagoWorldcon/planorama/pull/568
Update version number to 2.0.0 by @dcterman in https://github.com/ChicagoWorldcon/planorama/pull/566
PLAN-673 Update sessions tags by @LVerhulst4321 in https://github.com/ChicagoWorldcon/planorama/pull/570
1.7.1 to dev by @Gailbear in https://github.com/ChicagoWorldcon/planorama/pull/575
Merge pull request #572 from ChicagoWorldcon/staging by @Gailbear in https://github.com/ChicagoWorldcon/planorama/pull/576
PLAN-672 Check that attendance type exists by @LVerhulst4321 in https://github.com/ChicagoWorldcon/planorama/pull/573
Update schedule. by @MattArnold in https://github.com/ChicagoWorldcon/planorama/pull/578
PLAN-673 Update conclar tags output to use new format by @LVerhulst4321 in https://github.com/ChicagoWorldcon/planorama/pull/577
PLAN-678 Adjustxml by @balen in https://github.com/ChicagoWorldcon/planorama/pull/579
PLAN-667, PLAN-668 publish diffs by @balen in https://github.com/ChicagoWorldcon/planorama/pull/580
plan-659 moderators for publishable sessions by @balen in https://github.com/ChicagoWorldcon/planorama/pull/581
plan-670 daily grid by @balen in https://github.com/ChicagoWorldcon/planorama/pull/582
PLAN-608 publish controls by @Gailbear in https://github.com/ChicagoWorldcon/planorama/pull/584
plan-662 make staging visually different by @Gailbear in https://github.com/ChicagoWorldcon/planorama/pull/585
Add descriptive info for 2 new reports PLAN-680 by @dcterman in https://github.com/ChicagoWorldcon/planorama/pull/583
plan-676 sessions streamed ane recorded by @balen in https://github.com/ChicagoWorldcon/planorama/pull/586
Plan 663 mail merge report - back of badges by @balen in https://github.com/ChicagoWorldcon/planorama/pull/587
2.0.0-rc1 by @Gailbear in https://github.com/ChicagoWorldcon/planorama/pull/588
Restore apostrophes by @MattArnold in https://github.com/ChicagoWorldcon/planorama/pull/589
fix so that draft sessions included in reports by @balen in https://github.com/ChicagoWorldcon/planorama/pull/598
Prep for when integrations are added to models by @balen in https://github.com/ChicagoWorldcon/planorama/pull/590
PLAN-682 pub reset endpoint by @balen in https://github.com/ChicagoWorldcon/planorama/pull/591
signups should use audience size by @balen in https://github.com/ChicagoWorldcon/planorama/pull/596
2.0.0-rc2 by @Gailbear in https://github.com/ChicagoWorldcon/planorama/pull/600
PLAN-682 publication reset button (2.0) by @Gailbear in https://github.com/ChicagoWorldcon/planorama/pull/601
PLAN-681 refresh button for pubs table (2.0) by @Gailbear in https://github.com/ChicagoWorldcon/planorama/pull/602
PLAN-689 PLAN-690 fix conclar fields by @LVerhulst4321 in https://github.com/ChicagoWorldcon/planorama/pull/603
PLAN-695 fix saving signups (audience size) 2.0 by @balen in https://github.com/ChicagoWorldcon/planorama/pull/606
PLAN-693 tweak column headings by @dcterman in https://github.com/ChicagoWorldcon/planorama/pull/604
Add descriptions to new reports PLAN-683 by @dcterman in https://github.com/ChicagoWorldcon/planorama/pull/605
PLAN-697 adjust pub diffs so add and drops do not also create info about title… 2.0 by @balen in https://github.com/ChicagoWorldcon/planorama/pull/607
PLAN-696 should use fetchall and on paged (2.0) by @balen in https://github.com/ChicagoWorldcon/planorama/pull/608
no-more-manual-versions by @Gailbear in https://github.com/ChicagoWorldcon/planorama/pull/610
/PLAN-701 deal with dropped and draft session state changes by @balen in https://github.com/ChicagoWorldcon/planorama/pull/616
PLAN-702 Remove venue from data sent to conclar by @LVerhulst4321 in https://github.com/ChicagoWorldcon/planorama/pull/617
PLAN-699 change date format for excel to be 12 hour (2.0) by @balen in https://github.com/ChicagoWorldcon/planorama/pull/614
Plan-700 add duration to grid (2.0) by @balen in https://github.com/ChicagoWorldcon/planorama/pull/615
plan-705, plan-706 fix diff report issues (2.0) by @balen in https://github.com/ChicagoWorldcon/planorama/pull/627

New Contributors

@MattArnold made their first contribution in https://github.com/ChicagoWorldcon/planorama/pull/554

Full Changelog: https://github.com/ChicagoWorldcon/planorama/compare/v1.7.1...2.0.0

Did a quick glance over the changelog, looks reasonable, and works in staging.


Running WASI binaries

Let me start off by saying how much I like wasmtime. I always liked the concept, but six months ago the implementation wasn't quite there. I was pretty sure it'll get better, and today I ran the exact same binary I was trying to run back then in no time and it works better than my most optimistic expectations. Right now it seems like wasmtime is going to solve the single biggest obstacle new nMigen developers encounter, especially on Windows but elsewhere too.
That aside, is there any way I'm missing to run WASI binaries with wasmtime-py? I really only need stdin and stdout, though the binary I'm interested in, I think, is expecting to be able to load from $PREFIX/share/ so some sort of virtualized fileystem would be ideal.
Is this something I can easily implement myself? (I'm only really familiar with WASM/WASI as a toolchain user, not a library developer.) Or, is it something in scope for wasmtime-py? Do I have to implement every single function WASI provides, or just the ones my application calls?

To answer one of my own questions, it looks like I only need to implement the functions that are actually used, and it doesn't seem very hard! I'm going to leave this open because it looks like the C and Rust embeddings offer built-in WASI bindings, and I'm wondering if Python embedding should, too.

I just read through the docs again and found out that there were, in fact, WASI bindings all this time. I apologize for the disturbance.

Thanks for the report! Glad you were able to find an answer, but if anything else comes up don't hesitate to ask!


Fix table tile updates in 4 up view and undo/redo

Another middle ground we could take here is to keep the efficient minimal updates that were in place previously, and only do the inefficient rebuild when we discover we're out of sync. Given the size of the datasets we're dealing with, I don't think this would have a significant impact on performance, but it would keep the code that precisely modifies the maps which could be useful when we're able to return to this and do it better someday.

@kswenson @scytacki It seems like this solution isn't quite working. The cypress tests that are failing are because attrNameMap isn't updating when a column's name changes.

@kswenson I agree that if this same DataSet is shared with CODAP then we should add the complexity to make it more efficient. Are you using this exact implementation in CODAP3? Are there tests in CODAP3 for large tables so we can keep track of how changes to it impact performance?  Ideally you could test both speed and memory usage.
I think the biggest impact of this PR will be memory not speed. I don't think it will too much be slower. MST will cache the result of these functions and only recompute the result if something accessed by the function changed. I'd expect attributes don't change much in practice so that won't slow things down. The worse problem would be cases, if something is pumping cases into the DataSet (simulation or sensor) then the cases cache will be rebuilt on each new case. This rebuild should be fast though since it is a tight loop just working with integers. So I think the number cases will have to get into the millions  before it is noticeable.
I think the memory increase will be significant. It means MobX will be adding a listener to each case.__id__ property write. And these listeners in the case of MST are not small (based on that other article I read before).  So I'd guess it could be as bad as 10k per case.
Since the DataSet is using observable objects for the caseIds and attribute values array, it should be expected that views and models might observe numberOfCases objects. I would guess the Table component is already observing every case. So that will have the same memory impact as the views added in this PR. So if the size of the DataSet is going to be large enough that this is a concern then maybe we need to not even use observable objects for things related to cases. That way they can't be observed.  I don't know how to use MST's serialization mechanism with non observable objects, but I'd guess that it is related to the frozen type.  But I'd guess no matter what we do if we are working with millions of cases developers will need to be careful with their code not to eat up memory.
I don't think it is a requirement of the new document model that models cannot cannot distinguish between locally-driven changes and externally-driven changes. It is currently, because we don't have the right hooks in place for models and views to know the difference. I think your previous PR is the right way to go to add this. We just have to track down all of the places that modify the models externally. It was nice living in a world where that didn't matter. Even though I think it is a good idea to add this support, any models (and views) that work this way will be more complex and error prone.  So I think it should only be used when there is no other option.

@kswenson I agree that if this same DataSet is shared with CODAP then we should add the complexity to make it more efficient. Are you using this exact implementation in CODAP3? Are there tests in CODAP3 for large tables so we can keep track of how changes to it impact performance? Ideally you could test both speed and memory usage.

CODAP3 has a slightly evolved version of the DataSet model with a streamlined API and some other changes, but it's the same at the level of these cached maps.

I think the biggest impact of this PR will be memory not speed. I don't think it will too much be slower. MST will cache the result of these functions and only recompute the result if something accessed by the function changed. I'd expect attributes don't change much in practice so that won't slow things down. The worse problem would be cases, if something is pumping cases into the DataSet (simulation or sensor) then the cases cache will be rebuilt on each new case. This rebuild should be fast though since it is a tight loop just working with integers and strings. So I think the number cases will have to get into the millions before it is noticeable.
I think the memory increase will be significant. It means MobX will be adding a listener to each case.__id__ property write. And these listeners in the case of MST are not small (based on that other article I read before). So I'd guess it could be as bad as 10k per case.
Since the DataSet is using observable objects for the caseIds and attribute values array, it should be expected that views and models might observe numberOfCases objects. I would guess the Table component is already observing every case. So that will have the same memory impact as the views added in this PR. So if the size of the DataSet is going to be large enough that this is a concern then maybe we need to not even use observable objects for things related to cases. That way they can't be observed. I don't know how to use MST's serialization mechanism with non observable objects, but I'd guess that it is related to the frozen type. But I'd guess no matter what we do if we are working with millions of cases developers will need to be careful with their code not to eat up memory.

As currently implemented, the cases are essentially an observable array of IDs, the attributes are an observable array of attribute models, but the values stored in the attributes use types.frozen() and a bit of sleight-of-hand to make sure that the bulk of the data values are just dumb data. I was anticipating that a single observable per-case structure would be acceptable but that there shouldn't need to be more than one. Having the cases (i.e. the case IDs) be serializable but not observable would require yet more sleight-of-hand.

I don't think it is a requirement of the new document model that models cannot cannot distinguish between locally-driven changes and externally-driven changes. It is currently, because we don't have the right hooks in place for models and views to know the difference. I think your previous PR is the right way to go to add this. We just have to track down all of the places that modify the models externally. It was nice living in a world where that didn't matter. Even though I think it is a good idea to add this support, any models (and views) that work this way will be more complex and error prone. So I think it should only be used when there is no other option.

I think it's worth the effort to implement those hooks -- this PR was an attempt to identify some of them -- sooner rather than later so that we can figure out what they are and what restrictions need to be placed on them, e.g. not making changes to the observable model properties.


Fix Identifier value used when configuring Boomi integration.

Requires content (text) edit and a screenshot (image) change.

@sjwaight : Thanks for your contribution to the Azure documentation! The author, @jeevansd, has been notified to review your proposed change.

#sign-off


Vimeo Password protected videos RegexNotFoundError

Checklist

[x] I'm reporting a broken site support
[x] I've verified that I'm running youtube-dl version 2020.03.01
[x] I've checked that all provided URLs are alive and playable in a browser
[x] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[x] I've searched the bugtracker for similar issues including closed ones

Verbose log

./youtube-dl --verbose -F https://vimeo.com/showcase/6733390/video/389861851
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'--verbose', u'-F', u'https://vimeo.com/showcase/6733390/video/389861851']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2020.03.01
[debug] Python version 2.7.16 (CPython) - Darwin-18.7.0-x86_64-i386-64bit
[debug] exe versions: none
[debug] Proxy map: {}
[vimeo] 389861851: Downloading webpage
[vimeo] 389861851: Extracting information
ERROR: Unable to extract info section (caused by RegexNotFoundError(u'Unable to extract \x1b[0;34minfo section\x1b[0m; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.',)); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "./youtube-dl/youtube_dl/extractor/vimeo.py", line 656, in _real_extract
    flags=re.DOTALL)
  File "./youtube-dl/youtube_dl/extractor/common.py", line 1005, in _search_regex
    raise RegexNotFoundError('Unable to extract %s' % _name)
RegexNotFoundError: Unable to extract info section; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "./youtube-dl/youtube_dl/YoutubeDL.py", line 797, in extract_info
    ie_result = ie.extract(url)
  File "./youtube-dl/youtube_dl/extractor/common.py", line 530, in extract
    ie_result = self._real_extract(url)
  File "./youtube-dl/youtube_dl/extractor/vimeo.py", line 670, in _real_extract
    cause=e)
ExtractorError: Unable to extract info section (caused by RegexNotFoundError(u'Unable to extract \x1b[0;34minfo section\x1b[0m; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.',)); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

Description

Regular videos download just fine. Password protected are giving Regex error. Will DM a password to that video in case needed. Using the latest version of YT-dl with Python 2.7

you did not pass the password to the command in the first place.

OK, my bad. Pasted the password with --video-password the log is pretty much the same.
./youtube-dl --verbose --video-password XXXXX -F https://vimeo.com/showcase/6733390/video/389861851
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'--verbose', u'--video-password', u'PRIVATE', u'-F', u'https://vimeo.com/showcase/6733390/video/389861851']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2020.03.01
[debug] Python version 2.7.16 (CPython) - Darwin-18.7.0-x86_64-i386-64bit
[debug] exe versions: none
[debug] Proxy map: {}
[vimeo] 389861851: Downloading webpage
[vimeo] 389861851: Extracting information
ERROR: Unable to extract info section (caused by RegexNotFoundError(u'Unable to extract \x1b[0;34minfo section\x1b[0m; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.',)); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
File "./youtube-dl/youtube_dl/extractor/vimeo.py", line 656, in _real_extract
flags=re.DOTALL)
File "./youtube-dl/youtube_dl/extractor/common.py", line 1005, in _search_regex
raise RegexNotFoundError('Unable to extract %s' % _name)
RegexNotFoundError: Unable to extract info section; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
File "./youtube-dl/youtube_dl/YoutubeDL.py", line 797, in extract_info
ie_result = ie.extract(url)
File "./youtube-dl/youtube_dl/extractor/common.py", line 530, in extract
ie_result = self._real_extract(url)
File "./youtube-dl/youtube_dl/extractor/vimeo.py", line 670, in _real_extract
cause=e)
ExtractorError: Unable to extract info section (caused by RegexNotFoundError(u'Unable to extract \x1b[0;34minfo section\x1b[0m; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.',)); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

if the URL is not accessible to a developer, there is no way to fix the problem.

Where can I send you a password? It's not mine, so I can not share it for everyone to see.

you can find my email address in my profile.

Sent. Thank you so much for everything you do and apologies for the confusion!

Will this fix be reimplemented into the next version of youtube-dl? Any specific reason, why has it been reverted? Just curious.

it has caused a regression #24338, a proper fix would require more changes.

This issue ranks high in google for "youtube-dl vimeo showcase" phrase, so here's a workaround for those looking for a solution.
I managed to download password protected Vimeo videos by passing XXXXXX_albumpassword cookie to youtube-dl, where xxxxx is the id of Vimeo showcase (in the example provided by the OP it is 6733390 - the first number from the url). You need to get the value of the cookie by accessing Vimeo in your browser and copying the value from browser's dev tools.
Sticking to OP's example, the full command is:
youtube-dl --add-header 'Cookie: 6733390_albumpassword=PASTE.VALUE.HERE;' -F https://vimeo.com/showcase/6733390/video/389861851


pubsub will throw during Subscription.pull when autoAck is on and no messages avail

I'm porting to v1beta2 of the pubsub service. In recent versions of gcloud-node, my previous code will abort in Subscription.pull with:
Error: At least one ID must be specified before it can be acknowledged.
    at Subscription.ack     (/Users/jared/work/other/deep/node_modules/gcloud/lib/pubsub/subscription.js:258:11)

I use autoAck and right now there are very few messages on the topic. So I think what's happening is the long-poll completes without receiving a message, and this block in Subscription.pull:
  this.makeReq_('POST', path, null, body, function(err, response) {
    if (err) {
      callback(err);
      return;
    }

    var messages = response.receivedMessages || [];
    messages = messages.map(Subscription.formatMessage_);

    if (self.autoAck) {
      var ackIds = messages.map(function(message) {
        return message.ackId;
      });

      self.ack(ackIds, function(err) {
        callback(err, messages);
      });

calls self.ack with ackIds empty because receivedMessages is empty. I fixed this in my local copy with:
if (self.autoAck && messages.length != 0) {

since there's no point in trying to ack if there are no messages. If I had a bit more time right now, I'd offer a PR with a test, but let's get an issue opened at least to see if the problem has been characterized correctly first.

Oh, I think the issue was caused in https://github.com/GoogleCloudPlatform/gcloud-node/commit/1233ac63f6a794a08339bf5956ee798552a6ce9c during the 1beta2 API bump by @ryanseys (that you for that fix! I'd be dead in the water without that commit, and I'd encourage a 0.13.0 release to pick up those changes). The array always had at least one item in it previously, but now it can be empty.

To clarify, is this still a bug in master?

I realize we do need to release 0.13.0 soon. We've been wrestling with some regression tests for a while that have distracted from a next release. I hope to have something out soon.

Yes, the behavior described (throw+exit) happens on gcloud-node master as of today, and the code snippets in comment 1 are from master. Thanks!

Yep totally makes sense, I'll get a fix out for this.

Fix, as you suggested, in #456.


Unhandled error, when running cf html5-delete --content with non-existing application host id

Description
I tried to execute cf html5-delete --content with the service instance name instead of the app host id and triggered a crash with this.
It would be great if the output could read sth like: "App host ID not found/invalid".
cf html5-delete --content mta_modul_html5_repo_host
Deleting content of service instances with app-host-id mta_modul_html5_repo_host in org D056949trial_newsubaccountd056949 / space dev as<EMAIL_ADDRESS>panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x10 pc=0x135a5fa]

goroutine 1 [running]:
cf-html5-apps-repo-cli-plugin/clients.CreateServiceKey(0x14ec9c0, 0xc000056180, 0x7ffeefbffc05, 0x19, 0x1, 0x7ffeefbffc05, 0x19)
	/Users/i070970/go/src/cf-html5-apps-repo-cli-plugin/clients/create_service_key.go:37 +0x33a
cf-html5-apps-repo-cli-plugin/commands.(*DeleteCommand).DeleteServiceInstancesContent(0x1801ce0, 0xc0000a0040, 0x1, 0x1, 0x0, 0x0, 0x0, 0xc0000bdd38)
	/Users/i070970/go/src/cf-html5-apps-repo-cli-plugin/commands/html5_delete_command.go:94 +0x808
cf-html5-apps-repo-cli-plugin/commands.(*DeleteCommand).Execute(0x1801ce0, 0xc0000a0030, 0x2, 0x2, 0xc000056180)
	/Users/i070970/go/src/cf-html5-apps-repo-cli-plugin/commands/html5_delete_command.go:51 +0x2b9
main.(*HTML5Plugin).Run(0x181ff48, 0x14ec9c0, 0xc000056180, 0xc0000a0020, 0x3, 0x3)
	/Users/i070970/go/src/cf-html5-apps-repo-cli-plugin/html5_plugin.go:45 +0x1b1
cf-html5-apps-repo-cli-plugin/vendor/github.com/cloudfoundry/cli/plugin.Start(0x14e5fc0, 0x181ff48)
	/Users/i070970/go/src/cf-html5-apps-repo-cli-plugin/vendor/github.com/cloudfoundry/cli/plugin/plugin_shim.go:30 +0x276
main.main()
	/Users/i070970/go/src/cf-html5-apps-repo-cli-plugin/html5_plugin.go:65 +0x3d

PS: It's also strange that I see @micellius user ID in this log.
Qualities of Change

Functional -  y
User Interface - y
Documentation - n
Introduces new dependency (y/n) - n
Incompatible change (y/n) - n

@IObert , thanks for reporting an issue!
I've tried to reproduce the issue by running the same command, but got a user friendly error message
$ cf html5-delete --content mta_modul_html5_repo_host
Deleting content of service instances with app-host-id mta_modul_html5_repo_host in org *** / space *** as ***...
FAILED
Could not create service key for service instance with id 'mta_modul_html5_repo_host' :
The service instance could not be found: mta_modul_html5_repo_host

Looks like it was fixed in the latest version (1.3.0).
Which version of CLI Plugin you are running?
You can check with cf plugins.

Closing the issue, since v1.4.0 and above are already available via cf install-plugin html5-plugin


Project recovery and autosave feature#2957

Fixes #2957  #34
Describe the changes you have made in this PR -
General Logic for Auto-save or Recovery :-

The Auto-save or Recovery only starts after some changes are made to current project because the user might be just viewing the project.
The are 4 variables to alter -
- changeCounter - counts the number of changes, increments by 1 with all necessary mouse events, cut, copy, delete, paste, undo, redo..etc.
- changeCounterLimit : number of changes after previous save on which to call auto-save/recover
- smallInterval : the number of seconds after which we check if changeCounter > changeCounterLimit then call auto-save/recover.
- bigInterval : the number of seconds after which we will do autosave/recover
As of now - smallInterval - 5 sec, bigInterval - 5 min,  changeCounterLimit - 5

Why use 2 Intervals ?

Autosaving after each necessary change event wont be feasible. so after every x sec (smallInterval) we check if some number (changeCounterLimit) of changes has been done then autosave.
Autosaving only after some y sec (bigInterval) is not feasible as the user might have done a lot of changes in that bigInterval and just before the bigInterval completes somethng happens and looses his data.

So why not only change counter with small interval ?
Because on using both change Counter with small Interval and Big Interval together it gives us the flexibility of having a relatively big value of changeCounterLimit. Eg. If we have some limit of 20 then if user stops after 10 to 15 changes the big interval will make sure that after some time those 10 to 15 changes are saved.
For autosave/recovery we have to call the generateSaveData() function and it takes some time.
So main reason to use them together is to make call for auto-save/recovery effectively.
Why not use update() function as reference for auto-save ?
Scheduled Update function is called every 0.5 seconds which calls update along with other calls so it will call auto-save (generateSaveData() function) every 0.5 sec so it is not feasible.

Task1 : Project Recovery -
When the project is neither saved online or offline.
Just keeping data of the latest unsaved project in local storage using the logic described above.
https://user-images.githubusercontent.com/61665451/156211554-2b97970e-fba9-4b9d-8f7a-9da2f837bdb9.mov

Task2 : Auto-save
Keep updating the data of the previously saved circuit according to the logic described above.
If project is saved online, creating a copy of it offline and keep updating the data in save offline.
https://user-images.githubusercontent.com/61665451/156212073-7669748c-ae34-46c1-8854-ef7892c850c7.mov
Also Save successful message in bottom is displayed only when we manually save, not on every auto-save call
On saving, removing the recover from local storage.

Task3 : Checking for latest edits on opening project from server
On opening a project from server, check if same project is present in Local Storage.
If found, the compare the time of the last saved changes. (also added a field in data object to store time of the last saved changes).
If time for last saved changes
(online) == (offline)
Show message - ${projectName} is up to date
(online) < (offline)
Show message  - {projectName} has some recent unsaved edits which are saved offline
https://user-images.githubusercontent.com/61665451/156213473-88213af6-7856-4179-bb39-e7db02ca20c0.mov
@tachyons @gr455 @ayan-biswas0412 @abstrekt can you please review it and suggest the changes required.

https://user-images.githubusercontent.com/61665451/156903565-9a888fd2-877e-46c2-b25a-b8d88bef323d.mov
Added a dialog box for Recover Project.


Fix sonar dup and logout bug

attempt to fix the sonar duplication issue
fix a bug where UI breaks when logging out while on a public shared photo list page

Looks good to me 👍 This time I even looked at sonarcloud 😄

I think it would be nice to enable status checks in branch protection


[file_selector_web] Cannot stream chunks from large files (~2gb)

Steps to Reproduce
Step 1:
Run the following code on the web:
Snippet (complete code attached below):
    final fileSize = await file.length();
    final chunkStarts = List.generate((fileSize/chunkSize.toDouble()).ceil(), (i) => i * chunkSize);

    for (var chunkStart in chunkStarts){
      final bytes = await file.openRead(chunkStart, min(fileSize, chunkStart + chunkSize)).single;
      print("Streamed ${bytes.length} bytes");
    }

main.dart
import 'dart:math';

import 'package:file_selector/file_selector.dart';
import 'package:flutter/material.dart';

void main() async {
  WidgetsFlutterBinding.ensureInitialized();

  runApp(MaterialApp(home: OpenImagePage()));
}

/// Screen that shows an example of openFiles
class OpenImagePage extends StatelessWidget {
  void _openImageFile(BuildContext context) async {
    final XTypeGroup typeGroup = XTypeGroup(
      label: 'Video files',
      extensions: ['mp4'],
    );
    final XFile file = (await openFile(acceptedTypeGroups: [typeGroup]));
    print(file.name);
    print(file.path);

    print('Will attempt to iterate through stream in chunks');
    final chunkSize = 5000000;
    final fileSize = await file.length();
    print("File size: ${fileSize}");
    final chunkStarts = List.generate((fileSize/chunkSize.toDouble()).ceil(), (i) => i * chunkSize);

    var totalBytesStreamed = 0;
    for (var chunkStart in chunkStarts){
      final chunkEnd = min(fileSize, chunkStart + chunkSize);
      print("Opening chunk [$chunkStart, $chunkEnd]");
      final bytes = await file.openRead(chunkStart, min(fileSize, chunkStart + chunkSize)).single;
      print("Streamed ${bytes.length} bytes");
      totalBytesStreamed += bytes.length;
    }
    print("Total bytes streamed: $totalBytesStreamed");
  }

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(
        title: const Text("Open video file_selector"),
      ),
      body: Center(
        child: Column(
          mainAxisAlignment: MainAxisAlignment.center,
          children: <Widget>[
            ElevatedButton(
              style: ElevatedButton.styleFrom(
                primary: Colors.blue,
                onPrimary: Colors.white,
              ),
              child: const Text('Press to open a file'),
              onPressed: () => _openImageFile(context),
            ),
          ],
        ),
      ),
    );
  }
}

Step 2:
Open a large file mp4 (2gb or larger file)
Expected results:
All chunks can be streamed regardless of file size.
Actual results:
The code works on smaller files, but larger files produce the following "RangeError":
Uncaught (in promise) Error: RangeError (end): Invalid value: Only valid value is 0: 5000000

Logs

> flutter run -d chrome --verbose --web-renderer html --web-hostname localhost --web-port 7357

[ +135 ms] executing: [C:\evo\apps\flutter/] git -c log.showSignature=false log -n 1 --pretty=format:%H
[ +835 ms] Exit code 0 from: git -c log.showSignature=false log -n 1 --pretty=format:%H
[        ] 225a43d941ddd1753765fec3682011e9ddffb11d
[   +1 ms] executing: [C:\evo\apps\flutter/] git tag --points-at 225a43d941ddd1753765fec3682011e9ddffb11d
[ +322 ms] Exit code 0 from: git tag --points-at 225a43d941ddd1753765fec3682011e9ddffb11d
[        ] 2.5.0-6.0.pre
[  +18 ms] executing: [C:\evo\apps\flutter/] git rev-parse --abbrev-ref --symbolic @{u}
[  +37 ms] Exit code 0 from: git rev-parse --abbrev-ref --symbolic @{u}
[        ] origin/dev
[        ] executing: [C:\evo\apps\flutter/] git ls-remote --get-url origin
[  +30 ms] Exit code 0 from: git ls-remote --get-url origin
[        ] https://github.com/flutter/flutter.git
[ +189 ms] executing: [C:\evo\apps\flutter/] git rev-parse --abbrev-ref HEAD
[  +38 ms] Exit code 0 from: git rev-parse --abbrev-ref HEAD
[        ] dev
[ +145 ms] Artifact Instance of 'AndroidGenSnapshotArtifacts' is not required, skipping update.
[        ] Artifact Instance of 'AndroidInternalBuildArtifacts' is not required, skipping update.
[        ] Artifact Instance of 'IOSEngineArtifacts' is not required, skipping update.
[        ] Artifact Instance of 'FlutterWebSdk' is not required, skipping update.
[   +6 ms] Artifact Instance of 'WindowsEngineArtifacts' is not required, skipping update.
[        ] Artifact Instance of 'WindowsUwpEngineArtifacts' is not required, skipping update.
[        ] Artifact Instance of 'MacOSEngineArtifacts' is not required, skipping update.
[        ] Artifact Instance of 'LinuxEngineArtifacts' is not required, skipping update.
[        ] Artifact Instance of 'LinuxFuchsiaSDKArtifacts' is not required, skipping update.
[        ] Artifact Instance of 'MacOSFuchsiaSDKArtifacts' is not required, skipping update.
[        ] Artifact Instance of 'FlutterRunnerSDKArtifacts' is not required, skipping update.
[        ] Artifact Instance of 'FlutterRunnerDebugSymbols' is not required, skipping update.
[ +218 ms] Artifact Instance of 'AndroidGenSnapshotArtifacts' is not required, skipping update.
[        ] Artifact Instance of 'AndroidInternalBuildArtifacts' is not required, skipping update.
[        ] Artifact Instance of 'IOSEngineArtifacts' is not required, skipping update.
[   +5 ms] Artifact Instance of 'WindowsEngineArtifacts' is not required, skipping update.
[        ] Artifact Instance of 'WindowsUwpEngineArtifacts' is not required, skipping update.
[        ] Artifact Instance of 'MacOSEngineArtifacts' is not required, skipping update.
[        ] Artifact Instance of 'LinuxEngineArtifacts' is not required, skipping update.
[        ] Artifact Instance of 'LinuxFuchsiaSDKArtifacts' is not required, skipping update.
[        ] Artifact Instance of 'MacOSFuchsiaSDKArtifacts' is not required, skipping update.
[        ] Artifact Instance of 'FlutterRunnerSDKArtifacts' is not required, skipping update.
[        ] Artifact Instance of 'FlutterRunnerDebugSymbols' is not required, skipping update.
[ +191 ms] Skipping pub get: version match.
[   +2 ms] Found plugin file_selector at C:\Users\neil\AppData\Roaming\Pub\Cache\hosted\pub.dartlang.org\file_selector-0.8.2\
[   +3 ms] Found plugin file_selector_web at C:\Users\neil\AppData\Roaming\Pub\Cache\hosted\pub.dartlang.org\file_selector_web-0.8.1+1\
[   +2 ms] Generating C:\projects\lc3\er\flutter\android\app\src\main\java\io\flutter\plugins\GeneratedPluginRegistrant.java
[ +348 ms] Launching lib\main.dart on Chrome in debug mode...
[ +226 ms] Updating assets
[ +744 ms] Manifest contained wildcard assets. Inserting missing file into build graph to force rerun. for more information see #56466.
[  +32 ms] Waiting for connection from debug service on Chrome...
[        ] Found plugin file_selector at C:\Users\neil\AppData\Roaming\Pub\Cache\hosted\pub.dartlang.org\file_selector-0.8.2\
[   +1 ms] Found plugin file_selector_web at C:\Users\neil\AppData\Roaming\Pub\Cache\hosted\pub.dartlang.org\file_selector_web-0.8.1+1\
[ +785 ms] <- reset
[  +34 ms] C:\evo\apps\flutter\bin\cache\dart-sdk\bin\dart.exe --disable-dart-dev C:\evo\apps\flutter\bin\cache\artifacts\engine\windows-x64\frontend_server.dart.snapshot --sdk-root C:\evo\apps\flutter\bin\cache\flutter_web_sdk/ --incremental --target=dartdevc --debugger-module-names --experimental-emit-debug-metadata -DLC_EVENT_DOT_DOMAIN=https://er-staging.web.app -DFLUTTER_WEB_AUTO_DETECT=false -DFLUTTER_WEB_USE_SKIA=false --output-dill C:\Users\neil\AppData\Local\Temp\flutter_tools.f4959f5e\flutter_tool.238c0e14\app.dill --libraries-spec file:///C:/evo/apps/flutter/bin/cache/flutter_web_sdk/libraries.json --packages C:\projects\lc3\er\flutter\.dart_tool\package_config.json -Ddart.vm.profile=false -Ddart.vm.product=false --enable-asserts --track-widget-creation --filesystem-root C:\Users\neil\AppData\Local\Temp\flutter_tools.f4959f5e\flutter_tools.2dee180e --filesystem-scheme org-dartlang-app --initialize-from-dill build\41295e4f4bb0f57ff3d594f134282bfe.cache.dill.track.dill --platform file:///C:/evo/apps/flutter/bin/cache/flutter_web_sdk/kernel/flutter_ddc_sdk.dill --no-sound-null-safety
[  +17 ms] <- compile org-dartlang-app:/web_entrypoint.dart
[+38169 ms] Waiting for connection from debug service on Chrome... (completed in 39.4s)
[   +1 ms] Synced 31.7MB.
[        ] <- accept
[        ] Caching compiled dill
[+1117 ms] [CHROME]:
[   +1 ms] [CHROME]:DevTools listening on ws://<IP_ADDRESS>:56482/devtools/browser/ca5b0fee-277b-4df8-a067-2f4a78dd0fe4
[ +833 ms] DwdsInjector: Received request for entrypoint at http://localhost:7357/main_module.bootstrap.js
[  +12 ms] MetadataProvider: Loading debug metadata...
[  +35 ms] MetadataProvider: Loaded debug metadata (no sound null safety)
[  +14 ms] DwdsInjector: Injected debugging metadata for entrypoint at http://localhost:7357/main_module.bootstrap.js
[+3445 ms] ChromeProxyService: Initializing expression compiler for main_module.bootstrap.js with sound null safety: false
[ +178 ms] DevHandler: Debug service listening on ws://<IP_ADDRESS>:59694/LfAz_hv6DKw=/ws

[  +41 ms] This app is linked to the debug service: ws://<IP_ADDRESS>:59694/LfAz_hv6DKw=/ws
[  +16 ms] Debug service listening on ws://<IP_ADDRESS>:59694/LfAz_hv6DKw=/ws
[        ] Running with unsound null safety
[        ] For more information see https://dart.dev/null-safety/unsound-null-safety
[   +4 ms]   To hot restart changes while running, press "r" or "R".
[        ] For a more detailed help message, press "h". To quit, press "q".
[   +1 ms] An Observatory debugger and profiler on Chrome is available at: http://<IP_ADDRESS>:59694/LfAz_hv6DKw=
[ +687 ms] DevTools activation throttled until 2021-08-19 00:19:19.716492.
[ +929 ms] The Flutter DevTools debugger and profiler on Chrome is available at: http://<IP_ADDRESS>:9109?uri=http://<IP_ADDRESS>:59694/LfAz_hv6DKw=
[+19232 ms] 2g.mp4
[        ] blob:http://localhost:7357/7a70f400-7f5a-4996-b3c7-db6cf35a4869
[        ] Will attempt to iterate through stream in chunks
[        ] File size:<PHONE_NUMBER>
[        ] Opening chunk [0, 5000000]
[+11264 ms] Error: RangeError (end): Invalid value: Only valid value is 0: 5000000
                at Object.throw_ [as throw] (http://localhost:7357/dart_sdk.js:5400:11)
                at Object._checkValidRange (http://localhost:7357/dart_sdk.js:33343:17)
                at Uint8Array.[dartx.sublist] (http://localhost:7357/dart_sdk.js:32469:37)
                at html$.XFile.new.openRead (http://localhost:7357/packages/cross_file/src/types/html.dart.lib.js:131:39)
                at openRead.next (<anonymous>)
                at _AsyncStarImpl.new.runBody (http://localhost:7357/dart_sdk.js:33428:40)
                at _RootZone.runUnary (http://localhost:7357/dart_sdk.js:40515:58)
                at _FutureListener.thenAwait.handleValue (http://localhost:7357/dart_sdk.js:35444:29)
                at handleValueCallback (http://localhost:7357/dart_sdk.js:36029:49)
                at Function._propagateToListeners (http://localhost:7357/dart_sdk.js:36067:17)
                at _Future.new.[_completeWithValue] (http://localhost:7357/dart_sdk.js:35909:23)
                at async._AsyncCallbackEntry.new.callback (http://localhost:7357/dart_sdk.js:35932:35)
                at Object._microtaskLoop (http://localhost:7357/dart_sdk.js:40820:13)
                at _startMicrotaskLoop (http://localhost:7357/dart_sdk.js:40826:13)
                at http://localhost:7357/dart_sdk.js:36291:9

n/a

[√] Flutter (Channel dev, 2.5.0-6.0.pre, on Microsoft Windows [Version 10.0.19043.1165], locale en-US)
    • Flutter version 2.5.0-6.0.pre
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 225a43d941 (13 days ago), 2021-08-05 11:32:48 -0700
    • Engine revision 07ec4b82c7
    • Dart version 2.14.0 (build 2.14.0-383.0.dev)

[√] Chrome - develop for the web
    • Chrome

[√] Android Studio (version 4.1)
    • Android Studio
    • Flutter plugin can be installed from:
       https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
       https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b01)

[√] Connected device (2 available)
    • Chrome (web)      • chrome               • web-javascript • Google Chrome 92.0.4515.159
    • Edge (web)        • edge                 • web-javascript • Microsoft Edge 92.0.902.73

This issue is similar to #76036

Hi @nwparker
I can reproduce the issue with 3.5GB video file, recorded from my phone, however this seems confirmed in https://github.com/flutter/flutter/issues/76036#issuecomment-893828596 too and the issue was closed
[        ] Will attempt to iterate through stream in chunks
[   +1 ms] File size:<PHONE_NUMBER>
[        ] Opening chunk [0, 5000000]
[+7051 ms] Error: RangeError (end): Invalid value: Only valid value is 0: 5000000
                        at Object.throw_ [as throw] (http://localhost:56404/dart_sdk.js:5348:11)
                        at Object._checkValidRange (http://localhost:56404/dart_sdk.js:31972:17)
                        at Uint8Array.[dartx.sublist] (http://localhost:56404/dart_sdk.js:31098:37)
                        at html$.XFile.new.openRead
(http://localhost:56404/packages/cross_file/src/types/html.dart.lib.js:131:39)
                        at openRead.next (<anonymous>)
                        at _AsyncStarImpl.new.runBody (http://localhost:56404/dart_sdk.js:32057:40)
                        at _RootZone.runUnary (http://localhost:56404/dart_sdk.js:39087:58)
                        at _FutureListener.thenAwait.handleValue (http://localhost:56404/dart_sdk.js:34073:29)
                        at handleValueCallback (http://localhost:56404/dart_sdk.js:34633:49)
                        at Function._propagateToListeners (http://localhost:56404/dart_sdk.js:34671:17)
                        at _Future.new.[_completeWithValue] (http://localhost:56404/dart_sdk.js:34513:23)
                        at async._AsyncCallbackEntry.new.callback (http://localhost:56404/dart_sdk.js:34536:35)
                        at Object._microtaskLoop (http://localhost:56404/dart_sdk.js:39374:13)
                        at _startMicrotaskLoop (http://localhost:56404/dart_sdk.js:39380:13)
                        at http://localhost:56404/dart_sdk.js:34887:9

smaller video files work fine
[+409354 ms] ForBiggerBlazes.mp4
[   +1 ms] blob:http://localhost:56404/c7c09b83-043a-4584-8ff5-6c40aaa4c7b1
[        ] Will attempt to iterate through stream in chunks
[   +1 ms] File size: 2498125
[        ] Opening chunk [0, 2498125]
[  +40 ms] Streamed 2498125 bytes
[   +1 ms] Total bytes streamed: 2498125

flutter doctor -v
[✓] Flutter (Channel stable, 2.2.3, on Microsoft Windows [Version 10.0.22000.132], locale en-US)
    • Flutter version 2.2.3 at C:\Users\Taha\Code\flutter_stable
    • Framework revision f4abaa0735 (7 weeks ago), 2021-07-01 12:46:11 -0700
    • Engine revision 241c87ad80
    • Dart version 2.13.4

[✓] Android toolchain - develop for Android devices (Android SDK version 31.0.0)
    • Android SDK at C:\Users\Taha\Code\android-sdk
    • Platform android-31, build-tools 31.0.0
    • ANDROID_SDK_ROOT = C:\Users\Taha\Code\android-sdk
    • Java binary at: C:\Users\Taha\Code\android-studio\jre\bin\java
    • Java version OpenJDK Runtime Environment (build 11.0.10+0-b96-7249189)
    • All Android licenses accepted.

[✓] Chrome - develop for the web
    • Chrome at C:\Program Files\Google\Chrome\Application\chrome.exe

[✓] Visual Studio - develop for Windows (Visual Studio Community 2019 16.11.0)
    • Visual Studio at C:\Program Files (x86)\Microsoft Visual Studio\2019\Community
    • Visual Studio Community 2019 version 16.11.31605.320
    • Windows 10 SDK version 10.0.19041.0

[!] Android Studio (not installed)
    • Android Studio not found; download from https://developer.android.com/studio/index.html
      (or visit https://flutter.dev/docs/get-started/install/windows#android-setup for detailed instructions).

[✓] VS Code (version 1.59.0)
    • VS Code at C:\Users\Taha\AppData\Local\Programs\Microsoft VS Code
    • Flutter extension version 3.25.0

[✓] Connected device (4 available)
    • IN2011 (mobile)   • c9d8ee0c • android-arm64  • Android 11 (API 30)
    • Windows (desktop) • windows  • windows-x64    • Microsoft Windows [Version 10.0.22000.132]
    • Chrome (web)      • chrome   • web-javascript • Google Chrome 92.0.4515.159
    • Edge (web)        • edge     • web-javascript • Microsoft Edge 92.0.902.73

! Doctor found issues in 1 category.

Should such big files be streamed by this package or Is it a general web issue?
cc: @ditman

@TahaTesser  -- I see the "waiting for customer response" label was added.
Does this mean I should supply additional information?

@nwparker
I have added a wait label as the original issue was closed even after it is not with > 2GB so I am not sure this issue is valid.
I will label for @ditman to respond

@nwparker I'll try to rewrite the streaming implementation with something like this: https://stackoverflow.com/questions/14438187/javascript-filereader-parsing-long-file-in-chunks

any progress on this issue?

All right, I found a fundamental problem with the current implementation of cross_file (and as a side effect, on how it's being used by the plugins):

We discard the blob that is used to create the file (in fact: we never show the blob to the XFile constructor, so we can't fix this issue "in one go")

This causes problems because it's hard to restore a Blob object from a blobUrl (it requires a fetch and uses 2x the memory! interesting SO answer). For very large files, this goes over the limit of memory of the browser (I'll experiment some more, but this seems to be the case).
In order to robustly fix this, we need to do two things:

[ ] Create a new html.File or html.Blob (possibly both) constructors for the XFile object, that are web-only.

[ ] For dart:io we could create a constructor that takes a io.File too.

[ ] Ideally, deprecate the default String path constructor, pointing users to the new platform-dependent constructors.
[ ] Reimplement the methods in XFile to use the new backing object(s).

[ ] This would allow us to implement slicing of the Blob in the web without much fuss, since the Blob is contained in the cross_file object.

Once the above is done, we need to:

[ ] Revisit the plugins that use cross_file, and fix the initialization of the XFile object to use the new backing objects: html.File/Blob or io.File.

Consumers of the XFile objects wouldn't be affected by this, unless they're creating their own XFile objects (they'd need to migrate to the non-deprecated constructors). To mitigate this, the cross_file version can be bumped one major version when adding the new constructors.

My assessment above of not creating the XFile from the Blob is still true. With my current patch, accessing the bytes of the blob the first time takes a while because the browser needs to "re-load" it. We could make large files in the web slightly faster by initializing them with the raw Blob, instead of its documentUrl.

I have a WIP version of a Blob-backed XFile here:
https://github.com/ditman/flutter-packages/tree/blob-backed-xfile
But in my (manual) testing, my code is making Safari on macOS reload the page (with @nwparker's example app) when selecting a file larger than 1 or 1.5GB. (The rest of the browsers seem to handle bigger files better with it, though.)
Can somebody else give the above branch a try and see if it works better/worse for them? I don't have too many test cases!

I've posted a PR here, if anybody is interested, please take a look!
https://github.com/flutter/packages/pull/494

In Chrome (desktop), it worked with a 10gb file. Although it did hang (loading forever on a 17gb file).

Very interesting data point! Let's see if some more bugs come about large files, and we'll try to find these extreme values! Thanks for testing with ludicrously large files! I ended up having to use mkfile, I couldn't download files that were large enough from anywhere :P

cross_file: ^0.3.2 might alleviate this issue? Now XFile on the web is implemented with a backing Blob, instead of an Array of bytes.

Thanks for the work!
I think we'll have to bump this dep (in File Selector) to use the new cross_file version:
https://github.com/flutter/plugins/blob/master/packages/file_selector/file_selector_platform_interface/pubspec.yaml

@nwparker yes, we can force File Selector and other plugins to pick a newer (non buggy) version of cross_file. However, pub should be able to resolve 0.3.2 from a dependency written as ^0.3.0.
If apps are not picking up the latest version of the library (check your pubspec.lock file to see exactly what version is being used), I'd recommend doing flutter pub upgrade so pub ignores the lock file and grabs the latest available in pub.dev. That should upgrade cross_file to 0.3.2, without having to publish any other package.


Detect x86 emulation for Windows on Arm

This PR detects x86 emulation for Windows on Arm and picks x64 or x86 binary base on what's supported.

This logic does not work on JRuby due to jffi missing prebuilt libffi for aarch64-Windows. Current on Windows Arm64, JRuby would always fallback to ia32 binary to ensure the compatibility with Windows 10 Arm64. Once jffi provides correct libffi.dll, the detection logic should work and it will fallback to x64 binary on Windows 11 Arm64.
cc @chadlwilson


How to use BatchNorm

Hello :wave:
I'm trying to port a neural net which makes use of some BatchNorm layers. But I'm confused how to use the BatchNorm op in Gorgonia.
Which nodes are trainable?
I'm assuming the scale (γ) and bias (β). However, the comment above the BatchNorm function tells a different story:
// BatchNorm applies a batchnormalization. This operator can be used in forward pass or for training.
// In an evaluation only, the "op" output can be discared.
// In training phase, γ, β can be discarded and the op should be used.
func BatchNorm(x, scale, bias *Node, momentum, epsilon float64) (retVal, γ, β *Node, op *BatchNormOp, err error) {

How is BatchNormOp used during training? I see that the SetTraining and SetTesting methods exist, but it is set to training by default already.
How do I save the op's state?
The mean, variance etc. aren't exported or accessible.
The default scale and bias keep the network from learning
When I started using the op I set the scale and bias to nil. BatchNorm will create these tensors for me and return them as γ and β. Using these defaults, my network doesn't train at all. If I manually initialize the scale with ones and the bias with zeroes it does work. Is this just weird or am I doing something wrong?
How to do validation during training?
Here's a simplified version of my training loop:
for i := 1; i <= epochs; i++ {
    m.SetTraining()
    m.Train(trainData.Load())
    cost, acc := m.Metrics()

    m.SetInference()
    m.Predict(validationData.Load())
    valCost, valAcc := m.Metrics()

    // print costs and accuracies
}

Every time I call the SetTraining function on the BatchNorm op it will implicitly call Reset too. With my loop, the train accuracy still goes up, but the validation accuracy stays around its initial value.
Edit: It seems that calling Reset manually at the start once keeps both training and validation accuracy from increasing, or at least not as fast.
Thanks for the help :)

Hi @MarkKremer -- just curious if you ultimately found some guidance on this?

Saw today's comment and was reminded of this.  I also was looking for examples or guidance on using BatchNorm and the difference in setting it up for training / testing.  IIRC the setups in test were not illuminating, though I admit I haven't looked recently.

It has been a long time :sweat_smile: . I think I found some bugs in the op (some of which I mentioned in this issue) but I'm not sure if I completely fixed it in the end and if those fixes were correct. See my old branches:

https://github.com/MarkKremer/gorgonia/commits/batchnorm-2 (diff)
https://github.com/MarkKremer/gorgonia/commits/batchnorm (diff)

I think the batchnorm branch was my first attempt with a bunch of comments to figure out what things were doing. The batchnorm-2 branch is a tidied up version of that (possibly incomplete) + some other changes with the intention of making a PR.
I also have a vague memory that one of the ops didn't match the CUDA version of that op. That could have been BatchNorm.
To answer my own questions as best I can:
Which nodes are trainable?
γ and β
How is BatchNormOp used during training? / How to do validation during training?
I think I was using op.SetTraining() and op.SetTesting() in the way you would expect (but see fixes in my branch). But in some other parts of my projects I wasn't. So I'm not sure anymore.
How do I save the op's state?
I don't know. γ and β can be saved. But not sure about mean and variance.
The default scale and bias keep the network from learning
I think this is still correct. See my changes to scale and bias in my batchnorm-2 branch. You can pass nil as parameter and add the returned γ and β to the learnables.
Hope this helps.

hello, BN is currently broken for evaluation mode due to a problem when calculating the gradient. We're working to fix it this week.
Training works well but you'll need to use the master to be able to access the mean and variance: https://github.com/gorgonia/gorgonia/blob/master/op_nn.go#L1256


Support user-defined colors

Users should be allowed to

Set the array of possible colors
Set their names so that they can

create bindings
see the tooltips

This is a distinct feature from #169, but if combining them, may wish to allow users to also modify the "free color" options (i.e. put free color options, including default values and what not, in the same array as fixed color options)

this is one of the things i wanted to be in #158


ngx-restangular v1.0.13, rxjs v5.5.0  Incompatible

it can work at ngx-restangular v1.0.13, rxjs v5.4.3

+1

+1

I had the exact same issue
Refering to https://github.com/2muchcoffeecom/ngx-restangular/issues/109

+1

Please update to 1.1.0 version!

still remains on 1.1.0 with rxjs v5.5.2

@AndreyNikolayev thanks for letting us know. We've fixed that.


Add ability to distinguish pending items

This needs to be done without matching the pending type
val s: ScalaVirtualMachine = _
val pendingBreakpoints = s match {
  case p: PendingBreakpointSupportLike => p.pendingBreapointRequests
  case _  => Nil
}

and without iterating through a list of pending items (although that should also be possible).

Resolved by #184 and #187.


Update Post “how-do-termites-appear-out-of-nowhere”

Automatically generated by Netlify CMS

👷 Deploy Preview for chic-shortbread-66a66b processing.

Name
Link

🔨 Latest commit
bf3fc94dfeea4da2aa3622fa8a7c97cfc8d388a3

🔍 Latest deploy log
https://app.netlify.com/sites/chic-shortbread-66a66b/deploys/654875cef7fbaf0008e21045


[Script]: Notification App(GUI)

Aim
The notification app in Python is a simple graphical user interface (GUI) application that allows users to set a notification and receive a pop-up notification with a custom message after a specified delay.
Details
The app provides a dark-themed interface and uses the tkinter library for GUI development and the winsound module (available on Windows) to play a system notification sound.
Record

[X] I agree to follow this project's Code of Conduct
[X] I'm a GSSoC'23 contributor
[X] I want to work on this issue

/assign


Create contributors' guide

Per @jquesada2016 in https://github.com/gbj/leptos/issues/53#issuecomment-1304910693 it would be a great idea to put together a short guide to make it easier for people to contribute.
I'm curious to hear from others what sort of stuff would be useful in there.
Outline of the various packages and what goes where? General approach of each? Normal flow of execution in the program?
What would be useful?

I personally have two suggestions. I believe that since the crate is in the early stages, and major changes are likely to ensue, an overview of the approaches, or architecture of the overall project would be most useful. This can be, for example, explaining the core principles on how the reactive system manages to be Copy, the purpose of the Scope, etc. Also, what techniques are used for updating the DOM reactively. Are components treated specially? Etc. No need to be specific to current code, as this will likely change. Perhaps once 0.1.0, then more specifics could be included.
The thing I believe would help the most, especially because one would only need to do this once upfront, and then gradually as things change, is to add #[deny(missing_docs)].

@jquesada2016 had really the most important bits in my mind, especially given the nature of the reactive system being so drastically different to most modern mental models...as well as your included bits.
I don't think I have that level of input just yet, but thinking more to ease scaling  (which is likely in my opinion) without you pulling your hair out.
I think setting up a PR syntax is a good idea so future maintainers can more easily skim PRs, also include some form of Prettier, etc. so the code is unified in layout.
Maybe an iterative FAQ page for contributors -- for those specific instances of misunderstanding of whatever version the contributor guide will be (I can almost guarantee I'll be asking one of those FAQs xD)
If there is a specific way you are interpreting a library like Solidjs and extracting the required information to re-create in Rust, a quick example might also be nice.


Add Who is using Deployer?

Just add my credits

Hi, are you using Deployer in company or organization?

Hi,
actually I'm a freelance so I do not have a "company"...

Sorry, a have to say no then.


pypi missing acoular logo and result image

README includes local file references. This causes problems on PyPI, namely, the images cannot be accessed.

In GitLab by @esarradj on Mar 12, 2024, 17:14
mentioned in commit c5e7e843cc6e6767604244c64030e7af48524466


Please consider fixing issues detected by repository checker

Notification from ioBroker Check and Service Bot
Dear adapter developer,
I'm the ioBroker Check and Service Bot. I'm an automated tool processing routine tasks for the ioBroker infrastructure. I have recently checked the repository for your adapter vis-2-widgets-ovarious for common errors and appropiate suggestions to keep this adapter up to date.
Please see the result of the check below.
ioBroker.vis-2-widgets-ovarious
   -

:thumbsup: No errors found
WARNINGS:

[ ] :eyes: [W162] js-controller 5.0.17 listed as dependency but 5.0.19 is recommended. Please consider updating dependency at io-package.json.
[ ] :eyes: [W400] Cannot find "vis-2-widgets-ovarious" in latest repository
[ ] :eyes: [W519] Consider adding plugin "@alcalzone/release-script-plugin-manual-review".

Please review issues reported and consider fixing them as soon as appropiate.
Warnings reported by repository checker should be reviewed. While some warnings can be considered as a suggestion and be ignored due to good reasons or a dedicated decision of the developer, most warnings should be fixed as soon as appropiate.
Feel free to contact me (@iobroker-bot) if you have any questions or feel that an issue is incorrectly flagged.
And THANKS A LOT for maintaining this adapter from me and all users.
Let's work together for the best user experience.
your
ioBroker Check and Service Bot
@mcm1957 for evidence

This issue has been replaced by new isse #2
This issue can be closed.
your
ioBroker Check and Service Bot


KC Migration from Silver to Gold [#3002]

Migration of KC (Gold-Dev)

[x] Setup of the identity provider GitHub
[x] Setup of the identity Provider IDIR
[x] Setup of the identity provider BCSC
[x] Migration of client forms-flow-bpm
[x] Migration of client forms-flow-web
[x] Migration of client forms-flow-analytics
[x] Migration of client foi-web
[x] Migration of client foi-lob-api
[x] Migration of client foi-help-frontend
[x] Migration of client AXISFOIAPI
[x] Migration of client foi-document-redaction
[x] Migration of saml client for reports
[x] Migration of Groups
[x] Migration of Users
[x] Migration of existing User Group Mappings
[x] Admin Setup

NOTE: forms-flow-bpm needs its dependencies to be upgraded for supporting the new KC.
forms-flow-bpm

[x] Upgrade of forms-flow-bpm dependencies to support Gold KC

Migration of Users

[x] IDIR
[x] Test Accounts [Basic]
[x] BCSC [Not Applicable - Need Confirmation]

Supported Authentication mechanism - Dev

[x] IDIR
[x] Github
[x] BCSC
[x] Basic

Supported Authentication mechanism - Test

[x] IDIR
[x] Github
[x] BCSC
[x] Basic

Supported Authentication mechanism - Prod

[ ] IDIR
[ ] BCSC

Unit Testing

[x] FOI Reports
[x] FOI Web
[x] FOI Backend
[x] FOI BPM
[x] FOI formsflow.ai

Integration Testing - Dev Marshal

[x] FOI Reports
[x] FOI Web
[x] FOI Backend
[x] FOI BPM
[x] FOI formsflow.ai
[x] Redaction app

Integration Testing - Test Marshal

[x] FOI Reports
[x] FOI Web
[x] FOI Backend
[x] FOI BPM
[ ] FOI formsflow.ai

Migration Entities (In order to approach)

IDP
Clients
Groups
Users
User – Group mapping

How is it Done ?

Step-1 : Got admin access to realms: dev, test, prof : Completed
Step-2: Submitted request to get IDPs (IDIR & BCGov github) using link https://bcgov.github.io/sso-requests/my-dashboard/integrations : Completed
Step-3: Setup IDP & send email to get setup completed for BC Services Card
a.	Reference to reach team to have a new redirect URI setup
https://stackoverflow.developer.gov.bc.ca/questions/512/515#515
Import Clients - Leverage admin privilege to export and import [Ensure to remove all IDs with are random generated]
Import Groups - Leverage admin privilege to export and import [Ensure to remove all IDs with are random generated]
Import Users
a.	IDIR : Leverage the script given by the sso -> reference : https://github.com/bcgov/sso-keycloak/tree/dev/scripts/custom-realm-users
b.	Local Accounts : Leverage postman API
c.	BCSC : Not required
User-Group Mapping : Leveraged custom plugin (.jmx) to do it

Issues Faced:

Unable to logout from the session (Tested in browsers – Edge, Chrome and FF) -> Issue persists in FF
Root cause – SSO request to have logout URI configured
Variation in the value format of preferred_username
Resolution - Custom mapper in IDP & client
Migration of users with script
Observations:
a.	Script enforces to give PAT; even though the silver has only IDIR and BASIC users.
Note: Giving PAT is mandatory (The script is driven by IDP and not .env)
b.	Script did not run with user ID with admin privileges whereas; it requires a service account with realm administration
c.	Script creates the user with “Provider User ID” in upper case. [ Blocker -> New user gets created every time with the additional setting of “Duplicate emails” enabled].
Resolution: Deleted all users and re-ran the modified script.

Realm Settings
Not advisable to have "Duplicate emails" at the realm level. Turned it off !!!! - Sumathi

TO DO MANUAL
For BPM: The client service account is to be manually provided with realm user and group related permissions.

TO DO MANUAL
For client "forms-flow-web". Manually add the mapper "Preffered_username".

TO DO MANUAL
Setup session management properties

TO BE MANUAL
Set password for local accounts.


Unable to perform asset reset

I am using the below json to make an API request to the CP API, then signing and broadcasting the transactions, but unfortunately dogeparty does not appear to be picking up the transaction.
{
    "method": "create_issuance",
    "params": {
        "source": "DJDog8zPScfL6dXawkdYDfiQDnGqSUrA4P",
        "asset": "TWERK",
        "quantity": 100,
        "divisible": 0,
        "reset": true,
        "description": "imgur/RiV3bCX.gif;Dogeparty Twerk",
        "transfer_destination": null,
        "lock": false,
        "fee": 100000000,
        "allow_unconfirmed_inputs": true
    },
    "jsonrpc": "2.0",
    "id": 0
}

issue fixed with pull request #9


¿Money values in dataset?

This is  a feature request.

Prerequisites

[X] Are you running the latest version?
[X] Are you reporting to the correct repository?
[X] Did you check the documentation?

Description

Good morning, I would like to make a linear chart of the sales of the month. At the moment I have no problem doing it, just that I would like the values to be shown in money format, it would give a better appearance to my chart. I have tried in many ways without being able to get it :(
Do you think there is a way to achieve it?.

Chart.vue has a formatter prop where you can pass a function to achieve exactly that.
The formatter is for the datalabels plugin (already used in the component)

And, you can search for numberFormat or shortNumber in enso-ui/ui and find maybe useful examples.

@heyner29 can we close this?

Yes, excuse me.  I solved it as you suggested, once again a thousand thanks.