For yield returning more than one Node causes GC overhead limit exceeded

Dependencies

OS: Mac 10.14 (Mojave)
Java Zulu JDK: <IP_ADDRESS>-macosx
SBT: 1.3.7
scalajs: 1.2.0
Binding + html.scala: latest

Steps to reproduce
{ for (conversation <- conversations) yield {

}}

Currently, my workaround is to wrap those sequence of elements in
This is the stacktrace
java.lang.OutOfMemoryError: GC overhead limit exceeded

  | => hat scala.runtime.IntRef.create(IntRef.java:23)
	at scala.util.hashing.MurmurHash3.unorderedHash(MurmurHash3.scala:93)
	at scala.util.hashing.MurmurHash3$.setHash(MurmurHash3.scala:230)
	at scala.collection.GenSetLike.hashCode(GenSetLike.scala:135)
	at scala.collection.GenSetLike.hashCode$(GenSetLike.scala:135)
	at scala.collection.AbstractSet.hashCode(Set.scala:51)
	at scala.runtime.Statics.anyHash(Statics.java:122)
	at scala.util.hashing.MurmurHash3.productHash(MurmurHash3.scala:68)
	at scala.util.hashing.MurmurHash3$.productHash(MurmurHash3.scala:215)
	at scala.runtime.ScalaRunTime$._hashCode(ScalaRunTime.scala:149)
	at scala.Tuple2.hashCode(Tuple2.scala:24)
	at scala.runtime.Statics.anyHash(Statics.java:122)
	at scala.collection.immutable.HashMap.elemHashCode(HashMap.scala:87)
	at scala.collection.immutable.HashMap.computeHash(HashMap.scala:96)
	at scala.collection.immutable.HashMap.updated(HashMap.scala:62)
	at scala.collection.immutable.Map$Map4.updated(Map.scala:227)
	at scala.collection.immutable.Map$Map4.$plus(Map.scala:228)
	at scala.collection.immutable.Map$Map4.$plus(Map.scala:200)
	at scala.collection.mutable.MapBuilder.$plus$eq(MapBuilder.scala:32)
	at scala.collection.mutable.MapBuilder.$plus$eq(MapBuilder.scala:28)
	at scala.collection.TraversableLike.$anonfun$groupBy$4(TraversableLike.scala:415)
	at scala.collection.TraversableLike$$Lambda$505/1556162268.apply(Unknown Source)
	at scala.collection.TraversableLike$WithFilter.$anonfun$foreach$1(TraversableLike.scala:877)
	at scala.collection.TraversableLike$WithFilter$$Lambda$401/1152507263.apply(Unknown Source)
	at scala.collection.mutable.HashMap.$anonfun$foreach$1(HashMap.scala:149)
	at scala.collection.mutable.HashMap$$Lambda$506/1650332494.apply(Unknown Source)
	at scala.collection.mutable.HashTable.foreachEntry(HashTable.scala:237)
	at scala.collection.mutable.HashTable.foreachEntry$(HashTable.scala:230)
	at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:44)
	at scala.collection.mutable.HashMap.foreach(HashMap.scala:149)
	at scala.collection.TraversableLike$WithFilter.foreach(TraversableLike.scala:876)
	at scala.collection.TraversableLike.groupBy(TraversableLike.scala:414)
Exception in thread "task-progress-report-thread" java.lang.OutOfMemoryError: GC overhead limit exceeded
	at java.util.Arrays.copyOf(Arrays.java:3332)
	at java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:124)
	at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:448)
	at java.lang.StringBuilder.append(StringBuilder.java:136)
	at sbt.internal.util.ObjectEvent.toString(ObjectEvent.scala:19)
	at java.lang.String.valueOf(String.java:2994)
	at org.apache.logging.log4j.message.ObjectMessage.getFormattedMessage(ObjectMessage.java:55)
	at org.apache.logging.log4j.core.async.InternalAsyncUtil.makeMessageImmutable(InternalAsyncUtil.java:41)
	at org.apache.logging.log4j.core.impl.MutableLogEvent.setMessage(MutableLogEvent.java:220)
	at org.apache.logging.log4j.core.impl.ReusableLogEventFactory.createEvent(ReusableLogEventFactory.java:80)
	at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:404)
	at org.apache.logging.log4j.core.config.AwaitCompletionReliabilityStrategy.log(AwaitCompletionReliabilityStrategy.java:63)
	at org.apache.logging.log4j.core.Logger.logMessage(Logger.java:146)
	at org.apache.logging.log4j.spi.AbstractLogger.tryLogMessage(AbstractLogger.java:2170)
	at org.apache.logging.log4j.spi.AbstractLogger.logMessageTrackRecursion(AbstractLogger.java:2125)
	at org.apache.logging.log4j.spi.AbstractLogger.logMessageSafely(AbstractLogger.java:2108)
	at org.apache.logging.log4j.spi.AbstractLogger.logIfEnabled(AbstractLogger.java:1827)
	at org.apache.logging.log4j.spi.AbstractLogger.log(AbstractLogger.java:1613)
	at sbt.internal.util.ManagedLogger.logEvent(ManagedLogger.scala:53)
	at sbt.internal.TaskProgress.report0$1(TaskProgress.scala:116)
	at sbt.internal.TaskProgress.sbt$internal$TaskProgress$$report(TaskProgress.scala:125)
	at sbt.internal.TaskProgress$ProgressThread.run(TaskProgress.scala:36)

What is the Scala version?

2.13.3

I have another issue that is related to this. Not sure that it is caused by the same bug.
If there is an unrecognized attribute in a tag, this GC overhead limit will be thrown as well.

Summary

From the above snippet, data-toggle will cause the GC issue.
The solution is adding data: prefix to that attribute, but I think this kind of error should be handled.

Does this bug affect Scala 2.12?
Hussachai Puripunpinyo<EMAIL_ADDRESS>于2020年10月14日周三 上午11:44写道：

I have another issue that is related to this. Not sure that it is caused
by the same bug.
If there is an unrecognized attribute in a tag, this GC overhead limit
will be thrown as well.
Summary
From the above snippet, data-toggle will cause the GC issue.
The solution is adding data: prefix to that attribute, but I think this
kind of error should be handled.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/GlasslabGames/html.scala/issues/97#issuecomment-708590492,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAES3OQ6GRKLCQFB3PFIERDSKXWSTANCNFSM4SQCTOYQ
.

Does this bug affect Scala 2.13.1?
Yang, Bo<EMAIL_ADDRESS>于2020年10月14日周三 下午12:26写道：

Does this bug affect Scala 2.12?
Hussachai Puripunpinyo<EMAIL_ADDRESS>于2020年10月14日周三
上午11:44写道：

I have another issue that is related to this. Not sure that it is caused
by the same bug.
If there is an unrecognized attribute in a tag, this GC overhead limit
will be thrown as well.
Summary
From the above snippet, data-toggle will cause the GC issue.
The solution is adding data: prefix to that attribute, but I think this
kind of error should be handled.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/GlasslabGames/html.scala/issues/97#issuecomment-708590492,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAES3OQ6GRKLCQFB3PFIERDSKXWSTANCNFSM4SQCTOYQ
.

I'm not sure about 2.12. I'm using Scala.js 1.2 and html.scala is not crossed compile for Scala.js 1.2 and Scala 2.12
This is the error I got Error downloading org.lrng.binding:html_sjs1_2.12:latest.release
Let me try 2.13.1

It works fine for 2.13.1 👍

Maybe related to https://github.com/scala/bug/issues/12021
Hussachai Puripunpinyo<EMAIL_ADDRESS>于2020年10月14日周三 下午12:38写道：

It works fine for 2.13.1 👍 Thank you!
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/GlasslabGames/html.scala/issues/97#issuecomment-708617677,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAES3OUMRHRRHURNPYXCWMLSKX42PANCNFSM4SQCTOYQ
.

ok, looks like it will be fixed in Scala 2.13.4.  I will close this issue. Thank you.

It should have been fixed in 2.13.3. Since it's not actually fixed, I don't
think 2.13.4 would be any better.
Hussachai Puripunpinyo<EMAIL_ADDRESS>于2020年10月14日周三 下午1:01写道：

ok, looks like it will be fixed in Scala 2.13.4. I will close this issue.
Thank you.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/GlasslabGames/html.scala/issues/97#issuecomment-708628804,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAES3OWAPXGIOA5S2AAEDCTSKX7QFANCNFSM4SQCTOYQ
.

You're right. The supposedly fix was merged before 2.13.3.
Actually, the issue you mentioned was Stackoverflow, but this issue is more like consuming too much heap space.
Should I re-open this issue?

I think it's a regression in Scala, which could be reported at
https://github.com/scala/bug
Hussachai Puripunpinyo<EMAIL_ADDRESS>于2020年10月14日周三 下午6:17写道：

You're right. The supposedly fix was merged before 2.13.3.
Actually, the issue you mentioned was Stackoverflow, but this issue is
more like consuming too much heap space.
Should I re-open this issue?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/GlasslabGames/html.scala/issues/97#issuecomment-708824907,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAES3OVDH7JPAEA6I3FIK63SKZERLANCNFSM4SQCTOYQ
.

Fair enough. Thank you.


iOS: widget is slow to reload

I think we should separate out performance from styling. In fact, I think this is responsible for this bug I saw today:

I'm doing some analysis to see if I can find bottlenecks, and use that to improve the loading performance. A more complex solution might be to use data loaded by the main app, perhaps if sufficiently recent, but there are some issues with that. I'll see what I can come up with.
That message should mean what it says; no eligible stories for the sites. I don't think it'd appear due to loading time.

The two time-consuming parts are (unsurprisingly) fetching the stories, and fetching the images (feed and story). So a solution could be (as you suggested via email) to move that to server side, with a single endpoint call to get JSON data with the relevant number of stories with their images.
Parameters:

Number of stories to fetch (currently 3 for small, 6 for large)
The user token
An array of feed IDs for which to fetch stories.
An array of feed IDs that have already been fetched, so don't need to include favicon images for them.
An array of story hashes that have already been fetched, so don’t need to include thumbnail images for them.

Response:
A JSON object (dictionary) containing:

An array of feed info:

id
feed_title
favicon_color
favicon_fade
favicon_image

An array of story info:

story_hash
story_feed_id
short_parsed_date
story_title
thumbnail_image: only included if not in the parameter array

Can omit feed info for feeds specified in the parameter of already fetched feeds.
Alternatively could return a single array of stories, with the feed info embedded, if only including the feed image once, if needed.
Or could simplify by skipping those parameters, and always return all of the images; should be negligible extra server or data transfer time.
Also the story images can be pre-scaled to 192x192 pixels, which will reduce the time to fetch them.
What do you think?

Nice job, I'll build that endpoint tomorrow. As for story images, you can use the new story image proxy, which will happily resize images based on the url, which is provided per-thumbnails in the secure_image_thumbnails dictionary per story.

I do fetch images from secure_image_thumbnails, though may not provide a resizing parameter? Though doing multiple requests has a cost, too; combining them into one would be faster.
 The widget update takes a total of 10 seconds.
Combining all of that into one request, with properly scaled images, should reduce it to only a few seconds, even on my slow connection.

In the secure story image url, the 192,sc are the scaling parameters. But what I'll do is do the fetch on the server-side (we already have the infrastructure for 3-5 second requests) and base64 encode them.
I wish I supported Python 3's async libs. This may be the time to implement the first one!

Ok, here's the URL: /reader/river_stories_widget. Works the same as river_stories (in fact, it uses river_stories), plus it adds the select_thumbnail_data key to each story if it found the image in less than 3 seconds. Make sure to set limit=6 so you only get 6 stories back.
This uses the imageproxy which also caches images, so it'll be faster for subsequent users. The image is base64 encoded. Happy to use another encoding, but it's json so we're a bit limited.
Let's see if having a single request and centralized image fetching takes care of 90% of performance without having to dip into a more custom endpoint.

Thanks. Does the limit=6 account for hidden stories too?

Oh, and I guess it just includes story images, not feed images? Which is probably fine, since there will be a limited number of feed favicons, they're cached, and they're small.

Does the limit=6 account for hidden stories too?

No, but you also won't receive hidden stories unless you also pass in include_hidden=true. So you may have to page through this to fill it up. It wouldn't be all that difficult to do on the backend. Just build some recursive footer that builds up the list until its either full or out of tries. Maybe a replace_hidden_stories=true flag, knowing it'll take a while.

No, it just requests one page. Which would be why it sometimes didn't find enough for you. So in the interest of reducing network calls, if you can fill it up on the server, that'd help.

I didn't see any image scaling in the server code (though perhaps I missed it?). If not, please add that too; no point in sending back full-sized images. I'll pass a thumbnail_size parameter for the desired size. To allow for the different display sizes and retina, I'll probably use thumbnail_size=300.
I'll also pass replace_hidden_stories=true, to fill up to the requested number of stories.

Better performance, sometimes; it's loading a lot more data with the stories, presumably because the images are not pre-scaled, so should be faster once you do that:
	15:14:33.003711-0800	getTimeline: loadCachedStories took 0.068593 seconds (total 0.068593 seconds) 🚨🚨
	15:14:35.992250-0800	/reader/river_stories_widget data: 49541 bytes
	15:14:36.112763-0800	/reader/river_stories_widget data: 8996 bytes
	15:14:36.223042-0800	/reader/river_stories_widget data: 19657 bytes
	15:14:36.418979-0800	/reader/river_stories_widget data: 21653 bytes
	15:14:36.530838-0800	/reader/river_stories_widget data: 20638 bytes
	15:14:36.681008-0800	/reader/river_stories_widget data: 20841 bytes
	15:14:36.681580-0800	/reader/river_stories_widget data: 11 bytes
	15:14:36.789336-0800	/reader/river_stories_widget data: 21323 bytes
	15:14:36.904323-0800	/reader/river_stories_widget data: 28976 bytes
	15:14:37.019175-0800	/reader/river_stories_widget data: 16307 bytes
	15:14:37.029264-0800	getTimeline: cache.load() took 4.025544 seconds (total 4.094137 seconds) 🚨🚨🚨🚨
	15:14:37.029432-0800	getTimeline: making timeline took 0.000201 seconds (total 4.094338 seconds)
	15:14:37.886876-0800	/reader/favicons data: 73 bytes
	15:14:37.888847-0800	getTimeline: requesting images took 0.859388 seconds (total 4.953726 seconds) 🚨🚨🚨

Also, sometimes the thumbnail image is some sort of placeholder; the server should return no image rather than this:

Those are pre-scaled images, at 192px. I think that's why they're all around 20k. Compare this image that has a lot of white background:
(9.8k, 192px) https://imageproxy.newsblur.com/192,sc,sh4rvrCkwJtXWlh2n3jdgvnxX6GSI7P1jG-VsnD95GNM=/https://techcrunch.com/wp-content/uploads/2022/02/Amira-Yahyaoui_.jpg
(1.5MB, original) https://imageproxy.newsblur.com/sc,sh4rvrCkwJtXWlh2n3jdgvnxX6GSI7P1jG-VsnD95GNM=/https://techcrunch.com/wp-content/uploads/2022/02/Amira-Yahyaoui_.jpg
As for choosing a better image, the setup I have doesn't actually know the original dimensions, so it will naively scale up images. Thinking about ways of addressing that, but it's a known issue.

We can probably close this now, unless you want to try any further tweaks?

After switching to the staging server, I can't get it to load. Rebooted my phone and still showing this load screen. I'm watching the staging logs and the widget river endpoint is never called. And when I tapped on the widget, no sites were checked. Makes sense, given I changed servers, but we should probably handle this by deleting things or something.
I committed the staging to the testflight branch, but it's the PROD_DEBUG flag.

Should be https

I'm seeing this still with the main www as well.
https://user-images.githubusercontent.com/44229/156235718-218ae1f7-5212-4d63-96cc-1d44cb3f85ea.MP4

I sometimes get that too.
The cause is the /reader/river_stories_widget/ request is getting the error:
Error Domain=NSURLErrorDomain Code=-999 "cancelled"

This is with this request, with lots of feeds:
https://www.newsblur.com/reader/river_stories_widget/?include_hidden=false&replace_hidden_stories=true&thumbnail_size=192&page=1&infrequent=false&order=newest&read_filter=unread&limit=6&f=8264795&f=3581&f=8264796&f=109755&f=1003963&f=8264797&f=569&f=8337202&f=183139&f=7808652&f=7300406&f=1186180&f=2186&f=192575&f=4353&f=3233139&f=76&f=26931&f=8429586&f=1340342&f=1485973&f=5994357&f=1590355&f=6037101&f=5719953&f=7049261&f=2998&f=5168&f=1665281&f=7101685&f=6342410&f=1272495&f=3240&f=7505145&f=2308&f=6038475&f=6238772&f=1138885&f=2907440&f=6238774&f=212379&f=1010775&f=6556107&f=146&f=4153&f=3530&f=8261266&f=6476&f=38&f=644144&f=8264790&f=8264791&f=8264792&f=8264785&secret_token=2785fe5513db

Error 999 can be caused by doing the exact same request twice (which doesn't happen), losing the reference to the request (which also doesn't happen), or by a certificate issue on the server (which doesn't seem to be the case, since it works sometimes).
Experimentation shows that it is more likely to happen with lots of feeds, and less likely with only a few. Which suggests it could be a timeout issue, of the server taking too long.
I'll experiment some more.

 It's a 500 exception and I've got the traceback in Sentry. Just deployed a fix and that url should work now.
Problem was that some images in stories are encoded as data-uri, so the backend was naively sending them to the imageproxy, which choked: https://imageproxy.newsblur.com/192,sc,sEermKyn22l4_mZxwHOr2dW9Esbadb7MZWomoqKJquj0=/data:image/svg+xml;base64,PHN2ZyB3aWR0aD0iNDAiIGhlaWdodD0iNDAiIHhtbG5zPSJodHRwOi8vd3d3LnczLm9yZy8yMDAwL3N2ZyIgdmVyc2lvbj0iMS4xIi8+
So I set it to ignore data-uri images and it now that url is working for me. The widget is still a single grey line, but i moved back to the testflight version (which is why I think I regressed on the folder highlight bug).
One thing that would personally help me is to hide muted feeds in the widget list. I have 900 feeds but only 400 aren't muted. And that Widget dialog is a bit wonky as it'll show up all checked off and then suddenly all the checkmarks disappear. Would be nice to show a loading indicator.

I'm glad you were able to spot the issue. That does seem to have fixed it.
Hiding muted feeds in the widget sites preference is a good idea.
Another concept that I've been thinking about, and might try if you don't object, is to add a new smart folder for widget sites, showing only the feeds selected in the widget prefs. And show that feed when loading from a widget. That would make finding a story much faster, since the ones shown in the widget should be right at the top.
I don't see the widget sites preference loading wonkiness; it loads immediately. I tried logging in as you, but it still loads immediately, at least in the Simulator. But hiding muted sites might help.

I feel like the widget feed selection is too fine-grained. I either want a single feed or a folder, defaulting to the all folder. And I'm struggling with the widget feed list right now because the widget is now showing a "Tap to setup in newsblur" message, and sometimes the widget dialog shows all of my sites selected and sometimes deselected.
I just released this on TestFlight and will promote to prod tomorrow if I don't hear any complaints. The widget may not work for me but we'll fix it quickly enough if others have the same issue.

After playing with the Widget Site Stories folder, I'm more convinced this should only feature the option for a single feed or a folder. These can be shown in two lists, so folders are shown first.
There's also a bug with the Widget Site Stories folder that shows all sites, including muted sites, on the feed list when go into the Widget Site Stories folder and swiping back to the feed list.
The use case for the Widget Site Stories folder is better served by allowing multiple instances of the widget with different single feed/folder settings.

I agree ideally supporting multiple instances of the widget, each with their own single feed/folder settings, would be the ideal. I could work on that next week if you wanted to hold off the release for it. It will take some time, though.
Alternatively, I could rip out (or just hide) the the new Widget Site Stories smart folder, replace the Widget Sites preferences with a picker for a single folder/feed (like when moving a feed), and display that folder/feed when tapping a story in the widget. That would be much faster. (And if you reply promptly, I could probably do it tonight or tomorrow.)
Or we could go with the current implementation for now.
Let me know what you'd prefer.

Alternatively, I could rip out (or just hide) the the new Widget Site Stories smart folder, replace the Widget Sites preferences with a picker for a single folder/feed (like when moving a feed), and display that folder/feed when tapping a story in the widget. That would be much faster. (And if you reply promptly, I could probably do it tonight or tomorrow.)

Yep, let's do this for now. And we can ship multiple widget later, although that should probably be in the dejal branch and any hotfixes should go in testflight.

Im still not seeing stories. The widget doesn't make any requests, so there's a hint that's somethings wrong.

Weird. Trying to think of possibilities... you have done a clean build, to ensure the widget is up-to-date? You have set some sites in the Widget Sites prefs? You are using the new widget, not the old Today widget?
Have you tried selecting different sites, to see if certain sites are causing it?
When running in Xcode, you could attach a debugger to the widget to see what it is doing; perhaps something in the data causing an exception, or something.

🎉


ImageLoader with 2 differents URL

I would like to display a thumbnail (URL 1) in the message preview and implement an onClickListener in the imageLoader and use another URL(URL 2) to show it.
Is there a way to do that?

Nvm found the solution with the method setIncomingImageConfig()


[Event Request ] Codeunit 1535 Approvals Mgt. - PrePostApprovalCheckSales procedure, change event subscriber behaviour

Hi guys, according to our business needs we are going to override logic in PrePostApprovalCheckSales function by using event subscriber for OnBeforePrePostApprovalCheckSales event.
procedure PrePostApprovalCheckSales(var SalesHeader: Record "Sales Header"): Boolean
    var
        IsHandled: Boolean;
    begin
        IsHandled := false;
        OnBeforePrePostApprovalCheckSales(SalesHeader, IsHandled);
        if IsHandled then
            exit;

        if IsSalesHeaderPendingApproval(SalesHeader) then
            Error(SalesPrePostCheckErr, SalesHeader."Document Type", SalesHeader."No.");

        exit(true);
    end;

We have figured out the following if use the mentioned subscriber and put IsHandled = true, Business Central stops positng process as this function returns false. To resume posting function should return true.
We noticed that same issue with the same request was raised some time ago https://github.com/microsoft/ALAppExtensions/issues/12250.
Thus, we suggest to this function in the following way:
procedure PrePostApprovalCheckSales(var SalesHeader: Record "Sales Header"): Boolean
    var
        Result: Boolean;
        IsHandled: Boolean;
    begin
        IsHandled := false;
        OnBeforePrePostApprovalCheckSales(SalesHeader, Result, IsHandled);
        if IsHandled then
            exit(Result);

        if IsSalesHeaderPendingApproval(SalesHeader) then
            Error(SalesPrePostCheckErr, SalesHeader."Document Type", SalesHeader."No.");

        exit(true);
    end;

For Purchase, we already have the similar functionality:
    procedure PrePostApprovalCheckPurch(var PurchaseHeader: Record "Purchase Header") Result: Boolean
    var
        IsHandled: Boolean;
    begin
        IsHandled := false;
        OnBeforePrePostApprovalCheckPurch(PurchaseHeader, Result, IsHandled);
        if IsHandled then
            exit(Result);

        if IsPurchaseHeaderPendingApproval(PurchaseHeader) then
            Error(PurchPrePostCheckErr, PurchaseHeader."Document Type", PurchaseHeader."No.");

        exit(true);
    end;

this modification will allow use customized check and post sales document
Thank you in advance.

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. Please do not reply to this, as we do not monitor closed issues. If you have follow-up questions or requests, please create a new issue where you reference this one.
Build ID: 44362.


read specs args from stdin pipe

Closing this. We should use a file to allow this instead. See conversation above.


[F4 - Auto Generated AI Art] Complete "AI Art Generation" feature

Description

Remove unused codes in aiCreator API.

Add fake data response as a comment in aiCreator API.

Tweak AICreator components in FE to support the followings:

display the generated AI image to the user
add loading progress bar when the API is processing with the request
show success/error message when making the AI art generating request
be able to download the image
be able to hit a 'create' button and start to create a new asset with the AI art image as the 'assetUrl' value
be able to select from different style and generate different style art work
enable the input field only when the user is logged in
enable the input field only when there's at least one NFT collection created by the user
be able to show image data

Trello

Link: https://trello.com/c/0ToqNr2r/58-f4-digital-image-auto-generated-by-ai-models-7

Extra Info

 F4 - Image meta data 2

 F4 - Input parameters 5

 F4 - Digital image auto generated by AI models 7


Option to skip n-directories creation when generating packages

Codecov Report

Merging #65 (5dc1075) into main (fc85260) will increase coverage by 0.04%.
The diff coverage is 100.00%.

:exclamation: Current head 5dc1075 differs from pull request most recent head 011cd35. Consider uploading reports for the commit 011cd35 to get more accurate results

Additional details and impacted files

@@            Coverage Diff             @@
##             main      #65      +/-   ##
==========================================
+ Coverage   93.59%   93.63%   +0.04%
==========================================
  Files          16       16
  Lines        2779     2797      +18
==========================================
+ Hits         2601     2619      +18
  Misses        133      133
  Partials       45       45

Impacted Files
Coverage Δ

pkg/internal/generator.go
100.00% <100.00%> (ø)

Codecov Report

Merging #65 (011cd35) into main (fc85260) will increase coverage by 0.04%.
The diff coverage is 100.00%.

Additional details and impacted files

@@            Coverage Diff             @@
##             main      #65      +/-   ##
==========================================
+ Coverage   93.59%   93.63%   +0.04%
==========================================
  Files          16       16
  Lines        2779     2797      +18
==========================================
+ Hits         2601     2619      +18
  Misses        133      133
  Partials       45       45

Impacted Files
Coverage Δ

pkg/internal/generator.go
100.00% <100.00%> (ø)

Codecov Report

Merging #65 (011cd35) into main (fc85260) will increase coverage by 0.04%.
The diff coverage is 100.00%.

:exclamation: Current head 011cd35 differs from pull request most recent head cf45468. Consider uploading reports for the commit cf45468 to get more accurate results

Additional details and impacted files

@@            Coverage Diff             @@
##             main      #65      +/-   ##
==========================================
+ Coverage   93.59%   93.63%   +0.04%
==========================================
  Files          16       16
  Lines        2779     2797      +18
==========================================
+ Hits         2601     2619      +18
  Misses        133      133
  Partials       45       45

Impacted Files
Coverage Δ

pkg/internal/generator.go
100.00% <100.00%> (ø)

Codecov Report

Merging #65 (cf45468) into main (fc85260) will increase coverage by 0.04%.
The diff coverage is 100.00%.

Additional details and impacted files

@@            Coverage Diff             @@
##             main      #65      +/-   ##
==========================================
+ Coverage   93.59%   93.63%   +0.04%
==========================================
  Files          16       16
  Lines        2779     2797      +18
==========================================
+ Hits         2601     2619      +18
  Misses        133      133
  Partials       45       45

Impacted Files
Coverage Δ

generator/generator.go
50.98% <ø> (ø)

pkg/internal/generator.go
100.00% <100.00%> (ø)

Fixes #66


Remove existing_layers, check from save_to_cache

Sorry, thought this submitted days ago

@laurieroy Yes, you had submitted it, but I cannot merge the PR until you update it with the changes mentioned. Once merged, your change will be part of the code base and you will be one of the contributors to the project! It would be awesome if you could edit your change so I can merge it.


Authorizer on included resources

Currently I have to write such code in some Schemas:
            'users' => [
                self::SHOW_SELF => true,
                self::SHOW_RELATED => true,
                self::SHOW_DATA => isset($includedRelationships['users']),
                self::DATA => function () use ($resource) {

                    if(!Auth::user()->can('view.users'))
                        throw new UnauthorizedException();

                    return $resource->users;
                },
                self::META => function () use ($resource) {
                    return [
                        'count' => $resource->users()->count(),
                    ];
                },
            ],

Is there any better way where the authorizers of the included resource can be used?

So we'd approach this via validation... i.e. only allow the user include path if a user is signed in. Basically this approach is a lot better than doing it in schemas, because by the time the serialization is occurring, all checking and logic should have been executed... because to do logic in the schemas is highly inefficient, particularly when serializing lots of models.
To implement this validation approach, overload the allowedIncludePaths() method on your resource's Validators class:
    protected function allowedIncludePaths(): AllowedIncludePaths
    {
        if (Auth::check()) {
                $this->allowedIncludePaths[] = 'user';
        }

        return parent::allowedIncludePaths();
    }

Closing this as I think the question is answered. Feel free to reopen if you need additional info.


Unsuccessful Generic Findings Import

At URL:
http://<IP_ADDRESS>:8000/engagement/1/import_scan_results
Attempts to import CSV file containing generic finding results in message:
Generic Findings Import processed, a total of 1 finding was processed
However, the CSV file contains the following entries:
2017-01-23,Outdated OS,1,ball.com,High,"A cross site scripting vulnerabilitiy exists within each of us. By opening our ports to the world, we can type in more test entries.",None known for the issue.,High,ball.com,True,True,False,False
2017-01-23,"Reflected XSS Test Entry",2,ball.com,High,5,None known for the issue.,High,ball.com,True,True,False,False
2017-01-23,CSRF,3,ball.com,Medium,"A cross site request forgery vulnerabilitiy exists within each of us. By opening our ports to the world, we can type in more test entries.",None known for the issue.,High,ball.com,True,True,False,False

I created a CSV file based on the content in the issue.  Unfortunately, it successfully imports in my development environment.
Does the 1 finding that was processed have information for the first record?  Did the CSV file have headers? Was the CSV file created on a Linux, Mac, or Windows environment?  Is it possible to attach a test file that you're having issues with?

@a1ec ^ thank you for looking into it @jpbowie!

@jpbowie Thanks for the speedy response.
The copy and paste in the original post is the actual file contents; there are no headers, that may be the issue. I'll note it was not apparent in the documentation that headers were required. My mistake but perhaps an example in the docs might help other users.
I've since looked at the generic import unit tests and successfully imported based on a passing test.


Change the jwt package

Take a look at https://jwt.io/
Whould be nice to change the jwt package to use github.com/SermoDigital/jose.

Can you elaborate as to why switch from current jwt-go?

On Jan 22, 2017, at 5:50 PM, Diego Bernardes<EMAIL_ADDRESS>wrote:
Take a look at https://jwt.io/
Whould be nice to change the jwt package to use github.com/SermoDigital/jose.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.

I'm sorry for the hype, was a unnecessary ticket.
It's my first time using jwt, was reading about it and when i got into the claims, realized that most of the claims are not implemented in current package, this was why i suggest the move, but they are all optional and isn't that hard to implement.
https://tools.ietf.org/html/rfc7519#section-4.1


[PSRE-3596] Check compatibility against ruby 3.2

PSRE-3596
Add test matrix ruby 3.2

@toptal-anvil ping reviewers


Change the banner text

Prerequisites:
Please answer the following questions for yourself before submitting an issue. YOU MAY DELETE THE PREREQUISITES SECTION.

[x] I checked to make sure that this issue has not already been filed.

Expected Behavior:
Meaningful text on the banner
Current Behavior:
Sample text
Solution:
Change the text on the banner to "Your Guide to learn and apply in-demand technologies, all under one HUT !!"

Please assign me this issue as a part of LGMSOC.

I would like to work on this

@shrutigrover25 assigned!

I would like to work on this

I am assigning issues on First Come First Serve basis!
You can find or create any other issue.


Added wordcamp udaipur

Details of the conference:

Website: https://2019.udaipur.wordcamp.org/
Date: 23 Nov
Venue: The Artist House Near, Ashoka Cinema Rd, (Max Store) Surajpole, Udaipur
Description: Primarily a Wordpress Conference, covering everything related to WordPress and include things such as blogging, coding, branding, and security.

Does the conference have a CFP? I can't find it on the website :thinking:

https://2019.udaipur.wordcamp.org/2019/07/31/call-for-speakers/


Inspector color picker bug when working with a list or array of Colors

Hi there,
This issue occurs when the alchemy plugin is installed. Is there a fix available? Are there any plans to update the plugin?
https://youtube.com/shorts/G2JfaZMnE-E?feature=share
Code:
[SerializeField] private Color[] sunColors;
My Unity version: 6000.0.17f1 Mac Silicon

We were using Unity 2022.3.29f and had a lot of problems with losing the content after editing (All collections + Color). Unity 2022.3.51f doesn't have this anymore. Probably fixed in later versions 6 too


fix reading decimal values

This patch addresses #4.

Thank you for your MR. Would you be so kind to add a sample and a spec for your fix?

@bigzed When I run spec I get the following error and I don't know how to resolve it:
Failure/Error: require 'mediainfo_native.so'

LoadError:
  cannot load such file -- mediainfo_native.so

You are missing the C-extension for the gem. You have to compile it to run the specs.
If you have libmediainfo-dev installed, it should be as easy as running bundle exec rake spec, it should compile it implicitly.
Run  bundle exec rake clean compile to just compile the extension and display any errors while trying to compile it.

Thank you for your help with compiling the C extension.
Conveniently, the project already has a CAF file, which I used in my spec to demonstrate the issue.

Thank you.


Where Lambda expression ignoring NULLs

Hi,
Our team is currently in the final stages of migrating a large solution from Entity Framework to NPoco and we stumbled upon some quirky behaviour:
If the Appointments table has the following rows

id
hidden

A
True

B
False

C
NULL

Query 1. [Entity Framework] var appointments = new DBEntities().Appointments.Where(c=> c.hidden != true).ToList();
Query 2. [NPoco] var appointments = new QueryProvider< Appointment>(db1).Where(c => c.hidden == null || c.hidden != true).ToList()
Query 3. [NPoco] var appointments = new QueryProvider< Appointment>(db1).Where(c => c.hidden != true),ToList();
Query 1 will return 2 rows;
Query 2 will return 2 rows;
Query 3 will return 1 row;
We were expecting all three queries to return 2 given null is also not equal to true. Is this by design?
Thanks, and keep up the great work!

NPoco directly translates LINQ to sql. It doesn't try and be all fancy like EntityFramework. Its actually scares me that this is the case. As can be seen below, only 1 result is return when running this against SQL Server.
select * from (
	select 1 as id, 1 as value
	union
	select 2 as id, 0 as value
	union
	select 3 as id, null as value
) a
where value != 1

Fair enough. Alright, thanks.


add currentSrc support detection (fixes #478)

While I think we can simply just remove currentSrc polyfill (Because it is broken and has no value), we can maintain backwards compatibility by simply doing an additional support check.

+1

Big question: Do we want to do a "fast" bugfix release (2.3.1) with this patch. While I don't think, that there are not so much spartan users, I really get the fact, that currentSrc (although not so useful) is part of the responsive image base and should be implemented with any partial native responsive images implementation. Therefore this bug was kind of foreseeable (see also this commit).

I think a 2.3.1 patch would be good. Not a lot of work for a lot of potential good.


Fix compatibility issue with Httparty >= 0.15.0

Backwards compatible with Httparty 0.10 which is the min version in the gemspec.

Could you please release a version with these changes?


Heat spells

Pull Request Checklist:

[ ] This PR addresses an already opened issue (for bug fixes / features)

This PR fixes #xyz

[x] Tests for the changes have been added (for bug fixes / features)

[x] (If applicable) Documentation has been added / updated (for bug fixes / features)

[x] CHANGELOG.rst has been updated (with summary of main changes)

[x] Link to issue (:issue:number) and pull request (:pull:number) has been added

What kind of change does this PR introduce?

3 new indicators heat_spell_frequency, heat_spell_max_length and heat_spell_total_length. The "heat_wave" indicators test that a bivariate condition on each day and then find consecutive runs of a minimum length. The new functions adhere to xclim's concept of "spell" : the condition is tested against a rolling N-day window and if it is fulfilled all days of the window are considered part of the spell.
New generic function spell_mask that encapsulate the spell notion. It returns a boolean mask of the days part of spells. The code has been added two features:

Multivariate : input data can be a list (and so must be the thresholds). The way we check for the conditions across variables is decided by new argument var_reducer ('all' or 'any').
Weights : if the spell stat is a mean, weights can be given (used by the INSPQ, but not for the actual PC indicators).

Indicator parameters can now be assigned new names. See examples in how the heat spell indicators are created. We could already do something similar with data variables. The internals of Indicator have changed accordingly (sorry).

Does this PR introduce a breaking change?
No.
Other information:
I kinda decided that "heat" meant "tasmax and tasmin".

coverage: 89.114% (+0.01%) from 89.1%
when pulling 3d276a07aad907ef6c7c8c1d57b643540ae7f593 on heat_spell
into 7acc410dc34f878ace5230b3e34ed4eee25dae03 on main.


NewClientWithJSONSlogger

Please review the below changes included with this PR,
-- Added NewClientWithJSONSlogger to print logs in JSON format.
-- Some changes in the Readme file for the above function.
-- An example under cmd/examples for the above function.

LGTM!


Choose from a list of keywords

Hello,
Thanks for this. It's working quite well, but is there a way to get it to choose only from a pre-defined list of keywords?

Hello,
Thanks for this. It's working quite well, but is there a way to get it to choose only from a pre-defined list of keywords?

Hello,  I would like to ask if you have any progress regarding the idea you said, I am also very interested and want to ask you for advice.
Looking forward to your reply, thanks a lot!!!


Fix CFStreamTests

Pass extra param to grpc_endpoint_read() as the API has changed.
Fixed build error seen with Xcode 10.
Enable pipefail to xcodebuild errors are propagated to the caller.

The changes in CFStreamTests.xcodeproj/project.pbxproj are to fix the following compilation error seen with Xcode 10.
error: Multiple commands produce '/Users/pjaikumar/Library/Developer/Xcode/DerivedData/CFStreamTests-gwdwuolxcdrsidefaoashkjzhrnr/Build/Products/Debug-iphonesimulator/CFStreamTests.xctest/Info.plist':

1) Target 'CFStreamTests' (project 'CFStreamTests') has copy command from '/Users/pjaikumar/grpc/test/core/iomgr/ios/CFStreamTests/Info.plist' to '/Users/pjaikumar/Library/Developer/Xcode/DerivedData/CFStreamTests-gwdwuolxcdrsidefaoashkjzhrnr/Build/Products/Debug-iphonesimulator/CFStreamTests.xctest/Info.plist'

2) Target 'CFStreamTests' (project 'CFStreamTests') has process command with output '/Users/pjaikumar/Library/Developer/Xcode/DerivedData/CFStreamTests-gwdwuolxcdrsidefaoashkjzhrnr/Build/Products/Debug-iphonesimulator/CFStreamTests.xctest/Info.plist'

The fix for this is to remove Info.plist from CFStreamTests -> Build phases > Copy Bundle Resources.
More in this Stackoverflow thread: https://stackoverflow.com/questions/50718018/xcode-10-error-multiple-commands-produce

Thanks @vjpai for the review!

cpp_linux_opt_native failure: #18424 is unrelated to this PR


Allow Alliterative Results

This PR would allow an option to request alliterative results.
Also fixes #6

@jdforsythe Thanks! Merged


Fixed: perf tests were failing to build

After merging #4550, perf tests were not building due to a changed method signature

/ok-to-perf

Thanks @ItalyPaleAle


Move the jumbotron default styles to be up with the @mobile rules to fix cascade issue affecting mobile display

I trust you on this change, but am curious, what was the cascade issue?

The @media query doesn't add specificity to the rules it contains. It just decides whether or not the selectors/styles are added to the cascade.
So, even if the query triggers and enables the image width, a later selector was overruling it. The two rules had equal specificity, so the one later in the file took precedence.

Generally, should we be putting @media queries at the end of our CSS?

Generally, when not using SASS, I extract media queries into their own stylesheet and load it last.


Small changes to spacing on fake bridge rules and rules section

Mainly to see if I can successfully create a pull request for this project. (Attempt 5 using a CLA email after push)

I signed it!?


New Slack channel

Hey @renou @julianpistorius @subokita @yarbelkgazia @rogoman @pouledodue @danielccyr @jellevandehaterd @leonh @lukaszb!
In an attempt to improve communications around this project (perhaps we could organise meetings about this project) there's a new Slack channel: https://eventsourcinginpython.slack.com
There's a sign up page which is available for 7 days. Please sign up here:
https://eventsourcinginpython.slack.com/join/shared_invite/MjA2ODA4MTI0OTYyLTE0OTg5ODM1MDMtZjUyNTI1ZWIyZg
Sorry this has taken so long!

Hey,
Could you resend this to my regular (non-work) github account: yarbelk
James
On Sun, Jul 2, 2017 at 4:55 PM, John Bywater<EMAIL_ADDRESS>wrote:

Hey @renou https://github.com/renou @julianpistorius
https://github.com/julianpistorius @subokita
https://github.com/subokita @yarbelkgazia
https://github.com/yarbelkgazia @rogoman https://github.com/rogoman
@pouledodue https://github.com/pouledodue @danielccyr
https://github.com/danielccyr @jellevandehaterd
https://github.com/jellevandehaterd @leonh https://github.com/leonh
@lukaszb https://github.com/lukaszb!
In an attempt to improve communications around this project (perhaps we
could organise meetings about this project) there's a new Slack channel:
https://eventsourcinginpython.slack.com
There's a sign up page which is available for 7 days. Please sign up here:
https://eventsourcinginpython.slack.com/join/shared_invite/
MjA2ODA4MTI0OTYyLTE0OTg5ODM1MDMtZjUyNTI1ZWIyZg
Sorry this has taken so long!
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/johnbywater/eventsourcing/issues/102, or mute the
thread
https://github.com/notifications/unsubscribe-auth/AYfLBvwltim3XCo22rdv6aea6qyYdmJqks5sJ1scgaJpZM4OLh4X
.

Hi @yarbelk, good to hear from you. I think the link above expired by now. But if you send an email to the author email in this file (below), so I get your email address, then I'll send you a Slack invite. Hope all's well. John
https://github.com/johnbywater/eventsourcing/blob/develop/setup.py

@yarbelk: new invitation link, valid for 7 days or something:
https://join.slack.com/eventsourcinginpython/shared_invite/MjEyMTMzODkxOTkwLTE0OTk4MDQ4MzctMDcyOTg5MjllYw

@julianpistorius here's a new invitation link, valid for 30 days: https://join.slack.com/t/eventsourcinginpython/shared_invite/MjI0NDEzODA5NzM0LTE1MDIxNzUwMjctNTNmYWMzZTI2Mg

Here's a new invitation link, valid for 30 days:
https://join.slack.com/t/eventsourcinginpython/shared_invite/enQtMjQyMTA5Njc5MTg1LWM0N2M3OTIzYmNmMGQ4NDk2ZWU3NTI5N2YwMjQ0MDk0MWQ4MGJhYzQ4MjM1ZWNkNGZkMmE5OWU2ZDhhN2ZkMjI

Here's a new invitation link, valid forever:
https://join.slack.com/t/eventsourcinginpython/shared_invite/enQtMjczNTc2MzcxNDI0LTUwZGQ4MDk0ZDJmZmU0MjM4MjdmOTBlZGI0ZTY4NWIxMGFkZTcwNmUxM2U4NGM3YjY5MTVmZTBiYzljZjI3ZTE


Simple way to provide URL query parameters for GET requests as in for POST request

@authorjapps Excited about this awesome project, I've ran into a scenario where it require to pass number of query parameters in url. Can't we have the support for GET requests as in this POST example. Please ignore if there is existing support, I couldn't find and unable to get my trial and error working.
sample json:
{
  "scenarioName": "A scenario which has huge number of url query parameters",
  "steps": [
    {
      "name": "get_user_details",
      "url": "/example",
      "operation": "GET",
      "request": {
        "queryParams": {
          "queryParam1": "...",
          "queryParam2": "...",
          "queryParam3": "...",
          "queryParam4": "...",
          "queryParam5": "..",
          "more": "..."
        }
      },
      "assertions": {
        "status": 200,
        "body": {
          "...": "...",
        }
      }
    }
  ]
}

otherwise url will be like "url":"/examplequeryParam1=...&queryParam2=...&queryParam3=...&queryParam4=...&queryParam5=..."
which can grow more and become harder to grasp. Can't we have the support for this kind of scenario? Once get more familiar with the code I'd love to contribute. Thanks.

@ruwanka That's a nice feature. Thanks for raising this. This was already supported in the earlier versions. Yes, you can kindly contribute and fix this, if this is missed in the recent versions.

You can please raise a PR by forking the repo, please read the CONTRIBUTING.md.
The code is pretty easy to change and test, once you become familiar.
Please go step by step. Please start with a unit test, then add an integration test, then a sample working code into the hello-word repo and finally a reference into the README file.

Initially I can help you to locate the code where this needs change/amendment(if you need).

@ruwanka , Can you please look at the queryParams code at the branch ISS-99_query_params and add some more tests to

org.jsmart.zerocode.core.httpclient.utils.UrlQueryParamsUtilsTest
org.jsmart.zerocode.core.verification.queryparams.QueryParamVerifyTest

when you get chance please. Test scenarios could be

If no "queryParams" are present
and You can think of any other scenarios

This will give some ideas into the framework in terms of how the query params are handled.

Version 1.2.5 has this feature now.


PS1 and PS2 games unable to import from Retroachievements

With the latest readding of Retroachievement features it is unable to import achievements for PS1 and PS2 games, I have tried everything from readding, manually triggering and automatically triggering the plugin to no avail.

SOLUTION
This will take a while for large libraries I have a total of 521 games between PS1 and PS2 and it took me anywhere from 30 minutes to an hour

Filter your library to the platforms that aren't working, in my case Playstation 1 and 2
Press Ctrl + A to select all of your games
Right click on any game and locate SuccessStory in the context menu
Click refresh data

Be sure that when you view your SuccessStory settings they look like this (except next to the platform name on the left there should also be a name on the right I need to change that color in my settings because my text is black there):

The left side is the name of the platform on RetroAchievements and the right side in the name in Playnite
The new API key location in in your profile settings on RA if you scroll down, it is the Web API Key
If this still doesn't work try manually triggering the refresh on an individual game that you know contains achievements


Scheduler changes to set a start month for year views

As per forum query - https://forum.radzen.com/t/scheduler-yearplanner-not-solar-year/16368
Also, in YearPlanner and YearTimeline, the slots that are at the beginning and end of each row are there for aesthetics only, to fill out the rows to the edges. They are not "real" slots. Although we remove the Click event from these slots, they were still reactive to style modifications in the SlotRender event. This has also been resolved.

Hi @paulo-rico,
This property should not be at RadzenScheduler level as it doesn't work for the other views. Move it to the views instead.

I suggest another rename - the enum from ScheduleStartMonth to just Month and the property from ScheduleStartMonth to just StartMonth.


Do not create a block frame for out-of-flow items only

Fixes #2297
This PR makes it so that, when a breakable block is broken into multiple frames, a frame containing only out-of-flow items (which correspond to place elements, counter updates, etc.) will move its items to the next frame if there is one.
Please review this carefully! I was not familiar with this part of the codebase when I started working on that, so please make sure I did not write anything stupid. All tests pass, so it should be ok, but there are still some things I'm not fully confident about (see comments). In particular, because this fix is implemented by modifying the way FlowElem is laid out, I think it may affect other things (although I could not think of any).

Just removing the last frame seems like a strange thing to do. It probably happens to work because the next frame is too large to fit into the first region and the parent flow handles it, but generally a multi-region layouter should produce one frame per region.
One thing that might work is to mem::take the only-out-of-flow items at the start of finish_region to effectively create a completely empty frame and then, once that's done readd them to the current items. This way, you also wouldn't need a new field in the layouter.

Oh god, what did I just do with the commit history!?

I'm not sure how to do that, because at at the start of finish_region, we no longer know whether the previous frame contained only out-of-flow elements (we would still need a boolean field, which I agree is better than cloning the elements).

What I meant is that rather than writing and then removing the frame with only out-of-flow elements, just don't create the frame at all, effectively deferring writing of the only out-of-flow elements. (Perhaps with the exception of the final finish_region call where we need to ensure that they are written.)

Turns out setting the frame size to self.initial instead of Size::zero() solved the issue of the last test not passing. Now all tests pass.
I will try to resolve the conflicts and solve the issue that frames with size zero are not necessarily out-of-flow, and then this will be ready to merge. In the meantime, I'm switching this PR to a draft.

I think this should now be safe to merge!

Thank you, also for the patience!


html file not found error in exe

i made an application using selenium , eel and other modules. i compiled it into an exe file using pyinstaller. however when i run the exe file it opens the chrome window but says the file isnt found. when i run the program on my idle , it is able to locate the file. please help me.

Can you provide a visualization the folder structure?


Missing bytes when calling ChannelBuffer.readableBytes()

After a few months of using netty without any issues i have came a across something very strange.
I am using a NioDatagramChannelFactory and on my pipeline i insert a decoder which extends OneToOneDecoder and i do something like :
    @Override
    protected Object decode( ChannelHandlerContext context, Channel channel, Object msg ) throws Exception
    {

            ChannelBuffer msgBuffer = ( ChannelBuffer ) msg;

            // Get the number of bytes for this message
            int length = msgBuffer.readableBytes();

I have my netty server bind to <IP_ADDRESS>/7474 and i then fire up a bunch of packets with :
while true
do
   nc -w1 -u  <IP_ADDRESS> 7474  <  data_file
done

It seems that aprox 4% of the packets return a smaller value of length, which i do not understand why.
Even stranger is that I can see this behavior in 4 different machines that i have this software running, but if i run it the exact same code on any other hardware i seem to be unable to reproduce this problem.

Just a guess but could it be that the rest of the bytes will be passed to the decode(..) method with a new buffer ? I think you need to extends FrameDecoder and make sure that you only pass the ChannelBuffer to the next handler in the pipeline if you received all bytes.

I though FrameDecoder was only applicable to TCP. This is UDP. Or am i wrong?


Table cell border line color is not same

What did you do?
Hi, I want to build a table pdf.
What did you expect to happen?
I expect the cell bottom line color is same black.
What happened instead?
But, the cell bottom line color is not same. Some lines are light gray. Some lines are black.

TPPDF Environment
**TPPDF version: 2.3.5
**Xcode version: 12.5
**Swift version: 5.4
Demo Code / Project
    table.style.outline = .none
    let tableCellStyle = PDFTableCellStyle(colors: (UIColor.clear, UIColor.black), borders: PDFTableCellBorders.init(left: .none, top: .none, right: .none, bottom: .init()), font: UIFont.boldSystemFont(ofSize: 12))
    table.style.rowHeaderStyle = tableCellStyle
    table.style.columnHeaderStyle = tableCellStyle
    table.style.contentStyle = tableCellStyle
    table.style.footerStyle = tableCellStyle
    table.style.alternatingContentStyle = tableCellStyle

thank you for reporting this bug.
I assume this issue is caused by <1.0 pixel line widths.
Can you please add the generated PDF too?

Closed due to inactivity


Secure boot check fail  / enabled secure boot v1 without flash encryption  (IDFGH-8159)

Answers checklist.

[X] I have read the documentation ESP-IDF Programming Guide and the issue is not addressed there.
[x] I have updated my IDF branch (master or release) to the latest version and checked that the issue is present there.
[X] I have searched the issue tracker for a similar issue and not found a similar issue.

IDF version.
esp-idf -release-v4.2 esp32-version
Operating System used.
Windows
How did you build your project?
Command line with idf.py
If you are using Windows, please specify command line type.
PowerShell
Development Kit.
ESP32-WROOM-32D
Power Supply used.
USB
What is the expected behavior?
I expected to flash the code in secure boot enabled chip using same secure boot key.
What is the actual behavior?
PS C:\Users\LENOVO\OneDrive\Desktop\abhith\template-app> idf.py flash
Executing action: flash
Serial port COM5
Connecting....
Detecting chip type... Unsupported detection protocol, switching and trying again...
Connecting....
Detecting chip type... ESP32
Running ninja in directory c:\users\lenovo\onedrive\desktop\abhith\template-app\build
Executing "ninja flash"...
[1/5] cmd.exe /C "cd /D C:\Users\LENOVO\OneDrive\Desktop\abhith\templ...s/LENOVO/OneDrive/Desktop/abhith/template-app/build/template-app.bin"
template-app.bin binary size 0x2fff4 bytes. Smallest app partition is 0x100000 bytes. 0xd000c bytes (81%) free.
[2/5] Performing build step for 'bootloader'
[1/1] cmd.exe /C "cd /D C:\Users\LENOVO\OneDrive\Desktop\abhith\template-app\build\bootloader\esp-idf\esptool_py && C:\Users\LENOVO.espressif\python_env\idf4.4_py3.8_env\Scripts\python.exe C:/Users/LENOVO/esp/esp-idf/components/partition_table/check_sizes.py --offset 0x10000 bootloader 0x1000 C:/Users/LENOVO/OneDrive/Desktop/abhith/template-app/build/bootloader/bootloader.bin"
Bootloader binary size 0xab90 bytes. 0x4470 bytes (29%) free.
[2/3] cmd.exe /C "cd /D C:\Users\LENOVO\esp\esp-idf\components\esptoo...Users/LENOVO/esp/esp-idf/components/esptool_py/run_serial_tool.cmake"
esptool.py esp32 -p COM5 -b 460800 --before=default_reset --after=no_reset write_flash --flash_mode dio --flash_freq 40m --flash_size 2MB 0x20000 template-app.bin 0x10000 partition_table/partition-table.bin
esptool.py v3.3.2-dev
Serial port COM5
Connecting.....
Chip is ESP32-D0WDQ6 (revision 1)
Features: WiFi, BT, Dual Core, 240MHz, VRef calibration in efuse, Coding Scheme None
Crystal is 40MHz
MAC: 0c:b8:15:c4:49:90
Uploading stub...
Running stub...
Stub running...
Changing baud rate to 460800
Changed.
Configuring flash size...
Flash will be erased from 0x00020000 to 0x0004ffff...
Flash will be erased from 0x00010000 to 0x00010fff...
Compressed 196596 bytes to 92754...
Writing at 0x00020000... (16 %)
Writing at 0x00030f63... (50 %)
Writing at 0x00036758... (66 %)
Writing at 0x0003ed7e... (83 %)
Writing at 0x00046f82... (100 %)
Wrote 196596 bytes (92754 compressed) at 0x00020000 in 2.4 seconds (effective 657.8 kbit/s)...
Hash of data verified.
Compressed 3140 bytes to 175...
Writing at 0x00010000... (100 %)
Wrote 3140 bytes (175 compressed) at 0x00010000 in 0.1 seconds (effective 322.3 kbit/s)...
Hash of data verified.
Leaving...
Staying in bootloader.
Done
PS C:\Users\LENOVO\OneDrive\Desktop\abhith\template-app> C:\Users\LENOVO.espressif\python_env\idf4.4_py3.8_env\Scripts\python.exe C:\Users\LENOVO\esp\esp-idf\tools\idf.py -p COM5 monitor
Executing action: monitor
Running idf_monitor in directory c:\users\lenovo\onedrive\desktop\abhith\template-app
Executing "C:\Users\LENOVO.espressif\python_env\idf4.4_py3.8_env\Scripts\python.exe C:\Users\LENOVO\esp\esp-idf\tools/idf_monitor.py -p COM5
-b 115200 --toolchain-prefix xtensa-esp32-elf- --target esp32 --revision 1 c:\users\lenovo\onedrive\desktop\abhith\template-app\build\template-app.elf -m 'C:\Users\LENOVO.espressif\python_env\idf4.4_py3.8_env\Scripts\python.exe' 'C:\Users\LENOVO\esp\esp-idf\tools\idf.py' '-p' 'COM5'"...
--- WARNING: GDB cannot open serial ports accessed as COMx
--- Using \.\COM5 instead...
--- idf_monitor on \.\COM5 115200 ---
--- Quit: Ctrl+] | Menu: Ctrl+T | Help: Ctrl+T followed by Ctrl+H ---
ets Jun  8 2016 00:22:57
rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3fff0030,len:6664
load:0x40078000,len:14848
load:0x40080400,len:3792
0x40080400: _init at ??:?
secure boot check fail
ets_main.c 371
ets Jun  8 2016 00:22:57
rst:0x10 (RTCWDT_RTC_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3fff0030,len:6664
load:0x40078000,len:14848
load:0x40080400,len:3792
0x40080400: _init at ??:?
secure boot check fail
ets_main.c 371

(REPEATING )

Steps to reproduce.
Hi,
The secure boot check is fail. I followed this documents ESP-IDF PROGRAMMING GUIDE for enabling secure boot v1 (one time flash) without flash encryption.
STEPS FOLLOWED

Step 1: Project Configuration Menu --> (a) changed revision 0 to 1( I am using ESP32-WROOM-32D it is revision  1)

(b) Security features --> Enable hardware Secure Boot in bootloader-->Enable secure boot version1
(c) Secure bootloader mode --> One-Time flash
(d) Partion Table --> Single factory app, no OTA -->Offset of partition table (0x10000)
All other configurations are in default mode
saved and exit from the config menu

Step 2:  Build command using  idf.py build
an error message showed as-  ninja: error: '../secure_boot_signing_key.pem', needed by 'signature_verification_key.bin', missing and no known rule to make it

Step 3: Generated secure boot key using command :  espsecure  generate_signing_key --version 1 secure_boot_signing_key.pem
key generated succesfully.

Step 4:  run command - idf.py bootloader
Terminal output :
Bootloader built. Secure boot enabled, so bootloader not flashed automatically.
One-time flash command is:
C:/Users/LENOVO/.espressif/python_env/idf4.4_py3.8_env/Scripts/python.exe  C:/Users/LENOVO/esp/esp-idf/components/esptool_py/esptool/esptool.py --chip esp32 --port=(PORT) --baud=(BAUD) --before=default_reset --after=no_reset write_flash --flash_mode dio --flash_freq 40m --flash_size 2MB 0x1000 C:/Users/LENOVO/OneDrive/Desktop/esp-idf-v4.4.2/examples/get-started/hello_world/build/bootloader/bootloader.bin

Step 5: Using above command, flashed bootloader in chip.

Step 6:  Flashed the code using - idf.py flash

Debug Logs.
Project build complete. To flash, run this command:
C:\Users\LENOVO\.espressif\python_env\idf4.4_py3.8_env\Scripts\python.exe ..\..\..\..\esp esp-idf\components esptool_py esptool esptool.py -p (PORT) -b 460880
--before default_reset --after no_reset --chip esp32 write_flash --flash_mode dio --flash_size detect --flash_freq 40m @x10000 build partiti
on_table\partiti on-table.bin 8x20000 build template-app.bin
PS C:\Users\LENOVO\OneDrive\Desktop\abhith\template-app> C:/Users/LENOVO/.espressif/python_env/idf4.4_py3.8_env/Scripts/python.exe C:/Users/LENOVO/esp/esp-id
f/components/esptool_py/esptool/esptool.py --chip esp32 --port=COMS --baud=115200 --before=default_reset --after=no_reset write_flash --flash_mode dio --flash
_freq 40m --flash_size 2MB @x1000 C:/Users/LENOVO/OneDrive/Desktop/abhith/template-app/build/bootloader/bootloader.bin
esptool.py V3.3.2-dev
Serial port COMS
Connecting....
Chip is ESP32-DOWDQ6 (revision 1)
Features: WiFi, BT, Dual Core, 240MHz, VRef calibration in efuse, Coding Scheme None
Crystal is 40MHz
MAC: Oc:b8:15:04:49:90 Uploading stub...
Running stub...
Stub running...
Configuring flash size...
Flash will be erased from @x60001000 to 8x8000afff...
Compressed 39248 bytes to 24710...
Wrote 39248 bytes (24710 compressed) at Ox00001000 in 2.8 seconds (effective 114.0 kbit/s)...
Hash of data verified.

Leaving...
Staying in bootloader.
PS C:\Users\LENOVO\OneDrive\Desktop\abhith\template-app> C:\Users\LENOVO\.espressif\python_env\idf4.4_py3.8_env\Scripts\python.exe ..\..\..\..\esp esp-idf.com ponents\esptool_pylesptool\esptool.py -P COMS -b 460800 --before default_reset --after no_reset --chip esp32 write_flash --flash_mode dio --flash_size detect
--flash_freq 40m 0x10000 build\partition_table partition-table.bin x20000 build\template-app.bin esptool.py V3.3.2-dev Serial port COMS Connecting........ Chip is ESP32-DOWDQ6 (revision 1) Features: WiFi, BT, Dual Core, 240MHz, VRef calibration in efuse, Coding Scheme None Crystal is 48MHz MAC: c:b8:15:04:49:90 Uploading stub... Running stub... Stub running... Changing baud rate to 460880 Changed. Configuring flash size... Auto-detected Flash size: 4MB Flash will be erased from Ox00010000 to Ox00010fff... Flash will be erased from Ox00020000 to 8x0084ffff... Compressed 3140 bytes to 175... Wrote 3140 bytes (175 compressed) at Ox00010080 in 0.0 seconds (effective 523.7 kbit/s)... Hash of data verified. Compressed 196596 bytes to 92752... Wrote 196596 bytes (92752 compressed) at Ox00020000 in 2.4 seconds (effective 647.3 kbit/s)...
Hash of data verified.

Leaving...
Staying in bootloader.
PS C:\Users\LENOVO OneDrive\Desktop\abhith\template-app> C:\Users\LENOVO\.espressif\python_env\idf4.4_py3.8_env\Scripts\python.exe C:\Users\LENOVO esplesp-idf \tools\idf.py -p COMS monitor
Executing action: monitor
Running idf_monitor in directory c:\users\lenovo\onedrive\desktop\abhith\template-app
Executing "C:\Users\LENOVO\.espressif\python_env\idf4.4_py3.8_env\Scripts\python.exe C:\Users\LENOVO\esp\esp-idf\tools/idf_monitor.py -P COMS - 115200 --tool
chain-prefix xtensa-esp32-elf- --target esp32 --revision 1 c:\users\lenovo \onedrive desktop abhith\template-app build\template-app.elf -m 'C:\Users\LENOVO\.es
pressif\python_env\idf4.4_py3.8_env\Scripts\python.exe' 'C:\Users\LENOVO esp esp-idf tools\idf.py' '-p' 'COMS""...
--- WARNING: GDB cannot open serial ports accessed as COMX
--- Using \\. \COMS instead... --- idf_monitor on \\. \COMS 115200 ---
--- Quit: Ctrl+] | Menu: Ctrl+T | Help: Ctrl+T followed by Ctrl+H ---
ets Jun 8 2016 00:22:57
rst: 8x1 (POWERON_RESET), boot:x13 (SPI_FAST_FLASH_BOOT)
configsip: , SPIWP: Oxee
clk_drv: 0x00,q_drv: 0x00,d_drv: 0x00,cs0 drv: 0x00, hd drv: 0x00,wp_drv: Ox00
mode:DIO, clock div:2
load: Ox3fffoeb8, len: 10768
load: Ox40078000, len: 24604
load: 0x40080400, len:3792
@x40088400: _init at ??:?

entry @x48080694
I (29) boot: ESP-IDF GIT-NOTFOUND 2nd stage bootloader
I (29) boot: compile time 03:56:45
I (29) boot: chip revision: 1
I (33) boot.esp32: SPI Speed : 48MHz
I (37) boot.esp32: SPI Mode : DIO
I (42) boot.esp32: SPI Flash Size : 2MB
I (46) boot: Enabling RNG early entropy source...
I (52) boot: Partition Table:
I (55) boot: ## Label          Usage         Type ST Offset Length
I (63) boot: nvs              WiFi data      01 02 00011000 00006000
I (70) boot: 1 phy_init       RF data        01 01 00017000 60001000
I (78) boot: 2 factory        factory app    00 00 88020000 60100000
I (85) boot: End of partition table
I (89) esp_image: segment @: paddr=88020020 vaddr=3f408020 size=88298h ( 33424) map
I (110) esp_image: segment 1: paddr=880282b8 vaddr=3ffb0000 size=82368h (9064) load
I (114) esp_image: segment 2: paddr=8802a628 vaddr=40080000 size=859f6h ( 23024) load
I (126) esp_image: segment 3: paddr=88030020 vaddr-400d0020 size=15558h ( 87384) map
I (158) esp_image: segment 4: paddr=00045580 vaddr=400859f0 size=6588ch ( 22668) load
I (168) esp_image: segment 5: paddr=0004ae14 vaddr=50000000 size=60010h 16) load
I (168) esp_image: segment 6: paddr=0004ae2c vaddr=00000000 size-05154h ( 20820)
I (182) esp_image: Verifying image signature
I (521) boot: Loaded app from partition at offset @x20000
I (521) esp_image: segment 6: paddr=88001020 vaddr=3fff0b8 size=82a10h ( 10768)
I (521) esp_image: segment &: paddr=00007028 vaddr=37978003 size=oza16h ( 28508)
I (526) esp_image: segment 1: paddr=88003a38 vaddr=40078000 size=8601ch ( 24684)
I (537) esp_image: segment 2: paddr=88009a5c vaddr=40088400 size=b0edoh (3792)
I (541) secure_boot_v1: Generating new secure boot key...
I (547) efuse: BURN BLOCK2
I (561) efuse: BURN BLOCK2 - OK (write block == read block)
I (561) secure_boot_v1: Generating secure boot digest...
I (602) secure_boot_v1: Digest generation complete.
I (602) boot: Checking secure boot....
I (602) efuse: Batch mode of writing fields is enabled
I (606) secure_boot_v1: blowing secure boot efuse...
I (612) secure_boot: Read & write protecting new key...
I (618) secure_boot: Disable JTAG...
I (622) secure_boot: Disable ROM BASIC interpreter fallback...
I (629) efuse: BURN BLOCKO
I (643) efuse: BURN BLOCKO - OK (all write block bits are set)
I (643) efuse: Batch mode. Prepared fields are committed
I (647) secure_boot_v1: secure boot is now enabled for bootloader image
I (654) boot: Disabling RNG early entropy source...
I (671) cpu start: Pro cpu up.
I (671) cpu_start: Starting app cpu, entry point is @x40080ffo
8x48080ffo: call_start_cpul at C:/Users/LENOVO/esp/esp-idf/components/esp_system/port/cpu_start.c:160

I (0) cpu_start: App cpu up.
I (685) cpu_start: Pro cpu start user code
I (685) cpu_start: cpu freq: 160000000
I (685) cpu_start: Application information:
I (690) cpu_start: Project name: template-app
I (695) cpu_start: App version: 1
I (699) cpu_start: Compile time: Aug 23 2022 03:56:41
I (705) cpu_start: ELF file SHA256: 12fa547789889536...
I (711) cpu_start: ESP-IDF: GIT-NOTFOUND
I (717) heap_init: Initializing. RAM available for dynamic allocation:
I (724) heap_init: At 3FFAE6E0 len 30001920 (6 KiB): DRAM
I (730) heap_init: At 3FFB2CDO len 8002D330 (180 KiB): DRAM
I (736) heap_init: At 3FFE0440 len 30003AED (14 KiB): D/IRAM
I (743) heap_init: At 3FFE4350 len 8001BCBO (111 KiB): D/IRAM
I (749) heap_init: At 4008B27C len 80014D84 (83 KiB): IRAM
I (756) spi_flash: detected chip: generic
I (760) spi_flash: flash io: dio
W (764) spi_flash: Detected size(4096k) larger than the size in the binary image header (2048k). Using the size in the binary image header.
I (778) cpu_start: Starting scheduler on PRO CPU.
I (0) cpu_start: Starting scheduler on APP CPU.
Hello world!
This is esp32 chip with 2 CPU core(s), WiFi/BT/BLE, silicon revision 1, 2MB external flash Minimum free heap size: 295072 bytes
Restarting in 10 seconds...
Restarting in 9 seconds...
Restarting in 8 seconds...
Restarting in 7 seconds...
Restarting in 6 seconds...
Restarting in 5 seconds...
Restarting in 4 seconds...
Restarting in 3 seconds...
Restarting in 3 seconds...
Restarting in 2 seconds...
Restarting in 1 seconds...
Restarting in 6 seconds...
Restarting now.
ets Jun 8 2016 00:22:57

rst:Oxc (SW_CPU_RESET), boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP: Oxee
clk_drv: 0x00,q_drv: 0x0,d_drv: Oxe, cs@_drv: 0x00, hd_drv: 0x00, wp_drv: 0x00
mode:DIO, clock div:2
load: Ox3fff0eb8, len: 10768
load: 0x40078000, len:24604
load: 0x48080400, len:3792
0x40088400: init at ??:?

entry @x40080694
I (29) boot: ESP-IDF GIT-NOTFOUND 2nd stage bootloader
I (29) boot: compile time 03:56:45
I (29) boot: chip revision: 1
I (33) boot.esp32: SPI Speed : 48MHz
I (37) boot.esp32: SPI Mode : DIO
I (42) boot.esp32: SPI Flash Size : 2MB
I (46) boot: Enabling RNG early entropy source...
I (52) boot: Partition Table:
I (55) boot: ## Label          Usage         Type ST Offset Length
I (63) boot: nvs              WiFi data      01 02 00011000 00006000
I (70) boot: 1 phy_init       RF data        01 01 00017000 60001000
I (78) boot: 2 factory        factory app    00 00 88020000 60100000
I (85) boot: End of partition table
I (89) esp_image: segment @: paddr=88020020 vaddr=3f408020 size=88298h ( 33424) map
I (110) esp_image: segment 1: paddr=880282b8 vaddr=3ffb0000 size=82368h (9064) load
I (114) esp_image: segment 2: paddr=8802a628 vaddr=40080000 size=859f6h ( 23024) load
I (126) esp_image: segment 3: paddr=88030020 vaddr-400d0020 size=15558h ( 87384) map
I (158) esp_image: segment 4: paddr=00045580 vaddr=400859f0 size=6588ch ( 22668) load
I (168) esp_image: segment 5: paddr=0004ae14 vaddr=50000000 size=60010h 16) load
I (168) esp_image: segment 6: paddr=0004ae2c vaddr=00000000 size-05154h ( 20820)
I (182) esp_image: Verifying image signature
I (521) boot: Loaded app from partition at offset @x20000

I (528) secure_boot_v1: bootloader secure boot is already enabled. No need to generate digest. continu
I (533) boot: Checking secure boot...
I (538) secure_boot_v1: bootloader secure boot is already enabled, continuing..
I (546) boot: Disabling RNG early entropy source...
I (562) cpu_start: Pro cpu up.
I (563) cpu_start: Starting app cpu, entry point is 0x40080ffo
0x40080ffo: call_start_cpul at C:/Users/LENOVO/esp/esp-idf/components/esp_system/port/cpu_start.c:160

I (545) cpu_start: App cpu up.
I (543) cpu_start: App cpu apont -- endo
I (577) cpu_start: Pro cpu start user code
I (577) cpu_start: cpu freq: 160000000
I (577) cpu_start: Application information:
I (581) cpu_start: Project name: template-app
I (587) cpu_start: App version: 1
I (591) cpu_start: Compile time: Aug 23 2022 03:56:41
I (597) cpu_start: ELF file SHA256: 12fa547789889536...
I (603) cpu_start: ESP-IDF:                GIT-NOTFOUND
I (609) heap_init: Initializing. RAM available for dynamic allocation:
I (616) heap_init: At 3FFAE6E0 len 80001920 (6 KiB): DRAM
I (622) heap_init: At 3FFB2CDO len 8802D330 (180 KiB): DRAM
I (628) heap_init: At 3FFE6440 len 60003AED (14 KiB): D/IRAM
I (635) heap_init: At 3FFE4350 len 8001BCBO (111 KiB): D/IRAM
I (641) heap_init: At 4008B27C len 80014084 (83 KiB): IRAM
I (648) spi_flash: detected chip: generic
I (652) spi_flash: flash io: dio
W (656) spi_flash: Detected size(4096k) larger than the size in the binary image header (2048k). Using
I (670) cpu_start: Starting scheduler on PRO CPU.
I (0) cpu_start: Starting scheduler on APP CPU.
Hello world!
This is esp32 chip with 2 CPU core(s), WiFi/BT/BLE, silicon revision 1, 2MB external flash Minimum free heap size: 295072 bytes
Restarting in 10 seconds...
Restarting in 9 seconds...
Restarting in 8 seconds...
Restarting in 7 seconds...
Restarting in 6 seconds...
Restarting in 5 seconds...
ets Jun 8 2016 00:22:57

More Information.
I try to flash the chip again  using this command : PS C:\Users\LENOVO\OneDrive\Desktop\abhith\template-app> C:\Users\LENOVO.espressif\python_env\idf4.4_py3.8_env\Scripts\python.exe ........\esp\esp-idf\components\esptool_py\esptool\esptool.py -p COM5 -b 460800 --before default_reset --after no_reset --chip esp32  write_flash --flash_mode dio --flash_size detect --flash_freq 40m 0x10000 build\partition_table\partition-table.bin 0x20000 build\template-app.bin
but it shows an error as:
ets Jun  8 2016 00:22:57
rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3fff0030,len:6664
load:0x40078000,len:14848
load:0x40080400,len:3792
0x40080400: _init at ??:?
secure boot check fail
ets_main.c 371
ets Jun  8 2016 00:22:57
rst:0x10 (RTCWDT_RTC_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3fff0030,len:6664
load:0x40078000,len:14848
load:0x40080400,len:3792
0x40080400: _init at ??:?
secure boot check fail
ets_main.c 371
repeating
EFUSE SUMMARY

Can anyone help me with this? What went wrong.

rst:0x10 (RTCWDT_RTC_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3fff0030,len:6664
load:0x40078000,len:14848
load:0x40080400,len:3792
0x40080400: _init at ??:?

rst:Oxc (SW_CPU_RESET), boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP: Oxee
clk_drv: 0x00,q_drv: 0x0,d_drv: Oxe, cs@_drv: 0x00, hd_drv: 0x00, wp_drv: 0x00
mode:DIO, clock div:2
load: Ox3fff0eb8, len: 10768
load: 0x40078000, len:24604
load: 0x48080400, len:3792
0x40088400: init at ??:?

You must have accidentally reflashed the bootloader because the logs show two different bootloader sizes

Thank you for the detailed and sudden helpful response @negativekelvin .
I think I made a mistake by tried to confim if the secure boot enabled properly by flashing another code without using secure key.
If you don't mind ,Some doubts which I want to clear.

Is it possible to reflash the bootloader after enabling secureboot ?
Below  commad is shown after build

One-time flash command is:
        C:/Users/LENOVO/.espressif/python_env/idf4.4_py3.8_env/Scripts/python.exe  C:/Users/LENOVO/esp/esp-idf/components/esptool_py/esptool/esptool.py --chip esp32 --port=(PORT) --baud=(BAUD) --before=default_reset --after=no_reset write_flash --flash_mode dio --flash_freq 40m --flash_size 2MB 0x1000 C:/Users/LENOVO/OneDrive/Desktop/abhith/template-app/build/bootloader/bootloader.bin
* IMPORTANT: After first boot, BOOTLOADER CANNOT BE RE-FLASHED on same device*

As the steps mentioned above(from the documment), I did not burn the key manually. Is it really necessary to burn the key with burn key command? If it is necessary to burn key manually then it should be after build project,right?
What is the main diffrence between One-time flash and reflashable in secure boot v1.
After enabling secure boot can we flash any code with signing with our private key?

Hi @ABHITHLALC,
I can help clear your doubts,

It is possible to reflash the bootloader but for that you need to select the "Reflashable" option for "Secure bootloader mode". I would suggest you to go through the Re-Flashable Software Bootloader section from the documentation. It has all the required details
If you decide to use the Reflashable option then you need to burn the key manually after the build
In one-time flash, you won't be able to flash any other bootloader as the secure bootloader key is generated on the device. Whereas, in reflashable mode, you provide the secure bootloader key and hence you can flash any bootloader as long as you generate the digest using the same secure bootloader key. All these details are provided in the documentation linked above.
Yes, you can flash any application as long as you sign it with the same secure boot signing key.

Hello @sachin0x18
Thanks for the reply and thank you for the information.

In one-time flash, you won't be able to flash any other bootloader as the secure bootloader key is generated on the device. Whereas, in reflashable mode, you provide the secure bootloader key and hence you can flash any bootloader as long as you generate the digest using the same secure bootloader key. All these details are provided in the documentation linked above.

In this case I used One-time flash mode. Can you please tell me which step went wrong.

Like negativekelvin pointed out, did you reflash some other bootloader ?

I think may be I did. But if I try to reflash the other bootloader on a system with secure boot enabled, will it damage the chip? Is there any option to recover this module?

If you flash any other bootloader then it simply won't boot up because it doesn't have the same digest. If you still have the same bootloader that you flashed when enabling secure boot then you can flash it again and the device should boot fine

Which files should be kept safe after enabling secure boot?

Only the signing key should be kept safe. You need to ensure that you don't erase/reflash the bootloader.

Thank you for all the informations.  It's really helpful. I will it out on new chip.

You were right about bootloader @sachin0x18 @negativekelvin , I accidentally flashed another bootloader. Now I have the old bootloader file that flashed earlier. And I reflashed it as you said

If you flash any other bootloader then it simply won't boot up because it doesn't have the same digest. If you still have the same bootloader that you flashed when enabling secure boot then you can flash it again and the device should boot fine

But the output is same secure boot check fail, but except the logs shows old bootloader logs.
But the problem is same.
ets Jun  8 2016 00:22:57

rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3fff00b8,len:10768
load:0x40078000,len:24604
load:0x40080400,len:3792
0x40080400: _init at ??:?

secure boot check fail
ets_main.c 371
ets Jun  8 2016 00:22:57

rst:0x10 (RTCWDT_RTC_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3fff00b8,len:10768
load:0x40078000,len:24604
load:0x40080400,len:3792
0x40080400: _init at ??:?

secure boot check fail
ets_main.c 371

Hi @ABHITHLALC,
Sorry for the delayed response.
Just want to check few things,
Did you re-build the old bootloader file, if so then its digest would change as well.
If you did not rebuild, did you change the digest that is flashed at 0x0 ?

Did you re-build the old bootloader file, if so then its digest would change as well.

Yes I rebuild the file. So maybe that will be the reson.

Yeah. Unfortunately, there is no way to recover this module now. You will have to try on a new chip.
Feel free to reopen or submit a new issue if you ever come across any other problem

Thank you for the support, really appreciated


Results of other dataset

Hi, @HaoyunHong
I'm very excited at your work of self-supervised learning in EA. According to your paper, SelfKG performs well in DBP15k and DWY100K. These datasets are relatively simple EA datasets, and I wonder how SelfKG performs in more complicated datasets, such as DBP100K or DBP1M. If you ever tried SelfKG in other datasets? Can you offer more experimental results?
best,
Meihao

Besides, since it is too slow to download the dataset from the website you provide(https://zenodo.org/record/6326870#.Y5nqdXa-tPZ), can you please provide me another way to download your datasets?


No message for project that has no third party dependencies

What happened
When running the program on a project that has no dependencies, no message is
presented
What would you like?
When running the program it would be nice if some message was displayed in the
case that there are no dependencies

I No longer thing this is a good idea. If you have no dependencies you don't want something to be introduced into a requirements file.


Fixed Parameter setstate bug

This is a somewhat subtle bug and I have no minimal example except for trying to run through the panel Param notebook and trying to set Example.unbounded_float = 3.14.

[x] Fixes https://github.com/ioam/param/issues/265

Coverage decreased (-0.2%) to 76.836% when pulling 9f3e0a4a603cf6f5ec81f462f45f69531a148290 on fix_parameter_copy into 56764fd599c86704ea9dd6636586e284744688fa on master.

I can't do anything about this right now or review in detail, so excuse the
brevity, but...
(a) That method is just a hack to allow watching of parameter metadata
(i.e. things other than value) - do we still need that? (If not, we could
remove it and leave metadata watching as something to consider in the
future.)
(b) I imagine there are various times when watching should be suspended.
Pickling, copying, initialization, etc. Respecting the 'initialized' flag
or whatever it is is one of the to-do items, and will appear in the
"depends cleanup" issue. (However, no doubt the setattr method is bad
anyway...)
(c) I can't immediately think what the problem would be with your minimal
example, i.e. how it would lead to the traceback you describe in the
corresponding issue. I'd need to run it (or if you have a session where you
already ran it, maybe you could copy/paste that...?)
On Tue, Sep 18, 2018 at 7:29 PM Philipp Rudiger<EMAIL_ADDRESS>wrote:

Actually, here is a minimal example:
import param
class A(param.Parameterized):
a = param.Number(default=0)
class B(A):

pass

B.param.watch(lambda x: x, 'a', 'bounds')
B.a = 3.14
—
You are receiving this because your review was requested.
Reply to this email directly, view it on GitHub
https://github.com/ioam/param/pull/266#issuecomment-422499292, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAAHib5wwgSHU91GdLC3-LVx1fie4zXJks5ucTt2gaJpZM4Wue5u
.

That method is just a hack to allow watching of parameter metadata
(i.e. things other than value) - do we still need that?

Yes, I use it in panel and it works well.

I'd need to run it (or if you have a session where you
already ran it, maybe you could copy/paste that...?)

The example above leads directly to the traceback, it occurs when setting the parameter value on a subclass which did not define the parameter, because this triggers a copy operation on the parameter which in turn triggers __setstate__.

By the way, I'd be fine with this being merged and a new dev release being made so you can proceed with panel.
We can discuss more if necessary when reviewing the whole mechanism later on.


do you know how i can use php code for the pintrest image

image: " ",
can this be used as the image for pintrest so that it share the image from the product i am on. ect.
if you could tell me how i would be most grateful.

also how to show share count as 0 ift here has been no shares

For pinterest you should specify the url of an image. Thus, just got the absolute url to you image and provide it to share button.

ok i would but the image is different on very page. so i need to use php to generate image url. sorry i don't have mega experience. maybe this one is not for me. pity.

Regarding zero count: currently, it's by design that zero counts are hidden (native sharers act the same AFAIK).
But you can do the trick adding following css:
.jssocials-share-count-box.jssocials-share-no-count  {
    display: inline-block;
}

.jssocials-share-no-count .jssocials-share-count:before {
    content: '0';
}

zero one worked fine ty for that one

Hard to say regarding image url issue without any details. Why is it a problem to generate it on server side?

i can generate the url using php but how to add it to the pintrest button for each product.
i am using open-cart shopping cart and have the buttons on the product page.
ie this is the code the generates the url for the image
and the only place i see to ad it is in the js script and php not work in js  if yu know what i mean
ie: logo:"fa fa-pinterest",media " php coce here    ",shareUrl:"https://pinterest.com/pin/create/bookmarklet/?media={media}&url={url}&description={text}",countUrl:"

What is about something like the following solution: iterate over your images on the server to render js array. Then you can iterate over this array with js and initialize sharing controls.
Another alternative could be: create a php script that returns the array of image urls. Than on the main page you can request these images with ajax and on done set imageUrl to sharing controls.

np. thanks i will get some one ot fix it for me. Thanks


How to verify MFA later and not inline when doing bank login?

Unless I'm mistaken there doesn't seem to be a method for verifying MFA for a bank login at a later point. Meaning I won't have the MFA answer when submitting the request. I could create an instance of SynapsePayRest::UnverifiedNode myself with the necessary data and verify it but I figure this would be private? Would it be preferred here to have a dedicated method/API for this? Something like user.verify_ach_us_node_mfa?

I didn't think to add this, since there isn't a way to fetch the node MFA info from the API again without resubmitting the bank login details. But I suppose if you stored the MFA access_token, the user could return to enter the MFA answer at a later time.
user.verify_ach_us_node_mfa seems reasonable. I would have it create an instance of UnverifiedNode and call unverified_node.answer_mfa.

This was resolved by #69


Graph: fraction of crates.io that depends on lazy_static

cc @kimundi

Heh, is that 30% now? Crazy!

:chart_with_upwards_trend: 38% now


Support for different UDP port used for SSDP multicast messages

Hello,
Just wanted to ask if it would be a valid request to be able to define the port for multicast messages manually. The default port for SSDP is 1900 but the manufacturer of the device I wish to discover listens to 1982 port. As he states in his documentation:

Different from SSDP protocol, we choose to send multicast messages to port 1982 instead of standard SSDP port 1900. This is to avoid excessive multicast messages being received by both smart LED and 3rd party devices. It's especially important if the 3rd party device is  power consumption sensitive (e.g. smart watch powered by battery).

I don't know if this is a  common practice manufacturers decide to follow, but I am unable to use RSSDP because of that port decision. I didn't find anything in the source code that enables me to override the multicast port. Nevertheless as I stated that is the decision of a single manufacturer, that knowingly decided to differentiate from the SSDP protocol, so I am not sure if such a change to RSSDP would be worth it.
If you consider that adding the port as another user defined parameter does not offer any significant value, please close this issue.
Thank you

Hi,
At the moment this isn't configurable, in theory it would be possible to change it though I haven't looked in detail. Is the port the only issue you have, or are there other differences too? What kind of devices are you trying to use this with?
See #48 for a prior discussion on this.

Just to follow up, the Yeelight stuff has different schema, different protocol, and different port. At that point just changing the port in RSSDP isn't going to help much. Yes, you would be able to search for the device, but you wouldn't be able to download or deserialise the device description document due to the custom protocol/document format, and none of the publishing stuff would work. A fork of this library specifically for Yeelight and with appropriate changes made for the devices seems best if all that is true, but I am not planning one. Happy for you or someone else to do so.
If all you need is to be able to change the port, then that's a different story and maybe something I can do in RSSDP.

Thank you Troy. #48 issue is exactly my case.
This is my first time playing with SSDP, so I didn't realize that yeelight was different in more ways than just the port.
I did change the port in the source code, but I still haven't been able to communicate with the lamp. Too bad, what could have been a walk in the park, suddenly needs re-eventing the wheel. I was under the impression that yeelight was implementing SSDP for the discovering/advertising part, before setting a TCP connection with the lamp for actually controlling it.
I will run a few tests to see what I can reuse from RSSDP awesomesauce.

No problem. I really don't why they changed from the standard. SSDP already has protections in it to prevent overload from the UDP packets. Even if they did have a problem there, (apparently) abandoning the device description document format and using a custom protocol to retrieve it all breaks the standard significantly.
The good news is that, as far as search goes, a lot of the stuff in RSSDP should be reusable. After changing the port in use, it should be a matter of downloading the doc using the custom protocol and deserialising whatever they give you into sensible objects. The rest of the stuff should work, but I have none of these devices to test with.
Let me know if I can help further.

Just a small follow up for anyone interested. I gave some feedback to the yeelight devs, who I must say are very eager in helping in every way. When I asked for future versions to support the proper SSDP standard, they gave some explanation around their decision for a custom protocol

Thanks for your comments and feedbacks!

Before we decide to develop the open API, we did some research. The result is: a full SSDP protocol implementation is too heavy for a embedded system, e.g. a WiFi bulb. The resource on a SoC is too constraint, the free RAM is normally around 6~8K in some low-end product. If too much SSDP packages are being received, it will impact the normal lighting control logic.
Suppose we follow the standard SSDP in discovery part, then what about next step? How should we expose our control interface? Normally or per some standard method XML/SOAP or Restful API will be needed to control the device, but all of them are too heavy for us. You may wonder why Hue or Lifx or Wemo can handle this, because they use a much stronger processor with much more resources.
Imagine we can implement Restful API or SOAP in our device, what about next step? Developer still need to check our spec and implement the API defined by us, pass each parameter carefully, which is way different than Hue/Lifx.

So current situation is: there is not a single standard in IOT world that everyone follows, that means developer has to build different handlers to handle devices from different vendors. We hope there will be some standard some day finally, but it will be a long way to go.

Hopefully you can understand my point. Currently, the IOT world is fragmented, we don't know which standard to follow, so we invented the standard that is most suitable for us.

It seems there is no way around the custom implementation for now. There are 2 or 3 C# working projects on github for controling yeelight bubls, so I am going to start from there.

Thanks for posting the follow up, good luck with the other libs.
It make sense to use low spec hardware for bulbs, and that they don't have the horse power to the full dependencies for the SSDP protocol. I was confused because I had previously understood they had concerns about the network, rather than their own hardware.


merge package from arlac77/npm-package-template

package.json

chore(package<EMAIL_ADDRESS>chore(scripts): cover@#overwrite c8 --temp-directory build/coverage ava && c8 report -r lcov --temp-directory build/coverage
chore(package): add nyc from template
chore(package): set $.ava.require='esm' as in template
chore(package): set $.ava.files='tests/-test.js,tests/-test.mjs' as in template
chore(package): set $.ava.extensions='js,mjs' as in template

Coverage remained the same at 83.951% when pulling 7471cf3b7e38239d35853ddef8445234dedb2043 on template-sync-1 into 21616dd22af8812a606406a880c8a8cfd022a183 on master.


ConnectionError [SSL: CERTIFICATE_VERIFY_FAILED] when connecting to AWS ElastiCache over TLS

We're experiencing connection issues with v3.x when trying to use our AWS managed ElastiCache Redis Cluster (version 4.0.10) over TLS (rediss://...). Rolling back to 2.10 fixes the issue immediately.
This is the stacktrace of our application:
ConnectionError: Error 1 connecting to master.**********.cache.amazonaws.com:6379. [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1051).
(8 additional frame(s) were not displayed)
...
File "redis/client.py", line 1182, in exists
return self.execute_command('EXISTS', *names)
File "redis/client.py", line 760, in execute_command
connection.send_command(*args)
File "redis/connection.py", line 619, in send_command
self.send_packed_command(self.pack_command(*args))
File "redis/connection.py", line 594, in send_packed_command
self.connect()
File "redis/connection.py", line 498, in connect
raise ConnectionError(self._error_message(e))

@u2mejc thoughts? This is related to the SSL 'strict' change in #1017

@dkms-its You can use redis-py 3.x with and create a client instance with the ssl_cert_reqs=None option. This will behave identical to redis-py 2.x with regard to encrypted connections.

Thanks for the quick reply and workaround, @andymccurdy. However, we're using redis.py not directly but through django-redis, so i guess i have to ask them for a corresponding option.

Had the same issue when accessing azure redis ( using Azure end point , not a CNAME ) .
specifying ssl_ca_certs='/etc/pki/tls/certs/ca-bundle.crt' when creating the redis client resolved the issue.

It seems like redis-py is behaving correctly. Closing this issue as there are ways to load the correct certs with both Azure and AWS.

I still think that the problem is that by default the ssl_cert_reqs is set to 'required'. If someone want to use ssl=true with password only without certs, one needs to change this parameter instead of setting it to 'required' only when you want to specify specific cert files.

Is there a known cert bundle for the elasticache service like how the rds service has it? How is the line truly encrypted in transit if we cannot validate the CA?


fix windows gpu build

Can one of the admins verify this patch?

This fix helps me! Thank you.

@tensorflow-jenkins test this please

Wonder if a comment makes sense for the T(2) thing. It is very common in this file and was only missing in the 2 places I changed.
I looked at the CI failure: the file must be read only for some reason. I remember seeing this 2 month back but not since. Never found out the root cause.
22:41:30          C:/tf_jenkins/home/workspace/tensorflow-pull-requests-windows-cmake/tensorflow/contrib/cmake/build/tensorflow/core/lib/core/error_codes.pb.cc: Permission denied
22:41:30     19>C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V140\Microsoft.CppCommon.targets(171,5): error MSB6006: