Solve warnings

Hello,
I recently pulled your repository and encountered a huge wave of warnings. I changed a few parts in that code, especially those were you are using uint/size_t inconsitently.
The only remaining warning type is: warning C4996: 'localtime': This function or variable may be unsafe. Consider using localtime_s instead. localtime_s is a msvc function, so this should be avoided to keep the code portable.

Thanks a lot for your contribution.
I am currently doing some refactoring in the "optimization" branch and there I am trying to use mostly uint32_t (uint32) whenever possible instead of uint. I will then try to use your changes to make sure that I don't have those warnings anymore (not sure I will able to directly merge your pull request but I will definitely use it to make sure I do not forget anything). I will probably do this for version v0.9.0 of the library.
Thanks !!

Glad that you like it.
It seems you try targeting c++11 and below, am I right?
Otherwise I would suggest to overhaul those nasty indexed for loops and potentially get rid of the custom containers and use the std versions. What is your point using these custom versions (hopefully that doesn't sound too rude :D )?

Yes, I am currently targeting c++11.
Many game/physics engines do not use the STL containers but custom ones.
Mostly for speed and being able to be in total control of the implementation (memory allocation, memory alignment, multi-threading). I have implemented custom list, hash map and hash set classes that are quite easy to implement. But if I need a method that is not available in the STL, it very easy for me to add it into my custom classes.
You can also read here in the book "Game Engine Architecture" by Jason Gregory some drawbacks when using the STL library in games.

Well, I get your point but I don't think I can fully agree with it. But to be honest, its totally your decision ;)
Don't get me wrong, I like your lib. If I wouldn't I would not invest time to make a pull request.
I own that book myself but, to be honest, didn't remember much of this chapter, so I quickly reread it. If I'm right, this book mostly refers to c++ prior c++11. For example the author talks about raw arrays instead of std::array (which doesn't do much more than a plain "c" array but provides a much nicer interface) which were introduced in c++11. I think I also saw a few raw arrays in your codebase, too. Each modern stl container (beside std::array of course) has the possibility to use a custom memory allocator; so it is tweakable to your needs (never did that myself).
Either way, I have two major complaints about your lib: your List implementation is more like a dynamic array (e.g. std::vector) than a list. And that confused me during my first step through your code.
The second is, that you are using the indexed loops a lot even if your container offers iterator support. "Range based for loops" where introduced in c++11 and they are a lot easier to write, maintain and read. From my personal experience with the stl I can tell, that using them is even faster than using the indexed approach , because they directly cache the end/size of the range and do not request this information during each iteration step, which is the case in most of your (and also others) hand written for loops. But that might be different with your containers.
I wouldn't complain about that, if you just provide the iterator interface to the user of your lib. But unfortunately you do not and thus the user is forced to use that indexed style, which is a pity because it prevents the use with common stl algorithms or c++20 ranges (to just name a few examples).
I hope you don't consider this text as a blame, because I really don't want to blame or tell you what you have to do with your library. I simply see the possibility to "modernize" the interface of this library a bit, so that it can be more enjoyable for the user ;)

Either way, I have two major complaints about your lib: your List implementation is more like a dynamic array (e.g. std::vector) than a list. And that confused me during my first step through your code.

That's right the reactphysics3d::List class is indeed for a dynamic array (with continuous memory) like std::vector but I really don't like the "vector" name for this. I would prefer to use reactphysics3d::Array. Maybe I will rename it to Array.

The second is, that you are using the indexed loops a lot even if your container offers iterator support. "Range based for loops" where introduced in c++11 and they are a lot easier to write, maintain and read. From my personal experience with the stl I can tell, that using them is even faster than using the indexed approach , because they directly cache the end/size of the range and do not request this information during each iteration step, which is the case in most of your (and also others) hand written for loops. But that might be different with your containers.

This is a good point. I can try to see if there is a performance benefit to use Iterators for the loops. If so, I have no problem to use iterators internally of course.
In the future, I plan to use multi-threading so that for instance on a 4 cores CPU and n bodies to update, the first core is responsible to compute the new positions for bodies 0 to n/4 in an array of positions. I think for this it is simpler to work with indices instead of iterators if I want to separate the work on an array on multiple threads.

I wouldn't complain about that, if you just provide the iterator interface to the user of your lib. But unfortunately you do not and thus the user is forced to use that indexed style, which is a pity because it prevents the use with common stl algorithms or c++20 ranges (to just name a few examples).

Yes, I do not see any problems to do that.

I hope you don't consider this text as a blame, because I really don't want to blame or tell you what you have to do with your library. I simply see the possibility to "modernize" the interface of this library a bit, so that it can be more enjoyable for the user ;)

No problem. It's always interesting to know what the users of the library like or dislike about it.

I was wondering something.
When you say:

From my personal experience with the stl I can tell, that using them is even faster than using the indexed approach , because they directly cache the end/size of the range and do not request this information during each iteration step, which is the case in most of your (and also others) hand written for loops. But that might be different with your containers.

I am not sure I understand this actually. When using an iterator, each time we advance the iterator we need to check against the size/end of the array right ? So it's exactly the same for the index loop. Am I missing something ?
Moreover, as described and benchmarked in this post there should be no huge difference between the two solutions in a loop.
When you say:

I wouldn't complain about that, if you just provide the iterator interface to the user of your lib. But unfortunately you do not and thus the user is forced to use that indexed style, which is a pity because it prevents the use with common stl algorithms or c++20 ranges (to just name a few examples).

What do you mean exactly ? As far as I know in the ReactPhysics3D library it is possible to iterate on rp3d::List, rp3d::Set and rp3d::Map using iterators as follows:
rp3d::List list = ....;
for (auto item: list) {
   ...
}

for (auto it = list.begin(); it != list.end(); ++it) {
   ...
}

Is this not working on your side ?

Hey, sorry for the late replay. Didn't check my notifications.

I was wondering something.
When you say:

From my personal experience with the stl I can tell, that using them is even faster than using the indexed approach , because they directly cache the end/size of the range and do not request this information during each iteration step, which is the case in most of your (and also others) hand written for loops. But that might be different with your containers.

I am not sure I understand this actually. When using an iterator, each time we advance the iterator we need to check against the size/end of the array right ? So it's exactly the same for the index loop. Am I missing something ?
Moreover, as described and benchmarked in this post there should be no huge difference between the two solutions in a loop.

Well, lately I performed some benchmark tests on quickbench and on my local machine. The outcome was quite interesting, because when I was using the usual setup like this:
for (std::size_t i = 0; i < cotainer.size(); ++i){ /*...*/ }
against
for (auto itr = container.begin(); itr != cotainer.end(); ++itr){ /*...*/ }
the indexed version was a lot slower than the index version. But when I changed to cached end versions:
for (auto size = cotainer.size(), i = 0; i < size ++i){ /*...*/ }
against
for (auto itr = container.begin(), end = cotainer.end(); itr != end; ++itr){ /*...*/ }
The index version gained a huge performance push but where still slower than the iterator version. Have a look here: https://quick-bench.com/q/-Z8vtdMWdF6uO1toADANd9FRLZ4
This might come from the fact, that the stl iterators are designed to work very well with iterators and therefore a begin/end instead of a begin/size member pair is cached inside of a vector, so the size needs to be calculated each time it is requested.
I saw a lot for loops with the non-cached size in your code base. Indeed, you could simply add the caching but from my perspective its preferable to use the range-based for, because you directly access the iterated member. But that might be a matter of taste. What I try to tell is, that even when it might seem odd, caching is king. With the range-based for loop you don't have to think about that detail, because it's doing it right under the hood; and it is easier to write.

When you say:

I wouldn't complain about that, if you just provide the iterator interface to the user of your lib. But unfortunately you do not and thus the user is forced to use that indexed style, which is a pity because it prevents the use with common stl algorithms or c++20 ranges (to just name a few examples).

What do you mean exactly ? As far as I know in the ReactPhysics3D library it is possible to iterate on rp3d::List, rp3d::Set and rp3d::Map using iterators as follows:
rp3d::List list = ....;
for (auto item: list) {
   ...
}

for (auto it = list.begin(); it != list.end(); ++it) {
   ...
}

Is this not working on your side ?
Yes, you are right. But you tend to hand over wrapper objects which exposes just a size and a indexed element access function. For example in your "CollisionCallback" you hand over a CallbackData object, which only exposes a size_t getNbContactPairs() const and ContactPair getContactPair(size_t index) const.

Well, lately I performed some benchmark tests on quickbench and on my local machine. The outcome was quite interesting, because when I was using the usual setup like this:
for (std::size_t i = 0; i < cotainer.size(); ++i){ /*...*/ }
against
for (auto itr = container.begin(); itr != cotainer.end(); ++itr){ /*...*/ }
the indexed version was a lot slower than the iterator version. But when I changed to cached size/end versions:
for (auto size = cotainer.size(), i = 0; i < size ++i){ /*...*/ }
against
for (auto itr = container.begin(), end = cotainer.end(); itr != end; ++itr){ /*...*/ }
The index version gained a huge performance push but where still slower than the iterator version. Have a look here: click me
This might come from the fact, that the stl containers are designed to work very well with iterators and therefore a begin/end instead of a begin/size member pair is cached inside of a vector, so the size needs to be calculated each time it is requested.

I saw a lot for loops with the non-cached size in your code base. Indeed, you could simply add the caching but from my perspective its preferable to use the range-based for, because you directly access the iterated member. But that might be a matter of taste. What I try to tell is, that even when it might seem odd, caching is king. With the range-based for loop you don't have to think about that detail, because it's doing it right under the hood; and it is easier to write.

Ok you mean caching the size of the container before the loop. I am currently doing some optimizations in this branch where I use the "perf" profiler with flame graph to find the places to optimize. I have already find (and fixed) some places in this branch where the caching you are talking about appeared in the profiler. Thanks for the interesting benchmarks.

Yes, you are right. But you tend to hand over wrapper objects which exposes just a size and a indexed element access function. For example in your "CollisionCallback" you hand over a CallbackData object, which only exposes a size_t getNbContactPairs() const and ContactPair getContactPair(size_t index) const.

Yes that's because my first idea what not to expose the List, Map, Set containers to the user of the library but only use them internally.

Yes, you are right. But you tend to hand over wrapper objects which exposes just a size and a indexed element access function. For example in your "CollisionCallback" you hand over a CallbackData object, which only exposes a size_t getNbContactPairs() const and ContactPair getContactPair(size_t index) const.

Yes that's because my first idea what not to expose the List, Map, Set containers to the user of the library but only use them internally.

How about simply exposing the iterator pair?
class CallbackData
{
public:
	auto rbegin() noexcept { return std::rbegin(m_Data); }
	auto rbegin() const noexcept { return std::rbegin(m_Data); }
	auto crbegin() const noexcept { return std::rbegin(m_Data); }
	auto rend() noexcept { return std::rend(m_Data); }
	auto rend() const noexcept { return std::rend(m_Data); }
	auto crend() const noexcept { return std::rend(m_Data); }

	auto rbegin() noexcept { return std::rbegin(m_Data); }
	auto rbegin() const noexcept { return std::rbegin(m_Data); }
	auto crbegin() const noexcept { return std::rbegin(m_Data); }
	auto rend() noexcept { return std::rend(m_Data); }
	auto rend() const noexcept { return std::rend(m_Data); }
	auto crend() const noexcept { return std::rend(m_Data); }

private:
	SomeMightyContainer m_Data;
};

It's kinda bloat, but on the caller side it definitely is more enjoyable (at least for me :D)

I am closing this pull request because most of the warnings have been fixed in the last version (v0.9.0) of the library. I have also used some fixes and comments from your pull request. Thanks a lot.


It should apply a limit for its aggregations?

When filtering by some value, it returns a set of aggregations without limit number.
https://github.com/puntmig/php-search/blob/3dea1271fd70a03348612e6bf3e883fb2dae8fe2/Query/Query.php#L421

This is a pretty fuzzy question is seems! I'm not sure I'm the right guy for the job here, but at the very least I can be a talking rubber duck for a second.
What problem are we trying to solve here? Is it a question of whether or not this piece of code will be able to handle very large results? Or is it a question of where to apply limits?
If it's the latter, then, well, it seems to me that the right place to do limiting is afterwards, i.e. something like Query::createMatchAll()->filterBy(something)->limit(10). This code is generalized in such a way that it's hard to get into the code base, so I can't really gather whether or not you use generators - if you do, then it would be natural to create a "limit" generator which stops yielding after a while. If you don't, well, then I guess you can do limiting afterwards anyway, since whatever is feeding filterBy data will need to store the entire result set in memory anyway, and you'd have bigger issues to handle.

Making a filter and an aggregation should be different things, although they can be related somehow.
So, filterBy method, and unless you specify the other way, will create a limitless aggregation.
If you want to create a limitted aggregation, then you specifically should call aggregateBy with the same filterName parameter. Then you'll be able to set the limit :)

Hmm, wait. Is filterBy an aggregation? Or do you mean aggregations applied after filterBy, e.g. Query()->filterBy()->sum() ? I'm not sure I follow.
Also, how do you want to separate aggregations and filters?

@dbeecham nopes. filterBy is just a filter. You can filter by X without the need of aggregating the data (for example, for a search box), or you can aggregate the data without the need of filtering anything (for example, for creating menus).
Both filter and aggregations are separated by different methods :)
BTW, are you using this library? I mean, do you want a sandbox to use it?
Anything, please, don't hesitate to write me at<EMAIL_ADDRESS>:)
Thanks for the feedback!

No, I just wanted to make github a bit more social. It's easier to get involved in ideas when you see others doing the same, so by just jumping in will at the very least make it easier for the next guy to actually help out.  :) I don't have time to work on this myself, sorry. Hopefully I managed to inspire an idea or two.


Relax some deps in general Modin env files

System information

OS Platform and Distribution (e.g., Linux Ubuntu 16.04):
Modin version (modin.__version__):
Python version:
Code we can use to reproduce:

Describe the problem

Some deps constraints in general Modin env files seem to be obsolete. Let's try to get rid of those constraints
Source code / logs

Related to #3615 in terms of keeping dependencies consistency. We should correct the reqs in all the env files.

Done in https://github.com/modin-project/modin/pull/5216.


Should HTP datasets provided by users be acknowledged as "community curated"

e.g.
https://www.pombase.org/reference/PMID:31077324

Should HTP datasets provided by users be acknowledged as "community curated"

I think if they help at all, even just putting a HTP file in the right format, they should be a acknowledged somehow.

It's quite nice to keep the browser-hosted datasets in a separate category. Could we have a 'subcategory' for user-provided ones maybe?
Other ideas?

Other ideas?

It's a bit of a hack, but you could make and approve an empty community curation session for each of the user-provided datasets.

We could do that, it would be fine. I was thinking that to show as curated they needed to be triaged as curatable? but maybe they don't?

OK work around is to ensure that the session is assigned to the user.
Check other HTP data sets have community attribution.

These publications have browser trackers are admin curated in Canto (according to Chado):
 PMID:18488015
 PMID:21511999
 PMID:25908789
 PMID:32101745
 PMID:18093330
 PMID:14566325
 PMID:17304213
 PMID:25747261
 PMID:24256300

These three are not curated in Canto for various reasons:
 PMID:11859360      (pombe genome paper)
 PMID:25361970      (PomBase NAR paper)
 PMID:26883383      (this one probably should go through Canto)

The publications for the other 16 tracks are community curated in Canto:
 PMID:25240800
 PMID:21981922
 PMID:23101633
 PMID:23163955
 PMID:31077324
 PMID:32152323
 PMID:30566651
 PMID:24152550
 PMID:21151114
 PMID:23900342
 PMID:22531001
 PMID:23395004
 PMID:27662899
 PMID:25665008
 PMID:29914874
 PMID:24709818

I checked the papers on Kim's "admin" list:

PMID
note

PMID:25908789
Antonia did the LTP curation; see below

PMID:32101745
Val did the LTP curation; see below

PMID:18093330
Antonia did the LTP curation, and paper is fairly old, with no indication that the browser data was an author submission

PMID:14566325
paper is fairly old, and there's no indication that the browser data was an author submission

PMID:17304213
Antonia did the LTP curation, and paper is fairly old, with no indication that the browser data was an author submission

PMID:25747261
reassigned session to Hua Li

PMID:24256300
reassigned session to Paco Antequera

So that leaves an outstanding question: do you want me to reassign the sessions for the ones that have both author submissions for the browser data and some annotations from us (PMID:25908789 and PMID:32101745)?
The rest are done (session reassigned, or admin classification was fine).
I also put a session through for PMID:26883383.

I checked the papers on Kim's "admin" list:

PMID
note

PMID:25908789
Antonia did the LTP curation; see below

PMID:32101745
Val did the LTP curation; see below

PMID:18093330
Antonia did the LTP curation, and paper is fairly old, with no indication that the browser data was an author submission

PMID:14566325
paper is fairly old, and there's no indication that the browser data was an author submission

PMID:17304213
Antonia did the LTP curation, and paper is fairly old, with no indication that the browser data was an author submission

PMID:25747261
reassigned session to Hua Li

PMID:24256300
reassigned session to Paco Antequera

So that leaves an outstanding question: do you want me to reassign the sessions for the ones that have both author submissions for the browser data and some annotations from us (PMID:25908789 and PMID:32101745)?
The rest are done (session reassigned, or admin classification was fine).
I also put a session through for PMID:26883383.

Maybe this isn't worth worrying about as it affects so few?
and we are just mindful going forward (and also try to get the authors to do their own small-scale stuff)

Maybe this isn't worth worrying about as it affects so few?
and we are just mindful going forward (and also try to get the authors to do their own small-scale stuff)

Your call ... I've got as far as is reasonable. Just those two papers to reassign, or not, and this can close.

Your call ... I've got as far as is reasonable. Just those two papers to reassign, or not, and this can close.

Closing if no indication...

Closing if no indication...


MonoMac backend Popups and Child Windows dont work

See this video:
https://1drv.ms/v/s!Av3c0ICwbVOLiN48bG4ffQiIn_TTNw

mono mac backend is being replaced.


Don't double count jsonl.gz

Tokenize-shuffle globs jsonl.gz and .gz separately, so we end up seeing some files more than once. This fixes that.

Superseded by #207


render() triggers website protections

Hi
I've just started working with requests-html, and render() seems to be triggering a website protection mechanism, and I'm not sure why.  The below URL loads a number of .js scripts with data I'm trying to access. For the first r.html.text below, this data just returns "Loading..." as the js scripts haven't yet run.  After r.html.render(), the page rejects the request.
Any advice on what is going wrong here and how to circumvent it would be very much appreciated - code below:
`from requests_html import HTMLSession
session = HTMLSession()
r = session.get("https://register.epo.org/application?number=EP16190441&lng=en&tab=federated")
print(r.html.text)
r.html.render()
print(r.html.text)`

I can render the page without any issue/rejection. Here is my example https://paste.ubuntu.com/p/ccRqzjBpSh/. Can you be more specific? Maybe it's about your connection. For example requests rate.

Thanks for looking at this. Actually, you have the same issue as me - the page you've rendered is the rejection page, not the actual page you get if you go via a browser. You'll also see if you print before using render() that you get the right web page, but just without the data that I'm looking to access (marked as 'loading').

This is a security system implemented by the site you're trying to scrape in order to prevent bots. The rejection message could be based on a number of aspects that they're watching traffic for, such as user agent, speed of requests, etc.
A possibility to circumvent would be to analyze the underlying network traffic on their site and hit the API directly, but respectfully. But this is an issue relating directly to the site you're trying to scrape, not this codebase.

Hi Ecript
The strange thing is though that it is only the render() function that triggers this. It isn't a speed of request issue presumably as this happens with just a single link.
Does the render () function use a different user agent? Is it possible to set this specifically?

Ah, I don't see a way to change it in the render() call from what I can see in the documentation and source. Before the render call you can set a different user agent via requests. The render() call defaults to the Chrome user agent.
from requests_html import HTMLSession
session = HTMLSession()
headers = {'user-agent': 'my-app/0.0.1'}
r = session.get(url, headers=headers)
http://docs.python-requests.org/en/master/user/quickstart/#custom-headers
But that doesn't solve your problem.
I would then recommend investigating the underlying network traffic that's going on when the javascript is rendered using your browser's dev tools. I've been kind of digging into that method. They're somehow identifying bots. Even just a simple requests get call triggers the denied message. This is going to be a touchy page to scrape.

Thanks Ecript. I suspect headers may be the issue with using render(), as you get the same bot issue just using requests.get(), unless you specify 'real' user_agent headers.
Investigating the underlying network traffic (and then doing something about it) sounds a bit beyond my capabilities at the minute!
Do you think adding the ability to specify the render() user agent would be a worthwhile feature to request?

I might be able to take a crack at it since I've already been reading over the source. I'll take a look and get back to you! Not 100% sure it will fix your issue, but it's worth a shot.

Amazing thank you!

Okay, I figured out how to change the user agent without altering the base code.
from requests_html import HTMLSession
# Sets the user agent to whatever you choose
session = HTMLSession(browser_args=["--no-sandbox", "--user-agent='Testing'"])
r = session.get(url)
r.html.render()

The "no-sandbox" option is passed in by default, so you include it there to make sure it still makes it through when you override the browser_args argument. Those two arguments get passed to the Chromium session that is created when render() is called, not before. Until you call render() it will still use the default Chromium user agent unless you change it using the method I outlined in previous comments.

I tested this solution with your original issue, and it fixed the problem!
I traded out the default Chromium user agent with a different one I found here: https://developers.whatismybrowser.com/useragents/explore/software_name/firefox/
I used the user agent Mozilla/5.0 (Windows NT 5.1; rv:7.0.1) Gecko/20100101 Firefox/7.0.1 and executed the code in your original comment, and that fixed your issue. I was loading content from the page that needed javascript to render. It was looking at the user agent to determine if you were a bot.
This begs the question of whether or not the default user agent in the source should be changed to something else, but I think that's dependent on how many of these issues crop up. I'll leave it to Kenneth to decide that.
Best of luck!

This is fantastic thank you!
Would you be able to post the code that you got working? I'm getting a series of errors (see below) when trying to run render() with those arguments. I assume I'm doing something silly, but have tried a number of different permutations and just can't get it to work:

from requests_html import HTMLSession
url = "https://register.epo.org/application?number=EP16190441&lng=en&tab=federated"
Testing = "Mozilla/5.0 (Windows NT 5.1; rv:7.0.1) Gecko/20100101 Firefox/7.0.1"
session = HTMLSession(browser_args=["--no-sandbox", "--user-agent='Testing'"])
r = session.get(url)
r.html.render()
print(r.html.text)

Traceback (most recent call last):
File "...test.py", line 8, in
r.html.render()
File "...\AppData\Roaming\Python\Python37\site-packages\requests_html.py", line 598, in render
content, result, page = self.session.loop.run_until_complete(self._async_render(url=self.url, script=script, sleep=sleep, wait=wait, content=self.html, reload=reload, scrolldown=scrolldown, timeout=timeout, keep_page=keep_page))
File "C:\Program Files\Python37\Lib\asyncio\base_events.py", line 584, in run_until_complete
return future.result()
File "...\AppData\Roaming\Python\Python37\site-packages\requests_html.py", line 537, in _async_render
await page.close()
File "~\AppData\Roaming\Python\Python37\site-packages\pyppeteer\page.py", line 1465, in close
{'targetId': self._target._targetId})
pyppeteer.errors.NetworkError: Protocol error Target.closeTarget: Target closed.

Yup! This part here: "--user-agent='Testing'" You made a variable Testing but it's a string in the sample I gave you. In my sample, I literally set my user agent to "Testing". You have to replace that with the user agent you want to use.

That was my first assumption, but replacing 'Testing' with 'Mozilla/5.0 (Windows NT 5.1; rv:7.0.1) Gecko/20100101 Firefox/7.0.1' (with or without those quotation marks, and also trying other user agents that I know work with Requests) gives the same error as the one I had above.  Sorry to keep dragging this out!

session= HTMLSession(browser_args=["--no-sandbox", "--user-agent='Mozilla/5.0 (Windows NT 5.1; rv:7.0.1) Gecko/20100101 Firefox/7.0.1'"])

It's impossible to use HTMLSession with browser_args, it returns an error init() got an unexpected keyword argument 'browser_args'

Thanks Trekma. That explains why I couldn't get it to work. Is there any workaround to change the default user agent as per the above?

Hi @LaurT . I'm facing a similar issue as yours and @Ecript 's solution worked perfectly for me! With a small addition I managed to get it working for your page as well.
By investigating the underlying traffic I found that the quotation marks around 'Testing' in session = HTMLSession(browser_args=["--no-sandbox", "--user-agent='Testing'"]) were being included in the GET requests headers. Removing the quotation marks did the trick.
The code below prints your webpage including javascript loaded content.
from requests_html import HTMLSession
session = HTMLSession(browser_args=["--no-sandbox", '--user-agent=Mozilla/5.0 (Windows NT 5.1; rv:7.0.1) Gecko/20100101 Firefox/7.0.1'])
r = session.get('https://register.epo.org/application?number=EP16190441&lng=en&tab=federated')
# This is necessary for your webpage in particular because it takes around 13 seconds for the page to load.
r.html.render(timeout=15)
print(r.html.text)


Using library with Java 8 leads to ByteBuffer NoSuchMethodErrors

When using the flatgeobuf jar published in maven with Java 8, you get NoSuchMethodErrors when ByteBuffers are used.
java.lang.NoSuchMethodError: java.nio.ByteBuffer.position(I)Ljava/nio/ByteBuffer;
This is a fairly well known issue:
https://github.com/geotools/geotools/pull/2878
https://issues.apache.org/jira/browse/UIMA-5904
https://github.com/eclipse/jetty.project/issues/3244
But instead of adding casts, we can just use a maven profile that adds the release 8 configuration when compiling with Java 9 or above.

Pull request here:  https://github.com/bjornharrtell/flatgeobuf/pull/58


end-to-end multioutput

Tensor.corealize attempts to fuse the outputs.
The scheduler groups buffers using a basic level ordering algorithm. Getting to 1 kernel Adam requires smarter realize grouping, tinygrad has all the infrastructure to run multioutput DAGs.

the JIT refactor seems to be wrong for CUDA, I need to isolate that change as a separate PR first.
https://github.com/tinygrad/tinygrad/actions/runs/8316267565/job/22755525144


[Accessibility] Color contrast fixed.

Fixed the color contrast of tab items, disabled headers and item description.
Related issues : https://github.com/Microsoft/pxt/issues/2519

@veler,
Thanks for your contribution as a Microsoft full-time employee or intern. You do not need to sign a CLA.
Thanks,
Microsoft Pull Request Bot

Screenshot of before and after please.

Be sure to go back to each of the bugs you've fixed and mark them as fixed. I believe the process is to assign them back to @v-amik, maybe a comment saying you've fixed them. (Note: you will need to bump before @v-amik can verify the issue has been fixed)

Alright, I will do a bump and assign @v-amik tomorrow morning for the PR that are merged.
I added a screenshot to the description.

@abchatra @Jaqster this changes UI colors, please review.

Also there are some UI changes here : https://github.com/Microsoft/pxt-microbit/pull/466

Looks ok to me

Can we make this change in semantic less. that way its available everywhere


Add PR templates

We should add a PR template with:

Guide to good commit titles
List of things to check when adding a new algorithm (example notebook, tests, docs, etc)
Etc

Hello, I would like to contribute to this issue and already create PR.
I am new the the OSS community and also Bayesian. I would like to to start making contribution in both fields and hope this is a good place to start.
Any feedback/suggestion is welcome. Thank you!


Hotspot Image page issues with latest develop code

As far as my testing goes it seems the hotspot image page is now broken for existing projects as well as not working fully in new projects. Here's some quick notes from my tests:
highlight hotpots doesn't show hotspots like it did previously
click explore only allows selection of first hotspot not any and all hotspots like it did previously
show me doesn't work as it did - now shows each question in turn but then waits for user interaction to click next and then click hotspot and doesn't allow re-selection of previous hotspots
now seems to be based on a question per hotspot without an option to just use original behaviour
question and feedback is shown above and below the image and the feedback is not shown in left panel as it was before
it seems we now can't drag to move hotspots when editing/adding them
users can no longer tab to hotspots using the keyboard this seems to have broken the accessibility
Not a bug but authors ideally need an option to set image size if only small, medium, large, full etc

I addressed these concerns, and also the concerns mentioned in #824

I created a new hotspot editor
hotspot connector and hotspot image support flexible hotspots and rotated rectangles
annotated diagram and drag drop hotspots are limited to rectangles

When you edit existing hotspots in hotspot image, these are restricted to ractangles. Creaet a new hotspot to create a flexible hotspot
Questions:

There is some help text below the image. Are these adequate? Sensible and correct English?
I introduced a 'Reset' icon, to restart drawing a polygon (or rectangle). Is the icon crear enough, and is the term 'Reset'clear enough.


is there any way I can have my app open on a specfic location and zoomed in

Hello,
I was wondering if I can have my app open on specific location and zoom in when I open it? I am looking at something like an uber app

`// Any value from 2 to 20
mapView.zoom = 15;
// Moves the map
mapView.latitude = lat;
mapView.longitude = lng;
// Places the marker at the point
marker.position = mapsModule.Position.positionFromLatLng(lat, lng);`

Use can use MapView  as :
/*map.component.html*/

<MapView
      (mapReady)="onMapReady($event)"
      [latitude]="latitude"
      [longitude]="longitude"
      [zoom]="zoom"
></MapView>

/*maps.component.ts*/
......
export class MapsComponent {
    @ViewChild("MapView") mapView: ElementRef;
    zoom:number = 15;
    latitude:number = 19.447667,;
    longitude:number =  -99.129888;
    constructor() { }
    private onMapReady = (args) => {
        console.log("Map Ready");
   }
}


bug fix when using fromTable option

not add checkbox in fields.blade.php when htmlType is boolean

$fieldTemplate is empty.
https://github.com/InfyOmLabs/laravel-generator/blob/5.2/src/Generators/Scaffold/ViewGenerator.php#L302
Because, $field->htmlType is not checkbox.
Example, $field->htmlType is boolean,hoge,1,

not cast tinyint to boolean

tinyint cast to boolean when $field->fieldType is boolean.
https://github.com/InfyOmLabs/laravel-generator/blob/5.2/src/Generators/ModelGenerator.php#L230
But, $field->fieldType is bigInteger.
https://github.com/InfyOmLabs/laravel-generator/blob/5.2/src/Utils/TableFieldsGenerator.php#L89

Can you put this same pull request in develop branch?


Does tile38 use CPU's single core?

Does tile38 use CPU's single core? The source code does not seem to set the use of multi-core, if set to multi-core, whether it can enhance the performance of it?

Yes. All cores should be used by default. You can use the GOMAXPROCS environment variable to control this.

I ask so many questions, thank you for your patience！

No worries. I appreciate the feedback. :)


Unable to update the registered DApp

I registered a DApp from my wallet, but I am unable to update its information.
I am encountering a transaction error when attempting to update the information.
I followed the same steps as the registration process.
I am encountering the same transaction error during the deregistration process.

I was able to update the registered DApp using different wallet.

I had the exact same issue. My first wallet was created using the step Restore a wallet from mnemonic with setup-new-wallet.sh but updating didn't work. I resolved by creating a new wallet using the setup-new-wallet.sh script.
I also used Blockfrost.


fix: Manually convert C++ vector to Swift Array

Replaces
array.map({ __item in String(__item) })

with
({
  var __array: [String] = []
  __array.reserveCapacity(array.count)
  for __i in 0...array.count {
    __array.append(String(array[__i]))
  }
  return __array
}())

to manually convert vectors from C++ to Swift.
map worked fine in Debug, but apparently broke in release. No idea why.
Created an issue for this bug in apple/swift compiler: https://github.com/swiftlang/swift/issues/76949

cc @perunt

This did actually not work. Simply accessing any property of a std::vector<std::string> (even .count) crashes the Swift compiler in release.
Workarounds:

For Taras' use-case, we just used enums (ContactKey[]) instead of string[]
For the case where you actually need string[], you can create a interface StringHolder { value: string }, then just use StringHolder[].

I agree that this is not ideal and will do my best to fix this - I can easily build a workaround in C++ and bridge to that for strings only, but ideally we want to keep it simple and have that compiler bug fixed. :)


Description
Certainly! Here's the corrected version of your message:
Hello,
Thank you for your great work.
I need a fast face detector with high accuracy, so I decided to convert the yolov8n model you provided in the GitHub repository to an ONNX model. However, the problem is that the ONNX model returns the keypoints as None in all examples.
Could you please help me with this?

Additional Info
No response

This thread was initiated for enhancement purposes, but it is unrelated to any enhancement. Please direct your questions to Stack Overflow, as indicated in the issue templates.


Merge Sort in C++

Addresses #100
Implementation of Merge Sort in C++.
Please review the code.

Hello @ashishlamsal , thank you for your contribution, we loved the clean code. Your pull request will be merged shortly.
Have a good day!


Document Proc.in, Proc.out, Proc.err

The problem
They are mentioned in the intro to the type, but not commented.
Suggestions
These are attributes, so maybe some special treatment is needed, although they are exactly the same as exitcode, signal and pid and the three are there.

[ ] .in
[ ] .out
[ ] .err

These are attributes, so maybe some special treatment is needed, although they are exactly the same as exitcode, signal and pid and the three are there.

They aren't "exactly the same", though, are they? Since, as you noted, they're attributes?
:in, :out, and :err are named arguments to Proc.new (or Proc.shell or run), and are available from object creation, but exitcode, signal, and pid aren't. So the three of .in, .out, and .err are acting much like public attributes, which elsewhere we don't explicitly document—and exactly like public attributes if you allow for the possibility of encapsulated code modifying the attributes even at a time consuming code isn't referring to them. (The implementational question of whether the attribute call is a straight accessor or returning a computation—the biggest salient difference here—isn't supposed to be relevant to consuming code.)
I don't think that this makes your suggestion invalid; I'm just voting in favor of the idea that maybe it should be treated a little specially. (Consider that a mock object or a subclass that was bridging to something that was not a process but could be treated with Proc semantics could, in fact, make in, out, and err straight accessors with zero difficulty, unlike the other methods.)


Create separate benchmark workflow for running large benchmarks

Creates a separate workflow for running large benchmarks. Adds ResNet50 at various batch sizes to this workflow. ResNet50 is part of our progression suite that we will be monitoring closely.
Also rearranges the model_groups to be specific to the target hardware.

Whoo thanks for the improvements! I really appreciate you trimming this down

Uh, whoops, that comment was intended for https://github.com/openxla/iree/pull/12908. I'm sure this one is great also, but I haven't looked at it yet :-D


possible bug in resolveCollisions

Changed line 244 in sankey.js, in order to properly fit chart (removing overlaps in links).
       if (dy > 0) {
to
       if (dy > -nodePadding) {

When reporting an issue, please include a link to a live example, preferably on bl.ocks.org or RunKit, or as a pull request in the appropriate repository with new unit tests, that demonstrates that the described behavior is not the expected behavior. Use only the minimum amount of code necessary to reproduce the unexpected behavior.
A good bug report should isolate specific methods that exhibit unexpected behavior and precisely define how expectations were violated. What did you expect the method or methods to do, and how did the observed behavior differ? The more precisely you isolate the issue, the faster I can investigate.
Non-actionable bugs may be temporarily closed until you provide additional information. If you can’t isolate the bug any further, please let me know and I will try to help. However, I beg your patience: I must balance your request against many other responsibilities—fixing other bugs, answering other questions, new features, new documentation, etc. Please also consider asking for help on Stack Overflow.
If you have a question about D3’s behavior and want to discuss it with other users, also consider the d3-js Google Group or joining the d3-js Slack.
Thank you! 🤗


Create a WS2812b sample using SPI

This is a new sample using a Raspberry Pi driven a WS2812b led matrix using SPI

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.:x: feokuma sign nowYou have signed the CLA already but the status is still pending? Let us recheck it.

Thanks. The agreement is signed. I still develop this and others samples. 👊😎


Added Pink effect

Give an input and ouput for demostration purpose in the PR description please.
Remove the pink effect. That was for your understanding so that you can see the issue with the negative effect logic you used


[Feature Request] Setting to not automatically suggest values while typing a url

Insomnia Version:

Operating System: Windows 10

Details

As I'm typing a url, the above appears. If I press enter in the ~800ms before it appears, it sends the request, which is what I want. Unfortunately if that box appears, pressing enter selects the first option and messes up my url.
Could there be an setting to make it only appear when I press Ctrl+Space?

Sure, this sounds like it would be useful to the general userbase. It would probably be best to have a setting to control the completion delay (with 0 meaning never). We could also change the number of match characters required for it to show automatically.
I won't have time to work on this right away but here's where the delay logic lives if you, or someone from the community, wants to work on it first
https://github.com/getinsomnia/insomnia/blob/26e9b646bd8f6ff0049b9fec71332fd66ae451c1/packages/insomnia-app/app/ui/components/codemirror/extensions/autocomplete.js#L152-L155


Better handling/Editing/creation of categories

Describe your suggested feature
It should be like this...

when you type the name of the category it should go to the alphabetical place in your current category list
maybe make it tree based where they can look like this

A
A 》AP Comics
A 》Anime
B
B 》Benny Breakiron
Hentai 》Anime-based
Hentai 》Manga-based
Other details
The current way of handling category is worse than the previous version of category creating editing
Acknowledgements

[X] I have searched the existing issues and this is a new ticket, NOT a duplicate or related to another open or closed issue.
[X] I have written a short but informative title.
[X] If this is an issue with an extension, I should be opening an issue in the extensions repository.
[X] I have updated the app to version 1.9.3.
[X] I will fill out all of the requested information in this form.

All category things are handled by main Tachi code, I am not going to do any additions in SY unless I feel its a good feature.


ordhook-sdk-js experiment: from proof of concept to production-readiness

The branch feat/improve-sdk is including a solid draft of the upcoming ordinals-api re-architecture.
Instead of starting ordhook and registering ordinals-api as an http observer receiving payloads including ordinals inscriptions / transfers, the incoming version of ordinals-api will pull a NPM @hirosystems/ordhoook-sdk-js library and implement the following skeleton (present in components/ordhook-sdk-js/lib/test.ts:

import { OrdinalsIndexer } from "./index";

const indexer = new OrdinalsIndexer({
    bitcoinRpcUrl: 'http://<IP_ADDRESS>:8332',
    bitcoinRpcUsername: 'devnet',
    bitcoinRpcPassword: 'devnet',
    workingDirectory: '/Users/ludovic/ordhook-sdk-js',
    logs: false
});

indexer.applyBlock(block => {
    console.log(`Hello from JS ${JSON.stringify(block)}`);
});

indexer.undoBlock(block => {
    console.log(`Hello from JS ${JSON.stringify(block)}`);
});

indexer.streamBlocks();

The OrdinalsIndexer struct can be fully piloted in Typescript. In case of migrations, the following actions can also be performed:
// Delete blocks and inscriptions
indexer.dropBlocks([32103, 32104]);

// Re-download + index blocks
indexer.rewriteBlocks([32103, 32104]);

// Sync blocks until chain tip
indexer.syncBlocks();

// Replay a list of blocks
indexer.replayBlocks([32103, 32104]);

The ordhook-sdk-js is producing a state (rocksdb+sqlite) identical to the one being produced by ordhook-cli. So a state produced by ordhook-cli can be reused by an app being built with ordhook-sdk-js and vice-versa.
Remaining work:

[ ] Wire dropBlocks method
[ ] Wire rewriteBlocks method
[ ] Wire syncBlocks method
[ ] Wire replayBlocks method
[ ] Manually publish ordhook-sdk-js on NPM for end to end testing
[ ] Automatically publish ordhook-sdk-js via CI
[ ] Prepare docker image

@lgalabru Is this still relevant & has accurate info?

Hello world

Changing ordhook to write to pg may solve this. Closing this optimistically.


Switch to iife format for pack and thunk bundles

Context
We used to manually add a closure when throwing a bundle into IVM context.
  const script = await isolate.compileScript(
    `(() => { ${stubName} = (() => { ${bundle} \n return exports; })(); })()`,
    {
      filename: path,
    },
  );
  await script.run(context);

why manual closure
The reason of doing so is that isolated-vm evaluation has a different behavior from loading / eval / requiring a module in NodeJS. Imagine that we have a simple module like this:
-> % cat test.js
'use strict';
function hello() {
  console.log('hello world');
}
module.exports = hello;

When the code is eval or required in node, no symbol leaks to global.
> require('./test')
[Function: hello]
> global.hello
undefined
> hello
Uncaught ReferenceError: hello is not defined
> module.hello
undefined

> const code = fs.readFileSync('./test.js'); eval(code.toString());
[Function: hello]
> hello
Uncaught ReferenceError: hello is not defined
> global.hello
undefined
> module.hello
undefined

This behavior is however different in isolated-vm. if we do
  const script = await isolate.compileScript(testJsBundle, {filename: path});
  await script.run(context);

hello will leak to global.hello. isolated-vm does support something like evalClosure which will bound local symbols. However  evalClosure doesn't allow us specify the filename of the code so that we can't map stack back with source map. (note that the documentation says evalClosure supports scriptOrigin but the code doesn't actually do it. as far as I can tell after looking into the cplus implementation of isolate_handle.cc used for script and context_handle.cc used for eval). This limits our option to only compileScript.
why does it break source map.
(() => { ${stubName} = (() => { ${bundle} \n return exports; })(); })() isn't compiled with sourcemap. it shifts the offset a few characters away. this is crucial to minified bundles.
what's IIFE & CJS
checkout https://betterprogramming.pub/what-are-cjs-amd-umd-esm-system-and-iife-3633a112db62 to see what the difference is between IIFE & CJS
in short, IIFE generates something like
module.exports = (() => { <actual code> })();

What does this PR do
In this PR does 3 things:

generate thunk bundle and pack bundle in IIFE format (these two bundles are loaded into IVM)
bump pack-sdk version to 0.7.0 (see below)
add an option requiresManualClosure that allows us to avoid manual closure if the pack bundle is compiled with packs-sdk after 0.7.0. see https://github.com/coda/coda/pull/63913 for the coda repo changes to make this possible.

Testing
Assuming that I'll bump this in the coda repo, there are a few things that we need to test.

[x] legacy pack bundles still run without a problem.
[x] new pack bundles will run with correct stack trace.
[x] new pack bundles will run on old runtime. (this isn't a big deal though)
[x] test with devapp. (see https://github.com/coda/coda/pull/63913)

Just for my knowledge, what kinds of issues were we seeing with source maps? Was this a long-standing issue with entirely broken source maps? Or just occasional issues? Or did things break when you added the extra closure recently?

oh, it's an occasional issue. since the code is shift a few characters, the source mapping sometimes will end up with missing or incorrect information. when the code is not minified (e.g. web editor), the source mapping works most of the time. When the bundle is minified (e.g. CLI packs), some frames might be missing or incorrect. Sometimes it's not obvious. For example, the stack below is from staging.
{
  category: 'Internal',
  type: 'Unknown error',
  message: "Cannot read property 'shipment' of undefined",
  details: 'Runtime error',
  stack: "TypeError: Cannot read property 'shipment' of undefined\n" +
    '    at parseTrackingInfoFromTrackResponse (packs/ups/main/helpers.ts:16:37)\n' +
    '    at <unknown> (packs/ups/main/formulas.ts:22:14)\n' +
    '    at Generator.next (<anonymous>:null:null)\n' +
    '    at fulfilled (null:null:null)\n'
}

This line at <unknown> (packs/ups/main/formulas.ts:22:14) is misinterpreted due to the sourcemap offset.

devapp tested see https://github.com/coda/coda/pull/63913.


Filter radius on the NearbyScreen

Currently it is not possible to set this value!

@agrim19
It could also be an introductory task for the new ones ^


XMonad.Config.Prime and RebindableSyntax

Okay, so I don't really get it. Why isn't Prime a Monad? From what I can tell, Prime is basically a (partially applied) function XConfig -> XConfig. While >> composition rules for function monad discard left operand entirely, it's possible to reformulate this problem in terms of state monad with XConfig being a state, and monadic actions in Prime acting upon said state. I don't think there's a huge difference performance-wise.
So, why the whole RebindableSyntax thing? It just makes working with actual monads (like IO) awkward.

Prime is a function from XConfig l1 -> XConfig l2, and state monad doesn't
allow for the type of the state to change.
It does make working with actual monads awkward. If you're doing a lot of
that, recall that you can stick other .hs files in your .xmonad and don't
need to enable RebindableSyntax for them.
That said, I'd love a version of (>>=) that works on both Prime and Monad.
I tried a bunch myself and couldn't get the compiler to like me. I
eventually asked a mailing list and got one potential solution on this, but
never took the time to try it out after that. Honestly, you're the first
person who's claimed to look at X.C.Prime; I thought I was the only user,
and it didn't bother me so much.
If there are a few +1s on this bug, I'll give it another go in the next
couple weeks. In the meantime, patches welcome.
On Fri, Apr 29, 2016 at 2:25 AM, Nikolay Yakimov<EMAIL_ADDRESS>wrote:

Okay, so I don't really get it. Why isn't Prime a Monad? From what I can
tell, Prime is basically a (partially applied) function XConfig -> XConfig.
While >> composition rules for function monad discard left operand
entirely, it's possible to reformulate this problem in terms of state monad
with XConfig being a state, and monadic actions in Prime acting upon said
state. I don't think there's a huge difference performance-wise.
So, why the whole RebindableSyntax thing? It just makes working with
actual monads (like IO) awkward.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub
https://github.com/xmonad/xmonad-contrib/issues/54

I'm thinking you may want to wait for ApplicativeDo to land in ghc8. You don't actually want Monad here, but Applicative.

I'm pretty sure applicative do won't help here, since it mostly works with cases like f <$> arg1 <*> arg2 ... , while what we want for Prime is more like expr1 >=> expr2 >=> ..., (due to it actually being XConfig l → IO (XConfig l')). I think <$> might work as well, but I couldn't make it work in my head with applicative do syntax (note that arrow notation doesn't work here either)
I've dug around a bit, and it's possible that indexed monads could be an answer (since regular monads are a subset). Couldn't make it work yet though, and I don't think it will be satisfactory (due to need to 'lift' regular monads into indexed monads for syntax to work for those).
Another option could be to build a config transformation AST and then apply it, so that AST building could be encapsulated into state monad. I think something like this was attempted in https://github.com/xmonad/xmonad-contrib/blob/master/XMonad/Config/Monad.hs, but that seems to have died off 7 years ago.
@twifkak, what was that potential solution that you got from mailing list? I might poke around to see if it's viable.

Huh. Using Layout existential type to wrap every layout type seems to work well enough.
That boils down to
type Prime = Arr (XConfig Layout)
type Arr x = StateT x IO ()

and a bogus instance
instance Read a => Read (Layout a) where
    readsPrec _ = const []

Only problem is, I don't think it's possible to construct a proper Read instance for an existential type -- unless you want to maintain a list of everything that could be inside. So layout persistence on config reload is lost (possibly something else as well)

Yes. I actually thought of that first, but the Read instance is already the source of a number of issues --- and is why the "public" XConfig is unwrapped, the wrapping only happening when it is converted to an XConf for internal use.

@lierdakil When all you have is a Layout a, you can still use a function like
the following (which is pretty much what XMonad has in Main.hs):
  lread :: Layout a -> String -> Maybe (Layout a)
  lread (Layout a) s = Layout <$> (readMaybe s `asTypeOf` Just a)

That code ends up using the Read dictionary hidden in the first argument,
trying it and giving Nothing if it can't be read (XMonad ends up using
the supplied (Layout a) in that case, but I don't think that's important
for your problem)
I thought the situation with layout modifiers was the problem: I think when
I wrote
<https://github.com/xmonad/xmonad-extras/blob/master/XMonad/Config/Alt.hs
XMonad.Config.Alt in xmonad-extras>, I kept the l type parameters in the
open (no existentials).  I didn't know how to express the idea that m l a is
a layout provided that l a is also a layout, without having being able to
mention the l. IE. I thought you couldn't write a
function wrap such that (wrap Mirror) :: Layout a -> Layout a.
https://gist.github.com/aavogt/6105a670be8dd0c48649118c8f4ed3c5 is a partial solution
I wrote today. The remaining problem is that you have to write
wrap (RLS (\_ -> Sub Dict)) Mirror :: Layout a -> Layout a
when wrap' Mirror :: Layout a -> Layout a would be ideal.

@aavogt, thanks, but the way I see it, applying layout modifiers to Layout a is non-issue. Problems arise only when one needs to prove that there is an instance of Read (Layout a), of which there are none. I don't think your code solves this problem (or maybe I'm just being stupid).
F.ex., try the following with your code:
test :: Layout Window -> IO ()
test l = xmonad $ def{layoutHook = wrappedMirror l}

Of course, one could require an instance in type signature. But that would just postpone the problem:
test :: (Read (Layout Window)) => Layout Window -> IO ()
test l = xmonad $ def{layoutHook = wrappedMirror l}

main' :: IO ()
main' = test $ Layout Full

I could thread a list of types that Layout could encapsulate, but that would require a type-level list, which would, again, require a different signature to (>>) :: m1 a -> m2 b -> m2 b, which essentially takes us out of monad category and into type-indexed monads. And that's something I'd prefer to avoid, for one, and there's an easier solution to the whole problem with type-indexed monads then threading a type list around.

@lierdakil the problem is that you have to write and maintain many functions like wrappedMirror. That seems to be acceptable to you, but I would rather say modify layout Mirror rather than having to find or define a wrappedMirror myself.
This is a way to write your test in a way that doesn't need that Read (Layout a) instance.
test :: Layout Window -> IO ()
test l = xmonad2 $ def{layoutHook = wrappedMirror l}

  where xmonad2 :: XConfig Layout -> IO ()
        xmonad2 cf@ XConfig { layoutHook = Layout l } =
            xmonad cf{ layoutHook = l }

@aavogt, okay, that's me being stupid then.
 (LayoutModifier m a, Read (m a))
      => (forall l. (LayoutClass l a, Read (l a)) => l a -> ModifiedLayout m l a)
      -> Layout a -> Layout a
wrapLM m (Layout la) =
  let (RLS' rS' :: RLS' m a) = RLS' $ const $ Sub Dict
  in case rS' la of
    Sub Dict -> Layout (m la)

which ultimately allows for syntax like this:
main = xmonad' $ do
    modifyLayout' minimize
    modifyLayout' borderResize
    modifyLayout' smartBorders
    modifyLayout' avoidStruts

Alas, at this point I have no idea how to generalize this to
main = xmonad' $ do
    modifyLayout' $ minimize . borderResize . smartBorders . avoidStruts

but that's at least some progress, I guess?

Okay, so I fiddled around a bit, and came to a more concise implementation that relies on typeclasses (or rather, a single typeclass) to build constraint entailment. Here's a gist with a minimal example.
It boils down to a single wrap :: (l a -> m l a) -> Layout a -> Layout a function, and a single instance for each thing like Mirror:
instance CC Mirror a where dict = Dict

Adding instances seems like less of a hassle. Of course it'd be better if it would derive automagically, as is the case with LayoutModifiers, but I found no meaningful way to do so. So if anyone has ideas, I'm eager to hear those. In the meantime, I'll see if I can make a sensible patch to Prime with this.

@lierdakil I like your solution. Here is some template haskell to find LayoutClass instances including those that wouldn't be covered by the CC instance for LayoutModifier. I was expecting more instances to need special treatment, and a dozen hand-written CC instances are probably less work than template haskell (to generate those instances).
Keep in mind that contrib has very rarely added dependencies, so I am going to follow suit and say: don't add the constraints package to the build-depends and instead copy the two data definitions you need. Contrib might still compile with ghcs that are older than -XConstraintKinds, so there's another question of whether the module that this code winds up in should be disabled with ghc<7.5 (or another version). But those details are secondary to having something that is well thought out and works on the configurations that travis is set up to test.

 So something like this works:
modifyLayout (Full |||)
modifyLayout Mirror

and something like this does not:
modifyLayout $ Mirror . (Full |||)

The real problem is it's entirely possible to have pure (or IO) functions that modify layout arbitrarily. So, for example, consider a function
stupidLayoutModifier = Mirror . (Full |||) -- yes, I know it's rather stupid, but it's an example

that's defined somewhere in the depths of xmonad-contrib.
User tries to use it, and typechecker explodes in his face:
modifyLayout stupidLayoutModifier

Couldn't match type ‘l’ with ‘Choose Full l’
  ‘l’ is a rigid type variable bound by
      a type expected by the context:
        LayoutClass l Window => l Window -> Mirror l Window
Expected type: l Window -> Mirror l Window
  Actual type: l Window -> Mirror (Choose Full l) Window

I was able to partially work around the problem by playing with type composition operator (similar to  TypeCompose library). But workaround ends up being rather annoying, e.g. to make above example work, one needs to change it as follows:
modifyLayout $ O . stupidLayoutModifier

In general, if layout modifier function applies n modifiers, O needs to be invoked n-1 times. Which gets tedious pretty fast.
Any insight is welcome.
A toy implementation is available as gist, if anyone wishes to poke at it.

After some more poking, I found a partial solution. It's possible to define a number of instances like this:
class S l f t | f l -> t where
  sq :: (l a -> f a) -> (l a -> t l a)

instance S l (m1 l) m1 where
  sq = id
instance S l (m1 (m2 l)) (m1 :. m2) where
  sq = sq . (O .)
instance S l (m1 (m2 (x l))) (m1 :. m2 :. x) where
  sq = sq . (O .)
instance S l (m1 (m2 (m3 (x l)))) (m1 :. m2 :. m3 :. x) where
  sq = sq . (O .)
instance S l (m1 (m2 (m3 (m4 (x l))))) (m1 :. m2 :. m3 :. m4 :. x) where
  sq = sq . (O .)
...

This seems to beg for an abstraction via recursive instance, but I can't seem to find a proper way to define one.

@lierdakil I figured out how to get those two instances to cover all uses of S here. I used IncoherentInstances so maybe there's still a way to avoid that unsavoury extension. On the other hand the extension seems to be doing the right thing so maybe that part is fine as-is.

@aavogt thank you. I was at my wit's end with this. I actually half-written a TH function to generate an absurd amount of instances like ones above. And I will happily take one incoherent instance over TH any day of the week!

I think I'm at the point where I can't do much more. It's still pretty awkward in places. A few examples:

Need separate functions to execute config transformers that change layout and that do not. I don't think it's possible to work around this. Perhaps if associated type families were less strict w.r.t. overlap. As a result, apply and applyIO have two variants each.
Need to 'squash' a stack of layout modifiers into single composed modifier. There are obviously different implementations for pure and IO functions on config and layout modifiers themselves. I couldn't figure out a way to hide this plumbing (I sincerely doubt it's possible), so it needs to be explicit. Thankfully, it's not required often.

However, I switched my config to use it, and it's not that bad. Besides, it's much easier to embed IO actions when >>= and lift are available in Prime.
Current work is available here. Not really sure if I should create a PR, due to reasons explained above. I will probably take a break from fighting with typechecker for now.

Since XMonad.Config.Prime is deprecated (see #679), I'm gonna close this issue.


inject our own marks into the browser to identify start and end of the benchmarks

@maxrjones / @katamartin,
as we discussed earlier today, this PR adds functionality for injecting our own marks into the browser. I'm calling these benchmark:start and benchmark:end. As you can see, we are able to measure the overall runtime (shown as benchmark in the timeline).
The benchmark:end identifies when we observe the onIdle callback is invoked.
https://github.com/carbonplan/benchmark-maps/assets/13301940/ecbd4718-bb4f-49be-b553-4e84c1f5b816

Love how neatly the performance.mark events integrate into the metrics panel!
From the screen recording, it looks like the the map had not finished rendering by the time benchmark:end was marked. Is that right? If so, could this have to do with the units/offsetting in the different time measures? Or is the idle callback simply getting invoked too early?

From the screen recording, it looks like the the map had not finished rendering by the time benchmark:end was marked. Is that right? If so, could this have to do with the units/offsetting in the different time measures? Or is the idle callback simply getting invoked too early?

It would be worthwhile if @andersy005 could confirm this, but I think this is an artifact of how the screenshots are rendered in the performance UI. I see the same thing zoomed out for some runs, but when I zoom in the first screenshot after benchmark:end shows a completed map.
Why do you think .zmetadata is requested before the start of the benchmark?

Why do you think .zmetadata is requested before the start of the benchmark?

Good question! Based on the way maps + the prototype are implemented, we should at least be confident that .zmetadata will not be requested until Display button is clicked. Looking at main.py, it seems like there might be a meaningful delay between these two events:
  page.click('//div[text()="Display"]/following-sibling::button')

  page.evaluate(
      """
      () => (window.performance.mark("benchmark:start"))
                """
  )

Reading the Node.js docs for the Page class, it looks like these are both natively async methods. Because we're using the playwright.sync_api, it seems likely that these methods are both getting automatically awaited, which would be responsible for the delay we're seeing. I think we could either swap the order of these events (and risk capturing the time between benchmark:start and activating the Display button) and/or switch to the playright.async_api and explicitly await the actions we want to occur in sequence and omit in cases where we don't (like this combination of events).

From the screen recording, it looks like the the map had not finished rendering by the time benchmark:end was marked. Is that right? If so, could this have to do with the units/offsetting in the different time measures? Or is the idle callback simply getting invoked too early?

i just recorded two runs, and i'm noticing different behaviors:
in the first screenshot, the idle callback seems to have been invoked early.

for the this run, i'm seeing a finished map.

I think we could either swap the order of these events (and risk capturing the time between benchmark:start and activating the Display button)

here are the results of this swap:

and/or switch to the playright.async_api and explicitly await the actions we want to occur in sequence and omit in cases where we don't (like this combination of events).

👍🏽  i'm happy to look 👀 into this...

After letting that digest for a little bit, I think that switching to playright.async_api and confirming which behavior we want for each of invocations of playright methods feels like the safer path. I think in that world, we should also probably still swap the two actions we're talking about for good measure.

i just recorded two runs, and i'm noticing different behaviors:

Are these behaviors confirmed when you "zoom in" on the screenshots in the trace like @maxrjones alluded to?

here's the latest run with async API:
https://github.com/carbonplan/benchmark-maps/assets/13301940/74b5bf5f-3750-4dcb-89b7-7e26d00b2304


Change tracking is considering edm:Date and edm:DateTime as same objects

Hello Colleagues,
We have change tracking implemented for entity Consignment in our project. There is a field EDI arrival date  which is included in Change History.
Issue is when it is updated, its time format changes in the change history. Also there is an another field EDI arrival time  which is also updated but it is not coming up along with EDI arrival date in change history tab.
I am attaching the snapshot for reference.

Hello @I590232 this is due to the new DB drivers of CAP Version 8 have changed their behavior. We are working to find a fix for this.
Best,
Nick

Hi @I590232 ,
Could you provide an app that can reproduce this issue? This way, I can understand how it arises. In addition, I guess you might be using @cap/js-sqlite or @cap/js-hana, which could lead to similar problems   .
Best Regards,
Wenjun


Create a Ruby gem wrapping libtorrent

All the available Ruby torrent gems are unmaintained. Create a new Ruby gem that wraps libtorrent. https://blog.jcoglan.com/2012/07/29/your-first-ruby-native-extension-c/
The torrent_client class can then ensure libtorrent and the new gem are installed, and use them for downloads and checking target file consistency.

Done at https://github.com/MikaelSmith/torrent_client.


Request: Diff with tracking branch

I do WIP commits on branches to make sure changes are kept when updating from tracking branch (i.e. rebase 'feature' on 'master'), making changes to the working tree, etc. Would be nice to have a combined diff to note every change, including unmerged commits, from current to tracking branch. Beyond the scope of git-diff maybe?

I don't think we're likely to prioritize work on this anytime soon. I'm closing out old issues because we've moved this package into the main Atom repository. FWIW, This may be something that we would accept a PR for, if you submitted one that met our contributing guidelines. Thanks!


added -m/-metricsFile flag to output duplication metrics

I decided to patch it myself. I'm not too familiar with C++ so I've basically imitated how the other files are opened, written to and closed. Example output:
TOTAL_READS  DISCORDANT_READ_PAIRS  SPLIT_READS  UNMAPPED_READS  UNMATED_READS  DUPLICATE_READS  PCT_DUPLICATE_READS
4596         0                      0            0               0              1229             26.74

cheers

Is this interesting to you? I think it's useful for many users.

BUMP

I would be interested in this as well.

Hello?

+1 for this feature..

I'm gonna close this, since it's been four years without a response.


HBASE-26107 MOB compaction with missing files catches incorrect exception

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
3m 59s
Docker mode activated.


ISPN-8870 Remove logger inheritance to reduce class metadata (more)

JCache loggers
Removed some more unused methods

https://issues.jboss.org/browse/ISPN-8870

merged! thanks


1.11.2 Dimension Portal Structure NPE in survival single player

https://www.dropbox.com/s/a3bnpdl73raaofc/fml-client-latest.log?dl=1
https://www.dropbox.com/s/za1e72x4d1kfulj/latest.log?dl=1

Thanks. I will check this out and see what's wrong

Thank you.
Konitor

Ah could you add your lostcity/general.cfg too?
I'd like to see if something went wrong there

Never mind
I know what's wrong

Will be fixed next release (in a few hours from now)

Sorry was afk. Do you work day and night?
https://www.dropbox.com/s/fpa8vwj7u58cmya/general.cfg?dl=1
Just to be complete....


Source Distributions for mitmproxy_rs

Problem Description
On Windows, mitmproxy_rs currently bundles windows-redirector.exe for OS proxy mode. This executable is built as part of CI, and then copied over into the Python package in build.rs. @emanuele-em is in the process of introducing similar shenanigans on macOS for traffic redirection (where it's going to be a Swift app).
As part of https://github.com/mitmproxy/mitmproxy/issues/6299, I realized that this actually is a problem for source distributions (sdists): As we currently build sdists on Linux, windows-redirector.exe is not built during CI and the produced sdist is incomplete for Windows users. The same problem will appear on macOS. We need to fix that somehow.
Some proposals to address this:

We could somehow add the platform-specific sources to the sdist and have platforms-specific build steps in the sdist. The downside of this approach is that we would basically need to hijack maturin build to add custom steps around it. I'd outright reject this approach due to the complexity it brings.
We could make them mitmproxy_rs CI a two stage process: The first stage builds the platform-specific parts (windows-redirector.exe on Windows, macos-redirector.app on macOS, etc.), and then the second stage pulls in all these artifacts and includes them all in the sdist. One downside here is that the sdist is partially a binary distribution and not really a source distribution. This would be the easiest approach for us, but downstream would rightfully complain I guess.
Another alternative would be to start producing mitmproxy_rs_windows/mitmproxy_rs_macos packages containing the platform-specific bits, and have mitmproxy_rs declare conditional dependencies on these. This would keep things nicely separated, and we could actually avoid the build.rs lets-copy-over-binaries nonsense. The downside is that we have more Python packages and more related compexity.

@mitmproxy/devs, @emanuele-em, any thoughts?

I don't have a strong opinion but the most natural solution seems to be the third one, keeping things strictly separate sounds good even for future maintenance, more complexity (although they are actually quite isolated from each other) but also more clear and more clean

Yeah, that's probably the way to go. Let me see how that leads us on Windows before we fully commit. :)

Ok, turns out that building a windows-redirector Python package individually is more complex than expected:

maturin would support distributing binary applications (https://www.maturin.rs/bindings.html#bin).
Unfortunately I don't think we can use this:

We need to place WinDivert.dll and friends in the same directory and there does not seem to be a good way to do that (maturin places the binary directly in Python's Scripts/bin directory).
PyInstaller does not include these scripts in its distribution (at least I think - haven't looked closer because of the WinDivert issue).
maturin won't help us for the (Swift-based) macOS redirector anyhow.

The next thing I looked at is Hatch with custom build hooks[^setuptools]: https://discuss.python.org/t/custom-build-steps-moving-bokeh-off-setup-py/16128/3. It's relatively straightforward to build a working bdist here. However, it is not as straightforward to build a working sdist because our Rust executables depend on local dependencies (the top-level "mitmproxy" crate) and Git dependencies (e.g. @emanuele-em's security-framework fork). We need to include at least the local dependencies in the sdist to build. I see two options here going forward:

Only provide a bdist on PyPI and let downstream build manually from the Git repo.
Run cargo vendor[^cargo-package] and then produce an sdist with that. The downside with this is that we will end up with a very manual build script...

I'm leaning towards only providing bdists for macOS and Windows. For the macOS redirector, the binaries need to be signed to work anyhow (so the sdist isn't very useful). For Windows, we don't have any downstream repackaging anyhow. For Linux, we don't have platform-specific stuff yet.
/cc @decathorpe - thoughts on this with your Fedora packaging background?
[^cargo-package]: We really don't want to ship a copy of crates.io in the sdist, but cargo package does not work for us because of the abovementioned Git dependencies which it refuses to handle.
[^setuptools]: Setuptools would probably work as well, but Hatch was more straightforward to get started with.

After pondering on this a bit more, I feel that the most useful approach for now would be to provide:

bdists for all major platforms,[^plats]
an sdist that works on Linux[^change], and
clear error messages in build.rs for all other platforms.

The reasoning behind that is as follows:

On macOS, pip install --no-binary will not work no matter what we do, because the redirector needs to be signed.[^oof]
On Linux, producing an sdist is still easy for us and it makes downstream's job easier.
On Windows, setting up working bdists would require a lot of complexity - and there is no real interest in building from source here anyways.

[^change]: This may change in the future similar to macOS/Windows if we require platform-specific external binaries.
[^plats]: Windows x64, Linux x64, Linux aarch64, macOS x64, macOS aarch64
[^oof]: We could include the binary in the sdist to overcome that, but that defeats the purpose of a source distribution in the first place.

This has mostly been fixed now:

mitmproxy-rs now is a pure Rust crate again that can simply be built with maturin. We're shipping an sdist compatible with all platforms, there is no build.rs anymore.
mitmproxy-rs now conditionally depends on the new mitmproxy-windows and mitmproxy-macos Python packages, which contain the platform-specific resources.
mitmproxy-windows and mitmproxy-macos are distributed as binary wheels only on PyPI.
mitmproxy-windows is LGPL (because of WinDivert), everything else is MIT.
For security-framework, we'll wait a bit; it looks like upstream is figuring out things and they'll have a new release eventually. :)


Add starknet auth

WIP

Will be kind of blocked by https://github.com/OpenZeppelin/cairo-contracts/issues/328 :(

Did we considered to not use any authenticator for StarkNet tx? And use space contract directly, since we don't really need to authenticate anything with a StarkNet tx

Did we considered to not use any authenticator for StarkNet tx? And use space contract directly, since we don't really need to authenticate anything with a StarkNet tx

We still to verify that the caller is the same as proposer_address / voter_address, so we indeed need to go through an authenticator I believe :(