Pull Requests listing is way slower on Windows

Gitea version (or commit ref): 1.14
Git version: 2.26.2
Operating system: Windows Server 2016 Standard
Database (use [x]):

[ ] PostgreSQL
[ ] MySQL
[x] MSSQL
[ ] SQLite

Can you reproduce the bug at https://try.gitea.io:

[ ] Yes (provide example URL)
[x] No

In 1.13.6 it was fast when going in the Pull Requests tab.
We've migrated to 1.14 today (2024-04-11) and now it's 20x slower.
We have "ISSUE_PAGING_NUM" set to 50. And it takes 30 seconds to load the pull requests in 1.14. In 1.13.6, it took less than 1 second.
I put back the default value to 10, it's better but it still takes 5-6 seconds. I think there is a missing index or something not retro-compatible.
How can I help you to investigate the issue ?
Kind regards.

It does not seems to be related to the DB. I've tried to move gitea service in another faster computer. The pull requests listing was 2x faster : 30 seconds => 15 seconds.
Moreoever, I've set SQL.Logs to true to check which query could take too much time, and actually none takes more than 1-2 ms.
So i'm pretty sure it's not comming from DB, but more from something new done after having retrieved the pull requests, while compiling informations.

Could you enable pprof and paste the profile file

Yes, here you are:
pprof.txt
You can see this:
`
2021/04/12 12:06:34 Completed GET /user/events 200 OK in 3.2254295s
/source/routers/routes/base.go:45 (0x299e785)
/usr/local/go/src/net/http/server.go:2069 (0xf5584a)
/source/vendor/github.com/go-chi/chi/middleware/strip.go:30 (0x2998b27)
/usr/local/go/src/net/http/server.go:2069 (0xf5584a)
/source/vendor/github.com/chi-middleware/proxy/middleware.go:37 (0x2994053)
/usr/local/go/src/net/http/server.go:2069 (0xf5584a)
/source/routers/routes/web.go:66 (0x29a2367)
/usr/local/go/src/net/http/server.go:2069 (0xf5584a)
/source/vendor/github.com/go-chi/chi/mux.go:87 (0x23140d0)
/source/modules/web/route.go:268 (0x27684fa)
/source/vendor/github.com/gorilla/context/context.go:141 (0x19657ba)
/usr/local/go/src/net/http/server.go:2069 (0xf5584a)
/usr/local/go/src/net/http/server.go:2887 (0xf58e89)
/usr/local/go/src/net/http/server.go:1952 (0xf5430c)
/usr/local/go/src/runtime/asm_amd64.s:1371 (0xbf65e0)
2021/04/12 12:06:34 Started GET /Ramasoft/Ramasoft.NET/pulls for [::1]:53100
/source/routers/routes/base.go:36 (0x299e34a)
/usr/local/go/src/net/http/server.go:2069 (0xf5584a)
/source/vendor/github.com/go-chi/chi/middleware/strip.go:30 (0x2998b27)
/usr/local/go/src/net/http/server.go:2069 (0xf5584a)
/source/vendor/github.com/chi-middleware/proxy/middleware.go:37 (0x2994053)
/usr/local/go/src/net/http/server.go:2069 (0xf5584a)
/source/routers/routes/web.go:66 (0x29a2367)
/usr/local/go/src/net/http/server.go:2069 (0xf5584a)
/source/vendor/github.com/go-chi/chi/mux.go:87 (0x23140d0)
/source/modules/web/route.go:268 (0x27684fa)
/source/vendor/github.com/gorilla/context/context.go:141 (0x19657ba)
/usr/local/go/src/net/http/server.go:2069 (0xf5584a)
/usr/local/go/src/net/http/server.go:2887 (0xf58e89)
/usr/local/go/src/net/http/server.go:1952 (0xf5430c)
/usr/local/go/src/runtime/asm_amd64.s:1371 (0xbf65e0)
2021/04/12 12:06:47 Completed GET /Ramasoft/Ramasoft.NET/pulls 200 OK in 13.7507922s
/source/routers/routes/base.go:45 (0x299e785)
/usr/local/go/src/net/http/server.go:2069 (0xf5584a)
/source/vendor/github.com/go-chi/chi/middleware/strip.go:30 (0x2998b27)
/usr/local/go/src/net/http/server.go:2069 (0xf5584a)
/source/vendor/github.com/chi-middleware/proxy/middleware.go:37 (0x2994053)
/usr/local/go/src/net/http/server.go:2069 (0xf5584a)
/source/routers/routes/web.go:66 (0x29a2367)
/usr/local/go/src/net/http/server.go:2069 (0xf5584a)
/source/vendor/github.com/go-chi/chi/mux.go:87 (0x23140d0)
/source/modules/web/route.go:268 (0x27684fa)
/source/vendor/github.com/gorilla/context/context.go:141 (0x19657ba)
/usr/local/go/src/net/http/server.go:2069 (0xf5584a)
/usr/local/go/src/net/http/server.go:2887 (0xf58e89)
/usr/local/go/src/net/http/server.go:1952 (0xf5430c)
/usr/local/go/src/runtime/asm_amd64.s:1371 (0xbf65e0)`

Have you load the pull request list page when you enable pprof? It seems there is no any functions about that.

Yes I did, but if it didn't worked, then i'm making it again.
Here you are:
profile.log

This is another profile.log:

I started gitea service : gitea.exe
I logged in with my user
I went on the specific repository (Ramasoft.NET)
I clicked on Pull Requests Tab
I went on http://localhost:6060/debug/pprof/, clicked on "profile" which downloaded me the following file:
profile.log

And this is what I can see in the page loading times:

And this is our app.ini file (a little bit modified to remove sensible info)
app.txt

Could you also paste your gitea logs?

Yes, of course. Here you are:

I changed the app.ini I provided you just earlier by defining the level of Logs to Trace
I started gitea service : gitea.exe
I logged in with my user
I went on the specific repository (Ramasoft.NET)
I clicked on Pull Requests Tab
I went on http://localhost:6060/debug/pprof/ and saved everything that was in this page.
Killed gitea service

These are the files:

Logs:  log.txt
pprof files:
block.txt
cmdline.txt
fullGoRoutineStackDump.txt
goroutine.txt
heap.txt
mutex.txt
profile.log
threadcreate.txt
trace.log
allocs.txt

And this is how much time it took to go to Pull Request Tab:

Kind regards,
David.

Thanks for extensive logging files and pprof files but please could you read https://docs.gitea.io/en-us/logging-configuration/#debugging-problems
STACKTRACE_LEVEL should generally be left unconfigured (and hence kept at none). There are only very specific occasions when it useful.

Your logs are essentially unreadable because of this setting.

Ok, no prob. I change the logging settings to the one in your documentation: Level=debug, and SQL.Log = False.
Thank you for the explanation of how pprof works. I was using it improperly.
This is what i did:

Change the app.ini to have the correct log level and other settings more suited for debugging
Run gitea service
Run your query (wget...)
Refresh the Pull Request tab

These are the files I can give to you:
30s-cpuprofile.log
logs.txt
It is ok now ?
Kind regards,
David.

One quick thought.
Most Windows users have seriously long path environment variables. Windows doesn't hash this path unlike Linux and Mac.
We're mostly using git directly these days.
Set your [git] PATH correctly as this could be a cause of serious slow downs.

Your logs still have stacktraces and are still missing the router logs. What is the contents of your [log] section?
Unfortunately it's now too late for me to look at your pprof output so I'll take a look tomorrow.

One quick thought.
Most Windows users have seriously long path environment variables. Windows doesn't hash this path unlike Linux and Mac.
We're mostly using git directly these days.
Set your [git] PATH correctly as this could be a cause of serious slow downs.

I'll try that.
To be noted: We didn't have any slowness in 1.13.6.

1.14 uses git itself, whereas 1.13 used a go library for some git operations, and git for the remaining operations. It is possible to build 1.14 using the go based library, it just isn't the default.

Your logs still have stacktraces and are still missing the router logs. What is the contents of your [log] section?
Unfortunately it's now too late for me to look at your pprof output so I'll take a look tomorrow.

Sorry for the inconvenience. Router was set to its default value "Console" and not File, that's why I could not provide you the needed information. I fixed this and also remove stack trace.
Here are the files:
router.log
30s-cpuprofile.log
log.txt

1.14 uses git itself, whereas 1.13 used a go library for some git operations, and git for the remaining operations. It is possible to build 1.14 using the go based library, it just isn't the default.

Is there a way to retrieve easily this version of gitea 1.14 using go library for git ops ? To test and see if it solves the slonesses.

Have you tried setting the path to git yet? I would be seriously interested to know if that improves things.
Gogit variants are set at build-time so you would have to build gitea yourself adding the tag gogit to your tags. (Likely this would be TAGS="gogit bindata")

The Log.txt you have provided is only at info level. We need debug logs.
Please could you read the link I sent you. (Actually also give us a copy of your app.ini - you never know if it's going to be enlightening.)

Have you tried setting the path to git yet? I would be seriously interested to know if that improves things.
Gogit variants are set at build-time so you would have to build gitea yourself adding the tag gogit to your tags. (Likely this would be TAGS="gogit bindata")

Not yet. I'm doing that in minutes, and tell you what afterwards :)

The Log.txt you have provided is only at info level. We need debug logs.
Please could you read the link I sent you. (Actually also give us a copy of your app.ini - you never know if it's going to be enlightening.)

Mmm, that's what is in your documentation. The log section is the following:
[log]
ROOT_PATH            = F:/git server/log
; Either "console", "file", "conn", "smtp" or "database", default is "console"
; Use comma to separate multiple modes, e.g. "console, file"
MODE                 = file
; Buffer length of the channel, keep it as it is if you don't know what it is.
BUFFER_LEN           = 10000
REDIRECT_MACARON_LOG = false
MACARON              = file
; Either "Trace", "Debug", "Info", "Warn", "Error", "Critical", default is "Info"
ROUTER_LOG_LEVEL     = Info
ROUTER               = file
ENABLE_ACCESS_LOG    = false
ACCESS_LOG_TEMPLATE  = {{.Ctx.RemoteAddr}} - {{.Identity}} {{.Start.Format "[02/Jan/2006:15:04:05 -0700]" }} "{{.Ctx.Req.Method}} {{.Ctx.Req.RequestURI}} {{.Ctx.Req.Proto}}" {{.ResponseWriter.Status}} {{.ResponseWriter.Size}} "{{.Ctx.Req.Referer}}\" \"{{.Ctx.Req.UserAgent}}"
ACCESS               = file
; Either "Trace", "Debug", "Info", "Warn", "Error", "Critical", default is "Trace"
LEVEL                = Debug
; Either "Trace", "Debug", "Info", "Warn", "Error", "Critical", default is "None"
STACKTRACE_LEVEL     = None

So Level = Debug and Router_Log_Level is set to Info like in the doc.
Am I missing something ?
You can find the app.ini in the first comments. I just have changed the log section with the previous pasted infos.

I've set the git path:
[git]
; The path of git executable. If empty, Gitea searches through the PATH environment.
PATH                          = "C:\Program Files\Git\bin\git.exe"

It's working, but no performance improvement.
I also set it in the PATH environment.
And tried also by restarting the computer.

Argh. You're not supposed to copy app.example.ini as your app.ini!
It doesn't work like that.
You're only supposed to use it for reference to know how to set things.
That's it tomorrow morning I'm going to open a PR to remove it - it seems no one can be trusted with the damned thing.
Remove the log.XXX sections. Remove settings in the log section that aren't mentioned in the documentation.
Look through the rest of your app.ini if it's unchanged from the app.example.ini consider if you really mean it to be set.
I think the queue sections are also incorrect. Certainly the Cron sections are likely to be unhelpful.
I'm not convinced that any of these things are causing your problem but they're unhelpful.

Shame about the git path thing - we're sort of stuck though until I can pprof and you give me some debug logs.

From 30s-cpuprofile.log, it seems most time is in pull.GetLastCommitStatus, it will open git repository to get compare information which v1.13 will not. This is introduced from #13975

Argh. You're not supposed to copy app.example.ini as your app.ini!
It doesn't work like that.
You're only supposed to use it for reference to know how to set things.
That's it tomorrow morning I'm going to open a PR to remove it - it seems no one can be trusted with the damned thing.
Remove the log.XXX sections. Remove settings in the log section that aren't mentioned in the documentation.
Look through the rest of your app.ini if it's unchanged from the app.example.ini consider if you really mean it to be set.
I think the queue sections are also incorrect. Certainly the Cron sections are likely to be unhelpful.
I'm not convinced that any of these things are causing your problem but they're unhelpful.
Shame about the git path thing - we're sort of stuck though until I can pprof and you give me some debug logs.

Thanks for your investigation. I'll do that :)

From 30s-cpuprofile.log, it seems most time is in pull.GetLastCommitStatus, it will open git repository to get compare information which v1.13 will not. This is introduced from #13975

Is there a way to disable this functionality to see if it solves the slownesses ?

Right because I want to rule out git as the causative problem here is a build of 1.14.0 with the gogit backend:
https://eldritchkitty.com/~andrew/gitea-1.14-gogit-windows-4.0-amd64.exe

Thank you for your build, but I have a 404 at write-time.

sorry try the new link

It's faster with the version you provided me: 14 seconds (1.14) => 9 seconds (your version). So I think your change are a good start.
But 1.13.6 was loading pull requests in less than 1 second.

Do you need pprof files ?

pprof would be useful I think

With 15447(git): https://eldritchkitty.com/~andrew/gitea-15447-git-windows-4.0-amd64.exe
=> from 14 seconds to 1,6 seconds (10x faster)
With 15447(go-git): https://eldritchkitty.com/~andrew/gitea-15447-git-windows-4.0-amd64.exe
=> from 14 seconds to 0,15 seconds (100x faster)
So the 15447 with go-git is way faster, and it's the same speed as in 1.13.6.

OK - The pprofs for the 2 new ones would be helpful - I think the issue is likely going to be that GetCommit() is being called far more often that I expected and hence the extra memory use of go-git will be helpful.

15447(git):
15447-git-30s-cpuprofile.log
15447(go-git):
15447-gogit-30s-cpuprofile.log

go-git's problem is at memory not cpu.

OK so this one it looks like we could reduce the calls to show-ref by collecting them together and calling show-ref for all of the refs.

OK another improvement:
https://eldritchkitty.com/~andrew/gitea-15447-15455-git-windows-4.0-amd64.exe
https://eldritchkitty.com/~andrew/gitea-15447-15455-gogit-windows-4.0-amd64.exe

15447-15455-git:
=> from 14 seconds to 1,5 seconds (10x faster)
=> it seems faster than 15447-git, which turned around 1,6 seconds
15447-15455-git-30s-cpuprofile.log
15447-15455-gogit:
=> from 14 seconds to 1,5 seconds (10x faster)
=> it's slower than 15447-git, which turned around 0,15 seconds. I was surprised, so I run again 15447-gogit, and yes the 15447-gogit is faster than the one with 15455. Maybe a problem with the version built ? (15447-15455-gogit)
15447-15455-gogit-30s-cpuprofile.log

looks like the go-git one is actually still a git version... weird.

Can you rebuilt the 15447-15455-gogit ? I will test it.
And also, if you want to make some other tests, do not hesitate.
Also, I found some slowness loading a specific PR, which was not that slow in 1.13.6. This PR has a lot of change requests, comments. I will create a specific issue ticket for this one, but maybe the subject is related.

Can you just confirm for me that you are not running with a random AV thing that's slowing down git and that your path for the git stuff is actually reasonable? Seriously set your environment PATH so that git's binaries etc are first in order to be found.
And can you tell me how many pulls you're trying to show?

I've rebuilt the go-git one but I don't expect that to improve or change at all as these PRs aren't hitting that.

I'm a little confused by the 15447-15455-git-30s-cpuprofile.log - it's a shame that it hasn't made much difference - I made a change that should have reduced the time spent in GetRefCommitID  if the refs weren't packed in a pack. I guess is that your pull refs are packed but I'm not certain. It would be useful to know if that's the case.
The reality is that the reason why the go-git one is faster is that it is just loading all the references in one shot - and keeping them in memory. Go-Git is really non-frugal with memory in general - but here it's potentially not uncalled for as realistically the number of PR heads is going to be somewhat constrained.
Now I think it may just be worth doing something similar - getting all the refs in one go as they're likely to be less than 4kb in general.

Can you just confirm for me that you are not running with a random AV thing that's slowing down git and that your path for the git stuff is actually reasonable? Seriously set your environment PATH so that git's binaries etc are first in order to be found.

Nope, on that computer there is only windows defender.
The path for git is short, you can see it in that comment: https://github.com/go-gitea/gitea/issues/15401#issuecomment-818282388

And can you tell me how many pulls you're trying to show?

50

I've rebuilt the go-git one but I don't expect that to improve or change at all as these PRs aren't hitting that.

With your rebuilt, it's now turning around 0,1s. So the 15447-15455-gogit is the fastest version you have provided me, right now.
Comparatively, the 15447-gogit was running the pull requests list in around 0,15s.

I'm a little confused by the 15447-15455-git-30s-cpuprofile.log - it's a shame that it hasn't made much difference - I made a change that should have reduced the time spent in GetRefCommitID if the refs weren't packed in a pack. I guess is that your pull refs are packed but I'm not certain. It would be useful to know if that's the case.
The reality is that the reason why the go-git one is faster is that it is just loading all the references in one shot - and keeping them in memory. Go-Git is really non-frugal with memory in general - but here it's potentially not uncalled for as realistically the number of PR heads is going to be somewhat constrained.
Now I think it may just be worth doing something similar - getting all the refs in one go as they're likely to be less than 4kb in general.

How can I help you to investigate this ?

By path I mean your environment path.
Git on Windows calls a lot of sub programs so it also needs to look these up on the path.

To check the refs pack status you would have to go in to the git repository directly and check which refs/pulls/XXX/head were files versus how many are returned from git for-each-ref refs/pulls (it might be pull rather than pulls though I can never remember)

By path I mean your environment path.
Git on Windows calls a lot of sub programs so it also needs to look these up on the path.

Yes it's already done. Please check the comment link.
So it's done in app.ini ans environment path.

To check the refs pack status you would have to go in to the git repository directly and check which refs/pulls/XXX/head were files versus how many are returned from git for-each-ref refs/pulls (it might be pull rather than pulls though I can never remember)

I'll try that later and tell you what.

still not completely solved I think

In my opinion with: 15447-15455-gogit, we have a solution for my problem.
But, we need to have both builds available for one who needs gogit in order to have something fast. Or maybe a way to switch between gogit and git ?
For "To check the refs pack status you would have to go in to the git repository directly and check which refs/pulls/XXX/head were files versus how many are returned from git for-each-ref refs/pulls (it might be pull rather than pulls though I can never remember)". I didn't have yet the time to check, sorry. I will have time on the weekend.

@ramasoft-dv was this fixed on 1.15?

Nope. I have 1.15.7 and it still fails.

On 2021-12-15 13:29, zeripath wrote:

@ramasoft-dv [1] was this fixed on 1.15?
--
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub [2], or unsubscribe
[3].
Triage notifications on the go with GitHub Mobile for iOS [4] or
Android [5].

Links:
[1] https://github.com/ramasoft-dv
[2]
https://github.com/go-gitea/gitea/issues/15401#issuecomment-994745709
[3]
https://github.com/notifications/unsubscribe-auth/ARREQNLKGATBRCAWGJP6RJLURCCZJANCNFSM42XVLTMQ
[4]
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
[5]
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub

Any chance you could try #17792 to see if that finally solves the issue?

Sorry that is #17992

No more feedback, I think this issue can be closed, feel free to reopen if there are more clues.


Auto-Resize Support?

Does not respond to resizing of browser.

You'll probably want to take a look at the RESIZE SCALE_MODE.
You can use the setupGameSize script to change the SCALE_MODE that'll be used. README

@rroylance
yeah thanks. RESIZE Scale mode is what Im looking.. Thanks
I just need to handle all my other UI to respond accordingly when screen size is changed.


Detect arch like distros in /etc/os-release check already

Even though /etc/arch-release is checked for existence as a fallback and is present on an - up to date as of writing - Manjora system, for some reason the arch variable is not set to 1. This fixes it, since os-release contains ID_LIKE=arch on Manjaro.

It is mysterious to me why the test -f fallback does not work but this proves that the "earlier" check for os-release does work with the fix:
Check results in "Unsupported platform" output:
$ mppinit

MusicPlayerPlus initialization has previously been performed
and there is an existing MusicPlayerPlus configuration.

Preserve existing, Overwrite existing, Skip ? (p/o/s) p

Initializing MusicPlayerPlus, please wait ...Unsupported platform

        No mpcplus release asset found for this platform ...
        No mpplus-bliss release asset found for this platform ...
...
...

With fix:
$ mppinit

MusicPlayerPlus initialization has previously been performed
and there is an existing MusicPlayerPlus configuration.

Preserve existing, Overwrite existing, Skip ? (p/o/s) p

Initializing MusicPlayerPlus, please wait ...
        Installing mpcplus ...
        Installing mpplus-bliss ...
...
...

For reference. These are the files on the system being tested on:
$ cat /etc/os-release
NAME="Manjaro Linux"
PRETTY_NAME="Manjaro Linux"
ID=manjaro
ID_LIKE=arch
BUILD_ID=rolling
ANSI_COLOR="32;1;24;144;200"
HOME_URL="https://manjaro.org/"
DOCUMENTATION_URL="https://wiki.manjaro.org/"
SUPPORT_URL="https://forum.manjaro.org/"
BUG_REPORT_URL="https://docs.manjaro.org/reporting-bugs/"
PRIVACY_POLICY_URL="https://manjaro.org/privacy-policy/"
LOGO=manjarolinux
$
$ cat /etc/arch-release
Manjaro Linux
$

Fixes an issue described in https://github.com/doctorfree/MusicPlayerPlus/discussions/13


postgresql: timestamp resolution bugs break parallel builds

Continues #51093 .
Background: on macOS, parallel builds of postgres would try to build libpgtypes.a, trip over each other and error out.
On the postgres mailing list, Tom Lane figured out that:

APFS has sub-second timestamp resolution
GNU Make can use sub-second timestamps if the filesystem supports them
ranlib truncates timestamps to the nearest second

Thomas Munro identifies a possible cause

Therefore, the .a file appears to be built before the .o files that comprise it
Therefore, multiple builds of libpgtypes.a are kicked off, trip over each other and error out.

GNU Make has a special target, .LOW_RESOLUTION_TIME, that can be used in this situation.
I won't have time to dig much further on this, but thought I'd log the issue for completeness. Ping @delroth , who wasn't thrilled at the blunt hammer of disabling parallel builds.

I wonder if we should just patch this in cctools' ranlib (using futimens/utimensat instead of utime when available). This is very broken and I'm guessing could lead to bad parallel builds across all packages.

Confirmed with a simple repro:
>>> import os
>>> os.utime('test.a', None)
>>> def test():
...   os.utime('test.a', None)
...   t1 = os.stat('test.a').st_mtime
...   os.system('ranlib test.a')
...   t2 = os.stat('test.a').st_mtime
...   return t2 - t1
...
>>> [test() for i in range(10)]
[-0.8948380947113037, -0.9246668815612793, -0.9467489719390869, -0.959259033203125, -0.9685111045837402, -0.9778580665588379, -0.9862489700317383, 0.0052928924560546875, -0.003030061721801758, -0.011502981185913086]

This is pretty ridiculous...

Yikes. Why doesn't it break more parallel builds? I posted this speculation (about the postgresql bug) to their mailing list:

This bug seems to be tickled because two different Makefiles use attempt to build the same target at the same time, using a tool (macOS libtool/ranlib, albeit through a recursive $(MAKE) invocation) that doesn't support subsecond timestamps, on a filesystem that does (APFS). That's a bit of a corner case. I speculate that building a static library with a nonrecursive Makefile would only kick off one build of the .a file, because make will only invoke the command once as it walks the DAG of dependencies.

Another option: any reason to not use ar/ranlib from binutils instead of using Apple's? I'm not fully aware of the particularities of Darwin so maybe there's an obvious reason why this is impossible. Might be worth giving it a shot.
If it's not possible, patching cctools' ranlib should be pretty easy anyway.

We don’t use cctools directly from apple but via a fork:
https://github.com/tpoechtrager/cctools-port
If this is causing the issue, i would open a pr there as it will probably effect many more tools!

The issue is in Apple's upstream code: https://github.com/opensource-apple/cctools/blob/master/misc/libtool.c#L2779

Yes but i think @tpoechtrager would gladly accept a patch to fix this (at least quicker than apple)

I've started working on this but right now our build environment targets builds at macOS 10.10, which means utimensat is not available at build time :( Not completely sure how that can be fixed unfortunately, I know close to nothing about macOS.

Actually thinking more about it we can probably just build cctools-port to target 10.13 (through $MACOSX_DEPLOYMENT_TARGET) -- since the one feature requiring 10.13 (utimensat) in that package is runtime-optional through weak linking, I think the builds will still work on older versions on MacOS. Will need to give it a shot.

That might be a good reason to up the minimum version. Right now we set 10.11 as the minimum but I am planning to up it to 10.12 with #47678. Maybe bumping all the way to 10.13 wouldn't be that bad of an idea? 10.13 was released just a little over a year ago so it's somewhat of a quick turnaround for us. I would prefer to support up to 2 years back.
You can try it out by adding the symbol to this list:
https://github.com/NixOS/nixpkgs/blob/master/pkgs/os-specific/darwin/apple-source-releases/Libsystem/system_kernel_symbols
As an alternative, I'm thinking maybe we should just disable nanosecond precision in make? I think you can override it with make_cv_file_timestamp_hi_res=no.

https://github.com/tpoechtrager/cctools-port/pull/62 has been merged.

Thanks for the update @alyssais, and @delroth for making the change. What else need to happen here before this gets closed?

Let's wait for #51408 to hit master.


[$250] Migrate AdditionalDetailsStep.js to function component

Class Component Migration
Filenames

[ ] src/pages/EnablePayments/AdditionalDetailsStep.js

type: React.Component
has state values: false
has refs: false
has context: false
lifecycle methods: none

Task

We currently have some class components in our codebase that we would like to refactor to a function component.
Here's a link with some general advice on how to refactor a class component to a function component: https://react.dev/reference/react/Component#alternatives
If you need additional guidance, please ask in #expensify-open-source
Test for any regressions and verify that there are no breaking changes

Upwork Automation - Do Not Edit

Upwork Job URL: https://www.upwork.com/jobs/~01ddb468a7b05692a6
Upwork Job ID:<PHONE_NUMBER>271044608
Last Price Increase: 2023-08-04

I would like to work on this.

I am ready to work on this ticket. :)

Can I work on this?

Contributor details
Your Expensify account email<EMAIL_ADDRESS>Upwork Profile Link: https://www.upwork.com/freelancers/~01b80b4cc418e7d5fb

I can work on this.

I'd like to work on this issue

can I work on this

I can work on this issue.

I am able to work on this task.

May I work on this?

Hi, I'm from Software Mansion, an expert agency. I'd like to work on this :)

here are the steps I will follow to convert the component to a function component:
Remove the class keyword.
Remove the render() method.
Convert all methods to functions.
Remove all references to this.
Remove the constructor.
Remove event handler bindings.
Replace this.setState() with useState().
Replace lifecycle methods with hooks.

I would like to work on this one

Contributor details
Your Expensify account email<EMAIL_ADDRESS>Upwork Profile Link: https://www.upwork.com/freelancers/~018cb7365e0e8298cf

here are the steps I will follow to convert the component to a function component:
Remove the class keyword.
Remove the render() method.
Convert all methods to functions.
Remove all references to this.
Remove the constructor.
Remove event handler bindings.
Replace this.setState() with useState().
Replace lifecycle methods with hooks.

Contributor details
Your Expensify account email<EMAIL_ADDRESS>Upwork Profile Link: https://www.upwork.com/freelancers/~0100ec54dabcbc04f0

Contributor details
Your Expensify account email<EMAIL_ADDRESS>Upwork Profile Link: https://www.upwork.com/freelancers/~0100ec54dabcbc04f0

GENERATED BY GPT : (Its the draft of functional component)
In PR i will provide you complete code with full functionality
import _ from 'underscore';
import React, { useState } from 'react';
import PropTypes from 'prop-types';
import { withOnyx } from 'react-native-onyx';
import { View } from 'react-native';
import moment from 'moment/moment';
import { parsePhoneNumber } from 'awesome-phonenumber';
import IdologyQuestions from './IdologyQuestions';
import ScreenWrapper from '../../components/ScreenWrapper';
import HeaderWithBackButton from '../../components/HeaderWithBackButton';
import withLocalize, { withLocalizePropTypes } from '../../components/withLocalize';
import styles from '../../styles/styles';
import Text from '../../components/Text';
import CONST from '../../CONST';
import compose from '../../libs/compose';
import ONYXKEYS from '../../ONYXKEYS';
import TextLink from '../../components/TextLink';
import TextInput from '../../components/TextInput';
import * as Wallet from '../../libs/actions/Wallet';
import * as ValidationUtils from '../../libs/ValidationUtils';
import * as ErrorUtils from '../../libs/ErrorUtils';
import AddressForm from '../ReimbursementAccount/AddressForm';
import DatePicker from '../../components/DatePicker';
import Form from '../../components/Form';
import withCurrentUserPersonalDetails, { withCurrentUserPersonalDetailsPropTypes, withCurrentUserPersonalDetailsDefaultProps } from '../../components/withCurrentUserPersonalDetails';
import * as PersonalDetails from '../../libs/actions/PersonalDetails';
import OfflineIndicator from '../../components/OfflineIndicator';
const propTypes = {
...withLocalizePropTypes,
...withCurrentUserPersonalDetailsPropTypes,
// Rest of the propTypes

};
const defaultProps = {
walletAdditionalDetails: {
errorFields: {},
isLoading: false,
errors: {},
questions: [],
idNumber: '',
errorCode: '',
},
...withCurrentUserPersonalDetailsDefaultProps,
};
const INPUT_IDS = {
// Rest of the INPUT_IDS
};
const AdditionalDetailsStep = (props) => {
const [minDate] = useState(moment().subtract(CONST.DATE_BIRTH.MAX_AGE, 'Y').toDate());
const [maxDate] = useState(moment().subtract(CONST.DATE_BIRTH.MIN_AGE_FOR_PAYMENT, 'Y').toDate());
const errorTranslationKeys = {
    // Rest of the errorTranslationKeys
};

const fieldNameTranslationKeys = {
    // Rest of the fieldNameTranslationKeys
};

const validate = (values) => {
    // Rest of the validate function
};

const activateWallet = (values) => {
    // Rest of the activateWallet function
};

if (!_.isEmpty(props.walletAdditionalDetails.questions)) {
    return (
        <ScreenWrapper
            style={[styles.flex1]}
            keyboardAvoidingViewBehavior="height"
        >
            <HeaderWithBackButton
                title={props.translate('additionalDetailsStep.headerTitle')}
                onBackButtonPress={() => Wallet.setAdditionalDetailsQuestions(null)}
            />
            <IdologyQuestions
                questions={props.walletAdditionalDetails.questions}
                idNumber={props.walletAdditionalDetails.idNumber}
            />
        </ScreenWrapper>
    );
}
const shouldAskForFullSSN = props.walletAdditionalDetails.errorCode === CONST.WALLET.ERROR.SSN;

return (
    <>
        <HeaderWithBackButton title={props.translate('additionalDetailsStep.headerTitle')} />
        <View style={[styles.flex1]}>
            <View style={[styles.ph5]}>
                <Text style={styles.mb3}>{props.translate('additionalDetailsStep.helpText')}</Text>
                <TextLink
                    style={styles.mb3}
                    href="https://use.expensify.com/usa-patriot-act"
                >
                    {props.translate('additionalDetailsStep.helpLink')}
                </TextLink>
            </View>
            <Form
                formID={ONYXKEYS.WALLET_ADDITIONAL_DETAILS}
                validate={validate}
                onSubmit={activateWallet}
                scrollContextEnabled
                submitButtonText={props.translate('common.saveAndContinue')}
                style={[styles.mh5, styles.flexGrow1]}
            >
                <TextInput
                    inputID={INPUT_IDS.LEGAL_FIRST_NAME}
                    containerStyles={[styles.mt4]}
                    label={props.translate(fieldNameTranslationKeys.legalFirstName)}
                    accessibilityLabel={props.translate(fieldNameTranslationKeys.legalFirstName)}
                    accessibilityRole={CONST.ACCESSIBILITY_ROLE.TEXT}
                    defaultValue={PersonalDetails.extractFirstAndLastNameFromAvailableDetails(props.currentUserPersonalDetails).firstName}
                    shouldSaveDraft
                />
                {/* Rest of the JSX */}
                <OfflineIndicator containerStyles={[styles.mh5, styles.mb3]} />
            </Form>
        </View>
    </>
);

};
AdditionalDetailsStep.propTypes = propTypes;
AdditionalDetailsStep.defaultProps = defaultProps;
export default compose(
withLocalize,
withCurrentUserPersonalDetails,
withOnyx({
walletAdditionalDetails: {
key: ONYXKEYS.WALLET_ADDITIONAL_DETAILS,
},
}),
)(AdditionalDetailsStep);

Contributor details
Your Expensify account email<EMAIL_ADDRESS>Upwork Profile Link: https://www.upwork.com/freelancers/~0118a3b63494ce86d7

Contributor details
Your Expensify account email<EMAIL_ADDRESS>Upwork Profile Link: https://www.upwork.com/freelancers/~0118a3b63494ce86d7

Contributor details
Your Expensify account email<EMAIL_ADDRESS>Upwork Profile Link: https://www.upwork.com/freelancers/~0118a3b63494ce86d7

Contributor details
Your Expensify account email<EMAIL_ADDRESS>Upwork Profile Link: https://www.upwork.com/freelancers/~0118a3b63494ce86d7

Contributor details
Your Expensify account email<EMAIL_ADDRESS>Upwork Profile Link: https://www.upwork.com/freelancers/~0118a3b63494ce86d7

Contributor details
Your Expensify account email<EMAIL_ADDRESS>Upwork Profile Link: https://www.upwork.com/freelancers/~0118a3b63494ce86d7

Why can I access? I think, I am correct.

Contributor details
Your Expensify account email<EMAIL_ADDRESS>Upwork Profile Link: https://upwork.com/freelancers/~01b7c2a45e48a0d1d1

Contributor details
Your Expensify account email<EMAIL_ADDRESS>Upwork Profile Link: https://upwork.com/freelancers/~01b7c2a45e48a0d1d1

can I work on this

@allroundexperts Do you mean that Anshu should do this issue? I'm already assigned and working on a PR.

No. @allroundexperts please look at the issue comment history in it's entirety before recommending a hire. Thanks!

Summary of payments:
@allroundexperts will request payment of $250 via NewDot
CC @JmillsExpensify

I think we're all set here? Gonna close. Feel free to reopen if there's anything left!

Reviewed the details for @allroundexperts. $250 approved for payment in NewDot based on the BZ summary above.


Bug ios issues

Description
crashes and wrong rendering fix on ios
About #3810
Checklist:

[x] PR title matches follow: (Feature|Bug|Chore) Task Name
[x] My code follows the style guidelines of this project
[x] I have followed all the instructions described in the initial task (check Definitions of Done)
[x] I have performed a self-review of my own code
[x] My changes generate no new warnings
[ ] I have added tests that prove my fix is effective or that my feature works
[x] New and existing unit tests pass locally with my changes
[x] I have added reference to a related issue in the repository
[ ] I have added a detailed description of the changes proposed in the pull request. I am as descriptive as possible, assisting reviewers as much as possible.
[ ] I have added screenshots related to my pull request (for frontend tasks)
[ ] I have pasted a gif showing the feature.
[x] @mentions of the person or team responsible for reviewing proposed changes

flip svg image from which you removed rotate transform

@johnsmith-gooddollar

We can't get current progress of animation when we use imperative methods of lottie, just rerun from the start. Will continue investigation
done
svg file is configured and importing/exporting it by figma breaks it. I added rotation slyle in another file, so it won't affect on the children

@roma-hladilka nice job, awaiting for the torus issue resolution


vimPlugins.aniseed: init at 2021-01-08

Motivation for this change
Add https://github.com/Olical/aniseed
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[ ] Fits CONTRIBUTING.md.

This is a semi-automatic executed nixpkgs-review which does not build all packages (e.g. lumo, tensorflow or pytorch)
If you find some bugs or got suggestions for further things to search or run please reach out to SuperSandro2000 on IRC.
Result of nixpkgs-review pr 108811 run on x86_64-darwin 1

1 package built:

spacevim

This is a semi-automatic executed nixpkgs-review which does not build all packages (e.g. lumo, tensorflow or pytorch)
If you find some bugs or got suggestions for further things to search or run please reach out to SuperSandro2000 on IRC.
Result of nixpkgs-review pr 108811 run on x86_64-darwin 1

1 package built:

spacevim

This is a semi-automatic executed nixpkgs-review which does not build all packages (e.g. lumo, tensorflow or pytorch)
If you find some bugs or got suggestions for further things to search or run please reach out to SuperSandro2000 on IRC.
Result of nixpkgs-review pr 108811 run on x86_64-linux 1

2 packages built:

spacevim
vimPlugins.aniseed

This is a semi-automatic executed nixpkgs-review which does not build all packages (e.g. lumo, tensorflow or pytorch)
If you find some bugs or got suggestions for further things to search or run please reach out to SuperSandro2000 on IRC.
Result of nixpkgs-review pr 108811 run on x86_64-linux 1

2 packages built:

spacevim
vimPlugins.aniseed


#3629 :: Check for last update in future

Proposed Code change to fix #3629 .

I don't think this is the solution to #3629 and it doesn't pass CI, so I'm closing it until we have found a better solution (and you are back from vacation).


When using advanced storage strategy, why copy the peer data file into the same dir as the output file?

Per the documentation, when using the io.d7y.storage.v2.advance storage strategy, the peer data file is copied into the same directory as the output file.
After running dfget <url> -O /tmp/eddie_test, I have observed that there are actually 3 hard links to the file. They are:

/tmp/eddie_test
/tmp/.eddie_test.dfget.cache.<req.PeerID>
<dataDir>/<req.TaskID>/<req.PeerID>/data

Why not just copy the file to the dataDir, and link from there?

To avoid copying the daemon cache across filesystems to the specified directory.

Hardlink is fast than copying the file. io.d7y.storage.v2.simple storage strategy will copy the file.

Yep I definitely like the hard link approach. But I don't see why links need to exist in  and in the output path.
If the  is on a different filesystem, I believe we could use a symlink (and I see there is code already to do this).
So just download straight to , and then either hard link or symlink to the output path?

The strategy  io.d7y.storage.v2.simple will make symlink if is on different filesystems

It appears that in https://github.com/dragonflyoss/Dragonfly2/blob/main/client/daemon/storage/storage_manager.go#L454 the symlink is done as a fallback if the hard link fails, when using io.d7y.storage.v2.advance.

@jim3ma  @gaius-qi To clarify, what I'd like to know is: Why is it not sufficient to download the file to the dataDir, and then link (either hardlink or symlink) to it?
Why do we need /tmp/.eddie_test.dfget.cache.<req.PeerID>?


syncer(dm): save table checkpoint after a DDL is filtered

Signed-off-by: lance6716<EMAIL_ADDRESS>
What problem does this PR solve?

Issue Number: ref #5272
What is changed and how it works?
as title
Check List
Tests

Integration test

Related changes

Need to cherry-pick to the release branch

Release note
save table checkpoint after a DDL is filtered

/hold
test is invalid

/run-dm-integration-test
/unhold

Codecov Report

Merging #5273 (b37cfac) into master (687e248) will increase coverage by 0.0145%.
The diff coverage is 53.8933%.

Flag
Coverage Δ

cdc
60.9608% <53.8933%> (+0.3621%)
:arrow_up:

dm
52.3346% <ø> (-0.1350%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.
@@               Coverage Diff                @@
##             master      #5273        +/-   ##
================================================
+ Coverage   56.1240%   56.1385%   +0.0145%
================================================
  Files           522        524         +2
  Lines         65325      68730      +3405
================================================
+ Hits          36663      38584      +1921
- Misses        25094      26412      +1318
- Partials       3568       3734       +166

/merge
/run-all-tests

/run-dm-integration-test


Updated wrong class name for UITableViewCell

In the "Subclassing UITableViewSource" section, when the docs describe the GetCell method, there is an error related to the UITableViewCell class name. It says UITableCellView.

@gentilijuanmanuel : Thanks for your contribution! The author(s) have been notified to review your proposed change.


Drop all pos related code from public ecom repo

👋 This notion doc contains the broader context of this PR stack. Please make sure to read it once!

What
Step 1 of Part 1.

Why
See here

Test Plan

❌ I've added unit tests for this change.  Did not add any new tests because this PR does not introduce any new functionality.
❌ The reviewer should manually test the changes in this PR.  No need to manual test because the top PR in the stack passes CI tests

Demo
N/A - no new functionality

How
This PR will not get merged into main. I am leaving it open to make it easier for folks to review the code changes in the whole stack. I will end up pointing #248 to main and merging that in. I will close this PR and #247 once folks have approved the stack. See here for more detail.

#246  👈
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @devinmorgan and the rest of your teammates on  Graphite

As discussed in the rollout planning, I merged #248 directly into main to avoid git and graphite drama. There we can close this PR


Picotorrent: Add file type association

Adds filetype association for .torrent and magnet links. The post_install is created from vscode's.
Will change https://raw.githubusercontent.com/BlackPowerade/scoop-extras/picotorrent-regs/ to https://raw.githubusercontent.com/lukesampson/scoop-extras/master/ if approved.

Forgot to mention, but another change here is Picotorrent.sqlite being added to the persist. This is what keeps the data for torrent progress, active torrents, where they are located/being saved to, etc.

Anyone tested actual installation, persisting of precreated sqlite file and registry files are correct and working?
I do not have nearby device, where I can manipulate with registry in this manner.

Everything seems to work fine here. The only thing I cannot verify is if the hashes for the registry files will still match after being merged.


[Priority: Low] Show extended public key for NTumbleBit support

If we show up the extended public key, users can configure Breeze as an output wallet for NTumbleBit. It should be on a different keypath in order to avoid some unseen issues.
Illustration:

If this is for TB only then maybe the TB UI can expose this field found at Stratis.Bitcoin.Features.Wallet.HdAccount.ExtendedPubKey.
Do you think there is any value in showing this for the regular Breeze Wallet (without TB)?

As ZeroLink's post mix mallet.

The API is ready to do such feature as soon as PR https://github.com/stratisproject/StratisBitcoinFullNode/pull/506 merged.
Endpoint: extpubkey

Any update on this? stratisproject/StratisBitcoinFullNode#506 has already been merged on Sep 27, 2017


reports.adguard.com

Energized Protection - block ⚡

Let's make an annoyance free, better open internet, altogether!

Issue Submit Form

  Provide the following infos properly, which will help us to resolve your issue quickly.

Issue(s):
Type x in between [ ] and make sure there isn't any space between brackets. Example; for Your Selected Issue(s), type like this - [x]
You can select more than one category of issues if you need to!

[ ] Whitelist
[ ] Blacklist
[x] App Broken
[ ] Website Broken
[ ] Request
[ ] Other Issue

Pack(s):
Write the name of the pack(s) you are using.

Pack(s) Name: Blu

Extension(s):
Also name the extension(s) if you are using any. If you don't then leave it blank.

Extension(s) Name: AdGuard

Domain(s):
If you are submitting this issue for whitelist/blacklist issue, send us the domain(s) for whitelisting/blacklisting here. Kindly use the Code Tag to prevent tracking.

Domain(s):

reports.adguard.com

Your Config:
Just to ensure there is no issues or conflicts with other app/software/magisk module/extension/source list.
Make sure you are running Energized Protection Service only.

Client: PiHole

Version: 20.03.091

Details:
Write us a lil bit more about your issue or query. You can attach any screen shot or log of the issue or advert, this will help to highlight it.

Your Issue Detail:
reports.adguard.com is needed for the browser extension "AdGuard" feature "safety report" which open a new tab, like https://reports.adguard.com/youtube.com/report.html

Blocking this domain, block that feature.

Thank you for making Energized Protection great with your kind help!

A project by Ador with ❤

@AdroitAdorKhan


Add functions to extract and analyze chat logs

@trgardos I've reviewed and merged your PR into mine


Adapt index creation to support both sync and async mode

What changes are proposed in this pull request?
Index creation can take a long time, especially for large datasets. This change allows dataset.create_index to support an async mode when using_async is true.
How is this patch tested? If it is not, please explain why.

Manual test (✅ )
Unit test (in progress)

Release Notes
Is this a user-facing change that should be mentioned in the release notes?

[x] No. You can skip the rest of this section.
[ ] Yes. Give a description of this change to be included in the release
notes for FiftyOne users.

(Details in 1-2 sentences. You can just refer to another PR with a description
if this PR is part of a larger change.)
What areas of FiftyOne does this PR affect?

[ ] App: FiftyOne application changes
[ ] Build: Build and test infrastructure changes
[x] Core: Core fiftyone Python library changes
[ ] Documentation: FiftyOne documentation changes
[ ] Other

Summary by CodeRabbit

New Features

Introduced asynchronous capabilities for the create_index method, allowing for non-blocking index creation.
Added new asynchronous properties for accessing sample and frame collections, enhancing non-blocking data retrieval.

Bug Fixes

Improved functionality for handling asynchronous operations in collections and datasets, optimizing performance.

@coderabbitai generate unit testing code for this file

@brimoor usage example
Synchronous execution:
IPython 8.24.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import fiftyone as fo
   ...:
   ...: import fiftyone.zoo as foz

In [2]: import logging

In [3]: logging.basicConfig(
   ...:     level=logging.INFO,  # Set the logging level
   ...:     format='%(asctime)s - %(levelname)s - %(message)s',  # Set the format of the log message
   ...:     datefmt='%Y-%m-%d %H:%M:%S'  # Set the format of the date/time
   ...: )

In [4]: dataset = fo.load_dataset("dave-quickstart")
2024-08-07 13:04:14 - INFO - {"message": "Waiting for suitable server to become available", "selector": "Primary()", "operation": "buildinfo", "topologyDescription": "<TopologyDescription id: 66b3d33e902ba2daae4614da, topology_type: Unknown, servers: [<ServerDescription ('staging-mongodb.fiftyone.ai', 29017) server_type: Unknown, rtt: None>]>", "clientId": {"$oid": "66b3d33e902ba2daae4614da"}, "remainingTimeMS": 29}

In [5]: dataset.create_index("ground_truth.detections.bounding_box", using_async=False)
Out[5]: 'ground_truth.detections.bounding_box'

In [6]: dataset.create_index("ground_truth.detections.index", using_async=False)
2024-08-07 13:08:10 - INFO - Creating index 'ground_truth.detections.index' synchronously...
Out[6]: 'ground_truth.detections.index'

In [7]: dataset.create_index("ground_truth.detections.index", using_async=True)
2024-08-07 13:08:38 - INFO - Running async, checking for running event loop
Out[7]: <coroutine object SampleCollection.create_index at 0x102a5b8b0>

In [8]: exit

Async execution:
(voxel310) minhtuevo@minh-tues-mbp fiftyone % apython
Python 3.10.14 (main, May  6 2024, 14:42:37) [Clang 14.0.6 ] on darwin
Type "help", "copyright", "credits" or "license" for more information.
---
This console is running in an asyncio event loop.
It allows you to wait for coroutines using the 'await' syntax.
Try: await asyncio.sleep(1, result=3)
---
>>> import fiftyone as fo
>>> dataset = fo.load_dataset("dave-quickstart")
>>> dataset.create_index("ground_truth.detections.index", using_async=True)
<coroutine object SampleCollection.create_index at 0x29c6ba8f0>
>>> import logging
>>> logging.basicConfig(level=logging.INFO)
>>> await dataset.create_index("ground_truth.detections.index", using_async=True)
'ground_truth.detections.index'
>>> await dataset.create_index("ground_truth.detections.area", using_async=True)
'ground_truth.detections.area'

Unless you issue additional commands to drop the index, once you call create index synchronously, can you just crl+c to avoid waiting for it to complete? (Never tried it) Or just call it in another window?
Note that for large index builds, calling create index could impact everyone using that cluster, and you probably shouldn't continue working on that dataset while an index is being built so not quite following the need/desire to wrap this method.

@kaixi-wang : I think once the cluster has successfully received the command it is okay to terminate, but it is a little bit tricky to time the execution code since the client might take sometimes to successfully open a connection to the DB cluster :D I wish they just have a background early return mode haha!
In [4]: dataset = fo.load_dataset("dave-quickstart")
2024-08-07 13:04:14 - INFO - {"message": "Waiting for suitable server to become available", "selector": "Primary()", "operation": "buildinfo", "topologyDescription": "<TopologyDescription id: 66b3d33e902ba2daae4614da, topology_type: Unknown, servers: [<ServerDescription ('staging-mongodb.fiftyone.ai', 29017) server_type: Unknown, rtt: None>]>", "clientId": {"$oid": "66b3d33e902ba2daae4614da"}, "remainingTimeMS": 29}

@brimoor @benjaminpkane @kaixi-wang : here is a different take on this, adding a timeout decorator to dataset.create_index(...) so that it will timeout after x seconds. We can also add some retry logic so that it will do a longer retry, capping at max_retry after verifying that the index is not created for some reasons :)
https://github.com/voxel51/fiftyone/pull/4641

Closing in favor of https://github.com/voxel51/fiftyone/pull/4624


Error in cohort pages when no signals were found by h12-signal-detection

It seems that the h12-signal-detection notebook saves an empty file with no column names when no signals are found for a given cohort's contig.
This seems to cause an error when running the cohort page notebook for that cohort. For example, in the MZ-I_Morrumbene_gamb_2004_Q1 cohort, we get the following error:
----> 3 df_signals = [
      4     pd.read_csv(here() / "build/h12-signal-detection/" / f"{cohort_id}_{contig}.csv")
      5     for contig in contigs
      6 ]

EmptyDataError: No columns to parse from file

my current fix is
try:
    df_signals = [
        pd.read_csv(here() / "build/h12-signal-detection/" / f"{cohort_id}_{contig}.csv")
        for contig in contigs
        ]
    df_signals = pd.concat(df_signals).assign(bottom=0, top=1)
except pd.errors.EmptyDataError:
    df_signals = pd.DataFrame()

and then only add signals to the H12 plot if not df_signals.empty:

Unfortunately the same error then of course crops up in chromosome pages. Will sort later.


Are there screenshots of this project?

Just curious, trying to put together a list of OSS dashboards for Dokku :)

Hello 👋
Not at the moment. This project is currently under development and the UI hasn't been finished just yet. I've used Dokku for some time now and I love the project. I plan Rocket Dokku to be my contribution to the project as well as my first open source repo. I'll let you know when the first version gets released.
Thanks for reaching out!

Amazing


Merge branch Feature/conformance

Purpose
Fixes https://github.com/wso2/apk/issues/2403
Design doc - https://docs.google.com/document/d/1fwDWKK_yZ3TXfdcA3vSX4Kg_plcg-JBYoobAiwyudtY/edit#heading=h.35yjoemxqx92
Goals

Describe the solutions that this feature/fix will introduce to resolve the problems described above

Approach

Describe how you are implementing the solutions. Include an animated GIF or screenshot if the change affects the UI (email<EMAIL_ADDRESS>to review all UI text). Include a link to a Markdown file or Google doc if the feature write-up is too long to paste here.

User stories

Summary of user stories addressed by this change>

Release note

Brief description of the new feature or bug fix as it will appear in the release notes

Documentation

Link(s) to product documentation that addresses the changes of this PR. If no doc impact, enter “N/A” plus brief explanation of why there’s no doc impact

Training

Link to the PR for changes to the training content in https://github.com/wso2/WSO2-Training, if applicable

Certification

Type “Sent” when you have provided new/updated certification questions, plus four answers for each question (correct answer highlighted in bold), based on this change. Certification questions/answers should be sent to<EMAIL_ADDRESS>and NOT pasted in this PR. If there is no impact on certification exams, type “N/A” and explain why.

Marketing

Link to drafts of marketing content that will describe and promote this feature, including product page changes, technical articles, blog posts, videos, etc., if applicable

Automation tests

Unit tests

Code coverage information

Integration tests

Details about the test cases and coverage

Security checks

Followed secure coding standards in http://wso2.com/technical-reports/wso2-secure-engineering-guidelines? yes/no
Ran FindSecurityBugs plugin and verified report? yes/no
Confirmed that this PR doesn't commit any keys, passwords, tokens, usernames, or other secrets? yes/no

Samples

Provide high-level details about the samples related to this feature

Related PRs

List any other related PRs

Migrations (if applicable)

Describe migration steps and platforms on which migration has been tested

Test environment

List all JDK versions, operating systems, databases, and browser/versions on which this feature/fix was tested

Learning

Describe the research phase and any blog posts, patterns, libraries, or add-ons you used to solve the problem.

Closing as this is already sent in https://github.com/wso2/apk/pull/2495

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 18.87%. Comparing base (707328a) to head (587bd93).
Report is 248 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2455      +/-   ##
==========================================
- Coverage   26.08%   18.87%   -7.22%
==========================================
  Files          68      113      +45
  Lines        4769    18700   +13931
  Branches     2451        0    -2451
==========================================
+ Hits         1244     3529    +2285
- Misses       3525    14967   +11442
- Partials        0      204     +204

Flag
Coverage Δ

adapter
18.87% <ø> (?)

config-deployer-service
?

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Added vertical scroll direction event

Added $direction to callback, will return 'up' or 'down' #34

gosh a huge chunk of the code has beed changed with no apparent edits. Is that an indentation issue? It would be great if we could clean this up a bit

This feature is in v2. closing this


Sequel migration requiring single model file

Re #1520 & #1795 comment:

Tasks db:migrate and db:seed might not work properly without additional testing with sequel because of http://sequel.jeremyevans.net/rdoc/files/doc/migration_rdoc.html#label-Using+models+in+your+migrations.

rake sq:migrate is requiring one of my project models (agent_alias.rb). Others not required - oddly!
Impact is rake sq:migrate builds my User model as a table of columns with blank titles.
Workaround is to rake sq:migrate:to[n] sequentially - which currently works.
Can't see this model required in /lib/**/* or anywhere else, so maybe a bug?
Any help much appreciated. Sequel + PostgeSQL working well with your excellent framework otherwise.

What does the migration file look like?

The migration files are here Mitchell.
Bruce

I was able to migrate from a newly created database without any issues.  What error are you getting exactly?

The problem occurs when I sq:migrate:down and then sq:migrate at that point the model file is being loaded:

Macintosh:padfoot me$ padrino rake sq:migrate
=> Executing Rake sq:migrate ...
DEVEL -   LOADING (0.0109s) /Users/me/dev/padfoot/lib/sass_initializer.rb
DEVEL -   LOADING (0.0107s) /Users/me/dev/padfoot/lib/scrapers/app_details_scraper.rb
DEVEL -   LOADING (0.0215s) /Users/me/dev/padfoot/lib/scrapers/app_refs_scraper.rb
DEVEL -   LOADING (0.0113s) /Users/me/dev/padfoot/app/app.rb
DEVEL -   LOADING (0.0110s) /Users/me/dev/padfoot/app/models/agent_alias.rb
INFO -  (0.000240s) SET standard_conforming_strings = ON
INFO -  (0.000209s) SET client_min_messages = 'WARNING'
INFO -  (0.000226s) SET DateStyle = 'ISO'
ERROR -  PG::UndefinedTable: ERROR:  relation "agent_aliases" does not exist

I just went migrated down and then back up without issue.  I get those messages as well, but the migrations do finish running.

well that's typical; seems an intermittent problem as Users table is ok now - previously had all-blank column titles. Repeated a few times and looks ok. Would still like to know why the one model file is required - which gives the error messages. Closed as functionality not impaired - many thanks for looking at this Mitchell

I will leave it open and look into the error some time soon.

ok, thanks

Finaly got to look at the issue. The problem is this line in your Rakefile: https://github.com/MatzFan/padfoot/blob/17149c799307fdd6cb3205b30f49c249fc1a1d1c/Rakefile#L3
By calling it you load full application environment for all the rake tasks though it's recommended to not load models for your migrations. There's a patch in our code that prevents loading models for sequel migrations, but this code is run after your require hence no effect.

Aaaggh - of course. Many thanks Igor & sorry you spent time on this, will change it right away.
Cheers
Bruce
On 4 Mar 2015, at 11:38, Igor Bochkariov<EMAIL_ADDRESS>wrote:

Finaly got to look at the issue. The problem is this line in your Rakefile: https://github.com/MatzFan/padfoot/blob/17149c799307fdd6cb3205b30f49c249fc1a1d1c/Rakefile#L3
By calling it you load full application environment for all the rake tasks though it's recommended to not load models for your migrations. There's a patch in our code that prevents loading models for sequel migrations, but this code is run after your require hence no effect.
—
Reply to this email directly or view it on GitHub.


Add actor registry

An actor registry would remove the need for storing actor addresses in actors. The reason this would be a beneficial change is that creating two actors that need to communicate with each other is verbose and needlessly complex.
Case in point:
let a1 = PingActor::new(VinContextPingActor { addr: None, time_of_ping: Default::default() });
let a2 = PingActor::new(VinContextPingActor { addr: Some(a1.clone()), time_of_ping: Default::default() });
a1.ctx_mut().await.addr = Some(a2.clone());

With an actor registry, one would be able to query the registry with a key passed to the actor during his creation. A challenge that needs to be resolved, however, is pruning the registry of closed actors. Perhaps a task that passes the registry every so often and deletes expired addresses or closed actor?

Closed by #3


Use --es6 to generate proto .js

Blocked by https://github.com/protobufjs/protobuf.js/issues/1308

Will get back to this when my PR is merged upstream. But there is no real need for this right now.


Fire "beforeunload" confirm box only if document has changes

This box is annoying. The save process is complicated enough (save scene, capture pose, save article), if users can't rely on this to know if they forgot to save something they will just ignore the dialog because it's wrong most of the time anyway.
Serializing the CVDocument just after openDocument() and serializing again to compare the string value in CVStoryApplication.beforeunload() might do the trick but I'd like a solution that allows a better UI (eg: highlighting the save button as soon as the doc is changed). The tricky bit is that we have no representation of the document's data once it's been loaded into the graph, so a "changed" switch seems hard to implement.
I'd like to work on this issue, if anyone has an idea worth trying.

I agree the box would be better served only when the data is dirty, but still better there than not. Several components already have literal or effective 'changed' output events, so a state manager could subscribe to those events to flip the global changed state and clear on save. The hard part would be adding the missing events and making sure coverage is 100%.


Cumulative Update for Keras 2.2.0 and some fixes and Explanations in the README.md

This Pull request includes fixes for the issues addressed in #102 #101, #96, #94 and #76
along with the bug in farizrahman4u/seq2seq#177 and farizrahman4u/seq2seq#250
(both caused by calls to the deprecated  keras.engine.topology class.)

While waiting for a response from @farizrahman4u, you can use the installation instructions from pyrestone/recurrentshop to install the library for keras 2.2.0

I added another fix for the Tensorflow teacher-forcing bug, fixing #81 and making the tf-teacher-force branch obsolete.

Thanks for the effort. Couple of small changes required.

Lgtm, Thanks!

Thank you for your work!


Icons Volume 17

Summary
An Icon to distinguish between "users/customers" and "employees" is missing.
The "user" icon is already available in clarity: shape="user" and shape="users".
Whats missing is the equivalent for employee/s.

What is the icon?
an employee, a group of employees
What does the requested icon signify?
differentiation if an entity is assigned to customers or employees
In which product(s) will this icon be used?
business to customer application

Use case
differentiation if an entity (appointment, task, search-result, ...) is assigned to customers or employees
Examples
see screenshots, something like the exisiting user icon with a tie:

Alternatives
alternative, that of course does not transport any meaning:

us the solid and outline ones to distinguish between customers and employees
(shape="user" class="is-solid")

@lil-kim
What do you think about a volume 17 set in early 2020?

@d-m-s
How about the "id-badge" icon to represent employee?

An animation icon would be great in the new icon set. I'm building a tool in which you can animate elements. Clicking the icon would lead to the animation configurator.
Example:

There is the play icon already, but this goes more in the direction of actually starting the configured animation.

@d-m-s
How about the "id-badge" icon to represent employee?

nice idea! thanks!

as a workaround that works pretty good, but we also need icons like driver-licence or identification cards, there we use the "id-badge"
so on the long run,

some user-icon with tie or
an user icon with badges in the upper/lower right corner - where probably any other clr-icon could be placed, would be great!

Icons for Vol 17

Employee
Group of employees
Factory
Holiday / not available notification
File share
Color palette
Animation
CI/CD


update dependencies for 0.42 sdk release

Description
Updates provenance to use cosmos SDK 0.42
closes: #141

Before we can merge this PR, please make sure that all the following items have been
checked off. If any of the checklist items are not applicable, please leave them but
write a little note why.

[ ] Targeted PR against correct branch (see CONTRIBUTING.md)
[ ] Linked to Github issue with discussion and accepted design OR link to spec that describes this work.
[ ] Wrote unit and integration tests
[ ] Updated relevant documentation (docs/) or specification (x/<module>/spec/)
[ ] Added relevant godoc comments.
[ ] Added a relevant changelog entry to the Unreleased section in CHANGELOG.md
[ ] Re-reviewed Files changed in the Github PR explorer
[ ] Review Codecov Report in the comment section below once CI passes

DO NOT MERGE  this until @dpederson-figure tests/reviews the WASM changes.


Allow custom file icons

closes #11149
Based on the latest discussions this would be my suggestion for the custom file icons:

An admin provides icon packs in a data/icons/<my-icon-pack> folder
Users can select one of the provided icon packs or leave the basic / unstyled one (same way as selecting a template)

Would be interested in feedback before continuing 😊
TODO

[ ] allow user to select an icon pack from a list configured by admin
[ ] cache icons loaded from fs
[ ] add some sample repo with a prepared icon pack

It's better to have a configuration item on app.ini

Not in this PR but I would prefer to be able to select this as per-user option

I think making icons as new storage is better so that icons can be put in Minio or S3 compatible places and can be cached with CDN or similar tools.

Actually there are other attempts for this, for example,  Add material icons for file list #26236
But there are still no agreements about:

How to store these icon files

By a binary zip in git repo? Is binary file (hundreds of KBs to a few MBs) good to store in git repo?
By a binary zip in git-lfs? Is git-lfs acceptable for Gitae's development? Is git-lfs usable for GitHub's "allow edit by maintainers"?
By a lot of small files in git? There will be a lot. Is it good to bloat the codebase too much (600+ files => 800+ files => maybe soon 1000+ files? And the google official material repo has more than 10K src files)
Do not store them in Gitea repo, and only make site admin add them manually?

Then my PR should be good enough.

How to build new icon packs and publish/deliver them?

Should we teach end users how to build new icon packs? Or just provide "official" icon packs?
Should it use something like "market" or "theme center" to list the icon packs and provide the downloadable packs?

Actually there are other attempts for this, for example, Add material icons for file list #26236
But there are still no agreements about:
1. How to store these icon files

   * By a binary zip in git repo? Is binary file (hundreds of KBs to a few MBs) good to store in git repo?
   * By a binary zip in git-lfs? Is git-lfs acceptable for Gitae's development? Is git-lfs usable for GitHub's "allow edit by maintainers"?
   * By a lot of small files in git? There will be a lot. Is it good to bloat the codebase too much (600+ files => 800+ files => maybe soon 1000+ files? And the google official material repo has more than 10K src files)
   * Do not store them in Gitea repo, and only make site admin add them manually?

     * Then my PR should be good enough.

2. How to build new icon packs and publish/deliver them?

   * Should we teach end users how to build new icon packs? Or just provide "official" icon packs?
   * Should it use something like "market" or "theme center" to list the icon packs and provide the downloadable packs?

Currently, I prefer to put these pack files as standalone files and managed by storage, so that they can be stored in S3 compatible places. I think this could be an experiment feature first. We can provide official icon packs in another repository and CI can publish it to dl.gitea.com so that users can download an archived file and extract it to gitea. Or we can provide a upload file UI in the admin panel, so users can upload them to storage.

Currently, I prefer to put these pack files as standalone files and managed by storage, so that they can be stored in S3 compatible places. I think this could be an experiment feature first. We can provide official icon packs in another repository and CI can publish it to dl.gitea.com so that users can download an archived file and extract it to gitea. Or we can provide a upload file UI in the admin panel, so users can upload them to storage.

Then I think some maintainers who have infra access should take over Add material icons for file list #26236 and continue there.

Currently, I prefer to put these pack files as standalone files and managed by storage, so that they can be stored in S3 compatible places. I think this could be an experiment feature first. We can provide official icon packs in another repository and CI can publish it to dl.gitea.com so that users can download an archived file and extract it to gitea. Or we can provide a upload file UI in the admin panel, so users can upload them to storage.

Then I think some maintainers who have infra access should take over Add material icons for file list #26236 and continue there.

If you don't mind, I will create a new pull request from #26236

How to store these icon files

IMO definitely not as part of the Gitea repo. Those icon packs are often quite large and update frequently.

How to build new icon packs and publish/deliver them?
Should we teach end users how to build new icon packs? Or just provide "official" icon packs?

The docs should describe how icon packs should be created. Having a reference icon pack would be great as it will easily show the community how to create their own.

Should it use something like "market" or "theme center" to list the icon packs and provide the downloadable packs?

This PR is basically based on #26236 now with the difference of using plain files from a folder which seems to be extremely simple to me. I also thought of using an http proxy for icons. So  it would call http://my-sidecar:3000/icons/file_md.svg and return that content.
For shipping I would suggest to start with something simple. Just clone your icon pack into data/icons/... (which would kind of be similar to how templates work right now).

Then I think some maintainers who have infra access should take over Add material icons for file list #26236 and continue there.

If you don't mind, I will create a new pull request from #26236

Feel free to do.

This PR is basically based on #26236 now with the difference of using plain files from a folder which seems to be extremely simple to me. I also thought of using an http proxy for icons. So it would call http://my-sidecar:3000/icons/file_md.svg and return that content.

It won't work because:

Gitea might be deployed in a private environment without external network access (some customers for "security")
In some regions, the "github.com" is blocked.
If there is some network lag, the first requests would be horrible slow.

For shipping I would suggest to start with something simple. Just clone your icon pack into data/icons/... (which would kind of be similar to how templates work right now).

Just FYI: Gitea has a messy "path" system (a lot of hacky patches in history). So what you see might not be what it is. Although I have fixed many legacy path problems, I didn't really "break" it to make it correct. An real example: there is a "custom path" for templates and "options(licenses, labels, etc)", and there is a "data path" for user's data (repositories, avatars). In docker, the "custom path" and "data path" is unfortunately mixed together, so you could see "data/templates" and "data/repositories" in the same "data" directory. So data/icons/... is not "similar to how templates work", there could be different parent directories.


Use junit test fixture to start and stop rpc server

We must guarantee we can close rpc server at any time. So we use test fixture to solve this problem.

This is done by https://github.com/dst-project/dst/pull/123


feat: add new worktop/jwt module

Opened as a draft PR. I want to solve #86 first and some additional rearranging of the ArrayBuffer utilities

Decided to do #86 at a later time. It will still be during the next phase because it will involve breaking changes (renames/reorganize).

Remaining task(s) in #90


Float64 run_speedy doesn't output correctly

julia> run_speedy(Float64; n_days=20, model=:shallowwater, output=true, trunc=62, )
ERROR: MethodError: no method matching scale_coslat⁻¹!(::SubArray{Float32, 2, Array{Float32, 3}, Tuple{Base.Slice{Base.OneTo{Int64}}, Base.Slice{Base.OneTo{Int64}}, Int64}, true}, ::Geometry{Float64})
Closest candidates are:
  scale_coslat⁻¹!(::AbstractMatrix{NF}, ::Geometry{NF}) where NF at ~/.julia/packages/SpeedyWeather/UDuyp/src/spectral_gradients.jl:266
  scale_coslat⁻¹!(::AbstractArray{NF, 3}, ::Any...) where NF at ~/.julia/packages/SpeedyWeather/UDuyp/src/distributed_vertical.jl:5

Likely because they are being converted into Float32 while saving but before scaling.
https://github.com/milankl/SpeedyWeather.jl/blob/1881d47ab3f28a85e433ccaaf5fc7218e4792b8d/src/output.jl#L173-L181

Thanks for pointing this out. I’d suggest that we need to implement a corresponding, less specific, method that doesn’t require the array and the Geometry instance to have the same number format NF. In that case we could scale by converting explicitly every element of the coslat^n vectors to Float32. I suggest the explicit conversion as to not run into troubles when NF hasn’t a type promotion to Float32 defined. Given this is not performance critical code I prefer that solution over also precomputing Float32 versions of the coslat^n vectors in Geometry

On 2 Jul 2022, at 01:19, Jayesh K. Gupta @.***> wrote:

julia> run_speedy(Float64; n_days=20, model=:shallowwater, output=true, trunc=62, )
ERROR: MethodError: no method matching scale_coslat⁻¹!(::SubArray{Float32, 2, Array{Float32, 3}, Tuple{Base.Slice{Base.OneTo{Int64}}, Base.Slice{Base.OneTo{Int64}}, Int64}, true}, ::Geometry{Float64})
Closest candidates are:
scale_coslat⁻¹!(::AbstractMatrix{NF}, ::Geometry{NF}) where NF at ~/.julia/packages/SpeedyWeather/UDuyp/src/spectral_gradients.jl:266
scale_coslat⁻¹!(::AbstractArray{NF, 3}, ::Any...) where NF at ~/.julia/packages/SpeedyWeather/UDuyp/src/distributed_vertical.jl:5
Likely because they are being converted into Float32 while saving but before scaling.
https://github.com/milankl/SpeedyWeather.jl/blob/1881d47ab3f28a85e433ccaaf5fc7218e4792b8d/src/output.jl#L173-L181
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you are subscribed to this thread.

Addressed with #103


Clean up the whole container

This is the most important task right now, the clean up helps make the updates for new Zulip version much easier (and readable :smiley: ).

[x] Reformat the puppet manifests
[x] Search and destory uneeded files
[x] entrypoint.sh: Clean up the functions
[x] entrypoint.sh: One big "run" function, with better seperation for example manage.py runs, etc.
[x] entrypoint.sh:Test if it's still working, if not fix it!
[x] Wait for fix in Galexrt/zulip-postgresql-tsearchextras
[x] Release new container version (Branch: v1.3.7-1)

Related to issues #32, #35 and pull request #37.

Released the new version https://github.com/Galexrt/docker-zulip/releases/tag/1.3.7-1.


Various improvements (check each commit separately)

I know you're already working on another branch and already have a solution for the change YouTube endpoint, so these changes can't be merged as-is, but I think you can find use for them.

First commit can be ignored - it just adds basic debug logging, and I know your branch has its own more robust version.

https://github.com/xenova/chat-replay-downloader/pull/50/commits/24e0d2293e5c1130fce9433b48f744376069d2ad?w=1
The HTTP library already has built-in retry and timeout functionality - they just need to be enabled.
Also ensure output file is always created even on uncaught exception.

https://github.com/xenova/chat-replay-downloader/pull/50/commits/36b9f8660a85e1cb03e4cde75c578447b88b9902?w=1
Use UTF-8-BOM encoding for file output. This was only needed for MS Excel compatibility with CSV unicode files. I just applied the same encoding to the other file output types for consistency.
Also improve JSON parse error message.

https://github.com/xenova/chat-replay-downloader/pull/50/commits/0203b7f5b47c52463b3cd080a8ed0ba5b95af707?w=1
Update to the new YouTube endpoint reusing param data from the ytcfg from the original video page. This json snippet includes these key fields: INNERTUBE_API_VERSION, INNERTUBE_API_KEY, INNERTUBE_CONTEXT.
This should be more robust than your solution, since it avoids the need to stub in dummy param data.
This commit also improves regex handling: more robust regex, lenient parsing, compiled regex

https://github.com/xenova/chat-replay-downloader/pull/50/commits/f0679f363eb9e01c8c3491a7363106162e71a7bd?w=1
Add datetime (ISO format that's never an offset so it's always consistent between live and replay) and author_type (language-agnostic enum for member, moderator, owner, etc.) fields.
Also changes the text output format to use datetime (rather than time_text or converted timestamp) and author_type (rather than badges). I find this more useful for my own purposes, but ideally the text output format should be configurable.

Thanks so much for the pull request! I'll definitely include some of these changes in a following update!

Thanks so much for the pull request! I'll definitely include some of these changes in a following update!

Great!
I messed up partitioning the commits a bit, so I fixed and force-pushed them.

Great!
I messed up partitioning the commits a bit, so I fixed and force-pushed them.

No worries! Thanks again!
However, if I were to include the code on my own, you would not be credited as a "Contributor" on GitHub. If I am not mistaken, that only lists people who have contributed code via pull requests. If you would like to be included as a contributor, would you mind forking and creating a pull request with your code?

I don't mind not being listed as a contributor on github. I also currently don't have time to "rebase" the changes onto the new master atm 😦

I don't mind not being listed as a contributor on github. I also currently don't have time to "rebase" the changes onto the new master atm 😦

Okay no worries! Just wanted to give credit where credit was due! 😃 Thanks again.

Don't mind me... been tinkering with my fork for my own uses, and haven't had time to "rebase" them off the massive rework on master.


Starting Strider: died 1 null - loop

I did a fresh git clone, npm install (node v4.4) and setup my env vars correctly - I see this loop over and over with no description of what's failing:
admin@jenkins:~/strider$ ./bin/strider
1887 forked
1887 died 1 null
1900 forked
^C
admin@jenkins:~/strider$

Solution
It turns out this is because mongo was not running..
a better error message should be displayed.


Disable trackResize option

The trackResize options makes leaflet react to global browser window resizes, which is not needed in the context of JupyterLab.

Thanks!

Published as<EMAIL_ADDRESS>


Build problem

Hi. I have a problem with a building of the project

dm-mbp:rustfmt minsler$ rustc -V
rustc 1.0.0 (a59de37e9 2015-05-13) (built 2015-05-14)
dm-mbp:rustfmt minsler$ cargo -V
cargo 0.2.0-nightly (efb482d 2015-04-30) (built 2015-04-29)

dm-mbp:rustfmt minsler$ cargo build
   Compiling strings v0.0.1 (https://github.com/nrc/strings.rs.git#551331d0)
/Users/minsler/.cargo/git/checkouts/strings.rs-3047d54c6ac12a97/master/src/rope.rs:20:1: 20:28 error: can't find crate for `rustc_unicode`
/Users/minsler/.cargo/git/checkouts/strings.rs-3047d54c6ac12a97/master/src/rope.rs:20 extern crate rustc_unicode;
                                                                                ^~~~~~~~~~~~~~~~~~~~~~~~~~~
error: aborting due to previous error
Could not compile `strings`.

Thank you for help in advance.

I think it is because you are building with the release version of Rust. rustfmt requires a recent nightly. You can install nightly Rust with
curl -sSf https://static.rust-lang.org/rustup.sh | sh -s -- --channel=nightly

If you want to also run release builds, you should look into multirust.

Thanks,  it was built with a recent nightly version.


MacVim hangs for minutes with hundreds of threads

Describe the bug
I'm a daily user of MacVim and my work depends on it. MacVim occasionally hangs for minutes with hundreds of threads (500 or more) when editing some specific large file(s). I had this happen several times, but only once a day at the most. It could be a syntax highlighting problem perhaps. It's difficult to reproduce exactly.
To Reproduce
Detailed steps to reproduce the behavior:

Run open -a MacVim Forth500.s
Edit Forth500.s and perhaps some other files in other windows (but don't think that matters)
Change Syntax to Assembly/Turbo then navigate, search, type, etc.
Eventually the problem occurs, but I can't tell when and why. It may happen after a while, so it's not frequent, fortunately for me.

Expected behavior
No hot laptop!
Screenshots
N/A
Environment (please complete the following information):

MacVim version: Custom Version 8.2.2681 (171)
OS: MacOS 10.15.7
Terminal: N/A

Additional context
The Forth500.s file is here: https://github.com/Robert-van-Engelen/Forth500/blob/main/Forth500.s

Hi, could you enlighten us with a bit more information about environment and test a few more things?
Please tell us:

which MacVim version do you refer to?
which macOS version do you use?
how have you installed MacVim?
have you overwrote vim supplied by MacVim by any other version of vim?

Please test your issue in terminal:

using vim (in terminal, supplied by MacVim)
using vim (in terminal, other version available)

Please test your issue with no plugins involved:

using mvim --clean (no configurations or plugins involved)
vim --clean (in terminal, supplied by MacVim)
vim --clean (in terminal, other version available)

See earlier info on version and OS. I run MacVim GUI. I don't use plugins (.vimrc attached as vimrc.txt), nor am I doing anything special other than just editing a few files with source code: C/C++, assembly (.s with Turbo syntax hl), Forth, markdown. The assembly file is large (see my previous comment). All other files are small/tiny. With this, MacVim generally uses 4.71GB of virtual memory and has 31,728 ports open when I use it for a few days.
I took two MacVim process samples (with MacOS Activity Monitor) of the MacVim process when it went crazy again, hanging for several minutes with hundreds of threads and all 8 cores running hot. The samples may offer some insight into the problem, because this intermittent problem is not easy to debug I'm sure. It happens about once a day when working with MacVim for hours.
vimrc.txt
MacVimsamples.zip
I will also try and see if daily restarting MacVim and/or my system can avoid this problem, keeping an eye on the process stats.

Thank you for sampling and your vimrc.
I have the same macOS version,
I have x86_64 arch,
I have the same version of MacVim installed from GitHub releases (no HomeBrew or MacPorts is involved)
In your vimrc I made GUI part be enabled all the time to have same configuration in GUI and terminal.
I opened your assembly file in using MacVim GUI, terminal vim from the same app bundle and /usr/bin/vim to make sure I covered all cases
I have no work within the file, so I scrolled a lot for few minutes in each editor and did some editing here and there.
I see no deviation in Application Monitor as you describe.

I've also tried to install ugrep 3.3.7 with newest ctrlp.vim plugin (thanks!), but don't see any deviation at all.

With all this, could you update your ugrep (which is not compatible with your configuration) and ctrlp.vim and try again.
I suppose, the old plugin version causes the problem. My vimrc you can find here (be careful with the dot in repository name)

Thank you! I will follow your suggestions and will keep testing (I use MacVim a lot!). Like I said, it happens sometimes.

This version of MacVim may have a memory leak that may explain the problem I'm seeing. I ran MacVim for three-four days without quitting. When I open the laptop daily and return to editing with MacVim, sometimes I see the spinning wheel. When that happens, the memory usage increases by 300MB or so. I started with less than 300MB and now at about 1GB reported by Activity Monitor on MacOS 10.15.7. I have just been editing the same three files (assembly and two MD files) in two windows. Eventually the spinning wheel issue gets progressively worse, hanging for minutes with several GB of memory use. That happens randomly, roughly once a day as reported earlier. The only remedy is to restart MacVim.

could you try to edit this file in a terminal and see what happens, like /usr/bin/vim?
the beginning of your configuration file could be as following to run same commands as GUI. To test with specific configuration file use /usr/bin/vim -u "/path/to/vimrc" -U "/path/to/vimrc". On my system with similar version /usr/bin/vim has version 8.1.2292 which is pretty close to MacVim.
"if has("gui_running")
  " Get the value of $PATH from a login shell.
  " If your shell is not on this list, it may be just because we have not
  " tested it.  Try adding it to the list and see if it works.  If so,
  " please post a note to the vim-mac list!
  if $SHELL =~ '/\(sh\|csh\|bash\|tcsh\|zsh\)$'
    let s:path = system("echo echo VIMPATH'${PATH}' | $SHELL -l")
    let $PATH = matchstr(s:path, 'VIMPATH\zs.\{-}\ze\n')
  endif
  " Change directory on startup.
  autocmd VimEnter * if getcwd()=="/" | if strlen(@%) | cd %:p:h | else | cd | endif | endif
  " If running in a Terminal window, set the terminal type to allow syntax highlighting.
  set guioptions-=T
  " set toolbar=text
  " set toolbariconsize=medium
  " set mousefocus
  set number
  set cursorline
"else
"  set term=builtin_ansi
"endif
" Grepping

Maybe you have also an updated configuration to match newer versions of ugrep and ctrlp.vim?

I've already made the ugrep and ctrlp.vim changes a few days ago and tested it out.  I will try the GUI settings. It looks like you've commented out set term=builtin_ansi.
Besides increased memory, I also see a growing number of "Ports" in Activity Monitor: now 6273 open ports and 1GB memory. Restarting MacVim to edit the same files and the updated .vimrc shows 147MB with 276 ports. I'll run this for a few days and see what happens.

Thank you, could you share your new vimrc for me to test it simultaneously?

Could you share how did you installed MacVim?

Vimrc:
vimrc.txt
The problem is observed with the GUI, I have yet to test the terminal-based vim.
I installed MacVim a long time ago, pretty sure this was installed from a DMG file and later updated via the auto built-in updates (checking for updates reports fully up to date).

Thank you for an updated version of vimrc file

I installed MacVim a long time ago, pretty sure this was installed from a DMG file

Thank you, it's very important that we use the same MacVim

I still have this issue. I doubt it is the history mechanism that lets memory grow in big increments. I am not moving bug chunks of text around either and I've closed all windows except one (the assembly file). I am back to 2GB with 13,641 "ports". When the spinning disk appears, hundreds of threads will run for seconds to minutes. I noticed it happens when switching windows in MacVim and switching split screens with the cursor (with touchpad, I have not tried CTRL-W),

I'm sorry to hear that. I'll profile MacVim. If I may to ask, could this file be separated into few ones. From my experience with Assembly language, I almost always separate code, use includes or link them together on the link stage.

The original wasn't mine to begin with and I need/want to keep it this way for several reasons. I doubt it's the assembly file actually, since I witnessed the spinning wheel before when returning to work on a project with MacVim open, but didn't give it much attention. Restarting MacVim is something I usually do.

I see, I'd split it anyway. It's way easier for me to find things in many files than in a single one.
I forgot to ask, which preferences (not vimrc) do you have?

I need the output of the command:
$defaults read org.vim.MacVim

Here you go. This intermittent problem is probably difficult to fix. I can live with it by restarting MacVim. But it should not have to be that way.
{
    MMAutosaveColumns = 152;
    MMAutosaveRows = 70;
    MMCurrentPreferencePane = General;
    MMLastWindowClosedBehavior = 2;
    MMTopLeftPoint = "{159, 1026}";
    NSFullScreenMenuItemEverywhere = 0;
    NSQuotedKeystrokeBinding = "";
    NSRepeatCountBinding = "";
    "NSWindow Frame FindAndReplace" = "113 351 490 169 0 0 1680 1028 ";
    "NSWindow Frame SUUpdateAlert" = "540 476 620 392 0 0 1680 1027 ";
    SUAutomaticallyUpdate = 0;
    SUCheckAtStartup = 0;
    SUEnableAutomaticChecks = 1;
    SUHasLaunchedBefore = 1;
    SULastCheckTime = "2021-09-14 20:58:47 +0000";
    SUSendProfileInfo = 0;
    SUUpdateGroupIdentifier =<PHONE_NUMBER>;
    SUUpdateRelaunchingMarker = 0;
}

This is what I get when inspecting the MacVim process after it OK again (by comparison, when MacVim is unresponsive "spinning disk", hundreds of threads will run). I have very few other tasks running. MacVim consumes the more memory than all of them. The "real memory" size of MacVim is not bad, but Activity Monitor reports 2.18GB and this grows in big leaps suddenly (300MB or much more) when MacVim becomes unresponsive. Heap fragmentation may also be present, but that would not explain the unresponsiveness delay that gets worse over time when hundreds of threads are spawned.

Hi, I'm sorry for the late response. I'm taking a look, but since this seems to be a memory leak it's going to take a while to repro properly. If you see this again, can you do the following?
ps aux | grep -i vim

You should see something like this:
ychin            49390   0.0  0.0  4268424    720 s006  R+   12:16AM   0:00.00 grep --color=auto -i vim
ychin            49325   0.0  0.1  4467028  19704   ??  Ss   12:10AM   0:01.13 /Applications/MacVim.app/Contents/MacOS/Vim -g -f
ychin            49324   0.0  0.5  5200664  89640   ??  S    12:10AM   0:05.35 /Applications/MacVim.app/Contents/MacOS/MacVim

In MacVim, there is an overall macOS process called MacVim, and each window is a child process Vim (which is roughly the same process as terminal Vim). It would be interesting to see whether it's the parent process or the child process that's leaking.

As pet my tests, it seems to be parent process. Numbers obviously increasing, but I couldn't reproduce so huge growth as described, probably I do it wrong.

Huh ok that does make sense, given the thread count explosion. I couldn't really reproduce myself so far, but I did see thread count go up and down so that should give at least some lead. One thing that may be related is the CoreText stateful renderer but I'll need to take a look.

Still happening in the latest 8.2.3455 MacVim. Up to 2.4GB after a few days of editing some files. It's typically increasing memory when opening a new file from the terminal with open -a MacVim, which will open a new window in MacVim to edit the file.
The problem is not an issue the first day or so, but as time passes and more files are opened and closed (I keep only a handful open), this problem will always occur. It seems that MacVim tries to allocate memory while the process is swapped from physical to virtual memory. So when MacVim is not used for some time and other applications are active, swapping MacVim back to physical while opening a new file is probably the best symptom to describe this issue.


Create symlink pointing to workdir of the last test run

Minor change intended to simplify checking of logs from the last run (e.g. in case with large number of test runs is present).

Sorry for the late response. Seems this pull request slipped through the cracks. So the main motivation of the newly added symlink is to make it easy to check the last executed run. Isn't this a bit duplicating the --last option? For example:
tmt run --last report

Also, in order to check the log of the (last) running job the --follow option can be used. Unfortunately this is currently broken due to a recent regression but should be fixed by #2248:
tmt run --last --follow
tmt run --id 123 --follow

Btw, the --last option is also implemented using a symlink and suffers from the problem when multiple runs are executed in parallel, see #525. Thus I'm not sure whether the new symlink under /var/tmp/tmt is the right approach.
@lukaszachy, @happz, thoughts?

Sorry for the late response. Seems this pull request slipped through the cracks. So the main motivation of the newly added symlink is to make it easy to check the last executed run. Isn't this a bit duplicating the --last option?

I also have to sorry for my response :-/ Yes, you're right about the motivation. It seemed to me like something I would use that time (it still does).
Regarding --last option, I am aware of that. I consider this new functionality to be complementary to it. You can easily reach the latest result just via knowing the expected path (and can be simply embedded into some script executing tmt run etc) without necessity of triggering new tmt run.

Btw, the --last option is also implemented using a symlink and suffers from the problem when multiple runs are executed in parallel, see #525. Thus I'm not sure whether the new symlink under /var/tmp/tmt is the right approach.

That's true, this was about to address more basic scenarios other than parallel execution. Regarding where to place symlink, I am completely opened to ideas here. I don't have any strong preference, it just seemed logical to me.

What would be the difference between the last-run symlink created by tmt.utils.Config and this new symlink? They seem to be doing the same job AFAICT.

The motivation here was that symlink is created no matter any special option like --last is passed to tmt command. It is just created for every single run of tmt without necessity of thinking about getting the last run before.

The motivation here was that symlink is created no matter any special option like --last is passed to tmt command.

Actually, that is exactly the case of the symlink created by tmt.utils.Config, the symlink is created always:
> tmt run discover
/var/tmp/tmt/run-162

/plans/example
    discover
        how: shell
        summary: 1 test selected

> ls -l ~/.config/tmt/
celkem 12
-rw-rw-r--. 1 psss psss  61 16. led  2020 cli.fmf
lrwxrwxrwx. 1 psss psss  20  6. říj 18.20 last-run -> /var/tmp/tmt/run-162
-rw-rw-r--. 1 psss psss 652  4. dub  2022 tmt-complete.bash

Seems to duplicate existing functionality, closing. Feel free to reopen of you don't agree.