iowait and memory consumption are skyrocketing regularly

Gitea version (or commit ref): 1.10.0
Git version: 2.16.4
Operating system: openSUSE Leap 15.1
Database (use [x]):

[ ] PostgreSQL
[x] MySQL (MariaDB)
[ ] MSSQL
[ ] SQLite

Can you reproduce the bug at https://try.gitea.io:

[ ] Yes (provide example URL)
[ ] No
[x] Not relevant

Log gist:

Description
As written earlier in the Discourse forum, it regularly happens that gitea makes my server's iowait and memory consumption skyrocket, causing issues for all other services running within the same VM. Usually only killing the whole gitea service makes the server recover.
While I couldn't find any direct cause by parsing the logs or the PPROF data, I just discovered in the admin web interface (tab Monitoring) that the GetDiffRange operation seems to be responsible. It seems to get stuck at a single repo of just 138MiB size, clogging IO and making gitea consume more and more memory. iotop shows that gitea creates a lot of I/O, although I'm not sure whether swapping operations are allocated to the process causing them as well.
After this issue first occured, I limited the gitea systemd service to 600MiB and gitea is currently using all of it. But apparently this limit isn't working as the memory+swap usage increased by more than 3GiB during the GetDiffRange operation.
What does this GetDiffRange operation do? I wasn't able to find information on that.
Luckily I was able to allocate an additional 1GiB of memory to my VM, giving me the chance to let the operation run. So far it's running for more than 30 minutes though.
Any ideas about that operation, why it's getting stuck or any additional debug information I could provide?

After ~45min the GetDiffRange operation vanished from the Monitoring web UI, but both memory consumption and IOwait remain equally high. I see myself forced to kill gitea after additional 15 minutes.

Killing the gitea service immediately freed 4.5GiB of memory.

There is a bug about OpenRepository has been fixed on v1.9.6 or v1.10.0(1?). Could you try v1.10.1?

will do

I think it's unlikely to be fixed by that - the architecture of GetDiffRange keeps the whole diff in memory. (Probably multiple copies of it too.) If you have a huge diff you're going to break Gitea.

References:
https://github.blog/2016-12-06-how-we-made-diff-pages-3x-faster/
https://github.com/go-gitea/gitea/blob/master/services/pull/merge.go#L504-L545

We can also cache the diff sections to cache or files.

The issue is still happening regularly even with v1.10.1, but I haven't always seen the "GetDiffRange" operation showing up on the Monitoring page. So GetDiffRange isn't necessarily the cause of this issue.
The gitea logs only show a huge amount of mysql queries and I wasn't able to get anything from the PROF data (I'm not a go developer).
If anyone thinks they can get relevant information from there, I'm willing to pass you the logs and prof data.

So that maybe some cron tasks.

Hmm excessive Perm checks are a possible cause - see the comments on #8540?
There's another possible issue - testPatch was previously reading the whole patch in to memory. That should stop now.

So that maybe some cron tasks.

Judging from the (non?)periodicty-time the issue appears, is at least none of the tasks with a periodicty of 10min or 24h listed in the monitoring tab.

Hmm excessive Perm checks are a possible cause - see the comments on
#8540?

I don't have any repos with an excessively large number of owners, which seems to be the precondition for this issue.

There's another possible issue - testPatch was previously reading the
whole patch in to memory. That should stop now.

Under which circumstances does testPatch get triggered?

Changes to PRs would cause that

Do you have repo indexer on? Maybe that could be a cause?

I didn't modify any PRs in the meantime and do not have the indexer enabled (if it's disabled by default). git-lfs is enabled though.

If the PROF files may be useful, I can provide you with those. I was too dumb to analyse them myself.

Any ideas on what I can do to figure out what the memory is even used for?
Maybe taking the memory dump of the gitea process(es) via /proc/ and somehow analysing it?

OK so on master I recently made a change that means that PR patches are not stored in memory - that could have been to blame.
The other places that are currently poorly written are diff and blame as these both read the whole thing into memory.
I've never used the prof output so I'd be starting from scratch with that.

OK so on master I recently made a change that means that PR patches are not stored in memory - that could have been to blame.

I really doubt that, as my repos have nearly no PRs.
Once I find the time I'll try disabling individual repos, maybe I can pin down the issue to a single repo.

I'd like to disable individual repos to see whether a single certain repo causes this issue.
Is there a convenient way to do this or do I need to delete the repos individually and later re-add them and restore issues + PRs from backup?

If you're using Linux, maybe you can try the following command to check what processes are using what repos at a given time:
find /proc -type l -lname '*gitea-repositories*' 2>/dev/null |
egrep '^/proc/[0-9]{1,}/fd/[0-9]{1,}$' |
tr '/' ' ' |
while read dummy1 pid dummy2 fd
do ps -f -p $pid 2>/dev/null ; readlink /proc/$pid/fd/$fd 2>/dev/null
done

The above command requires using root credentials.
With Windows you'd need sysinternals' process monitor (MS official).

I deleted all repositories but one, now the memory issue vanished.
I am going to try to pin down which (single?) repository is causing the issue.
Unfortunately I am inly able to do this on my production instance. So if there is a more convenient way of disabling and restoring repos than gitea dump and restoring all repos afterwards, I'd be glad to hear them.

This is still an issue, but unfortunately I haven't been able to pin it down further.
The current memory stats of my gitea 1.11.5 instance:
According to htop, the gitea web process currently uses 2194MiB of memory (RES) and has 8190MiB allocated virtually (VIRT).
 I'm using the Docker image with version 1.13.0+dev-329-gd25f44285.
I see that one of my repositories has some huge diffs, so I'll try remove the repository from Gitea and report back if that changes anything.
Down below you see how I get weird spikes in memory usage, caused by Gitea. When the Docker container is stopped, memory recovers to somewhere between 30% and 35%.
Also note that on this instance, LFS is disabled.

@eikendev What do you categorise as huge diffs and how can I check whether one of my repos falls into that category?
Gitea is still acting up, eating all my memory and swap:
$ ps auxm |grep gitea
gitea     8630  1.4 82.6 9303548 3336128 ?
There are 13 gitea web threads running which, as I read somewhere, isn't supposed to be the case, right?
 I've got a few ideas here:

Our blame code used to potentially fail to close the git process - that should be fixed on 1.13 and I think it was backported to 1.12. So that would be a cause for defunct gits on 1.11.8.

Another cause of the defunct git processes could be git gc.

When we run git gc git will attempt to repack things. This process can take a lot of memory and would cause a regular spike.
It's potentially worth running git gc on the command line for the in turn repositories. It may be that we have incorrect options or you need to set the pack configuration options in your global git configuration. If these git gc processes die then they can go defunct - but you should have some information on the processes page when this is happening.

I see that huge heap in the process memory  - that's interesting because it implies that Gitea has actually read in and required that in one go. It would be really helpful if you could try to reproduce that or point to a public repo that causes Gitea to do that. If I had to guess at a cause it would be a compare page but it would be very helpful to prove it and be able to reproduce it - as then we can work to fix it.

@zeripath

It would be really helpful if you could try to reproduce that or point to a public repo that causes Gitea to do that. If I had to guess at a cause it would be a compare page but it would be very helpful to prove it and be able to reproduce it - as then we can work to fix it.

I've already tried to bisect the repos to find the responsible one, but the process for doing so is quite bothersome:
Due to the lack of repo-level import/export, I always had to backup the whole gitea instance, remove some repos, wait to see the result, and then nuke the whole instance before restoring the whole backup.
Is there any better way?

@eikendev What do you categorise as huge diffs and how can I check whether one of my repos falls into that category?

It was a repository where a file of over 100MB was added, something that actually belongs into lfs.
I guess my specific case can be detected by looking for large files inside the repository. About the general case, maybe git log --stat helps.

@eikendev that's bad - do you know if it was when the file was pushed or when someone looked at the diff page?

oo interesting I just pushed a file with no newlines and memory shot up.

@eikendev that's bad - do you know if it was when the file was pushed or when someone looked at the diff page?

In the graph I posted earlier I have neither pushed anything nor loaded any huge diff. I looked at the repo at some points, but not the diffs. When memory is freed in the graph, this would indicate I manually restarted the server.
To me this seems more like a product of a scheduled task.
I'm not sure right now if my issue and @schmittlauch's are related.

Do you see what is exact git command (with arguments) that is left dangling?

@lafriks Currently there is no dangling git command after having restarted gitea.
I'll keep that in mind for the future, but the growth of memory consumption happens independent of dangling git processes being present or not. Right now my 3.5GiB memory are nearly full, the growth will continue into swap the next day.

OK I think this is going to be related to go-git related issues.
I think therefore I'm going to pull the confirmed from this tag to see if 1.14 still has this issue.

I think I might have discovered the underlying issue here - if git cat-file is called on a broken git repository it will hang until stdin is closed instead of fatalling immediately. I think therefore #17991 will fix this.

This issue might even be resolved already, but I cannot say for sure as I moved my installation to a new installation, cleaned up some old repos, and thus might have removed such a broken repo coincidentally.
So I suggest to indeed close this once #17991 gets merged, unless some new instance of these symptoms here comes up.


Identify peer on OnReceive event

I'm changing some of my code to use the NetSerializer.Subscribe event. How can I tell for sure what peer sent the information since the OnReceive for the Subscribe only includes the deserialized struct?
...
_netSerializer.Subscribe(OnSamplePacketReceived);
...
private void OnSamplePacketReceived(SamplePacket samplePacket)
{
// How to determine peer that sent here?
}
If multiple peers are sending, the OnNetworkReceive(...) event is too far removed to reliably determine peer associated with the OnReceive Action. In my situation, I'm receiving a request that expects a response.

Ok. I will add this possibility.

Thank you. I'm enjoying this library a great deal and it has eased my Udp implementation so much.

Added support for any user data.


add demo for singularity

Demo of executing Singularity container. Using ESMValTool as the example.

Hi @meiertgrootes @fnattino and @SarahAlidoost , the demo notebook for the singularity container + dask is ready. Theoretically, it should be self-explained. Would you like to review it?

@rogerkuou nice work +1 I was able to run the notebook on Snellius. I noticed that the future jobs stay pending even though the esmvaltool runs are successfully done. And just some suggestions:

adding some logos in the first cell of the notebook, logos of the RS-dat, center and surf.
moving the cell "Set up the commands for execution" above the cell "Step 3: Execute recipe" because the explanations of step 3 are not relevant for creating the commands.
refactoring the cell  "Set up the commands for execution" using pathlib instead of os library (don't forget to move import os to the cell calling os.system) :

# Set up the commands for execution
from pathlib import Path

# Get the abusolte path for the sif image
sif_image = Path.cwd() / 'esmvaltool_stable.sif'

# Two recipes for two datasets
recipes = [Path.cwd() / 'recipes/recipe_dataset1.yml',
           Path.cwd() / 'recipes/recipe_dataset2.yml'
          ]

# Set up shells commands
commands = [f"singularity run {sif_image} run {recipe} --offline=False" for recipe in recipes]
commands

Note: Singularity has an option --pwd for setting the working directory inside the container. Therefore no need to make the paths absolute. The codes below work in the notebook, but not with  client.map. I couldn't find out why.
!singularity run --pwd {Path.cwd()} {sif_image} run {recipe} --offline=False

Update: the code including --pwd is now working too :bowtie: :
# Set up the commands for execution
from pathlib import Path

# Get the absolute path for the sif image
sif_image = 'esmvaltool_stable.sif'

# Two recipes for two datasets
recipes = ['recipes/recipe_dataset1.yml',
           'recipes/recipe_dataset2.yml'
          ]

# Set up shells commands
commands = [f"singularity run --pwd {Path.cwd()} {sif_image} run {recipe} --offline=False" for recipe in recipes]
commands

Hi @meiertgrootes @SarahAlidoost  @fnattino Thanks for all your comments! I applied almost all of them. Will merge it now.


ollama run qwen:0.5B, Reply exception, stuck in a loop.

>>> /show info
Model details:
Family              qwen2
Parameter Size      620M
Quantization Level  Q4_0

~ uname -m -s -r
Darwin 23.3.0 arm64

https://github.com/ollama/ollama/assets/13782141/746225cc-9147-40e3-b7c3-d40a963fa2d5
/label bug

I had the same behavior with phi2 model. I noticed that the model gives the right or the expected answer before going to a new line (\n). So I had to add "\n" in the stop list.
const stream = await generate({
        model: "phi",
        prompt: text,
        stream: true,
        options: {
            num_predict: 70,
            temperature: 0.65,
            penalize_newline: true,
            top_p: 0.9,
            // presence_penalty: 0.6,
            stop: ["\n", "User:", "Assistant:", "User:"] //["\n"]
        }
    })

It still cuts at a wrong place sometimes, but I can manage to just remove the words after the last punctuation: . or ,
This method will not work if the user ask for a list as a result (give me 3 recipes of cappuccino) -> then after generating the first, the model will try to add a new line for the second element of the list, and it becomes more complicated to control the level.
(any workaround for this use case?)

The infinite generation should be fixed now. As for the poor responses from smaller models – this may be from the prompt template, prompt or other reasons – all of which we are trying to improve.

@jmorganca So we should update the Ollama binary verison then right ?


Notification is not using app icon

Hello,
i have icons in flutter changed to my own, app is installing with my icon, but notification still using flutter default icons, i tried to change it in NotificationChannel  icon property but not works....
why there is always used  resource://drawable  in examples? this is android specific :|

Take a look at this video tutorial;
https://youtu.be/JAq9fVn3X7U

Do something like this
AwesomeNotifications().initialize(
       'resource://drawable/app_icon', // icon for your app notification
       [
         NotificationChannel(
             channelKey: 'key1',
             channelName: 'VaxCEM Care',
             channelDescription: "VaxCEM Care",
             ledColor: Colors.white,
             importance: NotificationImportance.High,
             playSound: true,
             enableLights: true,
             enableVibration: true)
       ]);

thanks, I found this kind of configurations in examples tutorials and so on, but will it work on IOS too?

from what I tested, yes it works

Hi! Sorry for taking so long to respond to your issue on GitHub. I was focused on releasing the new 0.7.0 version of awesome_notifications and awesome_notifications_fcm and i didn't had time enough to do both. But now I can answer all your questions.
So now i'm asking you to recreate this topic using the new issue template. There's a lot of missing informations that i need to understand your problem, and all the instructions are already in the new issue template. Also, remember to check beforehand if your issue was posted by another user.
So I will automatically close all previous issues so far.
Sorry for the inconvenience and i will be waiting for your new issue request.
Thank you so much for your support!


create_r_baseimage.py do not build the cointaner correctly

I have been trying using that code but it starts building the container but never finshes it, the name and tag in the docker image list it kept stuck as  about 300MB without going further.

Are you able to create Python base image?
R base image generation requires extra packages to be downloaded. Make sure that you have enough available space in the host where Docker daemon is running.

The python worked flawlessly.
Yes, I do have plenty of space. It's a localhost registry/daemon
/dev/sda3       299G  198G  102G  66%
I could build it using an r-base from docker hub directly from bash, is it the same file?
Just tried again, this is how the container kept stuck using this code.

REPOSITORY                              TAG                 IMAGE ID            CREATED             SIZE
<none>                                  <none>              fed2b93076b7        2 minutes ago      394MB

I doubt it is due to centos version. When the Viya 3.5 was developed and released, the centos latest version was 7.0. Our base images were  based on centos:latest. So we didn’t notice any problem in the testing that time. Now the latest centos is 8.0. It may cause issue downloading some R dependency packages. Here is the workaround:
First, login into the machine where Docker daemon is being installed;
Second, Execute
docker image rm centos:latest
to remove centos:latest from local cache;
Last, run the following two commands to pull down ‘centos:7 image’ and tag it as ‘centos:latest’
docker pull centos:7
docker tag centos:7 centos:latest
Then restart your base image generation script.

Now it got stucked in even earlier stage of the container, but I don't know if it makes any difference. 203MB

Yeah, didn't work. 20 minutes building and got to 491MB, little more than before.
Even though this time there is a container running which i think it's the centos, before it wouldn't show at all.


schemaVersion not exported

related: #753
with the latest duffle built from master, the schemaVersion from duffle.json is not exported in the "bundle.json" file.
To reproduce:
duffle create foo

add the line: "schemaVersion": "v1.0.0-WD", to duffle.json
duffle build .
duffle export -t foo
tar -xvzf foo-0.1.0.tgz

verify that schemaVersion is not set:
  "name": "foo",
  "schemaVersion": "",
  "version": "0.1.0"

This is bad because a bundle that was built & exported with the latest master now fails to install with the message:
Error: invalid bundle schema version "": Invalid Semantic Version


Reveal in Side Bar not working

Issue Type: Bug
I opened index.d.ts file from my node_modules folder with command + click on the imported object, then when I try to right click on the editor tab title and select "Reveal in Side Bar" nothing happens, side bar doesn't respond on this.
 What are you doing differnetly than me?
Can you repro with vscode insiders https://code.visualstudio.com/insiders/
If yes, can you share a repository for which you can always reproduce?
Also do you see some error if you open F1 > developer tools > debug conosle?

@isidorn Try to open file from node_modules folder, I checked debug conosle, no errors. Here is small video: https://screencast.com/t/9054kgKXy8

Also works fro mnode_modules.
Please answer my other questions. Thanks

It's the same in insiders, it also works only with index.d.ts file from node_modules, here is video from vscode insiders: https://screencast.com/t/4ALaicCVUUQ here is that repo I work with https://github.com/vedmant/running-time-react

I have just cloned your repo. Can you just write the exact steps you do when you open that repo?
I do not have flash player and do not see your videos. So please provide steps, thanks

Open screens/Entries/EditEntryScreen.js, click on import { StyleSheet, View } from 'react-native' View with command button pressed. It will open<EMAIL_ADDRESS>this is where Reveal in Side Bar not working.
Here I made mp4 video https://a.uguu.se/7VmCKk1MAvm2_Nov-16-201900-02-32.mp4

That file is outside of your workspace, thus the reveal in side bar does not work. It is not in the sidebar, it is not part of your workspace.
Reveal in Finder for example nicely revelas it in the Finder.
The issue is that Typescript cmd + click, navigates to the index.d.ts that is outside of the workspace, and not inside node_modules.
The issue for this is that you do not have react-native types in your node_modules. So please setup the dependencies in your project proprely.
Closing since this is designed behavior

Yeah, I understand that, I didn't expect it will open anything outside of the project. It also was not clear why it didn't work cause there were no any message, it just silently didn't work. I think it might be a good idea to notify user why file can't be revealed in the sidebar with some message.

Yes, the sidebar could notify the user why it does not work.
However I am not a fan of notifications and too much spam and will only add it if more users encounter the same issue. Hope that makes sense.


(#7) Sync with code in mcorpc-ruby-support

Fix #7

Thanks, I still would rather this file isnt here at all - but been completely snowed under and had no time to check it :(


Fix getDfsShareAclView broken by bcc6807

Oh wait, please s/bcc6807/commit bcc6807/ in the commit log so that GitHub will make it a link.
John L

The unit test I added does replicate the breakage as reported in the bug.  The other breakage in subsequent additional entries are failures in the callback, followed by the original failure.


[Bug] Water Bottle Duplication

The liquids from Big Reactors are being registered as Material.water thus allowing players to create a unlimited water source from the fluids. This affects modpacks which rely on making the player think about hydration and so forth, as the player just needs to use one of your fluids and can just keep taking all the water they want.
Would it be possible for you to create a custom fluid material to register your fluids under to prevent this issue from happening?

How are your players gaining access to those fluids? They're placeholders, not intended for actual use.
The BR fluids aren't exposed anywhere via recipes in BR itself; to my knowledge, you can only obtain them via admin actions.

I'm not totally sure myself I will have the ask the players of my modpacks. But I know some mods allow you to pump the fluid's into the world, and some other mods are registering your blocks to be smeltable  (Example being AOBD) which then you can pump the fluid into the world ; so this could be the method that they are using. I understand that the fluids are not "survival" getable but I'd thought I would bring this up to you.

The actual bug, then, is that other mods are erroneously exposing placeholder/debug content that's not intended for consumption. Please inform them.

It should not be left up to other mod developers to fix YOUR liquids that are registered wrong. This could fixed with half a line of code (change the material type). It is also not feasible for developers to have to ask every other developer whether certain liquids/items/blocks should or shouldn't be accessed from their mods just because they left unfinished placeholder/debug content there.

I disagree. If you're integrating with another mod, you damned well need to know that mod - including whether something should be made user-accessible or not.
You don't expect an external mod to make creative mode items accessible either - this is an identical case.


Generate parsable ObjectDetection JSON format

Resolves #17
Depends on #15
- Change JSON response format. Instead "person" : 66.66 use "name": "person", "confidence": 66.66 instead
- Introduce ObjecDetection class to represent the returned JSON format
- Replace all Tuple usages by plain JSON (leverage the ObjectDetection class) - related to #18
- Update README doc to reflect the json format change

LGTM, merging.

Rebased, cherry-picked the changes applicable to this PR and merged as 506ede3


[BUG] replace lower quality and broken images on Vue site

Describe the bug
We have a couple of blurry and broken quality images that need replacing on the Vue site.
Replace the blurry Vue Amsterdam image in the communities section

Replace broken image for Nuxt Nation Event

Note
If you are outside of the core This Dot team and interested in working on this issue, please leave a comment below and we can assign it to you.

Can I work on this issue?

@aksy2512
The issue has been assigned to you 👍


Progress2

Hi, I've see this thread: http://forum.sa-mp.com/showthread.php?t=629401
You have write this:

The include also support previous version (1.0 and 2.0) syntax and function names.

I'll try your include and is not present none of the written functions, especially that for global bars.
Why?

Hey, sorry for late reply.
So i am planning on restoring the old version which supports old syntax with some internal changes and probably new design for progress bars. Expect a thread on forums includes section soon.


[Candidate 1v1 Tragedy Tracks] Water has very saturated blue color

Issue Description
Water has very saturated blue color.
Expected behavior
Water has lighter color like in previous version "Arab Tradegy ZHv3" which fits terrain better.
Screenshots

Fixed with https://github.com/xezon/GeneralsRankedMaps/commit/8e311c8cf1238a2572d3a953559b01fd392f2c5f


Multiple search tabs in bottom panel

Hello,
I have worked a lot of years with the Eclipse-based editors, and now at the firm we work for we have to evaluate the VS Code in order to make a switch.
After a few hours of using the VS Code (version 1.59.0) I have noticed what is for me a major step back compared to the Eclipse: the search function.
Eclipse has a very well organized visual interface

the list of files in the left part
the opened code files in the middle of the screen
the search tabs in the bottom panel.
I have managed to put in VS code the search tab in the bottom panel:

But if I try to open a new one, it will open in the main window, in between the code files:

Also, for the search in the bottom panel I can expand/contract the files in which the search item was found. In the additional search windows opened afterwards I do not have this option.

vs

I do not like this mix: code and search tabs in the same space. Code should be separate.
In Eclipse you can either choose to have one search window, or you can pin them and  open several search tabs in the bottom panel:

They are all the same as features (for all you can expand/contract the file names)
It is much easier to navigate this way.  Also now in VS code the bottom tab is a waste of space. The code area is overcrowded with the search part, and the bottom tab is unused.
Is there any way to have this also in VS code?

No, we don't support multiple search tabs like that. You've found our search editors, and you can have multiple of those if you want. You can also put them in a separate editor group if you don't want them mixed with your code files.


Huge jump in t_done median when moving to the latest boomerang version

I've been running with the boomerang script from this version (https://github.com/lognormal/boomerang/commit/035b7ea69157de6524acf019a04479eabaadf52c), and I recently updated to the latest (https://github.com/lognormal/boomerang/commit/1614c7b84856397b7dbeeca17021ea461fdd72a6).  I'm using the RT plugin.  Since doing that, the t_done measurements have jumped up, the median has gone from 2400ms to around 5000ms.
You can see in the graph below, I turned it on for a few hours, then rolled back to the previous version.  Also you can see that the t_page and t_resp numbers don't move.

Is this something I should have expected to happen?  Has their been a significant change in the way the t_done is calculated?  The only other change that I made when I upgraded was to stop using the navtiming plugin, I was using that before but not in version I deployed with the latest boomerang code.

Closing this as there hasn't been any activity for over a year.  Please re-open if you have additional details.


Test(web-react): Unnecessary use of Date.now refs #DS-766

Description
Replacing Date.now with static value.
Additional context

Issue reference
https://jira.lmc.cz/browse/DS-766

Before submitting the PR, please make sure you do the following

[ ] Read the Contributing Guidelines.
[ ] Follow the PR Title/Commit Message Convention.
[ ] Provide a description in this PR that addresses what the PR is solving, or reference the issue that it solves (e.g. fixes #123).
[ ] Ideally, include relevant tests that fail without this PR but pass with it.

LGTM. And can I ask why is it bad to use date here? Is it because the value changes over time? Or is it slow? Thanks!

Both. And performance reasons. Date.now generates every time a unique value and it is not repeatable. And the second reason is that it generates. It is an unnecessary operation so it slows down the tests a little bit. And of course, it can be a source of future problems in tests when some will start to work with dates and times on files and it will change on every run.


The highlighting of PascalCase and kebab-case cannot be distinguished

Using Vue-Official in VsCode, it is not possible to distinguish between the highlighting of PascalCase and kebab-case.

Vue-Official effect

Vetur effect

Why can't volar distinguish highlights?

Why you want to distinguish highlights?

Why do you want to distinguish highlights?

In the previous use of Vetur, it could distinguish between the highlighted colors of PascalCase and kebab-case. So I choose to use kebab-case when using third-party component libraries and PascalCase when using custom components to quickly distinguish the different sources of components by highlighting them. But when I switched from Vetur to Volar, I found that this feature that had helped me so much was gone.

Why do you want to distinguish highlights?

Distinguishing native HTML elements by using PascalCase is mentioned in vuejs.org. I wish Volar could distinguish div and MyDiv with different highlighted colors like Vetur does.

I wish Volar could distinguish div and MyDiv with different highlighted colors like Vetur does.

It does. Components have different highlight colors.

It doesn't work for me eathier:

On first screehshot I'm using the official extension, and the highlight isn't working.
On second screehshot I'm using a third-party extension, and the highlight is working

This is not a problem of PascalCase and kebab-case styles, it may be caused by your theme, you can try switching themes.
The same issue occurred in a previous version a long time ago, see #2252, return to the 1.x version of the token and it should behave normally.

This is not a planned feature or expected behavior. In this extension, semantic highlighting is used to highlight valid component tags rather than distinguish tag casing.


Ineffective check for OpenStack clone vm action

https://github.com/ManageIQ/manageiq/blob/ac2ce298aa9af7c34f791c15ffc95362a8ab106c/app/models/miq_provision_openstack/cloning.rb#L4
This will obtain potentially thousands of records over several API calls, just to filter one in ruby for some check before cloning.
There needs to be multi-tenant get call, that will grab max one record per each tenant, using it's uuid.

Is this still an issue? Current implementation: https://github.com/ManageIQ/manageiq/blob/master/app/models/manageiq/providers/openstack/cloud_manager/provision/cloning.rb#L4

@aufi yes, still an issue AFAIK

@miq-bot move_issue manageiq-providers-openstack


Self query retriever with Vector Store type <class 'langchain_postgres.vectorstores.PGVector'>  not supported.

Checked other resources

[X] I added a very descriptive title to this issue.
[X] I searched the LangChain documentation with the integrated search.
[X] I used the GitHub search to find a similar question and didn't find it.
[X] I am sure that this is a bug in LangChain rather than my code.
[X] The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Example Code
from langchain.retrievers.self_query.base import SelfQueryRetriever
from langchain_postgres.vectorstores import PGVector
from langchain_postgres import PGVector

connection = "postgresql+psycopg://langchain:langchain@localhost:6024/langchain"  # Uses psycopg3!
collection_name = "test_ddl1"

embeddings = DashScopeEmbeddings(model='text-embedding-v1', dashscope_api_key=DASHSCOPE_API_KEY)

pgvector = PGVector(
    embeddings=embeddings,
    collection_name=collection_name,
    connection=connection,
    use_jsonb=True,
)

metadata_field_info = [
    AttributeInfo(
        name="table",
        description="the name of the database table, should be the same as the one in query after 'TABLE' ",
        type="string",
    ),
]

#Create SelfQueryRetriever
document_content_description = "DDL for database table"
llm = Tongyi()

SelfQueryRetriever.from_llm(
    llm, vectordb, document_content_description, metadata_field_info, verbose=True
)
self_query_retriever = SelfQueryRetriever.from_llm(
    llm,
    vectordb,
    document_content_description,
    metadata_field_info,
    verbose = True,
    search_kwargs={"k": 1}
)

def get_relevant_commented_ddl(x):
    ddl = x.split("(")[0].split("CREATE")[1].split(" ")[1]
    return self_query_retriever.get_relevant_documents(ddl)[0].page_content

Error Message and Stack Trace (if applicable)
ValueError                                Traceback (most recent call last)
Cell In[45], line 32
30 llm = Tongyi() # Tongyi(temperature=0)
31 # llm = Ollama(model="mixtao", base_url="http://datascience.foundersc-inc.com/", temperature=0)
---> 32 SelfQueryRetriever.from_llm(
33     llm, vectordb, document_content_description, metadata_field_info, verbose=True
34 )
35 # self_query_retriever = SelfQueryRetriever.from_llm(
36 #     llm,
37 #     vectordb,
(...)
43
44 # 利用where 查询meta中table值一致的文档
45 def get_relevant_commented_ddl_by_where(x):
File ~/streamlit/.conda/lib/python3.11/site-packages/langchain/retrievers/self_query/base.py:245, in SelfQueryRetriever.from_llm(cls, llm, vectorstore, document_contents, metadata_field_info, structured_query_translator, chain_kwargs, enable_limit, use_original_query, **kwargs)
231 @classmethod
232 def from_llm(
233     cls,
(...)
242     **kwargs: Any,
243 ) -> "SelfQueryRetriever":
244     if structured_query_translator is None:
--> 245         structured_query_translator = _get_builtin_translator(vectorstore)
246     chain_kwargs = chain_kwargs or {}
248     if (
249         "allowed_comparators" not in chain_kwargs
250         and structured_query_translator.allowed_comparators is not None
251     ):
File ~/streamlit/.conda/lib/python3.11/site-packages/langchain/retrievers/self_query/base.py:119, in _get_builtin_translator(vectorstore)
116 except ImportError:
117     pass
--> 119 raise ValueError(
120     f"Self query retriever with Vector Store type {vectorstore.class}"
121     f" not supported."
122 )
ValueError: Self query retriever with Vector Store type <class 'langchain_postgres.vectorstores.PGVector'> not supported.
Description
Altough the self query retriver supports PGVector, however, I found that the source code (https://github.com/langchain-ai/langchain/blob/master/libs/langchain/langchain/retrievers/self_query/base.py) refers to the class in the langchain_community package source code which is going to get deprecated soon, the recommended PGVector code should be a new one from langchain_postgres.vectorstores import PGVector.
Can you fix this? Is there any workaround?
System Info
angchain==0.1.16
langchain-cohere==0.1.3
langchain-community==0.0.32
langchain-core==0.1.42
langchain-elasticsearch==0.1.2
langchain-openai==0.1.3
langchain-postgres==0.0.3
langchain-text-splitters==0.0.1

Just hit the same error message.  And it would be helpful if the documentation for SelfQuery using PGVector (Postgres) could be updated to reflect that we should be using the langchain_postgres.vectorstores import PGVector and not the deprecated langchain_community package.
The documentation is at:  https://python.langchain.com/docs/integrations/retrievers/self_query/pgvector_self_query/

Same on my side, one of the reasons that I moved my infra to PGVector was this...

Hi @eyurtsev. Thank you for your work on PGVector a few weeks ago. Could you confirm if we should be able to use SelfQuery with it or whether that is not an available feature with this vectorstore.  Thanks.

I was able to use PGVector by doing this
vectorstore = PGVector(
    embeddings=embeddings,
    collection_name=collection_name,
    connection=connection,
    use_jsonb=True,
)

from langchain.chains.query_constructor.base import (
    StructuredQueryOutputParser,
    get_query_constructor_prompt,
)

# Define allowed comparators list
allowed_comparators = [
    "$eq",  # Equal to (number, string, boolean)
    "$ne",  # Not equal to (number, string, boolean)
    "$gt",  # Greater than (number)
    "$gte",  # Greater than or equal to (number)
    "$lt",  # Less than (number)
    "$lte",  # Less than or equal to (number)
    "$in",  # In array (string or number)
    "$nin",  # Not in array (string or number)
    "$exists",  # Has the specified metadata field (boolean)
]

constructor_prompt = get_query_constructor_prompt(
    document_content_description,
    metadata_field_info,
    allowed_comparators=allowed_comparators,
    examples=examples,
)

output_parser = StructuredQueryOutputParser.from_components()

prompt = ChatPromptTemplate.from_template(
    "Answer questions based on the context below: {context} / Question: {question}"
)

model = ChatBedrock(
    client=bedrock_client, model_id=model_id, model_kwargs={"max_tokens": 1000}
)

# https://towardsdatascience.com/how-to-build-a-rag-system-with-a-self-querying-retriever-in-langchain-16b4fa23e9ad
output_parser = StructuredQueryOutputParser.from_components()
query_model = model
query_constructor = constructor_prompt | query_model | output_parser

retriever = SelfQueryRetriever(
    query_constructor=query_constructor,
    vectorstore=vectorstore,
    structured_query_translator=PGVectorTranslator(),
    search_kwargs={"k": 5},
)

Hi @jernkuan.  Thanks so much for this work around.  I wasn't able to get your code working directly so I would be very interested if you have a Github with the full script in or could provide the whole script (with your examples) here.
Having said that, your pointing to the article on movie recommendations led me to successfully create a version of what they did.  However I found that it didn't work with structured_query_translator=PGVectorTranslator() and I had to use ChromaTranslator() or PineconeTranslator().  I got that solution from @ravikumarmittal in this issue.
Given this issue and the one I linked to it feels like the PGVector (Postgres) connections could do with some testing and a review.

Hi,
Please change database column type json to json it ll work
Thanks
Ravi
On Fri, 7 Jun, 2024, 3:38 am Robs-Git-Hub, @.***> wrote:

Hi @jernkuan https://github.com/jernkuan. Thanks so much for this work
around. I wasn't able to get your code working directly so I would be very
interested if you have a Github with the full script in or could provide
the whole script (with your examples) here.
Having said that, your pointing to the article on movie recommendations
led me to successfully create a version of what they did. However I
found that it didn't work with
structured_query_translator=PGVectorTranslator() and I had to use
ChromaTranslator() or PineconeTranslator(). I got that solution from
@ravikumarmittal https://github.com/ravikumarmittal in this issue
https://github.com/langchain-ai/langchain/issues/9368.
Given this issue and the one I linked to it feels like the PGVector
(Postgres) connections could do with some testing and a review.
—
Reply to this email directly, view it on GitHub
https://github.com/langchain-ai/langchain/issues/20655#issuecomment-2153481563,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AUGQC7RKHBCMWE2KTKFXSL3ZGDMWDAVCNFSM6AAAAABGO4LCBWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNJTGQ4DCNJWGM
.
You are receiving this because you were mentioned.Message ID:
@.***>

Hi, Please change database column type json to json it ll work Thanks Ravi
…
On Fri, 7 Jun, 2024, 3:38 am Robs-Git-Hub, @.> wrote: Hi @jernkuan https://github.com/jernkuan. Thanks so much for this work around. I wasn't able to get your code working directly so I would be very interested if you have a Github with the full script in or could provide the whole script (with your examples) here. Having said that, your pointing to the article on movie recommendations led me to successfully create a version of what they did. However I found that it didn't work with structured_query_translator=PGVectorTranslator() and I had to use ChromaTranslator() or PineconeTranslator(). I got that solution from @ravikumarmittal https://github.com/ravikumarmittal in this issue <#9368>. Given this issue and the one I linked to it feels like the PGVector (Postgres) connections could do with some testing and a review. — Reply to this email directly, view it on GitHub <#20655 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AUGQC7RKHBCMWE2KTKFXSL3ZGDMWDAVCNFSM6AAAAABGO4LCBWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNJTGQ4DCNJWGM . You are receiving this because you were mentioned.Message ID: @.>

From json to json?

Specifying structured_query_translator when instantiating the retriever solved the issue for me (using langchain 0.1.20).
SelfQueryRetriever.from_llm(
...
 structured_query_translator=PGVectorTranslator(),
...
)

I had to add the lark dependency as well.

I am on langchain v0.2 and the issue still persists. I am using Qdrant, and QdrantVectorStore is not accepted by this


Open binary file in Windows

It seems that: "with open(filename, 'rw+b') as f:" in modify.py doesn't works in Windows.
I tried a number of solutions without success. How can we get around this?
I guess there is some issue with open file and seek() method on windows..
Thx Daniele

While I do not in any way condone running this, if the code is defective it's not educational. So I will fix this. Can you please post the traceback you received?

I did some improvements on your code, for example on changing also background. I’m working to have proof of concept working on Mac, Linux, and Win.
As soon as I’m at home I will share with you the code.
Daniele
Inviato da iPhone

Il giorno 18 apr 2018, alle ore 18:42, Skyler Curtis<EMAIL_ADDRESS>ha scritto:
While I do not in any way condone running this, if the code is defective it's not educational. So I will fix this. Can you please post the traceback you received?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.

This code works fine on Mac and Linux:
def modify_file_inplace(filename, crypto, blocksize=16):
with open(filename, 'r+wb') as f:
plaintext = f.read(blocksize)
while plaintext:
ciphertext = crypto(plaintext)
if len(plaintext) != len(ciphertext):
raise ValueError('''Ciphertext({})is not of the same length
of the Plaintext({}).
Not a stream cipher.'''.format(len(ciphertext),
len(plaintext)))
f.seek(-len(plaintext), 1) # return to same point before the
read
f.write(ciphertext)
plaintext = f.read(blocksize)
But in windows i get:
C:\Users\Daniele\Desktop\Ransomware_Test>python main.py
C:\Users\Daniele\Desktop\test\test.txt
Traceback (most recent call last):
File "main.py", line 70, in
main()
File "main.py", line 53, in main
modify.modify_file_inplace(file, crypt.encrypt)
File "C:\Users\Daniele\Desktop\Ransomware_Test\modify.py", line 8, in
modify_f
ile_inplace
with open(filename, 'r+wb') as f:
ValueError: Invalid mode ('r+wb')
Thx Daniele
2018-04-18 19:09 GMT+02:00 Daniele Votta<EMAIL_ADDRESS>
I did some improvements on your code, for example on changing also
background. I’m working to have proof of concept working on Mac, Linux, and
Win.
As soon as I’m at home I will share with you the code.
Daniele
Inviato da iPhone
Il giorno 18 apr 2018, alle ore 18:42, Skyler Curtis <
<EMAIL_ADDRESS>ha scritto:
While I do not in any way condone running this, if the code is defective
it's not educational. So I will fix this. Can you please post the traceback
you received?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/deadPix3l/CryptSky/issues/7#issuecomment-382451701,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AY6QdB1wYkYpnrQcEndHephwS3PM3X12ks5tp2zYgaJpZM4TaG7f
.

Yes, I'm with you on "r+b" probably is the way... but I did some tests on .txt files and something strange happened :). Let me know if you are able to figure out.
Daniele

that f.seek(f.tell()) seems to get the current position, and move to it? moving to the current position should not acheive anything, and be equivilent to a no-op. I have not replcated your Infinite loop. but i will keep an eye out. If it persists, please open a new issue.

Hi, f.seek(f.tell()) between read() and write() it seems the only way to get the function work also on Windows. On Windows with r+ it seems that he starts in any case from the beginning of the file so the f.seek() is not enought. With a combination of f.seek() and f.tell() you can force the read at the exact block position.
I suggest to test on windows and update your code in this way for better coverage :)
Thanks, Daniele

can you provide how you can tell this? my testing has not had this issue as far as i can tell. could you you please provide files, tracebacks, debug messages, etc that show this?


Regarding damage cause support in the latest version of Weapon Mechanic

Description
I am using [weaponmechanics] on my server and experiencing amazing quality.
However, starting from version 3.3.0, [death event] did not display the attacker and it was found out that this was due to damage cause.
Could you please support this?
Configuration
My_Feature:
  Feature_Option: <true/false>
  Other_Option: <amount>
  etc.

Payment (optional)

[x] I'm willing to pay for this feature

Going to close this.... This is just a change in the Spigot API and this is something that your plugin/script has to handle. Nothing I can do to fix that.


Recovery for lost executors 364

Pull request for the bug described in the issue https://github.com/oap-project/raydp/issues/364.

Please also fix the style lint errors, thanks

Correct me if I am wrong, but the fault tolerance mode is only for converting a Spark DataFrame to a Ray dataset. If the use case is to just run a Spark job and write the output to a destination (like S3), then enabling fault tolerance mode has no effect.
That being said, I understand that when the executor actors die, either because of OOM issues or node failures, Ray would restart the actors, and the restarted executors should be added back to Spark. However, currently, this is not happening. When an executor dies, it gets restarted, but it immediately dies before adding itself to the Spark application. This results in the Spark application getting stuck, where it indefinitely waits for the executors to come up without doing anything.
I will try to debug why this issue is happening and will attempt to raise a PR for it (in which case this current PR with changes in spark is not needed.)

However, currently, this is not happening. When an executor dies, it gets restarted, but it immediately dies before adding itself to the Spark application. This results in the Spark application getting stuck, where it indefinitely waits for the executors to come up without doing anything.

Do you mean even when fault tolerant mode is on, executors are not adding back as expected? I see, this might be some bug.
We add fault tolerance mode to recover dataframes, indeed, and it introduces some behavior which might not be wanted. We are happy to have this feature, but we should make sure this can be turned off when fault tolerance mode is on.

Do you mean even when fault tolerant mode is on, executors are not adding back as expected? I see, this might be some bug.

Yes, even when the fault tolerance mode is on, executors are not adding back.

I see.

We are happy to have this feature, but we should make sure this can be turned off when fault tolerance mode is on.

Is it possible to check if fault tolerance mode is on, and if so disable the periodic sent RPC?
@pang-wu , does this solve your issue？

Is it possible to check if fault tolerance mode is on, and if so disable the periodic sent RPC?

I will check and get back on this.

@kira-lin I need to check, but I am very exited about this feature -- it solve a big problem in our production. However I have a some concern on turn this feature off automatically, if possible, I would rather have an option for user to turn it off. The reasons are below:

Is it possible to check if fault tolerance mode is on, and if so disable the periodic sent RPC?

Let's say if fault tolerance mode is on, but I am not converting Spark dataframe to Ray dataset. Will RayDP still recovers failed executors?
Also, we have other usecases where Spark context is not initialized using Python code, but using Spark submit/raydp submimt. In this case, we don't have a way to turn fault tolerance on event it can solve the problem..?

Will RayDP still recovers failed executors?

With fault tolerance mode on, we basically turn on Ray's failed actor restart, so yes, it should recover the executors by design.

Also, we have other usecases where Spark context is not initialized using Python code, but using Spark submit/raydp submimt. In this case, we don't have a way to turn fault tolerance on event it can solve the problem..?

That's right.

@kira-lin I was able to debug and fix the issue with ray's fault tolerance.
I have raised another PR for this.
Closing this PR as its is not required.


Join/Part Option Unavailable

The option to toggle join/part messages would be nice.

Our team is using this to move away from our IRC clients and into Slack.  The messaging component works great, but users that enter the IRC channel see no one but the bot.  Also, Slack users are unaware that anyone has joined/parted the IRC channel. An option to enable join/part for IRC/Slack users would be greatly appreciated.

I'm not all that familiar with node, but could we add the following to lib\bot.js in Bot.prototype.attachListeners?
this.ircClient.on('join', this.sendToSlack.bind(this));
this.ircClient.on('part', this.sendToSlack.bind(this));

No, the join and part event callbacks don't use the same arguments as the message event, so you'd need to change it up a bit first. See https://node-irc.readthedocs.org/en/latest/API.html#'join' for more information.
It wouldn't require that much changes though, so it's certainly doable. However, is it really that useful? Users that enter the IRC channel will still see no one but the bot (see #34). The only real usecase I could see would be if you're talking with someone, and that someone disconnects/parts, but is that really worth the join/part spam an IRC channel often comes with?
It would be possible to show join/parts from the Slack channels too, but aren't these quite seldom anyway? I mean, it would only show the first time a user joined a channel, and not when they disconnect/connect to Slack later on. For this you'd have to listen for precense_change-events, but I think that'd be way too much spam if you have a decently sized Slack team (it would also fire for everyone in the Slack team, not just the people in the channel).

I found that huge switch/case in nose-irc that provides the proper arguments. I have a successful implementation in my fork if you're interested. Perhaps my team has a unique need for the feature, but we find it useful for monitoring our rarely-used public irc channel. We're using slack-irc to move away from irc while still offering the option to users that prefer it.
----- Reply message -----
From: "Martin Ek"<EMAIL_ADDRESS>To: "ekmartin/slack-irc"<EMAIL_ADDRESS>Cc: "Robert Kety"<EMAIL_ADDRESS>Subject: [slack-irc] Join/Part Option Unavailable  (#32)
Date: Fri, May 15, 2015 02:14
No, the join and part event callbacks don't use the same arguments as the message event, so you'd need to change it up a bit first. See https://node-irc.readthedocs.org/en/latest/API.html#'join' for more information.
It wouldn't require that much changes though, so it's certainly doable. However, is it really that useful? Users that enter the IRC channel will still see no one but the bot (see #34). The only real usecase I could see would be if you're talking with someone, and that someone disconnects/parts, but is that really worth the join/part spam an IRC channel often comes with?
It would be possible to show join/parts from the Slack channels too, but aren't these quite seldom anyway? I mean, it would only show the first time a user joined a channel, and not when they disconnect/connect to Slack later on. For this you'd have to listen for precense_change-events, but I think that'd be way too much spam if you have a decently sized Slack team (it would also fire for everyone in the Slack team, not just the people in the channel).

Reply to this email directly or view it on GitHub:
https://github.com/ekmartin/slack-irc/issues/32#issuecomment-102334427

https://github.com/robertkety/slack-irc/tree/Join/Part

This has been fixed in master. Will publish a new version including it as soon as a few other changes have landed.


define and import shouldn't be lambdas

That will allow us to hide some nonessential impl details from the API.

Fixed by 75dcb4163c35528111dc58b79822bde9e854be35


cant re-enable any extension

Issue Type: Bug
I had an issue with my cursor in vscode so I used code --disable-extensions  to disable all my extensions. Problem is, there is no way to re-enable any of these extensions in the ui or from the command-line. Every option for enable extension is greyed out. furthermore, vscode reloads extensions to their previous state upon startup, this still doesnt re-enable my extensions
One more thing,  uninstalling then reinstalling an extension still doesnt re-enable it.
 Make sure you close all the windows.


Add support for preview 9

Please add support for preview 9.

on preview 9 it's broken

I know it is still in preview, but wonder when they will stop tinkering issue being this:

Update implementations of IJSRuntime to return ValueTask.

https://devblogs.microsoft.com/aspnet/asp-net-core-and-blazor-updates-in-net-core-3-0-preview-9/

@andrewwilkin preview literally mean they can do whatever they want. since preview means it is not ready for use. once it is GA released (3.0.0 for server side) and (5.0.0) for client side. then you can expect near 0 breaking changes in minor versions.

This is done, so I'm closing the issue.


Scone unresponsive to gain access to their private helm repo

This issue is for a: (mark with an x)
- [ ] bug report -> please search issues before submitting
- [ ] feature request
- [ ] documentation issue or request
- [x] regression (a behavior that used to work and stopped in a new release)

In order to deploy the confidential healthcare with scone sample on Kubernetes, one requires access to the private Scone Helm repo.
As mentioned in https://sconedocs.github.io/helm/, one needs to send the Scone maintainers an email. I requested access to this repo. They replied saying that I should just play with the Scone playground on the Azure marketplace (https://sconedocs.github.io/azure/scone-playground/). This demo is not made with the Scone playground in mind.
To this day, I haven't heard anything back from Scone (despite sending gentle reminders), making this demo not deployable on AKS using their Helm charts.
workarounds
I have gained access to the images, and one can write their own Helm charts (or equivalent). Unfortunately I have no clue on how to correctly configure them, nor do I want to spend time reverse-engineering it.
proposition
Azure confidential computing should push Scone to streamline the access to their private Helm chart repo. Or make the Scone helm repo public.
Or the demo should be adapted to use the relatively easy-to-access Scone playground.

Sorry to hear this I will email this issue directly to Scone maintainers and ask for access. Please feel free to drop an email to me as well<EMAIL_ADDRESS>

I'm very sorry about this! We granted you access to the community version the day you asked for access. I just checked the emails but I cannot find any of your emails regarding getting access to the helm charts. I will grant you access now to the helm charts.

Hi Scontain, thank you for your reply and granting me access straight away. I am unblocked at last to try out the awesome stuff you have built. 👌
The request was sent with my private email instead of the professional one. Maybe they ended up in the spam folder? Anyway, I still believe that the helm charts could perfectly be made public without incurring excessive costs or leaking Scontain's secret sauce.


[Data collection flow] Header doesn't show full label of LOI, only ID

For example, an LOI with only ID 5 shows as "5" rather than "Point 5" as on the card.
@shobhitagarwal1612 PTAL?

If LOI contains only "id", then the display name is " (id)" e.g. Point (5)
If LOI contains only "name", then the display name is "name" e.g. "Loi Name"
My guess is that 5 is the name of the LOI and not ID. Can you please share more details? (survey id, loi id, etc.)

@gino-m FYI

@shobhitagarwal1612 The data collection flow header doesn't actually use that logic. You can see the incomplete impl and related TODO here:
https://github.com/google/ground-android/blob/956e9f110d75430f27633abb70f8bc9198ec2787/ground/src/main/java/com/google/android/ground/ui/common/LocationOfInterestHelper.kt#L40-L44


Efficiency between tensorflow and caffe?

@hx173149
Did you notice the difference in efficiency between this code and original C3D code?
During testing, the batch size of caffe code can be 30 in GTX, while the batch size for this code is 15. When adding the batch size, the code will report 'Out of Memory' error. Can you tell me you GPU configuration and batch size during training? Thanks!

Hi @LongLong-Jing
Yes, maybe there have some points to optimize the efficiency.


New cleansedb command and adding a local settings for the vagrant.

The changes we discussed on slack are now in this PR


Bug Report - Issues deploying to Sweeden Central

Describe the bug
Unable to deploy hub and spoke reference architecture (arm templates) to Sweden Central
Steps to reproduce

Deploy enterprise at scale - hub and spoke arm templates to Sweden Central
Able to discuss / supply logs
NB - Sweden Central is a new Region probably missing services from my own troubleshooting - Supported regions for linked Log Analytics workspace | Microsoft Docs we may need to build this into our validation checking for all deployment methods.

Screenshots

Hi @martydevlin,
Can you provide some information about what is not working here for you so we can assist further?
Thanks
Jack

Also @martydevlin was the issue around VNET deployment by any chance and related to this https://github.com/Azure/Enterprise-Scale/issues/881 ?
Let us know or provide a correlation ID and we can investigate further (https://docs.microsoft.com/en-us/azure/azure-resource-manager/templates/deployment-history?tabs=azure-portal#correlation-id-and-support) 👍

Spoke with @martydevlin offline as FTE.
He will send me a list of things that were causing issues, but this is primarily related to not being able to link to LAW & Automation account in Sweeden Central today as not supported (https://docs.microsoft.com/en-us/azure/automation/how-to/region-mappings)
This is something we cannot resolve ourselves as requires PG teams to enable capabilities.
May add a note to portal experience about supported regions for LAW & Auto linking?

Trigger ADO Sync 1

Trigger ADO Sync 2

Closing this issue as linking automation account to Log Analytics workspaces is no longer required for ALZ, only required for legacy update management, change tracking and start/stop vms v1. Deployment to Sweden Central works as expected.


Sync eng/common directory with azure-sdk-tools for PR 4098

Sync eng/common directory with azure-sdk-tools for PR https://github.com/Azure/azure-sdk-tools/pull/4098  See eng/common workflow

@danieljurek I am safe to override this and merge right?

/check-enforcer override


🌱 fix dependabot workflow permissions

Dependabot itself wouldn't need more than pull-requests: write, but we have added on generated code part which does need to write contents on top of the PR, so restoring the permissions.
Symptoms are seen in #414

/override test-centos-e2e-integration-main test-ubuntu-integration-main
Irrelevant.


Optical sizes and small caps

From Chris Newton:
A little while ago, you were kind enough to help me with some problems using XeLaTeX/fontspec that I posted about on comp.text.tex. One of those was that \textsc was not selecting small caps when I was using Adobe's Cronos Pro Opticals font set, though several variations using \fontspec to specify the OpenType features more directly did seem to work.
After a bit of investigation, it seems that the cuplrit is some interaction with SizeFeatures. As requested, I am attaching a minimal example file. I've now worked around this problem myself, but if you want to change anything in light of this but don't have Cronos Pro yourself to test with, please feel free to e-mail me and I'll help if I can.
Thanks again for the help,
Chris
\documentclass{article}
\usepackage{fontspec}
\setsansfont[SizeFeatures={
{Size={-8.1}, OpticalSize=6},
{Size={8.1-14}, OpticalSize=10},
{Size={14-24}, OpticalSize=16},
{Size={24-}, OpticalSize=26}
}]{Cronos Pro}
% The above prevents use of small caps, but using \setsansfont{Cronos Pro} works fine.
\begin{document}
\sffamily\textsc{This does not work, but if you remove the SizeFeatures it does.}
\end{document}

Surely this is fixed by now?!


SourceLink.Git.dll missing from SourceLink.Fake nuget package v1.1.0

reference: https://github.com/ctaggart/SourceLink/blob/master/Fake/SourceLink.fsx#L13
https://github.com/ctaggart/SourceLink/blob/master/Fake/Fake.nuspec#L24-L26
tools folder does not have SourceLink.Git.exe, and therefore breaks all FAKE/ProjectScaffold builds.

This persisted after 3 deletions/reinstallations, but a 4th reinstall of the package solved it. No clue.


should init mbedtls_rsa_context before using

in programs\pkey\rsa_decrypt.c,
it use mbedtls_mpi_read_file, before init the mbedtls_rsa_context,
maybe ,should add
mbedtls_rsa_init(&rsa, MBEDTLS_RSA_PKCS_V15, 0);
in line 91.

Thanks for raising this issue. We'll look into it.

ARM Internal Ref: IOTSSL-1101

fixed on versions mbedtls-2.5.1 , mbedtls-2.1.8


refactor: enable usage of legacy Anoncrypt packer. Make nested forwards while creating forward message. Add legacyForward model

Title:
Change Msg field of Forward model. Make nested packed forwards while creating forward message
Summary:

Change Msg field of Forward from byte array to Envelope structure as defined in protocol
Make nested packed forwards(for each routing key) while creating forward message

@Abdulbois can you please squash your commits.
https://github.com/hyperledger/aries-framework-go/blob/main/.github/CONTRIBUTING.md#pull-request

@Abdulbois can you rebase

@Abdulbois can you rebase

pls check it


multiple files upload in dev console

Hi Team, in the files section of Developer console I can upload single file. I'm trying to transfer data from old (not hasura-based) system to Nhost, so I need to upload like a hundred files, is there a quick way to do it?

@szilarddoro Could we allow to select multiple files in the Nhost Storage UI?

Yes, it's definitely something that we can allow. 👍🏼 I'll create the relevant issue for this.

any update / estimation on this?

Nothing yet. We'll make sure to update here as soon as we have an update.


Reorganizing introductory pages (Home + Docs Guide) and Installation section

Consolidate home page with Docs Guide page (new "Other docs" section)
Copy edit and add lists / sublists for conciseness and clarity
Add subfolders to Installation section for easier browsing
Rename Installation section pages for conciseness

TODO: Add redirects after getting feedback on the changes above
New home page (with new "Other docs" section included):

New sidebar organization for Installation section:

cc: @amacneil @defunctzombie

@clalancette @amacneil - PR for just the home page + docs guide (consolidation and copy editing) open here: https://github.com/ros2/ros2_documentation/pull/2453


Using Rails cache to cache results of graphql-batch promises?

I currently have graphql-ruby acting as a front end for our LDAP directory.  graphql-batch works really great against directory data.  I've even gone so far as to cache the ldap query results that are formed by the batch loaders using Rails.cache statement.
But I'd like the caching to be a bit more granular (i.e. cache individual objects, and not entire sets of objects returned by batch queries.
The problem is when I try to wrap an ObjectLoader.for statement in Rails.cache.fetch block I get an
TypeError (no _dump_data is defined for class Proc):
error.
This seems be because Rails.cache can't cache a promise.  Any thoughts or approaches on how to cache the results of an individual fulfill from a batch loader?
Heres some sample code from one of my resolvers that causes the above error:
      def call(obj, args, ctx)

		# calculate environment from the top level env ctx
		myEnv = get_root_query_env(ctx,args)

		# lets setup a cache key for this LDAP query
		myCacheKey = "ldap/#{myEnv}/#{args[:dn]}/#{Base64.encode64(LDAP::LDAP_USER_READABLE_ATTRS.to_s)}"
		Rails.logger.debug "Checking Rails Cache for key for LDAP User DN: #{args[:dn]}"

		# get cached result or if not cached, do actual query
		Rails.cache.fetch(myCacheKey, expires_in: LDAP::LDAP_GLBL_CACHE_EXPIRE_MINS.minutes) do

			# send the request for this user DN to the batch loader
			get_user_from_batch(myEnv, args[:dn])
		end
	end

	private

	def get_user_from_batch( myEnv, dn)
		# Call Generic directory Object loader so that this query can be batched
		Dir::Loaders::ObjectLoader.for("ldap", myEnv, "user").load(dn).then(nil, lambda do |exc|
				# looks like this query errored or wasn't found
				Rails.logger.error exc
				GraphQL::ExecutionError.new("#{exc}: Object Not Found")
			end ) do |thisObj|
				# return the resolved object from the batch loader
				thisObj
		end
	end

But I'd like the caching to be a bit more granular (i.e. cache individual objects, and not entire sets of objects returned by batch queries.

I think you still want to do the caching in the batch loader.  Instead of using a single Rails.cache.fetch block in the perform method, you could use Rails.cache.read_multi, batch load the uncached objects, then Rails.cache.write the loaded objects.  We have a fetch_multi method in identity_cache that seems to be what you want, assuming you aren't manually deleting cache entries.

I actually ended up doing both (i.e. I cache the results of a single batched query as a set of objects), but then I also added each individual object into the cache as the loader fulfilled it.
So basically I just did away with the Rails.cache.fetch method and split the Rails.cache.write apart from the Rails.cache.exist/read.  So the Loader writes individual objects to the cache as they are fulfilled, and the Resolver that would send a query to the BatchLoader checks for the item in cache first.
This method seems to work quite well as the cache write/read are fully decoupled.

@clader do you happen to still have the code for that loader that you can share?


Remove basic-auth method from grafana

Only openshift auth will be allowed

Is this all standalone code, or does it depend on any of the other work you've been doing around authentication access?

Is this all standalone code, or does it depend on any of the other work you've been doing around authentication access?

Standalone with no other dependencies. The basic auth only existed to provide backwards-compatibility to what we had before we implemented oauth (https://github.com/infrawatch/service-telemetry-operator/pull/281/files#diff-ae71801975adb4f8dd4aa5479a66ad46e46f17de40f9d147b2e09e13ce26633eL19) . Now that OAuth has been in place for several releases (since 1.4.0), and our threat model is suggesting removing all basic-auth and passwords in CRs, we can just turn this all off.

Is this all standalone code, or does it depend on any of the other work you've been doing around authentication access?

Standalone with no other dependencies. The basic auth only existed to provide backwards-compatibility to what we had before we implemented oauth (https://github.com/infrawatch/service-telemetry-operator/pull/281/files#diff-ae71801975adb4f8dd4aa5479a66ad46e46f17de40f9d147b2e09e13ce26633eL19) . Now that OAuth has been in place for several releases (since 1.4.0), and our threat model is suggesting removing all basic-auth and passwords in CRs, we can just turn this all off.

Awesome! Then approved and ready for merge at your discretion :)

recheck


Initial implementation of driver enable.

I've implemented Driver Enable pins for the USART abstraction. Tested with RS485 hardware on an STM32G030. Happy for feedback. Thanks!

Could you run cargo fmt.

Formatting updated!

Could you also add rs485 example.

I've added an example of using the usart_with_driver_enable() function.

@dotcypress do you have any objections?

@justinlatimer  thank you for your help!
I'm not happy with paste! macro. Maybe we can implement this feature without macro hell.
I will take a deep look into it a bit later this week.

Superseded by #109


StoreFetchMiddleware

StoreFetchMiddleware allows the caller to rewrite fetches from the store (for example, to redirect a read to another node)
This also includes a reference implementation to cover the case outlined by #332 (and I suspect the primary use of this middleware):
import ApolloClient, { cachedFetchById } from 'apollo-client';
const client = new ApolloClient({
  dataIdFromObject: value => value.id,
  storeFetchMiddleware: cachedFetchById,
});

await client.query({
 query: gql`
    query fetchAll {
      tasks { id, name }
    }
  `,
});

// abc123 is fetched from cache, if fetchAll returned it!
await client.query({
 query: gql`
    query fetchOne($id: ID!) {
      task(id: $id) { id, name }
    }
  `,
  variables: { id: "abc123" },
});

Also, FYI, Mocha is promise-aware - just return the promise from a test, and it'll take care of the rest!  No need to use done

Update: I just rebased this on top of #377, for better (less brittle) handling of this middleware (and potentially other future callbacks).
You can see the old commit (no context, just an extra arg) at https://github.com/convoyinc/apollo-client/commit/7cf26408f7182b445e4ccff151a41e98f5b64a75

Alrighty, rebased & it now handles the new IdValue structure for cachedFetchById

Hi guys! I'm currently working on a React Native where we use Apollo Client and StoreFetchMiddleware is exactly what we need actually 😱 !! Are you planning to work on it (maybe merge it?) anytime soon?

+1

Curious what you guys think about this PR?

any update?

The test failure is node v4 only with:
=============================== FileSize summary ===============================
The total size of index.min.js is 139.77 kB.
The total gzipped size of index.min.js is 37.16 kB.
================================================================================
The total gzipped size of index.min.js exceeds 37 kB.

Could anyone provide feedback?

Can you please provide any estimations on when this pull request will be merged ?

Hey @nevir @stubailo, what is the state of this pr?
I have some time and need this feature, so would like to help if I can.

Hey, sorry I've been a blocker on this PR for a while. I think in order to avoid people manipulating the query AST directly, we should implement this feature after https://github.com/apollostack/apollo-client/issues/615, which will let us switch the internal execution implementation to use https://github.com/apollostack/graphql-anywhere
This will allow the store fetch middleware to instead be replaced with custom resolvers on the client, which is much more similar to how GraphQL works on the server, rather than using AST manipulation.
We're hoping to get this refactor done in the next week or two, so perhaps we can wait until then? However if it's really urgent we can add a more hacky solution first. I'd suggest that the hacky solution should not live in the execution layer, but instead somewhere in query manager, where the query can be transformed before being passed into diffQueryAgainstStore.

Waiting 2 weeks (or a bit more) is just fine for me... I was worried 'cause this is an essential feature.

Yeah, I agree it's important to be able to use data from the cache, even if you are running a different query. I think it will be way easier after a refactor though, which might also come with a bonus of using any client-side data from GraphQL.

Yeah, local resolvers would be a much better solution

@stubailo I thought @nevir was on to something useful with this PR. I understand that the implementation of something like this will change with #615, though. Is there something else planned for covering the use cases here?

Yeah, if you look at the conversation above we concluded that this feature can be implemented in a much better way after a refactor. Then, you don't have to manipulate the AST, instead you'll be able to define custom client-side resolvers.

Gotcha. I saw that. I presumed this PR would reworked accordingly. It would be better to submit an entirely new PR. Thanks!


Collapse BUG_RATIO and DISCOVERY_RATIO into a single introduced work ratio?

Are BUG_RATIO and DISCOVERY_RATIO fundamentally different, or could we simplify our lives by having a single ratio for “new work”? Does treating them separately give us some benefit? Greater accuracy?

I've got an uncommitted TODO comment related to this:
// TODO: Should we use a compounding formula to account for tickets that get created and
//       then cause new tickets themselves, e.g. bugs introduced by the new features? If so,
//       should we treat the two ratios differently, since more bugs tend to be created by
//       feature tickets and bugs usually don't take as long as features?


Copy/Cut then Paste doesn't preserve or fix indentation #6392

Copy/Cut then Paste doesn't preserve or fix indentation #6392
This issue was never fixed but was closed and it won't allow me to comment on it so making an issue to tell y'all to reopen it.

Here is an example of the issue with a simple, plain HTML file that was formatted by VS Code. The editor.formatOnPaste is set to the default value of false. When I paste a single 'p' element into another nesting level (the 'div'), the formatter not only indents it incorrectly, but it also messes up the indentation of the surrounding 'div', which was not part of the pasted content.

The only two workarounds seem to be hitting Ctrl+Z to undo the silent formatting step, as the folks in issue #6392 suggest, or to set editor.formatOnPaste to true, which, ironically, appears to stop any formatting. It seems like formatOnPaste flag does the opposite of what the boolean value would suggest. When it it is set to false, formatting is applied (which often doesn't work) and when set to true, there is no formatting.
There is a helpful button on the settings page called 'Show matching extensions', but I don't see any that apply to HTML.

Duplicate of #32320. (?)


Add support for multiple connection hops in tx chan-open-try

Part of: #3940
Summary
tx chan-open-try

[x] Replace call to tx_chan_cmd! macro with an inline function on TxChanOpenTryCmd
[x] In the run() function for TxChanOpenTryCmd, build the reverse channel path, from --dst-chain to --src-chain, by leveraging information available on the existing ChannelEnd that was opened with tx chan-open-init
[x] Add multihop support to Channel::build_chan_open_try_and_send

[x] Add build_multihop_chan_open_try() method to struct Channel
[x] Add update_channel_path_clients() method to struct Channel

[x] Add struct MultihopProofHeights to store the heights that are necessary when constructing multihop proofs.

[x] Add build_multihop_proofs() method to struct Channel

Tests

[ ] Write integration test for single hop channel

For Admin Use

[ ] Not duplicate issue
[ ] Appropriate labels applied
[ ] Appropriate milestone (priority) applied
[ ] Appropriate contributors tagged
[ ] Contributor assigned/self-assigned

Closed by 81123d2.


Overrides are turned into Members incorrectly on format

Given
type Foo =
  | Blah
  override this.ToString() = "Bar"

And then autoformatting in Rider gives
type Foo -
  | Blah
  member this.ToString() = "Bar"

It's not an error, only a warning, but it's incorrect.
I'm also quite possibly missing some kind of setting somewhere, but I couldn't find one, so here I am.

The code reformat is being done by Fantomas library and the issue is already fixed there. We've updated bundled Fantomas version in upcoming 2020.1 release. There are preview builds available if you want to make sure it's been fixed for you.

I've just checked it on the code example above and it behaves as expected. Please let me know if it reproduces for you on 2020.1 on some other example.


dropbox: No module named 'gi'

Describe the bug
Dropbox fails to install in new 23.05 install.

Screenshots
As above.
Additional context
This is a new install of nixos 23.05, and dropbox installed using home-manager.
Notify maintainers
@ttuegel
Metadata
Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.
 - system: `"x86_64-linux"`
 - host os: `Linux 6.1.31, NixOS, 23.05 (Stoat), 23.05.20230531.3a70dd9`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.13.3`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`

This is admittedly off-topic, but unless you absolutely need the actual dropbox client, I cannot strongly enough recommend maestral. Been using that for years as there is a limit to the number of clients you can use with the free plan, but maestral doesn't count against that.

@peterhoeg Do you by any chance have a sample home-manager config for maestral that will automatically start it in a systemd user based service?

I have a nixos module which still supports the proprietary dropbox package.
{ config, lib, pkgs, ... }:
let
cfg = config.hoeg.programs.dropbox;
inherit (lib)
mkEnableOption mkIf mkMerge mkOption types;
commonServiceConfig = {
KillMode = "control-group"; # dropbox upstream recommends process
NotifyAccess = "main";
Restart = "on-failure";
PrivateTmp = true;
ProtectSystem = "strict";
Nice = 10;
Slice = "background.slice";
};
environment = {
HOME = "%h";
};
wantedBy = [ "graphical-session.target" ];
ini = pkgs.formats.ini { };
defaultKeyring = "keyring.backends.kwallet.DBusKeyring";
defaultKeyring = "automatic";
defaultKeyring = "keyring.backends.SecretService.Keyring";
maestralIni = ini.generate "maestral-override.ini" {
app = {
# if we pass a bool here, it will end up as false which makes
maestral blow up
analytics = "False";
notification_level = 30; # SYNCISSUE
update_notification_interval = 0;
};
# kwallet has support for SecretService, but it doesn't seem to work
auth = {
  keyring = defaultKeyring;
};

};
keyringIni = ini.generate "keyringrc.cfg" {
backend.default-keyring = defaultKeyring;
};
in
{
meta.maintainers = with lib.maintainers; [ peterhoeg ];
options.hoeg.programs.dropbox = {
enable = mkEnableOption "Dropbox/Maestral";
preferMaestral = mkOption {
  description = "Prefer maestral";
  type = types.bool;
  default = true;
};

runGUI = mkEnableOption "Run GUI tool";

folderPath = mkOption {
  description = "Path to local dropbox folder";
  type = types.nullOr types.str;
  default = null;
};

};
config = mkIf cfg.enable (mkMerge [
(mkIf cfg.preferMaestral {
environment.systemPackages = [
(pkgs.maestral.overrideAttrs (old: {
# doCheck = false;
}))
];
  systemd.user.services = {
    maestral = rec {
      description = "Maestral - OpenSource Dropbox client";
      conflicts = [ "dropbox.service" ];
      restartTriggers = [ maestralIni ];
      wants = [ ]; # [ "kwalletd5.service" ];
      # we need kwallet to be running before maestral can fetch the auth
      # token, so we *need* this to be a graphical session
      after = wantedBy ++ wants;
      inherit environment wantedBy;

      unitConfig = {
        ConditionUser =<EMAIL_ADDRESS>        ConditionPathExists = [
          "%h/.config/maestral/maestral.ini"
          # do not set this here as each machine has its own path and

thus sets it itself
] ++ lib.optional (cfg.folderPath != null) cfg.folderPath;
};
      serviceConfig = with pkgs; (
        commonServiceConfig // {
          Type = "notify";
          ExecStartPre = "/bin/sh -c '${pkgs.crudini}/bin/crudini

--merge $HOME/.config/maestral/maestral.ini < ${maestralIni}'";
ExecStart = "${lib.getBin maestral}/bin/maestral start
--foreground";
ExecStop = "${lib.getBin maestral}/bin/maestral stop";
# ExecStopPost = ''/bin/sh -c "test "$SERVICE_RESULT" !=
"success" && ${pkgs.libnotify}/bin/notify-send Maestral 'Daemon failed'"'';
ExecReload = "${lib.getBin coreutils}/bin/kill -HUP $MAINPID";
EnvironmentFile = "-%t/keys/maestral";
WatchdogSec = "60s";
RuntimeDirectory = "maestral";
}
);
};
};
})
(mkIf (! cfg.preferMaestral) {
  networking.firewall = {
    allowedTCPPorts = [ 17500 ];
    allowedUDPPorts = [ 17500 ];
  };

  environment.systemPackages = [ pkgs.dropbox-cli ];

  systemd.user.services.dropbox = rec {
    description = "Dropbox";
    after = wants ++ wantedBy;
    wants = [ "xembed-sni-proxy.service" ];
    conflicts = [ "maestral.service" "maestral-gui.service" ];
    inherit environment wantedBy;

    unitConfig = {
      ConditionPathExists = mkIf (cfg.folderPath != null)

cfg.folderPath;
ConditionUser = @.***";
};
    serviceConfig = with pkgs; (
      commonServiceConfig // {
        ExecStart = "${lib.getBin dropbox}/bin/dropbox";
        ExecReload = "${lib.getBin coreutils}/bin/kill -HUP $MAINPID";
      }
    );
  };
})

]);
}

OK, so the previous mail looked terrible. Let me try this again from a proper mail client:
┌────
│ { config, lib, pkgs, ... }:
│
│ let
│   cfg = config.hoeg.programs.dropbox;
│
│   inherit (lib)
│     mkEnableOption mkIf mkMerge mkOption types;
│
│   commonServiceConfig = {
│     KillMode = "control-group"; # dropbox upstream recommends process
│     NotifyAccess = "main";
│     Restart = "on-failure";
│     PrivateTmp = true;
│     ProtectSystem = "strict";
│     Nice = 10;
│     Slice = "background.slice";
│   };
│
│   environment = {
│     HOME = "%h";
│   };
│
│   wantedBy = [ "graphical-session.target" ];
│
│   ini = pkgs.formats.ini { };
│
│   defaultKeyring = "keyring.backends.kwallet.DBusKeyring";
│   # defaultKeyring = "automatic";
│   # defaultKeyring = "keyring.backends.SecretService.Keyring";
│
│   maestralIni = ini.generate "maestral-override.ini" {
│     app = {
│       # if we pass a bool here, it will end up as false which makes maestral blow up
│       analytics = "False";
│       notification_level = 30; # SYNCISSUE
│       update_notification_interval = 0;
│     };
│
│     # kwallet has support for SecretService, but it doesn't seem to work
│     auth = {
│       keyring = defaultKeyring;
│     };
│   };
│
│   keyringIni = ini.generate "keyringrc.cfg" {
│     backend.default-keyring = defaultKeyring;
│   };
│
│ in
│ {
│   meta.maintainers = with lib.maintainers; [ peterhoeg ];
│
│   options.hoeg.programs.dropbox = {
│     enable = mkEnableOption "Dropbox/Maestral";
│
│     preferMaestral = mkOption {
│       description = "Prefer maestral";
│       type = types.bool;
│       default = true;
│     };
│
│     folderPath = mkOption {
│       description = "Path to local dropbox folder";
│       type = types.nullOr types.str;
│       default = null;
│     };
│   };
│
│   config = mkIf cfg.enable (mkMerge [
│     (mkIf cfg.preferMaestral {
│       environment.systemPackages = [
│         (pkgs.maestral.overrideAttrs (old: {
│           # doCheck = false;
│         }))
│       ];
│
│       systemd.user.services = {
│         maestral = rec {
│           description = "Maestral - OpenSource Dropbox client";
│           conflicts = [ "dropbox.service" ];
│           restartTriggers = [ maestralIni ];
│           wants = [ ]; # [ "kwalletd5.service" ];
│           # we need kwallet to be running before maestral can fetch the auth
│           # token, so we need this to be a graphical session
│           after = wantedBy ++ wants;
│           inherit environment wantedBy;
│
│           unitConfig = {
│             ConditionUser = @.";
│             ConditionPathExists = [
│               "%h/.config/maestral/maestral.ini"
│               # do not set this here as each machine has its own path and thus sets it itself
│             ] ++ lib.optional (cfg.folderPath != null) cfg.folderPath;
│           };
│
│           serviceConfig = with pkgs; (
│             commonServiceConfig // {
│               Type = "notify";
│               ExecStartPre = "/bin/sh -c '${pkgs.crudini}/bin/crudini --merge $HOME/.config/maestral/maestral.ini < ${maestralIni}'";
│               ExecStart = "${lib.getBin maestral}/bin/maestral start --foreground";
│               ExecStop = "${lib.getBin maestral}/bin/maestral stop";
│               # ExecStopPost = ''/bin/sh -c "test "$SERVICE_RESULT" != "success" && ${pkgs.libnotify}/bin/notify-send Maestral 'Daemon failed'"'';
│               ExecReload = "${lib.getBin coreutils}/bin/kill -HUP $MAINPID";
│               EnvironmentFile = "-%t/keys/maestral";
│               WatchdogSec = "60s";
│               RuntimeDirectory = "maestral";
│             }
│           );
│         };
│       };
│     })
│
│     (mkIf (! cfg.preferMaestral) {
│       networking.firewall = {
│         allowedTCPPorts = [ 17500 ];
│         allowedUDPPorts = [ 17500 ];
│       };
│
│       environment.systemPackages = [ pkgs.dropbox-cli ];
│
│       systemd.user.services.dropbox = rec {
│         description = "Dropbox";
│         after = wants ++ wantedBy;
│         wants = [ "xembed-sni-proxy.service" ];
│         conflicts = [ "maestral.service" "maestral-gui.service" ];
│         inherit environment wantedBy;
│
│         unitConfig = {
│           ConditionPathExists = mkIf (cfg.folderPath != null) cfg.folderPath;
│           ConditionUser = @.";
│         };
│
│         serviceConfig = with pkgs; (
│           commonServiceConfig // {
│             ExecStart = "${lib.getBin dropbox}/bin/dropbox";
│             ExecReload = "${lib.getBin coreutils}/bin/kill -HUP $MAINPID";
│           }
│         );
│       };
│     })
│   ]);
│ }
└────

Backticks for easier skimming:
{ config, lib, pkgs, ... }:

let
  cfg = config.hoeg.programs.dropbox;

  inherit (lib)
    mkEnableOption mkIf mkMerge mkOption types;

  commonServiceConfig = {
    KillMode = "control-group"; # dropbox upstream recommends process
    NotifyAccess = "main";
    Restart = "on-failure";
    PrivateTmp = true;
    ProtectSystem = "strict";
    Nice = 10;
    Slice = "background.slice";
  };

  environment = {
    HOME = "%h";
  };

  wantedBy = [ "graphical-session.target" ];

  ini = pkgs.formats.ini { };

  defaultKeyring = "keyring.backends.kwallet.DBusKeyring";
  # defaultKeyring = "automatic";
  # defaultKeyring = "keyring.backends.SecretService.Keyring";

  maestralIni = ini.generate "maestral-override.ini" {
    app = {
      # if we pass a bool here, it will end up as false which makes
maestral blow up
      analytics = "False";
      notification_level = 30; # SYNCISSUE
      update_notification_interval = 0;
    };

    # kwallet has support for SecretService, but it doesn't seem to work
    auth = {
      keyring = defaultKeyring;
    };
  };

  keyringIni = ini.generate "keyringrc.cfg" {
    backend.default-keyring = defaultKeyring;
  };

in
{
  meta.maintainers = with lib.maintainers; [ peterhoeg ];

  options.hoeg.programs.dropbox = {
    enable = mkEnableOption "Dropbox/Maestral";

    preferMaestral = mkOption {
      description = "Prefer maestral";
      type = types.bool;
      default = true;
    };

    runGUI = mkEnableOption "Run GUI tool";

    folderPath = mkOption {
      description = "Path to local dropbox folder";
      type = types.nullOr types.str;
      default = null;
    };
  };

  config = mkIf cfg.enable (mkMerge [
    (mkIf cfg.preferMaestral {
      environment.systemPackages = [
        (pkgs.maestral.overrideAttrs (old: {
          # doCheck = false;
        }))
      ];

      systemd.user.services = {
        maestral = rec {
          description = "Maestral - OpenSource Dropbox client";
          conflicts = [ "dropbox.service" ];
          restartTriggers = [ maestralIni ];
          wants = [ ]; # [ "kwalletd5.service" ];
          # we need kwallet to be running before maestral can fetch the auth
          # token, so we *need* this to be a graphical session
          after = wantedBy ++ wants;
          inherit environment wantedBy;

          unitConfig = {
            ConditionUser =<EMAIL_ADDRESS>            ConditionPathExists = [
              "%h/.config/maestral/maestral.ini"
              # do not set this here as each machine has its own path and
thus sets it itself
            ] ++ lib.optional (cfg.folderPath != null) cfg.folderPath;
          };

          serviceConfig = with pkgs; (
            commonServiceConfig // {
              Type = "notify";
              ExecStartPre = "/bin/sh -c '${pkgs.crudini}/bin/crudini
--merge $HOME/.config/maestral/maestral.ini < ${maestralIni}'";
              ExecStart = "${lib.getBin maestral}/bin/maestral start
--foreground";
              ExecStop = "${lib.getBin maestral}/bin/maestral stop";
              # ExecStopPost = ''/bin/sh -c "test "$SERVICE_RESULT" !=
"success" && ${pkgs.libnotify}/bin/notify-send Maestral 'Daemon failed'"'';
              ExecReload = "${lib.getBin coreutils}/bin/kill -HUP $MAINPID";
              EnvironmentFile = "-%t/keys/maestral";
              WatchdogSec = "60s";
              RuntimeDirectory = "maestral";
            }
          );
        };
      };
    })

    (mkIf (! cfg.preferMaestral) {
      networking.firewall = {
        allowedTCPPorts = [ 17500 ];
        allowedUDPPorts = [ 17500 ];
      };

      environment.systemPackages = [ pkgs.dropbox-cli ];

      systemd.user.services.dropbox = rec {
        description = "Dropbox";
        after = wants ++ wantedBy;
        wants = [ "xembed-sni-proxy.service" ];
        conflicts = [ "maestral.service" "maestral-gui.service" ];
        inherit environment wantedBy;

        unitConfig = {
          ConditionPathExists = mkIf (cfg.folderPath != null)
cfg.folderPath;
          ConditionUser =<EMAIL_ADDRESS>        };

        serviceConfig = with pkgs; (
          commonServiceConfig // {
            ExecStart = "${lib.getBin dropbox}/bin/dropbox";
            ExecReload = "${lib.getBin coreutils}/bin/kill -HUP $MAINPID";
          }
        );
      };
    })
  ]);
}

Hi @eyJhb, I see you are listed as the Dropbox nix package maintainer.
Can confirm that dropbox is broken on latest stable and unstable NixOS.
Would you be willing to take a look? I'm using home-manager which doesn't say anything, but a manual dropbox start -i results in No module named 'gi', which blocks Dropbox from running and syncing.

Same issue here.

Sorry, I no longer use Dropbox. I should most likely remove myself as maintainer :/

It would be great if a maestral user who's configured a working systemd user service to run the daemon could add a code snippet to https://nixos.wiki/wiki/Dropbox :pray:.

there's this but I haven't tried it myself

I'm also on 23.05 and dropbox (the actual client) is working fine for me. The only change is that dropbox-cli reports 'Dropbox isn't running!' if I call dropbox status, even though I can update files and manually verify the changes have synced. This is my configuration.nix:
  systemd.user.services.dropbox = {
    description = "Dropbox";
    wantedBy = [ "default.target" ];
    environment = {
      QT_PLUGIN_PATH = "/run/current-system/sw/" + pkgs.qt5.qtbase.qtPluginPrefix;
      QML2_IMPORT_PATH = "/run/current-system/sw/" + pkgs.qt5.qtbase.qtQmlPrefix;
    };
    serviceConfig = {
      ExecStart = "${lib.getBin pkgs.dropbox}/bin/dropbox";
      ExecReload = "${lib.getBin pkgs.coreutils}/bin/kill -HUP $MAINPID";
      KillMode = "control-group";
      Restart = "on-failure";
      PrivateTmp = true;
      ProtectSystem = "full";
      Nice = 10;
    };
  };

@amacfie I found a few snippets like that on github, but they seem to store credentials in plaintext by default. I don't think the native dropbox daemon does that?

I translated the maestral snippet above to work with home manager:
systemd.user.services.maestral = {
    Unit.Description = "Maestral daemon";
    Install.WantedBy = [ "default.target" ];
    Service = {
      ExecStart = "${pkgs.maestral}/bin/maestral start -f";
      ExecStop = "${pkgs.maestral}/bin/maestral stop";
      Restart = "on-failure";
      Nice = 10;
    };
  };

So far seems to work.
I think a built in module that lets you do services.maestral.enabled = true; would be better.
@AlexChalk not sure I understand your concern, this doesn't do anything with passwords. Unless you are saying Meastral is insecure in the way it stores the credentials?

@AlexChalk I note you are not using the home manager module, i.e. dropbox.service.enable = true; - wonder if this is why it works for you...

On migrating from the official Dropbox client to Maestral I've also been having troubles with needing to reauthenticate often (about weekly?), and not being able to sync files with sizes over 350 GB. Assume these are limitations enforced by Dropbox but might be good to be aware of that the clients don't have feature parity.

Perhaps Maestral needs it's own Github issue ;-)
I'm not planning on migrating away from the official Dropbox client and currently the official client is broken on stable NixOS.

I'm also experiencing this issue right on 23.05 (with home-manager).

ModuleNotFoundError: No module named 'gi'

This line https://github.com/NixOS/nixpkgs/blob/master/pkgs/applications/networking/dropbox/cli.nix#L47 is missing at runtime


Make Familiar Rings be able to be toggle on and off with a keybind.

Make Familiar Rings be able to be toggle on and off with a keybind.
Familiar abilities are great but for how powerful they are some of them can get a little annoying in some cases, for example the drikwing has the ability of slow falling, multi-jump and jump boost. these are all great but hitting the roof with every jump is not really useful in enclosed spaces. or for example the greedy familiar's ability is to pick up items in a larger radius but in some mob farms and such you can end up with lot's of junk in your inventory. You can take of the ring and that would disable the ability, but I think this is alot more handy and would be alot more convenient to not have to take off my drikwing ring every time I go through a door lol. An alternative would be to have the effects disable while holding shift while that's great for some stuff it's not really as good. and I think it would be alot better if the status effects didn't just linger around after toggling and would just clear, so it would be an instant effect. it also adds more reason to use the ring over the familiars themselves

every familiar has its own keybind to enable/disable already..


function GetUserByPhone always error "invalid character '<' looking for beginning of value"

I can't use this function, it always gives the error "invalid character '<' looking for beginning of value"

@tlthanh provider a curl example for demo site: https://door.casdoor.com/

@tlthanh tested and it has no issue

I am getting the same issue while using GetUserByUserId API. @tlthanh Did you figure out the root cause of the issue by any chance?

@tlthanh @maknahar fixed by: https://github.com/casdoor/casdoor/commit/cb200687dce15aa5723928b98c1df3353ca9b1af