Support for code-side model defaults

This PR is open for discussion primarily, not looking for immediate merge.
Continuing the conversation from here: https://github.com/laravel/framework/pull/10392#issuecomment-143513354
This PR allows setting default values on models without having to save them and retrieve them back from the database to have database defaults applied.
class Comment extends Model
{
    protected $defaults = [
        'visibility' => 'public',
    ];
}

$comment = new Comment;
$comment->visibility; // 'public'

Previously, newly instantiated or created models would not have database defaults applied, which although not surprising when you think about it, can be confusing if you aren't aware of it.
This makes model defaults a first class citizen in your codebase instead of being enforced only by constraints in the database.
Re: recently merged refresh-on-save behavior
This PR stops models from refreshing on save if $defaults is not null. Assume that if the user has set the $defaults array, that they want to be in control of a model's defaults through code instead of through the database and don't make the extra query. This also makes it easy to disable the refresh behavior with just protected $defaults = []; at the top of your model.
Open to thoughts ripping out the refresh behavior in favor of @crynobone's approach here:
https://github.com/laravel/framework/pull/10397
Personally I think that might be better as a model property, because you enable it by default and make it easier to turn off, whereas a trait can't really be mixed out.

How is this different from just setting your $attributes in the model?
class Comment extends Model
{
    protected $attributes = [
        'visibility' => 'public',
    ];
}

You know what, I don't think it's different at all, seems to work exactly the same, didn't even occur to me to try that :blush:
Is this a known use-case? I had actually talked with Taylor about the $defaults thing and even he didn't seem to realize you could already do it by just using $attributes.
I do think a property called $defaults is a little bit more expressive for this use case, but still, might just need to add some documentation instead and file this one away :blush:

I use it a lot. Dunno if it's well known :smile:

To be clear: I believe #10392 should be reverted, and people should put their default attributes in the $attributes property on their model.
That second query really rubs me the wrong way.
BTW, you mentioned rails in that thread. How do they handle it, implementation-wise?

@JosephSilber Rails handles it when the app boots. It reads the schema of your database and caches it for every model. I'm not sure where the exact lines of code are, but it's easier in that environment since the app is a long running process. Not sure how easy it would be to do in PHP, probably could cache the results in a file on deployment or something, but definitely would be bigger in complexity than the scope of approaches we've been playing with :/

Maybe the solution is to add a description of how to set defaults using the $attributes to the Eloquent documentation. As I see it the code does not need to change.

Yeah the $attributes thing is expected. Forgot about that as well. We can just use that for code defaults.


Create teamviewer.yaml

Closes #38

Again, thanks for your contribution @zypA13510. Added to ssotax.org.

@MMore Actually, the SSO price for both TeamViewer and AnyDesk is not publicly available ("contact us for quote"). The price I put there is the highest tier without SSO.
I do have a quote from them in private though I may not be able to disclose them. It's much higher than even the highest tier (of course).

Hey @zypA13510 , thanks for letting me know. I removed the price. Feel free to adjust whenever you want to add a more exact price tag to it.


Web site is so broken :(

[Enter feedback here]
Mind-blowing your web site is so broken/links go to nowhere - what is going on, Microsoft?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 59e840a0-92ac-3ba4-75b3-01a154d67899
Version Independent ID: 8f7fd243-8f3a-7d32-8433-03b8167aee6f
Content: Update Microsoft applications for Mac by using msupdate - Deploy Office
Content Source: DeployOffice/mac/update-office-for-mac-using-msupdate.md
Service: o365-proplus-itpro
GitHub Login: @pbowden-msft
Microsoft Alias: danbrown

@zignzag12   Which links aren't working? All the links in this article seem to be working.  Thanks.

https://docs.microsoft.com/en-us/deployoffice/mac/update-office-for-mac-using-msupdate https://docs.microsoft.com/en-us/deployoffice/mac/update-office-for-mac-using-msupdate -> Submit feedback for This product https://odinsiders.uservoice.com/ -> https://odinsiders.uservoice.com https://odinsiders.uservoice.com/ This site is not currently active
On September 12, 2021, at 18:06, Daniel H. Brown @.***> wrote:
@zignzag12 https://github.com/zignzag12 Which links aren't working? All the links in this article seem to be working. Thanks.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/MicrosoftDocs/OfficeDocs-DeployOffice/issues/1005#issuecomment-917727432, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJTO6UPMUNOHTUUBFZ6I5ZLUBUW7FANCNFSM5D4SNOTA.
Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

@zignzag12  Thanks for pointing that out. (I was looking at the links in the main portion of the article.) We'll work on getting that fixed.

Why is this error coming up?
You can’t open the application “Microsoft AutoUpdate.app” because the Classic environment is no longer supported.
You cannot simply change something this major in the middle of a release - my parents paid a lot of money for this product and they are not system administrators; they are end users! Very poor decision and why is it not telling you how to change? I thought computers were made to eliminate the guessing game - you can do better than this.
On September 12, 2021, at 18:42, Daniel H. Brown @.***> wrote:
@zignzag12 https://github.com/zignzag12 Thanks for pointing that out. (I was looking at the links in the main portion of the article.) We'll work on getting that fixed.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/MicrosoftDocs/OfficeDocs-DeployOffice/issues/1005#issuecomment-917734248, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJTO6ULGWF4FXHJUBYPLEQLUBU3F5ANCNFSM5D4SNOTA.
Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

And why is the average download speed 70KB/s for downloading https://officecdn-microsoft-com.akamaized.net/pr/C1297A47-86C4-4C1F-97FA-950631F94777/MacAutoupdate/Microsoft_Office_16.52.21080801_Installer.pkg https://officecdn-microsoft-com.akamaized.net/pr/C1297A47-86C4-4C1F-97FA-950631F94777/MacAutoupdate/Microsoft_Office_16.52.21080801_Installer.pkg ?
Are you using a DSL modem? My parents have 500 Mbps FiOS - what is happening here?
On September 13, 2021, at 07:12, Magnus Larsson @.***> wrote:
Why is this error coming up?
You can’t open the application “Microsoft AutoUpdate.app” because the Classic environment is no longer supported.
You cannot simply change something this major in the middle of a release - my parents paid a lot of money for this product and they are not system administrators; they are end users! Very poor decision and why is it not telling you how to change? I thought computers were made to eliminate the guessing game - you can do better than this.
On September 12, 2021, at 18:42, Daniel H. Brown @.*** @.***>> wrote:
@zignzag12 https://github.com/zignzag12 Thanks for pointing that out. (I was looking at the links in the main portion of the article.) We'll work on getting that fixed.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/MicrosoftDocs/OfficeDocs-DeployOffice/issues/1005#issuecomment-917734248, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJTO6ULGWF4FXHJUBYPLEQLUBU3F5ANCNFSM5D4SNOTA.
Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.


Install and configure radix

Radix-UI is necessary to continue to work on styling and designing issues.

As discovered reading the documentation, Radix is design system of incremental adoption, so, it's not an deep dive all-in.
Closing this issue


[DSP] Simplex solver avoids using Gas/Ice Giant mining to produce hydrogen, outputs inbalanced solutions instead

In DSP hydrogen is produced as a byproduct of some recipes and used in usually greater quantities than what is obtained this way when producing end-game products, i.e. universe matrix (final "white" science) and small carrier rockets (main component to build the Dyson Sphere)
The player is expected to use Orbital Collectors to passively mine Gas and Ice Giants planets, which directly produces hydrogen that can be used to balance the production lines.
The simplex solver avoids using this method, instead it tries to use the recipes having hydrogen as byproduct to balance it, resulting in an excess of the main product of these recipes which would clog up the factory and is also a much less practical way to obtain the required hydrogen
To Reproduce
Steps to reproduce the behavior:

Go to https://factoriolab.github.io and go to the DSP calculator
From the default 14 enabled recipes, add the 2 giant mining recipes
Enable the simplex solver
Ask to produce 1 universe matrix (white cube)
See error that excess anti-mater is expected
If you look at the hydrogen you'll see that most of it is produced using Mass Energy Storage instead of orbital mining of Ice or Gas Giants

For example: https://factoriolab.github.io/list?z=eJwrcKrSMjNQS.JQK9bScvKrcwqtc4qoc8quc8qpc3arc3avC6iLrEupy66rVCszBgBbRw.T#hydrogen
Expected behavior
The solver should balance the hydrogen with the orbital mining. Note that orbital mining from an Gas/Ice Giant does not need to output the same ratio of products: an orbital collector could be full of the secondary product and still mine hydrogen
Screenshots
https://factoriolab.github.io/list?z=eJwrcKrSMjNQS.JQK9bScvKrcwqtc4qoc8quc8qpc3arc3avC6iLrEupy66rVCszBgBbRw.T#hydrogen
Desktop (please complete the following information):
Not applicable
Smartphone (please complete the following information):
Not applicable
Additional context
I suspect that either the simplex calculator does not separate the orbital collector recipes' outputs (so it assumes that using it would result in an imbalance in fire ice or deuterium), or that the weight on the two Orbital Collector recipes are wrong, making the simplex favor an imbalanced solution over using them

I'd say that on a technical level it's actually a bit worse than what you have presented, like this one.
Simply speaking, having all recipes except x-ray cracking enabled suggest I create critical photons and split them to cover remaining hydrogen needs even though I don't use critical photons nor antimatter in any capacity.

The gas giant recipes are currently set to have a higher cost, it sounds like in general that just shouldn't be the case. My understanding in the past was that mining gas giants was rare and difficult to set up, but if it's the standard way to get hydrogen in the late game, then the cost should reflect that.

@dcbroad3 exactly. One way to adapt this would either to have a toggle or to link the weight of the gas giant recipes with the assembly machine used for example (if you are using mk1 machine you are probably in the early game, whereas mk3 requires you to be at a point where orbital mining is not too hard)

I never got the impression that mining gas giants was ever rare. Sure it might be a pain to setup but if you've automated every step you came across along the way it's essentially just a bit of time waiting for them to be produced.
If the weight on orbitals will be changeable/dynamic then I feel it must be user configurable. Mainly because you can unlock orbitals without unlocking mk2 assemblers. Sure, have the defaults be based on configured assembler tier but let me set it to match my exact situation.

To be fair, the cost is already configurable today via the Matrix tab. I'm treating this just as a request to change the defaults.

Oh I did not know you could change the default in the matrix tab.
While you are at it I would increase the cost of the Mass Energy Conversion recipe (the one producing antimatter from critical photon) a bit since this is late game and the only way to produce antimatter anyway so the solver should use it to produce just the right amount of required antimatter and never use it to produce excess of it

Having slept on this I feel that there is some form of disconnect here. Isn't the primary purpose of being able to enable and disable specific recipes/sources to indicate what of those are available and/or what you actually want to use? If so then shouldn't the weight effectively only reflect "cost of operation" (so to speak) and efficiency of the recipe itself, not how rare/difficult it would be to acquire the tech or how rare/difficult its setup would be?

@BlackestDawn this seems like a fair point. If you aren't interested in using gas giant mining as part of your solution, that's easier to accomplish by disabling the recipe than changing the cost. Therefore the cost itself should be reasonably low so that gas giant mining is a preferred solution if it is enabled.


Unable to use bevy after wgpu was updated to 0.8

Bevy version
b399a374cb407ac94aaeb6755c06b9a3a559ec6a compiles
afaf4ad3daa01a2be529c19fc183ae8ddd2a330f doesn't compile
Operating system & version
Windows 10
What you did
cargo new <project_name>
add
bevy = { git = "https://github.com/bevyengine/bevy", rev = "afaf4ad3daa01a2be529c19fc183ae8ddd2a330f" }

to Cargo.toml
Additional information
PS C:\Projects\<project_name>> cargo run
    Updating git repository `https://github.com/bevyengine/bevy`
    Updating crates.io index
error: failed to select a version for `web-sys`.
    ... required by package `wgpu v0.8.0`
    ... which is depended on by `bevy_wgpu v0.5.0 (https://github.com/bevyengine/bevy?rev=afaf4ad3daa01a2be529c19fc183ae8ddd2a330f#afaf4ad3)`
    ... which is depended on by `bevy_internal v0.5.0 (https://github.com/bevyengine/bevy?rev=afaf4ad3daa01a2be529c19fc183ae8ddd2a330f#afaf4ad3)`
    ... which is depended on by `bevy v0.5.0 (https://github.com/bevyengine/bevy?rev=afaf4ad3daa01a2be529c19fc183ae8ddd2a330f#afaf4ad3)`
    ... which is depended on by `<project_name> v0.1.0 (C:\Projects\<project_name>)`

all possible versions conflict with previously selected packages.

  previously selected package `web-sys v0.3.46`
    ... which is depended on by `bevy_app v0.5.0 (https://github.com/bevyengine/bevy?rev=afaf4ad3daa01a2be529c19fc183ae8ddd2a330f#afaf4ad3)`
    ... which is depended on by `bevy_asset v0.5.0 (https://github.com/bevyengine/bevy?rev=afaf4ad3daa01a2be529c19fc183ae8ddd2a330f#afaf4ad3)`
    ... which is depended on by `bevy_audio v0.5.0 (https://github.com/bevyengine/bevy?rev=afaf4ad3daa01a2be529c19fc183ae8ddd2a330f#afaf4ad3)`
    ... which is depended on by `bevy_internal v0.5.0 (https://github.com/bevyengine/bevy?rev=afaf4ad3daa01a2be529c19fc183ae8ddd2a330f#afaf4ad3)`
    ... which is depended on by `bevy v0.5.0 (https://github.com/bevyengine/bevy?rev=afaf4ad3daa01a2be529c19fc183ae8ddd2a330f#afaf4ad3)`
    ... which is depended on by `<project_name> v0.1.0 (C:\Projects\<project_name>)`

failed to select a version for `web-sys` which could resolve this conflict```

Wgpu 0.8 pins web-sys to 0.3.50. I don't know where the requirement for a different version comes from though. In any case pinning dependencies often causes errors like this. IMHO cargo shouldn't have allowed dependencies to pin their own dependencies.

Don't know if is the same issue, but I got a similar issue when switching between commits on bevy main. The fix was to do a cargo clean and remove the Cargo.lock. After building everything from scratch, it worked.

With the release of Bevy 0.6, which depends on wgpu 0.12, is this still an issue @nobbele? If so, can you close this issue?

I still get a similar problem:
Updating crates.io index
error: failed to select a version for web-sys.
... required by package wgpu v0.12.0
... which satisfies dependency wgpu = "^0.12.0" of package bevy_render v0.6.0
... which satisfies dependency bevy_render = "^0.6.0" of package bevy_core_pipeline v0.6.0
... which satisfies dependency bevy_core_pipeline = "^0.6.0" of package bevy_internal v0.6.0
... which satisfies dependency bevy_internal = "^0.6.0" of package bevy v0.6.0
... which satisfies dependency bevy = "^0.6.0" of package ast4 v0.1.0 (C:\Users\jmonasterio\rust\ast4)
versions that meet the requirements ^0.3.53 are: 0.3.55, 0.3.54, 0.3.53
all possible versions conflict with previously selected packages.
previously selected package web-sys v0.3.39
... which satisfies dependency web-sys = "^0.3.39" of package wgpu v0.6.0
... which satisfies dependency wgpu = "^0.6" of package bevy_wgpu v0.2.1
... which satisfies dependency bevy_wgpu = "^0.2.1" of package bevy v0.2.1
... which satisfies dependency bevy = "^0.2.1" of package physme v0.2.4
... which satisfies dependency physme = "^0.2.4" of package ast4 v0.1.0 (C:\Users\jmonasterio\rust\ast4)

@jmonasterio It looks like you have Bevy 0.6 and Bevy 0.2.1 in your dependency tree. Looks like physme/ast4 need to update to Bevy 0.6.

Closing this as it appears to be resolved. Feel free to continue discussing if you think otherwise.


[Docs] Update "How to Get Your Token" Section for Clarity on Local an…

…d Remote Providers
This update clarifies the process of obtaining the authentication token in Meshery by distinguishing between Local and Remote Providers. The "How to Get Your Token" section has been revised to provide specific instructions for each provider type in both the Meshery UI and Meshery CLI.
Changes include:

Detailed login instructions for Local and Remote Providers.
Clear steps for retrieving the token from the Meshery UI.
Enhanced guidance for using the Meshery CLI to obtain the token.

This addresses issue #12184.
Notes for Reviewers
This PR fixes #
Signed commits

[ ] Yes, I signed my commits.

@ianrwhitney might be interested in this.

🚀 Preview for commit ee7b669a6e28a15d7e537731120fe7e349881497 at: https://6706c6ed91cf1c2591e91d3d--meshery-docs-preview.netlify.app

Awesome. 🙌
@aryannargotra, thanks much for jumping and taking care of this. We have the "all clear" from @ianrwhitney. 🚀

Oh, I just noticed a couple items. I'll make a couple suggestions.

Oh, I just noticed a couple items. I'll make a couple suggestions.

Sure! Please go ahead.

🚀 Preview for commit a490ea10cd368b0da0163fef383d5cfc8187053b at: https://670941df2063a6a220ad1677--meshery-docs-preview.netlify.app

@aryannargotra
Thank you for your contributions! Let's discuss this during the Meshery Development call today at 9:00 AM CT | 7:30 PM IST. Adding it as an agenda item to the meeting minutes || meeting link.

🚀 Preview for commit bb3743ff0144800d4b68c50f5a32a04df93b532f at: https://670f9a69e21712480f2773c8--meshery-docs-preview.netlify.app


feat(es/minifier): Ignore closure in initializer from sequential inliner

Description:
let updateFunction = (oldValue, ...functionArgs) => {
    let interceptedValue = value2(isControlled ? stateRef.current : oldValue, ...functionArgs);
    onChangeCaller(interceptedValue, ...args);
    if (!isControlled) return interceptedValue;
    return oldValue;
};
setStateValue(updateFunction);

We can ignore the body of the arrow expression in the initializer because the function body is not evaluated before evaluating setStateValue.
Related issue (if exists):

Closes https://github.com/swc-project/swc/issues/5694

Should we make arrow function inline able? Seems like a more generic fix

@Austaras It's a more general fix (and I prefer it), but it will require much more analysis.
Also, this is not about inlining arrow functions. It's about just marking arrow functions as side-effect-free for sequential inliner


Fix pumphistory exit code ignored.

Fix issue where pumphistory exit code wasn't causing the if statement to fail, and loop was continuing with a stale histroy file

only tested in the normal flow, but i need to reproduce an error somehow.

I believe this is ready to review and merge

Confirmed this fix works in real life as well:
Continuing oref0-pump-loop at Thu  1 Mar 19:05:10 PST 2018
Preflight OK. Profile less than 60m old; Profile valid. Pump history update failed. Last record 2018-03-01T18:36:05-08:00
Retrying invoke_pumphistory_etc
Pump history updated through 2018-03-01T19:02:54-08:00; meal.json refreshed


CleanMyMac will cleanup the cache folder, which makes an error when run brew cask doctor

By default, CleanMyMac will remove all caches in the ~/Library/Caches folder, in which case, the brew cask doctor command will give an error like this:
Error: No such file or directory - ~/Library/Caches/Homebrew/Casks
  Most likely, this means you have an outdated version of homebrew-cask. Please run:
    brew uninstall --force brew-cask; brew untap phinze/cask; brew untap caskroom/cask; brew update; brew cleanup; brew cask cleanup
  If this doesn’t fix the problem, please report this bug:
    https://github.com/caskroom/homebrew-cask#reporting-bugs

Possible solution is, creating the cache directory if cask doctor cannot find the cache directory under homebrew, instead of reporting an error.

This will be fixed when https://github.com/caskroom/homebrew-cask/pull/23265 is merged.

Great! Thanks.


Access Logging for moxxi binary

The moxxi binary needs to write access logs (line for every HTTP request logged) to stdout.

This can be done by simply wrapping the handlers in http://www.gorillatoolkit.org/pkg/handlers#LoggingHandler.
Include is github.com/gorilla/handlers
Use it with handlers.LoggingHandler(blah)


Errors are not logged through custom logger

Is there an existing issue for this?

[X] I have searched the existing issues

Current behavior
When an error is raised

I am not recieving it within my errorHandler
The error output is not using the custom logger I have injected following https://github.com/jmcdo29/nest-commander/issues/852

My output will be like:
Error: TEST[22:09:29.800] INFO (21106): Starting Nest application...
    context: "NestFactory"
[22:09:29.800] INFO (21106): CommandRootModule dependencies initialized
    context: "InstanceLoader"
[22:09:29.800] INFO (21106): AppModule dependencies initialized
    context: "InstanceLoader"
[22:09:29.800] INFO (21106): InstrumentationModule dependencies initialized
    context: "InstanceLoader"
[22:09:29.800] INFO (21106): OpenTelemetryModule dependencies initialized
    context: "InstanceLoader"
..... more logs

The output starts with the error Error: TEST which is not formatted as it should be with my logger, and then the buffered logs continue afterwards...
Minimum reproduction code
Command that returns an error
import { Logger } from '@nestjs/common';
import { Command, CommandRunner } from 'nest-commander';

@Command({
  name: 'test',
})
export class SomeCommand extends CommandRunner {
  private readonly logger = new Logger(SomeCommand.name);

  async run(inputs: string[], options: Record<string, unknown>): Promise<void> {
    this.logger.log(inputs);
    this.logger.log(options);
    await Promise.resolve();
    throw new Error('TEST');
  }
}

Expected behavior
Error is written with custom logger
Package

[X] nest-commander
[ ] nest-commander-schematics
[ ] nest-commander-testing

Package version
3.12.5
Node.js version
20.11.1
In which operating systems have you tested?

[X] macOS
[ ] Windows
[ ] Linux

Other
No response

What is the difference between serviceErrorHandler and errorHandler

serviceErrorHandler was created to handle errors that happen during this.commander.parseAsync as detailed in this PR, whereas errorHandler is the general commander error handler and modifies how errors at the commander level are handled


Adopt new Service-Manager version

Pull Request Test Coverage Report for Build 246

2 of 2 (100.0%)  changed or added relevant lines in 2 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 93.802%

Totals

Change from base Build 244:
0.0%

Covered Lines:
454

Relevant Lines:
484

💛  - Coveralls

Maybe merge it as  "version 0.1.1"


clarifying the title name

Description
This pull request updates the title name in the Prefix Caching documentation of the vLLM library to ensure clarity.
Fixes
No existing issues are linked to this PR.
Checklist
PR Title and Classification: [Doc]
Code Quality: Not applicable as this PR only involves documentation.
Documentation: The title name has been updated in the relevant documentation files.

/ready

the current doc looks reasonable to me:

any reason to change it?

This is how it looks to me on Microsoft Edge & Chrome

Just thought it's not clear enough, especially with start of paragraph "The core idea of PagedAttention is to ... "

can you help figure it out, how to decouple the title shown in the side bar, and the title shown in the doc page?


Fix the allowed extensions for uploading .env files

What does this PR do?
It should not include the leading dot because the InputFile component adds the leading prefix so including the leading dot results in an extra dot like:

Test Plan
Before the change, .env files were not selectable:

and trying to drag a .env file into the modal results in:

After the change, I can drag and drop the .env file in:

Unfortunately, I still can't select .env files from Finder 🧐
Related PRs and Issues
None
Have you read the Contributing Guidelines on issues?
Yes

I think we should just allow all extensions. There is also cases of .env.production and co.


pdf export pages are not exact

When exporting to pdf, some notes will be left out. Probably due to export function scrolling an incorrect amount of pixels per page.

fixed in commit https://github.com/oakleyaidan21/kalimba-tabs/commit/bd8033fe3f41e409b7ac0dec4f3bde645891c19c


No space left on device error with singularity when using --cifti-output 170k command

What version of fMRIPrep are you using?
I use 20.2.3 but i also faced the same error with 20.1.1
What kind of installation are you using? Containers (Singularity, Docker), or "bare-metal"?

YES
Did fMRIPrep generate the visual report for this particular subject? If yes, could you share it?
no the error comes at the very beginning
Can you find some traces of the error reported in the visual report (at the bottom) or in crashfiles?
Downloading https://templateflow.s3.amazonaws.com/tpl-fsLR/tpl-fsLR_space-fsaverage_hemi-L_den-59k_sphere.surf.gii
  0%|          | 0.00/1.40M [00:00<?, ?B/s]
  2%|▏         | 25.6k/1.40M [00:00<00:09, 148kB/s]
 14%|█▍        | 200k/1.40M [00:00<00:01, 638kB/s]
 55%|█████▌    | 774k/1.40M [00:00<00:00, 1.82MB/s]
 75%|███████▌  | 1.05M/1.40M [00:00<00:00, 1.95MB/s]
Process Process-2:
Traceback (most recent call last):
  File "/usr/local/miniconda/lib/python3.7/site-packages/templateflow/api.py", line 233, in _s3_get
    f.write(data)
OSError: [Errno 28] [log.txt](https://github.com/nipreps/fmriprep/files/7065698/log.txt)
[log.txt](https://github.com/nipreps/fmriprep/files/7065701/log.txt)
[log.txt](https://github.com/nipreps/fmriprep/files/7065703/log.txt)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/miniconda/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
    self.run()
  File "/usr/local/miniconda/lib/python3.7/multiprocessing/process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/miniconda/lib/python3.7/site-packages/fmriprep/cli/workflow.py", line 82, in build_workflow
    retval["workflow"] = init_fmriprep_wf()
  File "/usr/local/miniconda/lib/python3.7/site-packages/fmriprep/workflows/base.py", line 64, in init_fmriprep_wf
    single_subject_wf = init_single_subject_wf(subject_id)
  File "/usr/local/miniconda/lib/python3.7/site-packages/fmriprep/workflows/base.py", line 304, in init_single_subject_wf
    func_preproc_wf = init_func_preproc_wf(bold_file)
  File "/usr/local/miniconda/lib/python3.7/site-packages/fmriprep/workflows/bold/base.py", line 803, in init_func_preproc_wf
    repetition_time=metadata['RepetitionTime'])
  File "/usr/local/miniconda/lib/python3.7/site-packages/fmriprep/workflows/bold/resampling.py", line 634, in init_bold_grayords_wf
    for hemi in 'LR'
  File "/usr/local/miniconda/lib/python3.7/site-packages/fmriprep/workflows/bold/resampling.py", line 634, in <listcomp>
    for hemi in 'LR'
  File "/usr/local/miniconda/lib/python3.7/site-packages/templateflow/api.py", line 78, in get
    _s3_get(filepath)
  File "/usr/local/miniconda/lib/python3.7/site-packages/templateflow/api.py", line 234, in _s3_get
    t.update(len(data))
OSError: [Errno 28] [log.txt](https://github.com/nipreps/fmriprep/files/7065698/log.txt)

Are you reusing previously computed results (e.g., FreeSurfer, Anatomical derivatives, work directory of previous run)?
Yes anatomy segmentation previously ran with fmriprep and put in the right folders.
fMRIPrep log
If you have access to the output logged by fMRIPrep, please make sure to attach it as a text file to this issue.
The bug is generated as long as i put the --cifti-output 170k command, otherwise there is no error.
I checked first over the web and found problems relative to the temp data folder specified using -w command. I tried without specifying (same error), with specifying the server temp folder (/tmp/mszinte/ same error) by binding another folder (see command above -> same error). I also tried on another server with the same data and replicated the error again.
I would need to ouput my data in this format so at the moment i'm getting stuck, did anyone face the same problem ?
Thank you very much for your help and for the tremendous work done with fmriprep !

Thanks @effigies
I used one of the method proposed on your link and it worked !
I thus created a folder where the template will be downloaded and used the follwing lines of code to bind it after specifying where these template are located.
export SINGULARITYENV_TEMPLATEFLOW_HOME=/opt/templateflow
in the singularity command : -B /scratch/mszinte/softwares/fmriprep_tf/:/opt/templateflow
Here is the full command:
export SINGULARITYENV_TEMPLATEFLOW_HOME=/opt/templateflow

singularity run --cleanenv -B /scratch/mszinte/softwares/fmriprep_tf/:/opt/templateflow
-B /scratch/mszinte/data:/work_dir /scratch/mszinte/softwares/fmriprep-20.2.3.simg
--fs-license-file /work_dir/freesurfer/license.txt /work_dir/PredictEye/bids_data/
/work_dir/PredictEye/deriv_data/fmriprep_new/ participant --participant-label 01
-w /work_dir/PredictEye/temp_data/
--bold2t1w-dof 12 --ignore sbref --output-spaces T1w fsnative fsaverage MNI152NLin2009cAsym:res-1
--cifti-output 170k --low-mem --mem-mb 64000 --nthreads 32

Thanks again.


Fixed issue with corr_terms and confidence envelopes, and more

Hi @wbalmer and @gotten,
This PR contains a bit more changes then I initially planned to include so please have a look if you agree with these changes 😉.
There are two issue that I fixed:

When corr_terms include both True and False, it was only including the uncorrelated terms in likelihood function because of if ~corr_terms.all():
The 1 sigma envelopes were not correctly implemented from what I understood. This I have fixed by taking 200 samples from the posterior and calculating the percentiles.
Added the samples (instead of envelopes) to the deltaRA-deltaDEC plot

A couple of changes to the parameters:

The step_size can be set with generate_plots to speed up trackplot
Removed the jd_tt parameter (see issue #19) and instead added ref_epoch to generate_plots. When set to None (=default), it uses the mean date form the input astrometry. Change the days forward and backward to 3 years.
Added fileprefix parameter to system class (e.g. for the dynesty files). Perhaps we could use a single fileprefix for all methods?

And some more updates:

Simplified the extraction of samples by using samples_equal from dynesty
Added Github Actions workflow
Updated .gitignore and init
Added unit tests to the tests folder

Great that the package is now available on PyPI! I noticed that the name has been changed to backtracks. I have changed the name also in the folder and filenames. @wbalmer probably best to also change the name of the Github repo?
About the package structure: I would suggest to simplify it to just having a backtracks folder (like before) instead of embedded in src. I think that is a quite common package structure, in particular because there are only a few files. But let me know in case you disagree then I will change it back!
Also, I would recommend to not add any binary files, so only include text files. Any changes to a binary file will store a full copy of the file so quickly increasing the repo size. I just checked and it is 70 MB already when cloning from Github. I have therefore removed all binary files that seemed not needed to include. The unit tests run in a few minutes so I think it is fine to run the actual fit instead of including a precalculated fit. If you want to run them, just use pytest in the main folder.
Finally, I tested the installation with Python 3.11 (with pip install .) and that seems to work fine as well!

This is really excellent, thank you so much for clarifying the package structure, the pytests, etc. I'll review in just a few minutes and merge.

Sorry for the many commits... It took me a while to figure out how to setup the Actions workflow because installing orbitize caused an error. You can squash the commits if you decide to merge. But let me know if you have any comments!


Design Python API

The problem is, there's no clear design, it's mix of different abstractions. Pandera DataFrames are high-level abstract things, but represent low-level tables (which can come from arrow or geopackage). There's a high-level abstraction of a nodetype in between, but it doesn't represent something low-level. The current Model is a mix of a representation of these nodetypes with a (partial) low-level config. Will make a separate issue about this, which needs a refinement.
Originally posted by @evetion in https://github.com/Deltares/Ribasim/issues/580#issuecomment-1715360255

Some thoughts:
Requirements
Ideally we have a clean and extensible low-level design (representing config files/tables on disk) that makes it easy to build a user-facing higher level design, which abstract things away like nodeids and the separate node tables. It would also be possible to iteratively start a model (from scratch) and add nodes one by one. This would also make some tests a bit easier, as one could do BasinStatic(id=1, foo=bar) and add it, instead of making numpy vectors.
Current issues
So the Config file is quite easy to represent, as it is autogenerated, and quite straightforward. The issue is with representing tables, as a single geopackage class doesn't work (can also be arrow files). The nodetype abstraction doesn't exist in the files, we essentially have a loose collection of tables, but these are not represented as such (but as DataFrames). Sort is now defined on a thing like Basin, which has multiple sort statements for each DataFrame. Everything else is done in the Model class, often reaching down to check things on its children (https://github.com/Deltares/Ribasim/blob/main/python/ribasim/ribasim/model.py#L360-L363), instead of letting the children be responsible. It basically is not SOLID.
Low-level design
We might try to go with the hydrolib design (by me 😅, see https://deltares.github.io/HYDROLIB-core/0.5.2/topics/principles/). It essentially represents each file on disk as a (Pydantic based) class, which link to one-another by the attribute that normally holds a filename. So to access the static basin table, one would use Config().Basin.static (normally pointing to a filename).
Not sure how the above design can use autogeneration, because these classes require some customization.

tables, but these are not represented as such but as DataFrames

Can you please explain what's the difference between a table and a DataFrame in that context?

Thanks for the writeup. One thing to add to the requirements is that it should easily scale to 1e4-1e5 nodes. I never really profiled, but I've had to wait 10s for writing a 1000 node model, which IMO is too much, that should be <1s.
Linking some issues that may be relevant:

[ ] https://github.com/Deltares/Ribasim/issues/382 not sure if needed, but let's design for the future (v2)
[ ] https://github.com/Deltares/Ribasim/issues/47 the issue talks about the TOML, but how do we do case management in ribasim-python?
[ ] https://github.com/Deltares/Ribasim/issues/252 would be nice to tackle this issue with a redesign
[ ] https://github.com/Deltares/Ribasim/issues/311 or more generally, how (or if) do we facilitate postprocessing / reading output. Or don't we need to take that into account when designing the API?

tables, but these are not represented as such but as DataFrames

Can you please explain what's the difference between a table and a DataFrame in that context?

Ah yeah, I was unclear. I meant these are file based database (or arrow) tables (edited the above comment to reflect that) and not a generic in-memory table format for data analysis.
While similar, I like to separate the concerns (each class with small interfaces). So a file based table class ideally only has a link with their source file and related read/write methods, maybe with a parent class having methods related to unique ids. The interface of a DataFrame is huge and apart from some I/O methods, doesn't give us what we need. Ideally we wrap them, or allow them as input/output.

Thanks for the writeup. One thing to add to the requirements is that it should easily scale to 1e4-1e5 nodes. I never really profiled, but I've had to wait 10s for writing a 1000 node model, which IMO is too much, that should be <1s.
Linking some issues that may be relevant:
* [ ]  [Start supporting pydantic v2 #382](https://github.com/Deltares/Ribasim/issues/382) not sure if needed, but let's design for the future (v2)

Pydantic V2 will help with that, as it is much faster (core rewritten in Rust). Then again, I'm not sure if we can improve your 1e5 nodes write if it depends on geopackage writing. And I think we also have the usecase of a few million rows for timeseries, which could be way slower to read/write?

Yes the 1 second target is for a normal size (~1000) node model, without very long timeseries.
For timeseries with millions of rows we should recommend Arrow instead of GeoPackage anyway. So if performance is bottlenecked on IO performance of C/C++ libraries that would probably be a good thing, because that would mean its not spending it's time on converting each row back and forth to a dict first, as mentioned in https://github.com/Deltares/Ribasim/pull/178#issuecomment-1517540061.

Discussion with @evetion and @visr:
Requirements:

Performance:

One thing to add to the requirements is that it should easily scale to 1e4-1e5 nodes
I never really profiled, but I've had to wait 10s for writing a 1000 node model, which IMO is too much, that should be <1s

Autogeneration:

Table Schema
Config Schema

"Better architecture"
Full support and tests for writing Geopackage AND Arrow tables
Optional names for nodes
Benchmark Pandera

Pydantic v1 or v2?
Potentially move to Pandera JSON Schemas

Dataframe:

Benchmark if numpy based pandas is a problem
Options if yes:

Print a warning if numpy backend of pandas is used and suggest to use the arrow backend
Use internally polars and convert pandas dataframes to polars if given with a printed warning

High level:

Stop manage node IDs manually
Nodetype level also has validation

This shows that almost all time is spent in validation. For reading that was expected, but for writing not so much. Based on the profile this is triggered by calling sort on the dataframe before writing. It would be nice if validation could be done faster.

The slow write (sorting triggered validation) was fixed by #597.

I got positive feedback on the use of Pydantic and the overall design of Hydrolib, noting that performance has not been an issue, except for parsing/serializing very large ascii files, unrelated to the architecture.
Therefore, I propose to roughly follow the same structure, having an API that closely follows the file formats (especially the config file). The difficulty is in having auto-generated code while customizing the classes extensively, and handling topology/id-relationships from otherwise ideally independent tables.
Implementation
handle = ContextVar(DataBaseHandle)

class Model(FileModel): # mirroring all attributes of config.toml
	output: Output
	basin: Basin
	# other nodetypes
	solver: Solver
	logging: Logging

	# Unique id counter
	_id: int

	"""Initialize empty or from file"""
	def __init__(self, fn=None):
		# get/set handle for recurring database operations
		# prevents opening/closing the db many times
		handle.set()

	"""Write to file(.toml)/folder(all?)"""
	def write(self, fn):
		pass

	"""Run the model."""
	def run(self, fn):
		pass

class Basin(NodeModel):
	time: Table
	# all other tables(!)

    def add_time(row: PydanticModel, geometry=None):
	    pass

# Could also be an ArrowTable, changing read/write
class Table():
	rowtype: SomeSchema
	df: DataFrame[SomeSchema]
	_io: Union[Arrow, GeoPackage]  # or inhereted from parent?

	"""Initialize empty or from file"""
	def __init__(self, fn=None):
		handle.get()

	def write(self, fn):
		self._io.write(self)

Will this design still use the pydantic backend of pandera considering its inefficient way of validating row-wise?

Will this design still use the pydantic backend of pandera considering its inefficient way of validating row-wise?

I'd say that's an implementation detail, it should validate, and ideally be fast, but that doesn't change the API. To fix the performance, we probably need to transpile into a pandera schema. I still like to keep using both pandera and pydantic in the package.

Agree on all three statements.

Could use a comment on how to work object/row based with things like time/profile. Those are individual rows, with repeated node ids. Do we need a new groupeddrow object? Like BasinTimes(time=[now(), now()], level=[1,2])?

Requirements for this package:

New model from scratch
Edit any existing model
Conversion entry point by providing dataframe
Plotting (existing)

@visr Can you provide input on the prevalence of these use cases?