Issue 25: Extend rllib examples to save, load, and render bots

This patch uses ray.tune to save the trained models from self_play_train.py and allows you to view their behaviour in a pygame instance by running view_model.py.
Disclaimer:
I am only ~1 month familiar with RLLib and Melting Pot.
Note:
It is unfortunate that I needed to pass RayModelPolicy a full rllib.agents.trainer.Trainer instance, rather than just a rllib.policy.policy.Policy. This is because rllib.agents.trainer does some pre-processing of the inputs before forwarding to the policy (see lines 1452-1457 in Ray 1.11.0).
Also, although against the spirit of the library, I wanted to demonstrate that the RayModelPolicy could play against the PermissiveModelPolicy. I was not able to do this however, as replacing some of the bots in view_model.py with bots created via:
bot_factory.build(bot_factory.get_config("ah3gs_bot_finding_berry_two_the_most_tasty_0"))
Resulted in errors about the observations that they received. Therefore rllib bots and existing meltingpots cannot be mixed at the moment.
Finally, would you be open to answering some questions I have about use of this repo? I intend to make some modifications to meltingpot for my experiments and I would appreciate it if I could run my proposed changes by you.

Regarding using our saved model bots for evaluation, it would work if you instead just instantiate an scenario rather than the substrate. Then the appropriate wrappers would be put around the bots, and you'd be able to get an environment that you only connect your RLLib policies to. Then you can just render the WORLD.RGB which would show all players. Makes sense?

Thank you for the pointer. I think it makes sense, though I will leave this as future work for now. (First I need successfully train some bots!)

Hi Richard!
On Tue, May 3, 2022 at 11:38 AM Richard Willis @.***>
wrote:

This patch uses ray.tune to save the trained models from
self_play_train.py and allows you to view their behaviour in a pygame
instance by running view_model.py.
Disclaimer:
I am only ~1 month familiar with RLLib and Melting Pot.
Note:
It is unfortunate that I needed to pass RayModelPolicy a full
rllib.agents.trainer.Trainer instance, rather than just a
rllib.policy.policy.Policy. This is because rllib.agents.trainer does some
pre-processing of the inputs before forwarding to the policy (see lines
1452-1457 in Ray 1.11.0).
Also, although against the spirit of the library, I wanted to demonstrate
that the RayModelPolicy could play against the PermissiveModelPolicy. I was
not able to do this however, as replacing some of the bots in view_model.py
with bots created via:
bot_factory.build(bot_factory.get_config("ah3gs_bot_finding_berry_two_the_most_tasty_0"))
Resulted in errors about the observations that they received. Therefore
rllib bots and existing meltingpots cannot be mixed at the moment.
Finally, would you be open to answering some questions I have about use of
this repo? I intend to make some modifications to meltingpot for my
experiments and I would appreciate it if I could run my proposed changes by
you.

I somehow lost track of this thread -- I remembered that someone had said
they were planning to make some modifications to meltingpot for their
experiments and wanted to run the changes by us. And I was even actively
searching around for this message thread to reply before, but I couldn't
for the life of me find it then.! Concluded I must have hallucinated the
original message.. Anyway, I've found it now :). Let me know if you still
want to chat sometime. I'm happy to schedule a call.
On Tue, May 3, 2022 at 11:38 AM Richard Willis @.***>
wrote:

This patch uses ray.tune to save the trained models from
self_play_train.py and allows you to view their behaviour in a pygame
instance by running view_model.py.
Disclaimer:
I am only ~1 month familiar with RLLib and Melting Pot.
Note:
It is unfortunate that I needed to pass RayModelPolicy a full
rllib.agents.trainer.Trainer instance, rather than just a
rllib.policy.policy.Policy. This is because rllib.agents.trainer does some
pre-processing of the inputs before forwarding to the policy (see lines
1452-1457 in Ray 1.11.0).
Also, although against the spirit of the library, I wanted to demonstrate
that the RayModelPolicy could play against the PermissiveModelPolicy. I was
not able to do this however, as replacing some of the bots in view_model.py
with bots created via:
bot_factory.build(bot_factory.get_config("ah3gs_bot_finding_berry_two_the_most_tasty_0"))
Resulted in errors about the observations that they received. Therefore
rllib bots and existing meltingpots cannot be mixed at the moment.
Finally, would you be open to answering some questions I have about use of
this repo? I intend to make some modifications to meltingpot for my
experiments and I would appreciate it if I could run my proposed changes by
you.
You can view, comment on, or merge this pull request online at:
https://github.com/deepmind/meltingpot/pull/31
Commit Summary

f9dfc16
https://github.com/deepmind/meltingpot/pull/31/commits/f9dfc16971d5f38f232f6bd020041da7ba58bf1e
Issue 25: Extend rllib examples to save, load, and render bots

File Changes
(4 files https://github.com/deepmind/meltingpot/pull/31/files)

M examples/rllib/self_play_train.py
https://github.com/deepmind/meltingpot/pull/31/files#diff-c80f0cabe6240e1cb97d07f4b9ce809de98cbedde81fd9681e926dad303f56ad
(53)
R examples/rllib/utils.py
https://github.com/deepmind/meltingpot/pull/31/files#diff-cf78c5db8de0c3b77884016a2fec83a3082f3686161243746af8d5402950fd61
(71)
A examples/rllib/view_models.py
https://github.com/deepmind/meltingpot/pull/31/files#diff-18e81b4f0161d1eac8566855f961770176ef4ee6ac0e5d46692a024ab0d81675
(100)
M pyproject.toml
https://github.com/deepmind/meltingpot/pull/31/files#diff-50c86b7ed8ac2cf95bd48334961bf0530cdc77b5a56f852c5c61b89d735fd711
(4)

Patch Links:

https://github.com/deepmind/meltingpot/pull/31.patch
https://github.com/deepmind/meltingpot/pull/31.diff

—
Reply to this email directly, view it on GitHub
https://github.com/deepmind/meltingpot/pull/31, or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAHFJWS6VX4CHRHQCF77JNLVID64BANCNFSM5U6QPFYA
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

@jzleibo thank you, that would be great. I shall send an email to the address given in your personal blog.


[Feature request] Show in sidebar option

Hey! Thanks for this plugin. Could you consider adding a "Show in sidebar" option in the plugin settings?
Example:

This is not possible due to technical limiting in uptime kuma. There is an upstream feature request open to make this possible:
../Frenck


chore(deps): upgrade configuration

Upgrades project dependencies. See details in workflow run.

Automatically created by projen via the "upgrade-configuration-k8s-25-main" workflow

⚪ Backport skipped
The pull request was not backported as there were no branches to backport to. If this is a mistake, please apply the desired version labels or run the backport tool manually.
Manual backport
To create the backport manually run:
backport --pr 2172

Questions ?
Please refer to the Backport tool documentation


Amplify Codegen does not generate output in a consistent/deterministic order

First of all, I'm surprised I couldn't find an existing bug, so if I've missed it, apologies. But as I couldn't find one I'm creating this ticket as as surely I'm not the only one this drives nuts over time...
Describe the bug
Running amplify push and regenerating GraphQL code bindings, or simply running amplify codegen on a GraphQL schema that hasn't changed results in unnecessary changes to files such as graphql/schema.json simply due to reordering of the same output due to apparent non-deterministic ordering in the generated code.
Amplify CLI Version
4.35.0
To Reproduce
Run the commands listed above a few times and see unnecessary and noisy changes made to files such as graphql/schema.json.
Expected behavior
All code generated should be generated in a consistent deterministic, possibly sorted, order so that:

It is clear that a change in the generated file is caused by a change to the GraphQL schema that has just been pushed up, rather than changes in generated files unrelated to the schema change.

Currently I have to either spend a lot of time eyeballing the diffs to and reverting hunks if I'm absolutely sure something didn't change, or otherwise include it in file history just to be safe.
This is especially true because updating the @aws-amplify/cli package can cause expected changes to the generated files that are required even if not caused directly by user changes to their GraphQL schema, and so running a codegen after updating the package version without any GraphQL schema changes should be able to show if the version has introduced changes or not; currently that's not clear, it could just be non-deterministic reordering.

Git Commit history is not polluted with a ton of unnecessary changes making it hard to browse file history.

Screenshots
An example diff highlighting one such apparently unnecessary reordering:

Desktop (please complete the following information):

OS: macOS 11.0.1
Node Version: 15.2.0

Additional context
Git commit history is important to in order to understand when particular bugs may have been introduced and by what changes, and so unnecessary noise in codegen files is more important than it may seem.

Not sure if there is any update on this ticket. It would be great from an infrastructure-as-code perspective if generated code was generated in a way that removed the noise of unchanged but moved code since it's committed into source control.
I don't know how feasible that is?

I've definitely run into this too, and see the same thing that "aws_api_key", "aws_iam" and "aws_cognito_user_pools" seem to shuffle around a lot which can be super confusing. I'd be super happy if the sort order could be well defined.
Just as a random example I've used JSON for Modern C++ in another project and it provides the type ordered_json which is perfect for these kinds of situations (see https://json.nlohmann.me/api/ordered_json/). If Amplify could use a similar mechanism inside whatever JSON library it's using that would be incredibly helpful! 🙂


removing interactive network

What do these changes do?

the network is defined in the simcore repo

Related issue/s
Related PR/s
Checklist

[ ] I tested and it works

@matusdrobuliak66 we need a related osparc-config update

@matusdrobuliak66 we need a related osparc-config update

You mean osparc-ops-deployment-configuration? the variable is already there I reuse the same one.


dynaconf v3.2.4 ❄

PKG-3352 dynaconf 3.2.4 ❄

upstream: https://github.com/dynaconf/dynaconf/tree/3.2.4
license: https://github.com/dynaconf/dynaconf/blob/3.2.4/LICENSE
dependency files:

https://github.com/dynaconf/dynaconf/blob/3.2.4/setup.py

conda-forge: https://github.com/conda-forge/dynaconf-feedstock
pypi: https://pypi.org/project/dynaconf

Destination channel: Snowflake
Changes

fix recipe
add abs file for destination channel

Dependency for

kedro v0.18.14 ❄

Linter check found the following problems:
The following problems have been found:
ERROR: clone/recipe/meta.yaml:23: missing_wheel: For pypi packages, wheel should be present in the host section
WARNING: clone/recipe/meta.yaml:43: missing_description: The recipe is missing a description
Errors were found


Add sub task system for tasks; resolved #195

Codecov Report

Merging #290 into dev will increase coverage by 0.05%.
The diff coverage is 100%.

@@             Coverage Diff              @@
##                dev     #290      +/-   ##
============================================
+ Coverage     78.85%   78.91%   +0.05%
- Complexity      232      233       +1
============================================
  Files            82       82
  Lines           752      754       +2
============================================
+ Hits            593      595       +2
  Misses          159      159

Impacted Files
Coverage Δ
Complexity Δ

app/Models/Task.php
80% <100%> (+5%)
5 <1> (+1)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f13c963...a0d29be. Read the comment docs.

@MehediDracula nice work :+1:


Create asmadrid.txt

@pukinano I am sorry, but your website seems to be hacked. I already sent a report to<EMAIL_ADDRESS>and it may not be a good idea to check the eligibility of your school for JetBrains free educational licenses, because I am not sure that the content of the website is original. At the moment, the website forces a download of update.js file, which is to run a script called Application.BitCoinMiner at the users' computer. I'll get back and check the website content in few days. Thank you for the understanding.

@pukinano Pull request merged. Please notice that free educational license for JetBrains tools is for high and higher schools students, i.e. those who are over 13 of age. Thank you!


Change subdomain?

Currently, www.binance.com isnt resolving for me, as per their ongoing issues.
I can load us.binance.com as directed from their twitter etc, but cannot connect using this API wrapper as it uses www. Is there any way to change it to us temporarily? IF not, could be a useful feature for the future. Perhaps something in appsettings.json or the endpoint to use?
(Actually, im not 100% sure the API is accessible from the us subdomain, though I would assume it is...)
For anyone else, I havent tried it yet but I am assuming a temporary fix would be to setup a hosts file entry redirecting www traffic to us traffic on binance domain, by getting the IP by pinging us.binance.com...

@pingu2k4, I recommend you upgrade to 0.2.0-alpha30. This issue should be resolved as of 0.2.0-alpha29.

Thats sorted it now, thanks. Was a lot of work changing from alpha24 to alpha30! (mainly websockets)


bedtools getfasta - Error: malformed BED entry

Is there a limit to the length of fasta sequence that bedtools can use? I'm using bedtools getfasta on sequences that are 2.4Gb long and I cannot get it to work extract sub-sequences towards the end of the input sequences.
I have 24 samples/sequences, all exactly the same length<PHONE_NUMBER> bps). I need to chop my genomes into ~250Mb chunks so that they will work with a downstream genome introgression tool.
Example of my samples.fasta.fai file (to confirm the length of my genomes) is:
weasel<PHONE_NUMBER>      8       60      61
LIB18995<PHONE_NUMBER><PHONE_NUMBER>      60      61
LIB18991<PHONE_NUMBER><PHONE_NUMBER>      60      61

I have created 10 bed files, which each provide 250Mb intervals, e.g.
1.bed
LLIB18989        0       250000000
LIB18990        0       250000000
LIB18991        0       250000000

2.bed
LIB18989        250000001       500000000
LIB18990        250000001       500000000
LIB18991        250000001       500000000

etc.
Running the command...
$ bedtools --version
bedtools v2.27.1
$ bedtools getfasta -fi samples.fasta -bed 1.bed -fo 1.fasta

..works as expected, but once I provide the following bed file (which is my penultimate bed file):
9.bed
LIB18989<PHONE_NUMBER><PHONE_NUMBER>
LIB18990<PHONE_NUMBER><PHONE_NUMBER>
LIB18991<PHONE_NUMBER><PHONE_NUMBER>

I get the error
Error: malformed BED entry at line 1. End Coordinate detected that is < 0. Exiting.

Also, when I provide my final bed file..
10.bed
LIB18989<PHONE_NUMBER><PHONE_NUMBER>
LIB18990<PHONE_NUMBER><PHONE_NUMBER>
LIB18991<PHONE_NUMBER><PHONE_NUMBER>

I get the following error
Error: malformed BED entry at line 1. Start Coordinate detected that is < 0. Exiting.

Note the end co-ordinate throws the error in the first instance and the start coordinate in the second.
It appears that bedtools getfasta doesn't like coordinates that are in excess of somewhere in the region of 2-2.5Gb. Is this a bug, or am I doing something wrong?
Many thanks,
Graham

I was initially worried that you are running into a limitation of 32 bit integers, but bedtools uses an unsigned, 32-bit integers which have a max of 4,294,967,295. The fact that you are running into this suggests we are using signed 32 bit integers (max of 2,147,483,647) somewhere in the getfasta logic. If you have the relevant FASTA handy somewhere for download, please share it. Otherwise, I will try to create a test set.

Unfortunately the full file is 55Gb in size and I'm also not sure my collaborators would be happy for me to share all the data. Would you be able to generate one?

Yep, can make a synthetic example easily. Thanks

Update: we have a fix for this coming soon, as we are refactoring the way coordinates are handled to support much larger chromosomes.

This is fixed - we now support massive chromosomes!

I have the same issue... what was the solution?

chrIV	WormBase	gene	-305	695	.	+	.	ID=Gene:WBGene00021406;Name=WBGene00021406;locus=frm-5.2;sequence_name=Y38C1AB.4;biotype=protein_coding;Alias=frm-5.2,Y38C1AB.4
I have the same issue. How did you solve it?
Thanks
Su


Introduce skip_proto_gen and skip_grpc_gen flags in artman yaml

Introduce skip_proto_gen and skip_grpc_gen flags to toggle proto and grpc code gen tasks in GapicClientPipeline (currently, for some languages, running gapic task will always run proto and grpc tasks as well). These flags will be useful for java longrunning, where only gapic but not proto/grpc artifact is needed.

/cc @garrettjonesgoogle

close this PR since the function is now already supported.


Add ListOptions for listing user migrations

Fix for https://github.com/google/go-github/issues/2416

@amitsaha - it looks like one of the examples needs to be updated. Please run go test ./... from the base directory of the repo to make sure all tests pass. Please see CONTRIBUTING.md for more details. Thanks.

Thanks. I did that before I pushed my branch:
$ go test ./...
go: downloading golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be
go: downloading golang.org/x/net v0.0.0-20210226172049-e18ecbb05110
ok  	github.com/google/go-github/v45/github	0.840s
?   	github.com/google/go-github/v45/test/fields	[no test files]
?   	github.com/google/go-github/v45/test/integration	[no test files]

From the linting failures above I can see that there is an example needed to be updated, so I can see the failure when I run go test from within the example directory.
Not sure if I am doing something wrong?
Updating with the fix.

Thanks @gmlewis !

@gmlewis I suppose you would want to me to add a test as suggested by @raynigon ?

@gmlewis Thanks! I know we had a new release just days back, is there any chance I can request you for a new release with this change?

@gmlewis Thanks! I know we had a new release just days back, is there any chance I can request you for a new release with this change?

Yes, I think the worst thing that would happen is I might annoy some people, but that's OK.  I've done it before and survived. :joy:

@gmlewis Thanks! I know we had a new release just days back, is there any chance I can request you for a new release with this change?

Yes, I think the worst thing that would happen is I might annoy some people, but that's OK. I've done it before and survived. joy

Thanks a lot!

https://github.com/google/go-github/releases/tag/v47.0.0 now contains this change.


Corrected programming language for article

Fixes #13331.
Corrected dev_langs meta data.
Followed coding conventions (using var instead of data type when data type is clear from right hand side)

I'm going to push more commits to this PR. Don't merge now.

@Youssef1313

I'm going to review some related articles and may push more commits to this PR. Don't merge now.

I changed the title to add “WIP” for “Work in progress” to the title. We recognize that tag for any PR that’s still being worked on and isn’t ready to merge. Just remove that tag when you’re ready for final review and merge.

@Youssef1313

I'm going to review some related articles and may push more commits to this PR. Don't merge now.

I changed the title to add “WIP” for “Work in progress” to the title. We recognize that tag for any PR that’s still being worked on and isn’t ready to merge. Just remove that tag when you’re ready for final review and merge.

Thanks for the info about "WIP".
I've updated a broken link, but not sure if the new link is good replacement or not.

Thanks for these changes @Youssef1313
I had one update, and once you make that change, I'll :shipit:.

Thanks for the update, @BillWagner. I've committed it.


ownCloud: Deployment details improvements

Description
The deployment data for ownCloud needs some improvements.

All of these fields should be the same size as the other fields.

The content of the Owncloud Admin Password field should be hidden by default.

Email/SMTP Fields shouldn't appear if SMTP is not used/configured.

If they include data then their size should be the same size as the other fields.

The content of the SMTP Password field should be hidden by default.

Verified on devnet.
The size of the fields was modified and the content of the password field is hidden. and the smtp fields don't appear if it's not configured.

and if it's configured they appear normally and their size is normal


Height adjustment when keyboard appears is wrong

Hi, I am trying to shorten the screen when the keyboard is active. But it always adjust it weirdly in android, it works fine in iOS.
Before keyboard is active:

After keyboard is active:

I am using RN 0.37, this also happens when i am starting from scratch:
This is my code:
import React from 'react';

import {
  Keyboard,
  Dimensions,
  ScrollView,
  StyleSheet,
  Text,
  TextInput,
  AppRegistry
} from 'react-native';

var {height, width} = Dimensions.get('window');
var _keyboardWillShowSubscription;
var _keyboardWillHideSubscription;
height = height - 20;

class awesomeproject extends React.Component {
  constructor(props) {
 		super(props);
 		this.state = {
			height: height,
		};
	}
	componentDidMount() {
		_keyboardWillShowSubscription = Keyboard.addListener('keyboardDidShow', (e) => this._keyboardWillShow(e));
		_keyboardWillHideSubscription = Keyboard.addListener('keyboardDidHide', (e) => this._keyboardWillHide(e));
	}
	componentWillUnmount() {
		_keyboardWillShowSubscription.remove();
		_keyboardWillHideSubscription.remove();
	}
	_keyboardWillShow(e) {
		this.setState({height: height - e.endCoordinates.height});
	}
	_keyboardWillHide(e) {
		this.setState({height: height});
	}
  render() {
    return (
		<View style={{height: this.state.height}}>
			<View style={{backgroundColor: 'blue', flex: 1}} />
			<View style={styles.fieldBox}>
				<TextInput
					style={styles.field}
					placeholder={'hello'}
					autoCapitalize={'none'}
					placeholderTextColor={'#afbccc'}
					blurOnSubmit={false}
					autoCorrect={false}
					ref={'textinput'}
					onChangeText={(data) => this.setState({message: data})} />
			</View>
		</View>
    );
  }
}

const styles = StyleSheet.create({
  field: {
		color:'black',
		fontWeight:'bold',
		fontSize:18,
		flex: 1,
		marginHorizontal: 10
	},
});

AppRegistry.registerComponent('awesomeproject', () => awesomeproject);

try the keyboardavoidingview

@vonovak I have tried using KeyboardAvoidingView. the problem is that the <View style={{backgroundColor: 'blue', flex: 1}} /> will be replaced with ScrollView component and I want the ScrollView container's height to be the one is adjusted, instead the whole main container to be shifted up. Thanks.

I meet this problem too. It is not related to the android version.

please，you solved this problem?

@MingYe-dz try adding android:windowSoftInputMode="adjustResize" to your Android Manifest android/app/src/main/AndroidManifest.xml

Did you get any solution to this?

Did you try KeyboardSpacer ?
https://github.com/Andr3wHur5t/react-native-keyboard-spacer

Seeing the same behavior; there's a visible gap between the keyboard and the view. @vonovak KeyboardAvoidingView does not work for me either. There are numerous open issues describing the same bugs I'm having (padding not being removed after keyboard is dismissed, etc.)
It's quite frustrating when I can't build my own when the out-of-the-box solution doesn't work. You guys are really tying my hands here.

Hi there! This issue is being closed because it has been inactive for a while. Maybe the issue has been fixed in a recent release, or perhaps it is not affecting a lot of people. Either way, we're automatically closing issues after a period of inactivity. Please do not take it personally!
If you think this issue should definitely remain open, please let us know. The following information is helpful when it comes to determining if the issue should be re-opened:

Does the issue still reproduce on the latest release candidate? Post a comment with the version you tested.
If so, is there any information missing from the bug report? Post a comment with all the information required by the issue template.
Is there a pull request that addresses this issue? Post a comment with the PR number so we can follow up.

If you would like to work on a patch to fix the issue, contributions are very welcome! Read through the contribution guide, and feel free to hop into #react-native if you need help planning your contribution.

kelvinaliyanto's advice isn't too far off. If that doesn't work, try replacing
android:windowSoftInputMode="adjustResize"
with
android:windowSoftInputMode="adjustPan"
in your android/app/src/main/Android.Manifest.xml.


[BUG] Missing link to wiki

Readme mentions a wiki but couldn't find one
Readme in question - https://github.com/ONEARMY/project-kamp-weather-station/blob/main/README.md

Resolved, see Readme.md

Thanks @Sani7 !


Lab and Lch values

Continuing discussion from #48.
Lab values are currently scaled, L by 100 and a and b by 128. Even though a and b are unbound, typical ranges are -128 to 128.
By my limited research so far, I have not seen scaled lab values used ( unlike rgb which is usually assumed in the range 0-1 ). For example Conversion formula's, Color Difference formula's and most research papers assume unscaled lab values (I am happy to be corrected on this).
Right now using the lab spaces implies unnecessary scaling when used practically. So I am proposing that the Lab values should be changed to be unscaled. Same for Lch values.
If the decision is to still scale the value, we should provide a new_from_unscaled and into_unscaled function, to get values in and out.

Color Difference formula's and most research papers assume unscaled lab values

I don't know why, but I didn't even think about that. The current scaling turns out to be even worse than I thought. This is probably one of those times when it's better to not be so clever.

Remember to change the default precision for Lab and Lch when scaling is removed if necessary.

With the blending additions, I am not sure whether storing the unscaled or scaled is better. Need to figure out if the blending or conversions are the more common operations.

Blending isn't implemented for Lab and Lch (so far only Rgb, Luma and Color + alpha wrappers). I'm not sure if it makes any sense at all for the other types.


Headers statement in config does not seem to be working

The headers statement in the config does not seem to work, I tried my own config first and when that did not work, I tried the example config which has
    headers:
      X-Dummy: my-test-header

but headers do simply not get sent.
Any ideas?

This test is passing so maybe you are missing something in your configuration? - https://github.com/prometheus-community/json_exporter/blob/75ae2b065ee16e4e0b2ac264427eec1eba9c39b2/cmd/main_test.go#L235-L269


Refactor local match inference

Here is the background for this PR: Unification is currently implemented in a way that is subtly wrong, and that will definitely break in the future when we extend it. The problem is that the unification algorithm assumes that there is only one LevelCtx which is valid for all constraints. What we should do instead is to make the LevelCtx part of each individual constraint. In order to do this we have to move the generation of the constraints in the inference of the local_match and local_comatch functions syntactically close to the call of unify(...) , and essentially under the context extension check_telescope, so that we can correctly annotate the constraints with the correct context in the future.

The diffs look horrible, unfortunately, but if I didn't make any mistakes then the reorganized code should behave exactly the same. There are no "logic changes", only the code has been shuffled around. I did inline the check_case function and I have permuted a match expression with a  params_inst.check_telescope( || ...) function. In the after  version you can observe that the let eqns = ... is close to the call to unify(), and that both are now under the context extension. In the next PR I will move the ctx.levels() call from the argument list of unify to the creation of the constraints in let eqns = ....

:+1:, though note that just making the LevelCtx part of each constraint won't be sufficient to distinguish different places in the source code as different places may have the same LevelCtx but these still cannot be treated as if they live under the same context.

👍, though note that just making the LevelCtx part of each constraint won't be sufficient to distinguish different places in the source code as different places may have the same LevelCtx but these still cannot be treated as if they live under the same context.

My main concern is not identifying a place where a constraint comes from, but computing subconstraints. E.g. if we have the constraint Gamma |- \x. e1 = \x. e2 then we might want to compute the subconstraint Gamma, x |- e1 = e2 where we have to extend the LevelCtx. We currently don't do that due to the restrictions in the paper, but we very likely might want to do something like that in the future.


Typo correction

Proposed changes in this pull request
phantom instead of phnatom on readme.md
@amir20 to review
phantom instead of phnatom

Thanks


fix(topology): adds wrapper to topology view for consistent border

Fixes: https://github.com/janus-idp/backstage-plugins/issues/325
Screenshots:
Dark Theme:

Light Theme:

Verified the changes
LGTM !


premium offline experience

to make this work offline really really well, here is what I want to happen (I havn't implemented this yet because i've been busy with other things)
create a lite registry server - this would be a back ground process which tailed all the npm metadata into a local database (leveldb, of course), not all modules are in the registry (some are on github) so it would need to cache those modules to resolve it. Luckily this is already implemented in npmd-cache. Basically this server would just use npmd-cache, and a local database.
The lite registry would expose a resolve service, (see npmd-resolve) it would give you a tree of modules to install, but it would only look in it's local cache/db so that it doesn't need to go over the network.
You could run this service on your laptop, just for your self, or you could run it on the local network for a team.

maybe you could even "follow" a user or package, and have the lite-registry eagerly cache new publishes from that user/package so you can always install the fresh stuff

Nice - It's highly likely that I'll get distracted by something shiny on the weekend, but if I do get some time to code for the betterment of JS developers everywhere I'll take a look at this at CampJS :)

ah, yeah a lite registry like this would be perfect for something like campjs


Add a "Scheduled" or "Usage" column to job listing that indicates it claiming capacity

Expected Behavior
It is currently unclear in the listing that a job claims capacity:
JOB       IMAGE              CREATED AT       STATUS
j-67hgk   nginx              1 hour ago       Running
j-6rpp6   hello-world        1 hour ago       Completed
j-7c75m   nginx              3 hours ago      Running
j-7pbkf   nginx              3 hours ago      Running
j-7w8ld   nginx              4 hours ago      Running
j-8bbhx   alkdsflkaskdlafl   1 hour ago       Starting: ImagePullFailed
j-9csj6   nginx              3 hours ago      Running
j-ccrh8   nginx              3 hours ago      Running
j-cf6rs   nginx              4 hours ago      Running
j-g8txt   nginx              3 hours ago      Running
j-hchml   nginx              4 hours ago      Running
j-jbnxx   nginx              43 minutes ago   Waiting: NotEnoughResources
j-ksx4h   nginx              1 hour ago       Running
j-l5q8g   nginx              3 hours ago      Running
j-lpx2n   nginx              4 hours ago      Running
j-lqm8z   nginx              4 hours ago      Running
j-lxkcq   nginx              4 hours ago      Running
j-mp78v   nginx              4 hours ago      Running
j-n5ksb   nginx              3 hours ago      Running
j-n9x74   nginx              3 hours ago      Running
j-nbb8l   nginx              4 hours ago      Running
j-ng9zs   nginx              3 hours ago      Running
j-p79dv   hello-world        2 hours ago      Stopped
j-p9xzq   nginx              4 hours ago      Running
j-pf7kt   nginx              4 hours ago      Running
j-pkfrs   nginx              4 hours ago      Running
j-qf5xv   nginx              4 hours ago      Running
j-r2pjj   nginx              3 hours ago      Running
j-rcmx9   nginx              43 minutes ago   Waiting: NotEnoughResources
j-tspfp   alksdlkfklasdf     27 minutes ago   Waiting: NotEnoughResources
j-vbj52   asldkfklasd        2 hours ago      Starting: ImagePullFailed
j-wk9kb   nginx              4 hours ago      Running
j-xz7c9   nginx              4 hours ago      Running
j-zh8rs   alksdlkfklasdf     26 minutes ago   Starting: ImagePullFailed

Actual Behavior
Add a column that indicates what jobs are claiming resources by showing the resources in this column.

Done in #343


Better UI to show how to run the down migration

Requests:

Noah https://serverless-stack.slack.com/archives/C01JG3B20RY/p1645616608490659

We are transition this repo to https://github.com/sst/v2 and are closing this issue for now. If it is still relevant please reopen it there - sorry for the trouble


Large tables performance

Package
filament/filament
Package Version
V3+
Laravel Version
N/A
Livewire Version
No response
PHP Version
PHP 8.2+
Problem description
When using bigger tables and using the table size of 25+ for example 50 or All (Yeah All)
Than the loading times are huge.
The issue is also in the demo: https://demo.filamentphp.com/shop/orders
Expected behavior
Since we have used other products and Vue.js / Laravel Backpack loading of tables was really quick with these kinds of datasets.
Would be awesome if we can see some huge improvements here.
The issue is most notable with 50+ items in a CRUD and sure All will be very heavy.
But a page size of 75 / 100 / 200 will sometimes with big datasets be a usecase.
Steps to reproduce
If you select All here at the bottom 1000 records will be loaded. You have to wait a bit, and after that's done and you refresh.
The loading of the table will take about 14-18 seconds.
Does not matter how powerful the server is, and also your browser page may crash...
Reproduction repository
https://github.com/filamentphp/filament
Relevant log output
No response

It does not matter what project we link its already present in the base of Filament: https://github.com/filamentphp/filament

i have same problem 😞

@zepfietje Can you maybe reopen this issue? Since its closed right after creation?

Thanks, talked about this already with our devs but no idea/ solution yet. Hope someone will be able to fix this / improve the speed.

I’m currently restricting the max results per page to 50 because of this. There are certain types if apps where viewing 100-200 records at a time would be desirable though.

I agree for certain apps it is important to view more than 100 records at once.
Running into this myself for picci.app too.

But you think it has to do with livewire right?
And not Filament itself?
Maybe cross post in Livewire git as well?

Not sure if it really classifies as a Livewire bug though.
Posting in Livewire repo only really adds value when we create a failing test PR for bugs to be honest.
Maybe this is something @danharrin could discuss with Caleb though?

Hope someone will find the issue / fix it. Coming from Backpack loading 500+ records was never an issue😉 Glad its not on our side only. But also in the base.

Hey @alecritson, I remember you mentioning wrangling with Livewire performance when creating Lunar’s datatables package. Maybe you have some insights you could share?

Makes sense with ecom you wil have many tables with order/products etc. And filters will also be essential. Altough filters speed is okay in smaller tables, don't know how that will be with 200 or even 500 lines and lets say 30k of records.

Alright, I've conducted some debugging to identify the root cause. I will share my findings to determine if they can contribute to performance improvement or not. I cloned the demo project, seeded it with data, and loaded a page with 150 records on each page on the Order page.
Upon investigation, it became apparent that the issue arises when Livewire is rendering the Order Page component within the update  function.
It takes more than 10 seconds to fully render the component and generates approximately 3MB of data!
Then I hooked into the composing event and saw that to render Order page it is rendering more that 5000 views!
I saw component names like this:
filament-tables::columns.text-column
filament-tables::components.cell
filament-tables::components.actions
filament-tables::components.actions.cell
filament-tables::components.cell
filament-tables::components.row
filament-tables::components.selection.checkbox
filament-tables::components.selection.cell
filament-tables::components.cell
filament-tables::components.columns.column

There are over 4000 filament-tables* rendering events, indicating a rendering process for each cell, row, and column.
you can review the entire rendering log that I have attached here: compose_events-2023-11-10.log
I believe there might be some room for optimization in rendering tables, so I've decided to share this information with you for further investigation and potential improvement.
In conclusion, there is a significant number of rendering processes, and the HTML output appears to be very large, contributing to an extended rendering time for pages with 150 records.

@zepfietje Is this than something you guys can fix in Filament or is this more related to Livewire?
4000+ Filament events is a lot, like we saw also with debugging.

There are over 4000 filament-tables* rendering events, indicating a rendering process for each cell, row, and column.

I don't think that should be an issue. The views should be cached anyway.

It takes more than 10 seconds to fully render the component and generates approximately 3MB of data!

3MB sounds like a lot of data and a huge table! I just compared this to the Filament Demo's "Brands" resource. With 400 (!) entries, it still sends only 127kB data and takes 2.3s to load for me.
It's hard to judge your "benchmark" without knowing your setup. If your table really generates 3MB of data, there is not much Filament can do.

Just tested this again with the OrderResource. Demo with up-to-date Filament. 400 Orders load in 3.6s and produce 195kB in data (Macbook Air M1).

Just tested this again with the OrderResource. Demo with up-to-date Filament. 400 Orders load in 3.6s and produce 195kB in data (Macbook Air M1).

That's still a long wait time right for 400 lines... I mean, i get Filament has some overhead and cannot compare that to "Backpack" that we used to use, but over there 500+ Lines loaded within <1s

Hello Dennis
I really dont know very much about internals of Filament and I just wanted to show that there is 4000 events and it might be problem or not. ofcource It need more investigation.
about taking 10 sec: it takes 10sec on my system with debug enable and i think it take half of it without it. I think anything above 2 sec or even 1 sec for 150 records is high.
and about size: in your demo site it produce 1.15MB for 50 records:

also in demo site it produce 20MB in 9 sec for 1000 records

The strange thing is that the 20MB is compressed into 162KB :))) you can see it in Transferred col.

It's compressed because the HTML to render table cells is repetitive

It's compressed because the HTML to render table cells is repetitive

Yeas but strange part for me was the compression ratio which is over 99%, it's became 125 times smaller (wow)
It seems everything in it is repetitive, If my calculation was not wrong!

Yes, Tailwind is quite repetitive 😉

Yes that's amazing and its impact is huge.
Okay, I want to delve deeper into the extensive response to see what I can find. Regardless of whether there is an easy way for optimization, I'll document it here. Perhaps I or someone else can come up with ideas and create a pull request for it, or share insights with our respected developers, Dan and Dennis.
I am currently examining a 150-record response weighing 3MB. After decoding it, I noticed some code snippets like this:
<!--[if ENDBLOCK]><![endif]-->
<!--[if BLOCK]><![endif]-->

"It relates to Livewire Morphing . Could we consider making an exception for tables and disabling it for them? You won't believe that there were more than 12,000 of each of them! By removing them, the response size reduced by about 22%
Afterward, I discovered some repeated classes in the output HTML that somehow might exceptionally could be compact for Tables or maybe not!
155 input with this classes:
fi-checkbox-input rounded border-none bg-white shadow-sm ring-1 transition duration-75 checked:ring-0 focus:ring-2 focus:ring-offset-0 disabled:pointer-events-none disabled:bg-gray-50 disabled:text-gray-50 disabled:checked:bg-current disabled:checked:text-gray-400 dark:bg-white/5 dark:disabled:bg-transparent dark:disabled:checked:bg-gray-600 text-primary-600 ring-gray-950/10 focus:ring-primary-600 checked:focus:ring-primary-500/50 dark:text-primary-500 dark:ring-white/20 dark:checked:bg-primary-500 dark:focus:ring-primary-500 dark:checked:focus:ring-primary-400/50 dark:disabled:ring-white/10
850 rows with this class
fi-ta-text-item inline-flex items-center gap-1.5 text-sm text-gray-950 dark:text-white
1000+:
flex w-full disabled:pointer-events-none justify-start text-start
1300+:
fi-ta-cell p-0 first-of-type:ps-1 last-of-type:pe-1 sm:first-of-type:ps-3 sm:last-of-type:pe-3
1000+:
flex w-full disabled:pointer-events-none justify-start text-start
150:
x-bind:class="{
            'hidden': false && isGroupCollapsed(''),
            'bg-gray-50 dark:bg-white/5': isRecordSelected('300'),
            '[&>*:first-child]:relative [&>*:first-child]:before:absolute [&>*:first-child]:before:start-0 [&>*:first-child]:before:inset-y-0 [&>*:first-child]:before:w-0.5 [&>*:first-child]:before:bg-primary-600 [&>*:first-child]:dark:before:bg-primary-500': isRecordSelected('300'),

and there is more ...

I don't think we should focus on the response size, since it's compressed anyway. It would be more interesting to see if we could improve rendering time by reducing classes etc. I don't think we can turn off Livewire internals for a specific part of the app.

We have the same issue, tables with 500+ data always timed-out in nginx (504 gateway timeout), other pages with less data are quick.

TLDR; If you're using Laravel Debug Bar, try disabling it. (use this env variable DEBUGBAR_ENABLED=false)
For what is worth. I was having the same performance issue while attempting to render a table with barely 25 items per page but with more than 20 columns in it.
I noticed that the page would perform slightly better the less columns I added to the table. Then I started looking at the browser's profiler just to realize that the problem wasn't Filament or Livewire, per se... It was in fact caused by Larevel's debug bar.
Like someone else pointed out in earlier comments, many views get rendered in the back-end side when building the table. All of those rendering events are captured by the debug bar which causes the performance issue when it builds its UI.

It also happens in Production and in the filament demo site.  I'll update it to make it clearer.

Understand, thought you were reffering to that the issue is only there when the debugbar is on😅

@danharrin @zepfietje I suggest deferring the loading of additional rows in a table until the user scrolls to the bottom of the viewport. This approach would enhance performance and user experience by loading more records only when necessary. I'm not sure if this has been considered as a solution, but it could be beneficial. Once the user reaches the bottom, more records can be loaded dynamically. However, I'm uncertain how this would impact existing pagination options. Perhaps this could be introduced as a separate feature for tables.

A similar solution instead of relying on the scroll position.

Would that really make a difference? Because I think DOM diffing is the main cause of performance issues?

Would that really make a difference? Because I think DOM diffing is the main cause of performance issues?

I understand the concern about DOM diffing potentially being the main cause of performance issues. However, I'd like to clarify the benefits of deferring the loading of rows until the user scrolls to the bottom (infinite scrolling or lazy loading):

Reduced Initial Load Time: By loading only a small subset of rows initially, we significantly reduce the amount of data rendered at once. This should improve the initial load time and make the page more responsive.
Incremental Data Loading: As users scroll, additional rows are loaded in smaller batches. This can help manage memory usage and render time more effectively, reducing the risk of browser crashes with large datasets.
Improved User Experience: Users cannot currently view hundreds of records at once in the current viewport (only around 10-18 at once at 100% zoom on a 1920x1080 resolution), so there is no reason to load 25/50/100+ or all records at once. Loading data incrementally as they scroll can provide a smoother and faster experience. This ensures that the visible records load quickly without overwhelming the browser.

Regarding the impact on existing pagination options, infinite scrolling can be implemented alongside traditional pagination seamlessly. For example, when users choose to view 25/50/100+ records per page or all records, my suggestion can take action. This allows both methods to coexist effectively, enhancing performance while maintaining user preference for pagination.
I understand that integrating this feature might require significant changes and careful consideration of its impact on other functionalities. It may also be beneficial to explore combining this approach with Alpine.js or other frontend optimizations as mentioned.
Would you be open to exploring this further as a potential enhancement? I’m happy to collaborate on this and help test any solutions we come up with.

Based on the performance testing I conducted, I'd like to address some additional points and suggestions:
Metrics Summary
10 records per page
Load timings (ms)

Event
Start
Duration
End

Redirect
0
0
0

DNS
0
0
0

Connect
0
0
0

Request
3
367
370

Response
370
1
371

DOM
371
135
506

Parse
371
37
408

Execute Scripts
408
13
421

Content loaded
421
20
441

Sub Resources
441
65
506

Load event
506
1
507

Total

507

25 records per page
Load timings (ms)

Event
Start
Duration
End

Redirect
0
0
0

DNS
1
0
1

Connect
1
0
1

Request
8
563
571

Response
571
0
571

DOM
571
162
733

Parse
571
67
638

Execute Scripts
638
1
639

Content loaded
639
15
654

Sub Resources
654
79
733

Load event
733
1
734

Total

734

50 records per page
Load timings (ms)

Event
Start
Duration
End

Redirect
0
0
0

DNS
1
0
1

Connect
1
0
1

Request
3
930
933

Response
933
0
933

DOM
933
192
1125

Parse
933
61
994

Execute Scripts
994
0
994

Content loaded
994
16
1010

Sub Resources
1010
115
1125

Load event
1125
1
1126

Total

1126

100 records per page
Load timings (ms)

Event
Start
Duration
End

Redirect
0
0
0

DNS
0
0
0

Connect
0
0
0

Request
3
1258
1261

Response
1261
0
1261

DOM
1261
252
1513

Parse
1261
93
1354

Execute Scripts
1354
0
1354

Content loaded
1354
19
1373

Sub Resources
1373
140
1513

Load event
1513
1
1514

Total

1514

Key Findings

Request Duration:

The time taken for the request to complete increases significantly with the number of records, suggesting that server-side processing and data retrieval times grow proportionally with the number of records.

DOM Processing:

The DOM processing time also increases with the number of records, indicating that rendering the additional records in the browser takes more time as the number of records grows.

Sub Resource Loading:

The time taken to load sub resources (like CSS, JavaScript, images) also increases, though to a lesser extent.

Relevance to DOM Diffing and Alpine.js Suggestion
While the primary bottleneck appears to be server-side processing, the increase in DOM processing time suggests that the efficiency of DOM updates (including DOM diffing) is also a factor.
Regarding DOM Diffing:

DOM diffing helps optimize updates to the DOM by only re-rendering elements that have changed. However, the increased DOM processing time with more records indicates that the sheer volume of data being rendered might still overwhelm this optimization.

Considering Alpine.js:

Alpine.js could be used to render cells in a JavaScript loop from JSON data instead of using Blade. This approach could improve client-side performance by optimizing DOM updates and reducing the overhead of server-side Blade rendering.
However, as mentioned by Dan, this would require a significant refactor of the existing codebase and could limit the flexibility provided by Blade, such as using Blade directives and custom server-side logic directly within the view.

Recommendations

Optimize Server-Side Processing:

Review and optimize database queries and server-side logic to handle large datasets more efficiently.

Implement Lazy Loading/Infinite Scrolling:

Loading data incrementally can significantly reduce the initial load time and improve overall performance by distributing the load over time as the user scrolls. This can be implemented alongside traditional pagination, activating when users choose to view larger numbers of records per page.

Efficient Client-Side Rendering:

Explore ways to reduce the complexity of the DOM generated for large datasets, such as using virtual DOM techniques or frontend frameworks like Alpine.js.

Leverage Caching:

Implement caching strategies both on the server-side and client-side to reduce redundant processing and data retrieval.

Conclusion
The suggestion to use Alpine.js to render cells from JSON data is a valid long-term solution to improve rendering performance. However, as Dan mentioned, it would require careful consideration of the trade-offs, including the significant refactor required and the loss of Blade's flexibility. Combining this approach with server-side optimizations and lazy loading could provide a balanced solution to improve performance.

Scroll loading and alpinejs seems increase the complexity.
why not consider the new wire:replace https://github.com/livewire/livewire/pull/8403

You guys as @wychoong mentions (Livewire v3.5.0) Adding wire:replace by @samlev in https://github.com/livewire/livewire/pull/8403
may solve this issue

You guys as @wychoong mentions (Livewire v3.5.0) Adding wire:replace by @samlev in livewire/livewire#8403 may solve this issue

Are there any plans to add this? I see it's got the label low priority, but I think this is for a lot of users a big issue.

You guys as @wychoong mentions (Livewire v3.5.0) Adding wire:replace by @samlev in livewire/livewire#8403 may solve this issue

Are there any plans to add this? I see it's got the label low priority, but I think this is for a lot of users a big issue.

It’s open source project and filament is provided for free. For “a lot of users” and “big issue”, there seems to be not enough interest to provide a repo with huge data slow performance table, benchmarking setup, let alone funding the issue or submit a PR.
Regarding my suggestion of wire:replace, one should understand that it’s a relatively new api in LW and it might be optional enable in v3 or only available in v4

You guys as @wychoong mentions (Livewire v3.5.0) Adding wire:replace by @samlev in livewire/livewire#8403 may solve this issue

Are there any plans to add this? I see it's got the label low priority, but I think this is for a lot of users a big issue.

It’s open source project and filament is provided for free. For “a lot of users” and “big issue”, there seems to be not enough interest to provide a repo with huge data slow performance table, benchmarking setup, let alone funding the issue or submit a PR.
Regarding my suggestion of wire:replace, one should understand that it’s a relatively new api in LW and it might be optional enable in v3 or only available in v4

Thanks for the explanation, I thought I would be a small change.
What you say about provide a demo, it's also a problem on https://demo.filamentphp.com/blog/categories, if you set the pagination to all (only 400 records), the page takes 4 seconds to load.

The amount of HTML that needs to be loaded for those 400 records is substantial enough to slow the page down. In most projects, I would remove the "All" option and limit it to 50 per page maximum.

To be honest, it is quite slow even with 50 entries.  400-500ms is pretty much for a table with a few entries.
The problem does not only apply to tables, but also more or less runs through the entire filament.
For example, if I have a form with a repeater field, the request time also increases unreasonably with each entry

To be honest, it is quite slow even with 50 entries. 400-500ms is pretty much for a table with a few entries.
The problem does not only apply to tables, but also more or less runs through the entire filament.
For example, if I have a form with a repeater field, the request time also increases unreasonably with each entry

Totally agree on this. We experience the same symptoms.

The more items you add to a repeater field, the more HTML is required to render that repeater

I am aware of that. But the difference with a few entries is quickly in the region of several 100ms.
If you were to render this with a few blade components, it would be much much faster.
Don't get me wrong. I really like filament, but I think performance is a problem that you can't just dismiss.
You very quickly reach request times of over 1s and you almost can't expect customers to accept that.

We're working on optimisations involving the HTML size and partial rendering for version 4, but there are some limitations that cannot be overcome without compromising in other areas

html is over 14mb. Is this expected?

html is over 14mb. Is this expected? This is loading 200 rows on a table most columns are hidden.

Even if the columns are hidden, I believe they're still calculated and returned down, no?  Maybe I'm wrong on that one.

They are not

@danharrin do you have any pointers on resolving this issue? I am willing to dedicate the time to develop a solution as this is a major issue.

The real solution for rendering tables faster is probably using Alpine.js to render cells in a JS loop from JSON data instead of with Blade, but its a huge breaking change and would also limit the stuff you could do in a cell as you wouldn't have full access to Blade

Are there any internal discussions about taking this approach? Personally I'd be fine with losing some functionality in return for client-side datatable performance. Might be worth doing a poll? I've never seen anyone complain about Nova or Backpack's datatables (or any of the popular Vue or React datatabes packages).

Personally I'd be fine with losing some functionality in return for client-side datatable performance

This is probably a huge breaking change. For 3 versions we fully rely on Livewire for all the logic and users can define their own conditions, interactions between actions and the components, etc.

Proposed path towards client side support without breaking existing tables:
Fully client side tables.

[ ] Add ->isClientSide() to existing tables. Renders a table view that makes API requests to get the table schema and data.
[ ] Build read only cells renderable with js/alpine based on filament table schema.
[ ] Investigation: Editable cells need a protocol to interact with the filament/livewire table to CRUD cell values.
[ ] Build editable cells using defined protocol.

And who is funding the maintenance of this new table builder 😁 I don't think it's practical to maintain both for us

If funding is a problem, what about making it a third party plugin instead?

It's just a lot of work to maintain, whoever does it, I am going to wait and see how much improvement we can make from the changes in v4

If funding is a problem, what about making it a third party plugin instead
Well, go ahead 😅

I just spent a considerable amount of time debugging and profiling the table performance issue using the orders list page in the demo and showing 100 records at once.
It seems the issue isn't related to Livewire at all, as initial page load times are almost identical to subsequent requests when visiting the next page of table records (~800ms).
A couple of potential solutions I've tried:
Removing CSS classes
Removing CSS classes from all components rendered in the table didn't make any significant difference in loading times.
Replacing Blade components
Replacing Blade components like table rows with plain HTML elements made somewhat of a difference, but not enough to solve the performance issue.
Simplifying Blade components
The text column Blade component currently contains a lot of nested HTML elements which aren't necessary when displaying simple text values. Removing these elements resulted in a slightly noticeable performance improvement (~800ms down to ~500ms).
Unfortunately it doesn't seem like any of these approaches can fully solve the issue.

@zepfietje I find it hard to believe that loading 100 rows only takes 800ms. Is this version of the table you're using very light? How many columns?

Edit: Also, of course loading all rows for a table would take a long time anyway. Filament couldn't really do anything about that other than using infinite scroll to reduce the amount of html being rendered at once. You wouldn’t need to incrementally load records from the database; you could simply use the existing dataset while loading the HTML in chunks.

If you think this would be a good idea I can try to implement something over the weekend and get back to you.

From my experience in working with the jQuery Datatables package a lot I can tell you that it easily becomes unworkable when you're loading 20000 rows or more; the browser either chokes or your PHP process runs out of memory (I'd rather not assign more than 1GB of memory to my PHP processes).
The main benefit of Datatables is that it blazingly fast once the data has been loaded 😉

Replacing Blade components
Replacing Blade components like table rows with plain HTML elements made somewhat of a difference, but not enough to solve the performance issue.
Simplifying Blade components
The text column Blade component currently contains a lot of nested HTML elements which aren't necessary when displaying simple text values. Removing these elements resulted in a slightly noticeable performance improvement (~800ms down to ~500ms).

I also dug into a profiling session and the rendering of each table cell adds up to the total response time. Isn't it possible to somehow cache the table row before rendering, and then render each row from cache? Table actions with icons or nested components. It would at least reduce the IO-operations on the disk.
I also noticed that large tables are killing responses on AWS Lambda, using Laravel Vapor. Resulting in a 502 which probably means you ran into a Lambda quotation.

Isn't it possible to somehow cache the table row before rendering

I don't think that works, because you can use fn ($record) =>  for most of the methods on columns, so each row and column must be reevaluated. We can't simple save the HTML and replace stuff.

I do have an idea about how we can refactor tables to render using an Alpine loop without breaking anything, but I haven't tested it and it would be a lot of work. And I don't know if it would necessarily provide much of a performance improvement (it would push some work from the backend into the frontend). Could be worth attempting though.

I do have an idea about how we can refactor tables to render using an Alpine loop without breaking anything, but I haven't tested it and it would be a lot of work. And I don't know if it would necessarily provide much of a performance improvement (it would push some work from the backend into the frontend). Could be worth attempting though.

Might be worth experimenting with, though from my experience using Alpine on large parts of a page results in performance issues too (make sure to check all browsers as this was especially bad in Safari IIRC).

The solution is to move to Svelte :)

Wondering if we've considered what Philos discussed in this video?
I know Filament renders thousands of blade components each time something changes in the table. One of the big performance issues he discussed is changing that (which is essentially HTML) into livewire components, as livewire is clever enough just to change the livewire component that changed, as opposed to doing diffing with all the document.
Is it something you think it's worth exploring?

The answer is no, in our situation it would not help. The overhead of booting so many more Livewire components would be more than we have now, and there would be a significant delay in communicating updates through events once something changes.
Once you start breaking up Livewire components, you cannot render them all at once again and just choose which ones need to change. Livewire is just assuming each component is an island that doesn't know anything about the others.

i use wamp server php 8.3  is there any config can speed filament table performance ?

Same performance issues here. Large table are a pain and I have tried everything (without success) to improve the performances.
A Livewire only approach makes the development super easy and reactive but for medium-large applications there is a big gap compared to JS DataTable performances.

@nocodelab For performance writing custom widgets seems like the best available option unfortunately.
you can reuse filament components in your widget code and write alinejs to handle everything in JavaScript. Much more performant.
yes, I know rewriting something that already exists is a pain but no other solution atm.

Same issue here. On a table with just 8 rows I get 250ms loading time even with filament cache enabled.
According to Sentry 100ms is being spent on rendering filament-tables::components.actions component (~10ms per table action)
I can confirm that because once I remove all table actions, loading is dropped to ~140ms.
I guess nesting views takes considerable time to render even when cached so this does not scale well.
I hope Filament 4 will try to solve this, otherwise it has no feature. At least for large-scale apps 😢

This has improved in version 4. I do not see any way to port those fixes to v3 without introducing breaking changes, so I am closing this thread. I hope you like the improvements when that new version is released. And no, as of today, I don't have an ETA for the release.


Chrome browser by default with Brackets.io

Chrome browser by default with Brackets.io
I just installed Brackets and I like a lot, but I have a little problem, maybe you can help me any.
When using the option "Live Preview" Chrome browser opens with the live document, but not shown me the extensions and bookmarks that I have installed on my browser.
This is when the window opens Brackets Live Preview:
http://prntscr.com/63d8m2
That is when I open myself (with extensions and bookmarks, but without automatic update), which is how I would like to have:
http://prntscr.com/63d8ru
How I can do so that when you open Chrome do as what I have configured?
Thank You.
Note: Use translator, sorry.

@Deorro I think I answered your question in the comments of the other issue that you commented on.
Since I made those other comments regarding Live Preview, we have added a new Live Preview MultiBrowser option that may behave more like you want, so give that a try.

You can spend a video to enable this feature.
What I have understood is that this configuration can see the live preview in the current window you have open.
Can you tell me how to change livedev.multibrowser to true, where income?

@Deorro In the How to Use Brackets document, look in the Preferences section to see how to manually set preferences.

Closing since I think the question has been answered.  The current behavior of Live Preview (launching in a separate Chrome profile) is unavoidable, per #8653; the future Multi-Browser Live Preview does not have this issue; and the instructions linked above explain how to enable Multi-Browser Live Preview.


Add rejectOnNotFound to findOne/findFirst

Problem
Instead of returning a nullable result when calling model.findOne, accept a new argument (rejectOnEmpty) that makes the returning type non-nullable and throws an error when the row was not found
Suggested solution
const person = await prisma.person.findOne({ where: { id } })

if (person) {
  // do stuff
}

Will be:
const person = await prisma.person.findOne({ where: { id }, rejectOnEmpty: true })

// do stuff

rejectOnEmpty will accept either boolean or an Error so the thrown error can be customized:
const person = await prisma.person.findOne({
  where: { id },
  rejectOnEmpty: new MyCustomNotFoundError(`person with id=${id} was not found`)
})

Additional context

This API is stolen from Sequelize

@timsuchanek I see rejectOnNotFound is added in 2.15.0, but only for one-off invocations (vs setting globally during prisma client instantiation) and without the ability the customize the error, as described above.
Will those be coming in the next release? 🤔

@timsuchanek I see rejectOnNotFound is added in 2.15.0, but only for one-off invocations (vs setting globally during prisma client instantiation) and without the ability the customize the error, as described above.
Will those be coming in the next release? 🤔

Hey @flybayer, they should both possible in 2.15 for findUnique and findFirst.
Passing in just a rejectOnEmpty: new Error() was removed in favour of  rejectOnEmpty: () => new Error()
// Sets the default behavior of findUnique
PrismaClient({rejectOnEmpty: true})
PrismaClient({
	rejectOnEmpty: () => new MyCustomNotFoundError(`person with id=${id} was not found`)
})
PrismaClient({
	rejectOnEmpty: {
		findUnique: true
	}
})
// Sets the default behavior of findUnique but only on specified models
PrismaClient({
	rejectOnEmpty: {
		findUnique: {
			  User:  () => new MyCustomNotFoundError(`person with id=${id} was not found`),
			  Post: true
		},
		findFirst: true
	}
})

// Allows you to set the behaviour on a per request basis
await prisma.person.findUnique({ where: { id }, rejectOnEmpty: true })
await prisma.person.findUnique({
	where: { id },
	rejectOnEmpty:  () => new MyCustomNotFoundError(`person with id=${id} was not found`)
})

Hey @flybayer, they should both possible in 2.15 for findUnique and findFirst.
Passing in just a rejectOnEmpty: new Error() was removed in favour of  rejectOnEmpty: () => new Error()
// Sets the default behavior of findUnique
PrismaClient({rejectOnEmpty: true})
PrismaClient({
	rejectOnEmpty: () => new MyCustomNotFoundError(`person with id=${id} was not found`)
})
PrismaClient({
	rejectOnEmpty: {
		findUnique: true
	}
})
// Sets the default behavior of findUnique but only on specified models
PrismaClient({
	rejectOnEmpty: {
		findUnique: {
			  User:  () => new MyCustomNotFoundError(`person with id=${id} was not found`),
			  Post: true
		},
		findFirst: true
	}
})

// Allows you to set the behaviour on a per request basis
await prisma.person.findUnique({ where: { id }, rejectOnEmpty: true })
await prisma.person.findUnique({
	where: { id },
	rejectOnEmpty:  () => new MyCustomNotFoundError(`person with id=${id} was not found`)
})

Ah, thanks!
Bummer it doesn't affect the result type yet. Waiting until that is fixed to use it.

Ah, thanks!
Bummer it doesn't affect the result type yet. Waiting until that is fixed to use it.

For those looking for ^^ in 2023 it's been deprecated and replaced


This page doesn't describe how to write data to datastores to solve the latency issues it discusses

This doc page references writing to datastores, but mysteriously leaves out code examples to do so and instead talks about writing to the outputs and logs directory that can cause latency issues for large datasets: https://docs.microsoft.com/en-us/azure/machine-learning/how-to-save-write-experiment-files
It links to this page that also doesn't describe how to write to datastores: https://docs.microsoft.com/en-us/azure/machine-learning/how-to-access-data
The outputs and logs directories are not great solutions for storing the results of GPU model training because it's hard to find how to sync these to permanent storage accounts, along with the latency issues described in the docs.
A solution for writing many checkpoint files to a local disk (fast) so that tensorboard profiling can be used, and then moving these checkpoints and model outputs to a Datastore would be really valuable.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 62a2a1c7-808f-e969-932c-9edc399071ee
Version Independent ID: 16f092af-03f0-852f-6b30-9b8267cdf1de
Content: Where to save & write experiment files - Azure Machine Learning
Content Source: articles/machine-learning/how-to-save-write-experiment-files.md
Service: machine-learning
Sub-service: core
GitHub Login: @rastala
Microsoft Alias: roastala

Hello @rbavery
Thank you for your feedback! We will review and update as appropriate.

@rbavery Some examples to use the ScriptRunConfig to write files and data is available in this notebook. @rastala FYI for feedback and doc-update.

#assign:@nibaccam

I think AzureML Pipelines are the answer here as well as these docs for the OutputFileDatasetConfig, which allows saving files at the end of a training job (or any script):https://docs.microsoft.com/en-us/python/api/azureml-core/azureml.data.output_dataset_config.outputfiledatasetconfig?preserve-view=true&view=azure-ml-py
I think the docs for ScriptRunConfig should reference Azure Pipelines and make it clear that this is a more recent and more fully featured way of interacting with AzureML. I'm not sure what the advantage is of ScriptRunConfig over Azure Pipelines. There should probably be a doc page explaining that, since it seems like Pipelines does a lot more.

This is an example of how I am using the PythonStep class that enables saving to an outputFileDatasetConfig: https://github.com/TZCRC/AIAIA/blob/main/aiaia_detector/azureml_train_detector.py
The ScriptRunConfig doesn't seem to support this but it's not intuitive why the PythonStep class would support this but ScriptRunConfig does not/there's no mention of this capability in the ScriptRunConfig docs.

@MayMSFT can you provide guidance here?

@rbavery thank you for pointing this out. The sample notebook that @RohitMungi-MSFT provided in his comment demonstrates that OutputFileDatasetConfig can be used with ScriptRunConfig.
I've created an internal task item for the documentation to be updated--particularly to remove the unhelpful reference link that points to datastores and instead point to guidance on how to use OutputFileDatasetConfig with ScriptRunConfig.
We will now proceed to close this thread #please-close


Build FeedbackDialog and FeedbackDialogModel into XH

It seems weird that these are provided by Hoist, and the underlying service they rely on is provided by Hoist, but we don't wire them into XH by default.
Seems like if we think its a core part of the framework, there ought to be a one-line way for any app to display these.  e.g. XH.showFeedbackDialog().

As well as a standardized feedbackButton and built-in appBar support pls!


⚠ Processors has degraded performance

In e28d482, Processors ($STATUS_URL) experienced degraded performance:

HTTP code: 200
Response time: 39 ms

Resolved: Processors performance has improved in ed32563.


Run the master without kubelet

Is it possible to run the master without the kubelet? I'd like to do k3s-in-k3s (eg. one cluster manages the other master), but I can't see to figure out how to disable the kubelet on the master nodes.

I haven't tested it myself, but there's a hidden option for that:
https://github.com/rancher/k3s/blob/master/pkg/cli/cmds/server.go#L302

@brandond that does the job! Memory usage is still 375 Mb on a 1-node master (without any configuration) :open_mouth:
Any way to cut this down (ideally to <100Mb RAM)?

If you can find a way to run the entire Kubernetes control plane plus datastore in less than 100mb, I think that would be newsworthy ;)

Agreed, anyways, I've started a separate issue for this: #1944. Target is 150-200MB.

I have personally tried running it with as little as 256M, but it gets OOM killed pretty quick. at 384M it runs but doesn't perform well. As far as I can tell it should really have at least 512M.

Permalink for the curious that stumble upon this page: https://github.com/k3s-io/k3s/blob/5a81fdbdc5d02527c886af9a31324a73573d1909/pkg/cli/cmds/server.go#L302

I haven't tested it myself, but there's a hidden option for that:
https://github.com/rancher/k3s/blob/master/pkg/cli/cmds/server.go#L302

link is obsolete, which line are you actually referring to?

Because we don't technically support it. Many operators (system upgrade controller, etcd controller, and others) assume that the control-plane hosts will have nodes that can be used to track operations.

Much thanks @FabianKramm at Loft.sh for keeping the flag functional, ref: https://github.com/k3s-io/k3s/issues/4873 and https://github.com/k3s-io/k3s/issues/5118
It was definitely broken for a few versions, but has been fixed since this issue was originally created. When it was initially fixed, I don't think that the fix was propagated back to older releases.

Well I was the one that fixed both of those, but yes thanks to everyone for reporting issues.

I definitely misremembered that. Much thanks to you @brandond. Sorry about that.


support @disable tag

When applied to a feature/scenario, the feature/scenario will be disabled

delivered with @ignore


feat: add support for Java docs

Closes #405

I've updated / refreshed the generated docs based on feedback and to fix some previous bugs. Here is a gist with some example Java docs: https://gist.github.com/Chriscbr/95ffac0eb4fc3531765e14c87924b965
Known issues that are not addressed in this PR:

not all referenced types have corresponding imports in snippets (existing issue)
full type names shown everywhere ("java.lang.String") might be unnecessarily verbose
invocation snippets for constructs and classes use the builder syntax, but the snippets assume that all parameters in the create() method are required (even though the method may be overloaded, i.e. some fields are optional)
[object Object] appears in some places (existing issue)
code snippets for methods just show their signatures (public foo()), which is inconsistent with all other snippets showing usage-like examples (existing issue)


release review period link

Signed-off-by: Kevin Lumbard<EMAIL_ADDRESS>This is time sensitive so please review and merge before July 1st

LGTM!
Signed-Of-By: Armstrong<EMAIL_ADDRESS>


Update and redeploy the app  | My Git Repository

Update and redeploy the app
How do i update the website with latest on git repository ?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: a825f031-250e-034d-e552-4b0d976d88b1
Version Independent ID: 626d7451-0b8e-051e-e512-0bf90be319a0
Content: Create a static HTML web app in Azure
Content Source: articles/app-service/app-service-web-get-started-html.md
Service: app-service-web
GitHub Login: @cephalin
Microsoft Alias: cephalin

@ankushbindlish2 Thanks for the feedback! We are currently investigating and will update you shortly.

@ankushbindlish2 Are you able to use the command git push azure master to push from your local git to your web app?

@MicrosoftDocs/azure-cxp-triage Can you track this as a doc-bug for now, but include the needs-more-info label?

@ankushbindlish2 Since we have not heard back from you we will now proceed to close this thread. If there are further questions regarding this matter, please reopen it and we will gladly continue the discussion.


feat: skipping fields that don't exist in the base_model annotations

This runs on later python versions, not sure why this fails on 3.8...


Podman fixes

Podman support fixes

e2e failing, seems to have been broken earlier (related: https://github.com/Kuadrant/multicluster-gateway-controller/pull/795)

/lgtm
/approve


Issue: Cost tracking for RunTree

Issue you'd like to raise.
Hi there, I'm using RunTree and want to log the cost, I figured out how to add the token count but not the cost. Can someone guide me please?
PS: The provider openai and gpt-4 are just values to test with. I figured if it works for them it'll work for the other custom models I'm using like Command-R
Reference: https://docs.smith.langchain.com/how_to_guides/tracing/log_llm_trace#manually-provide-token-counts
   trace.end({
      data,
      usage_metadata: {
        input_tokens: 27,
        output_tokens: 13,
        total_tokens: 40,
        total_cost: ((inCost ?? 0) + (outCost ?? 0)),
        prompt_cost: inCost,
        completion_cost: outCost,
      },
      ls_provider: 'openai',
      ls_model_name: 'gpt-4',
    });

Suggestion:
No response

Hello! You're almost there! All that's remaining is to configure the  model costs in the langsmith dashboard:


Preserve runchecks across deepcopy

Fixes #724

LGTM but I think we need to rebase for the tests to pass with the new dependencies.

thanks


rST markup for Haddock

I would really really like it if I could write Haskell API documentation with reStructuredText (rST). It's great for writing Python documentation, and would allow me to be consistent between my API docs and higher-level docs (I notice that Haddock uses rST for its docs).
Haddock markup is idiosyncratic, and unlike markup used elsewhere. It's hard to guess, and I always find myself consulting the manual.
I realise this is a big request, and that there are equally valid arguments for Markdown or other markup languages. I'd love to implement it myself, but step 0 is filing a bug here.

...it may be useful to review why attempts to add support for Markdown have failed in the past (see https://github.com/haskell/haddock/issues/244#issuecomment-261019664) and whether rST would have a better chance than Markdown.

I filed this already as #570.  Sorry for the duplication.


Use the ExpandingDot wihtout the FLatList

Hello
I would like to pass MaxPages and currentPage to the ExpandingDot component and control the pagination programmatically by updating the currentPage value.
I have tried to add this custom feature but without success.
Can you please give me some hints that can help me proceed.
I will post the final code if i achieved it here.
Thanks.

Hello
I would like to pass MaxPages and currentPage to the ExpandingDot component and control the pagination programmatically by updating the currentPage value.
I have tried to add this custom feature but without success.
Can you please give me some hints that can help me proceed.
I will post the final code if i achieved it here.
Thanks.

Yes, but need to rework for this. But if you increase scrollX value by witdh * activeIndex, it should work.

I tried to change the value of scrollX value with different methods but its didnt apply any change on the ExpandingDot Component.


Support AnyIO

Closes #1827.

Awesome, thanks! Let me know when you think it's ready for review or if you have any questions. The edge-triggered FD stuff is very fiddly.

It's still a WIP, but it would be nice to know what you think about it @minrk.
The API in sugar/socket.py should probably not be async, I'm not sure exactly where to have the sync/async split.

Yeah, sugar/socket.py, which is the base blocking API, should probably not need any changes at all, and definitely shouldn't introduce any async code.
It may be that the coroutine/wait_readable approach is different enough in design that it would be better to start zmq/anyio.py from scratch and only take inspiration from the current asyncio implementation (or copy/edit), rather than updating in-place. That would also be the easiest way to ensure backward compatibility, which is important. But if it seems easier to update in-place, that's a possibility, too. It's hard for me to say since I don't know what it's going to look like.
The hardest part, I think, will be correctly handling the edge-triggered FD with multiple awaiters, especially since it is hard to ensure we provoke this situation thoroughly in tests. Any event that consumes socket events must necessarily wake all tasks that are waiting on the FD or they can hang. We'll need to be thorough about the implications of a socket being used in multiple task groups, because all waiting tasks must wake. If that's a problem, we'll need to make sure to have locks to prevent the situations that won't work from happening.
Needing to wake tasks includes accessing the socket.EVENTS attribute, sending a message, and receiving a message. The FD being readable does not mean the socket is readable, it means there have been events on the socket, and socket.EVENTS must be checked. The socket not being readable does not mean there are not messages waiting, it means that events have not changed. so e.g. with multiple messages ready to recv, the FD will wake once, and all messages must be received before it is appropriate to call wait_readable again.
Plus we'll need a wait_readable implementation that works on proactor on Windows before it can be used. We have the add_reader version of this via tornado, which will need to be adapted to wait_readable. The principle is the same, so it can be built on top, I think.

Here's a very simple anyio example implementing async recv using only the blocking pyzmq API. It doesn't handle the edge-triggered nuances or timeouts, but it gets the very basics.

Yes I think you're right that it's safer to have a separate implementation just for AnyIO, for backwards-compatibility reasons. I'm not sure anything I've done will remain then, since I was hoping to replace the asyncio layer with AnyIO, but now we can start from scratch as you did in your example.
This also raises the question of whether to create a different package, or have the AnyIO support here in pyzmq?

I think either way is fine for the separate package or here. A separate package can also make sense as a prototyping pathway to release often and avoid needing to wait for pyzmq releases. It really shouldn't need changes in pyzmq, though I do occasionally have to do some cooperation to get all the serialization methods to work well with the async subclasses without too much duplication.
For example, outside pyzmq it would be totally appropriate to only support low-level send/recv_multipart, not all the serialization extensions. That doesn't fit in the pyzmq socket subclass approach where all methods need to work, and use a "has a" rather than "is a" relationship, which type systems like a lot better since subclass != subtype.
We can always merge upstream here if/when it seems like 'the way to go' for async pyzmq, or not. We did that with gevent_zeromq and pyzmq_static before. Or if we like maintaining the separate package, gently deprecate zmq.asyncio in favor of the new package.

(btw, what you have can still remain, and you can copy the changed files to zmq/anyio.py and then feel free to update without concerns around breaking APIs). But if you'd rather start the whole design from scratch, that's also fine by me. Getting all the waits/timeouts/cleanup right was hard, but maybe (hopefully!) it will be easier with an anyio approach. Timeouts and cleanup are usually a lot easier with task groups and coroutines than futures! And if we had normal level-triggered FDs, this whole thing would be pretty straightforward.
Maybe it should even be a function-based approach (async_send / async_recv) using blocking Socket objects instead of defining async Socket classes (this might make the bookkeeping of waking multiple waiters hard, I'm not sure). I'm absolutely not set on using the same approach I used for zmq.asyncio for anyio. I'm happy to pursue whatever you think would be suit anyio compatibility best.

One simple way would be to run pyzmq's blocking APIs in a thread, but I guess that would lead to very bad performances.

I have started https://github.com/davidbrochart/zmq-anyio so I'm closing this for now. Thanks for the feedback @minrk !


Commit generated opcodes to remove the need for ruby?

Hi :wave:
Could the dependency on ruby be removed maybe, by e.g. committing the generated opcodes to the repository?

Hi wave
Could the dependency on ruby be removed maybe, by e.g. committing the generated opcodes to the repository?

Hello! I would not remove it as opcodes might be changed in the future or new ones added, always rebuilding them manually is not a great idea IMO. What might be done instead is maybe rewrite the DSL in Rust itself, but it is a large task to do

I did not mean to rebuild them manually, only that the build script for example checked that an existing opcode_generated.rs is older than the opcode.opcodes file, and then automatically rebuild it.
That would allow to have the automation to keep the generated code up to date without needing to manually call the generator, while also allowing for people to use the library without ruby installed, which could also be achieved by publishing to crates.io with the generated opcodes there of course.


Provide custom openssl/curl version to cmake

What platform/OS are you using?
Ubuntu 16.04
What compiler are you using? what version?
GCC 5.4
What's your CMake arguments?
cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX:PATH=/home/user/aws-sdk-shared -DOPENSSL_INCLUDE_DIR=/home/user/openssl1.0.2l/include -DOPENSSL_LIBRARIES="/home/user/openssl1.0.2l/libshared/libssl.so;/home/user/openssl1.0.2l/libshared/libcrypto.so" ..
We compile our applications against a different version than what is on the system path and with the default compilation of the AWS SDK I get linker errors when compiling our applications because there is a mismatch in symbols from the system version openssl and the one we compiled ourselves.
I tried to supply custom include/lib paths to cmake but it doesn't appear to work correctly. Is there a way to correctly bypass the FindOpenSSL cmake module and supply our own version of OpenSSL and CURL?

One way is to install newer version of libcurl into /usr/local, e.g.:
cd ~
wget https://curl.haxx.se/download/curl-7.54.0.tar.gz
tar -xvf curl-7.54.0.tar.gz
cd curl-7.54.0
./configure --libdir=/usr/local/lib64
make -j
sudo make install
sudo ldconfig

and link it statically (to avoid problems during deployment):
cmake -DBUILD_SHARED_LIBS=0 ...

... or package libcurl with your app. I used static linking -- not ideal, but my app is used in controlled environment, no pressing need for security updates.

Try passing -DCMAKE_PREFIX_PATH=

I had to use DOPENSSL_ROOT_DIR instead of specifying separate paths for includes and lib directories. There wasn't an equivalent variable for CURL so I had to hack up external_dependencies.cmake to specify a different path.

Use the following for CURL:  -DCURL_INCLUDE_DIR="${DEPEND_BUILD_DIR}/curl-7.57.0/include" -DCURL_LIBRARY="${DEPEND_BUILD_DIR}/curl-7.57.0/lib"
This is what we're using and it's works for us.  We also had to set the DOPENSSL_ROOT_DIR.  In addition we needed to set our own zlib as follows:  -DZLIB_ROOT="${DEPEND_BUILD_DIR}/zlib-1.2.11"
Before this we were also hacking the external_dependencies.cmake, but with the above you should not have to do that anymore.

It might be because this thread is old, but the CURL_INCLUDE_DIR and CURL_LIBRARY are not supported in aws-sdk-cpp-1.8.3:
-- Configuring done
-- Generating done
CMake Warning:
  Manually-specified variables were not used by the project:

    CURL_INCLUDE_DIR

-- Configuring done
-- Generating done
CMake Warning:
  Manually-specified variables were not used by the project:

    CURL_LIBRARY

@rvernica Did you find workaround to this problem?

No, I tried various things but I could not make cmake to use my cURL build. I gave up on this and managed to get my application link against my cURL build. See here for my original issue and the workaround.

After examining the aws-sdk cmake files, I discovered that the aws-sdk cmake provides support for both system installed CURL and local installed curl, but the system installed CURL takes precedence by using include(FindCURL) first.
I have found that there is a system-wide setting, NO_CMAKE_SYSTEM_PATH, but that prevents finding any system installed libraries. I was able to use NO_CMAKE_SYSTEM_PATH as a workaround, but that would prevent use of other system libraries.


Suggestion to improve WordCount (line 194)

I would like to make a suggestion to avoid having to edit the .cls file in line 194. Currently, that line contains:
Word Count: \quickwordcount{trb_template}~words $+$ \total{table} table(s) $\times$ \@WordsPerTable\ $=$ \totalwordcount~words\\

My suggestion is modify the trbunofficial.cls file to include a new command \thisfilename{...} which would be used to provide the TeX file name to be included in line 194. Thus, in the LaTeX file preamble, one would include a line with:
\thisfilename{main} % main.tex is the file with the manuscript for TRB

The trbunofficial.cls file would have to be modified to replace the current \renewcommand{\maketitle} macro with this code:
\newcommand{\thisfilename}[1]{\newcommand{\@filename}{#1}}
\renewcommand{\maketitle}{%
	\thispagestyle{empty}
	\begin{flushleft}
		{\MakeUppercase{\bfseries\@title}}\\
		\hfill\break%
		\hfill\break%
		\hfill\break%
		\@author\\
		\hfill\break%
		\hfill\break%
		\if\@TotalWords 0
		Word Count: \quickwordcount{\@filename}~words $+$ \total{table} table(s) $\times$ \@WordsPerTable\ $=$ \totalwordcount~words\\
		\else
		Word Count: \@TotalWords\ words $+$ \total{table} table(s) $\times$ \@WordsPerTable\ $=$ \the\numexpr\@TotalWords + \totvalue{table}*\@WordsPerTable~words\\
		\fi
		\hfill\break%
		\hfill\break%
		\hfill\break%
		\hfill\break%
		\hfill\break%
		\hfill\break%
		Submission Date: \today
	\end{flushleft}
	\newpage
}

I hope this would help.

How would you like to send this in a pull request? Its an interesting idea that I would lie to try out.


can the script work from right to left?

Hi there,
This is an amazing plugin. is there a simple way to make slides move from right to left instead of left to right?

Currently slides are moving in both directions.

Thanks for your reply,
I think you misunderstood me, what I meant was, that when you start in slide #1 scrolling to the next slide is always to the right (1st slide moving to the left).
Is there a way to make it the opposite direction? next slides will come from left side and first slide will move right.

Last slide can be shown in the beginning (and scroll will go right after). Take a look on processing initial hash in url, this code will help you to set last slide by default.

thank you very much :)


Create new release

It has been a while wince we have done a new release. There are good bugfixes on master. We should release soon and not wait for the large refactor.

@LegNeato not many things left. We have a milestone for that.
However, the most work, not mentioned there, will take the Book rewrite, because it's too outdated.

Ah, I didn't see the milestone! Feel free to close this.

@LegNeato now it's only the Book rework left!

@davidpdrsn while reworking the Book in #1230, I've noticed that davidpdrsn/graphql-app-example sill dependends on juniper 0.14.
Would you mind adding me as a collaborator to juniper-from-schema, juniper-eager-loading and davidpdrsn/graphql-app-example, so I at least keep them up-to-date with latest juniper and its reworkings?

@davidpdrsn while reworking the Book in #1230, I've noticed that davidpdrsn/graphql-app-example still dependends on juniper 0.14.
Would you mind adding me as a collaborator to juniper-from-schema, juniper-eager-loading and davidpdrsn/graphql-app-example, so I at least keep them up-to-date with the latest juniper and its reworkings?

@tyranron
Since you want to be a collaborator i wonder if you already have some changes done on a juniper-from-schema fork or something similar?
I ask because i forked(https://github.com/Kinflou/juniper-from-schema) and started upgrading and reworking the crate, but i was unsure if i ask the author(@davidpdrsn) if i could take over (#juniper-from-schema/141) or if i make a new fork/crate with my changes, since they are quite a few.
I'm up to collaboration on it

@Kinflou wonderful! No, I haven't touched juniper-from-schema and its ecosystem yet, but I do plan too. Would like to review your PR and give feedback. However, we need @davidpdrsn for this.
I propose you to make a PR into the upstream once juniper 0.16 is released.

Hello @tyranron, hope you're doing fine.
I wanted to ask you if you have an ETA on the 0.16 release and if you needed help somewhere (I don't know the codebase but I still can try my best).
Much respect for your work


Fix invalid syntaxes in test code

Revert unintended (probably due to automation...?) test code changes in #4511.

Thanks! These seems like something that we should be able to detect automatically...


Fix buildout and traversal

Coverage remained the same at 28.38% when pulling 4d22071c19f64a73e058eed22fa1c75d1ed847cc on kukirokuk:a339649518471296_awarding_2.0_manual into e597d95ea5ba7e4b6cf6290ed2d3d388ad64a594 on prozorro-sale:master.