Sanctioned websites

Feature Request
Add a category for sanctioned websites
Description
Please see this file
Even that it is not updated, it can be a good starting point
Motivation
We can have whitelist and blacklist domains to help user to choose bypass or proxy specific domains
Example
chatgpt is blocked in Iran

The repo name is Iran Hosted Domains, so this enhancement is out of this repository context.

but it is related to iran so there is no better place than your repository- i think

freedomofdevelopers/fod already done this.

Great you are so lovely ❤
On Sunday, May 7, 2023, bootmortis @.***> wrote:

freedomofdevelopers/fod https://github.com/freedomofdevelopers/fod
already done this.
—
Reply to this email directly, view it on GitHub
https://github.com/bootmortis/iran-hosted-domains/issues/50#issuecomment-1537429410,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/A3HPTEKEYABAGPFWVY24D2LXE6IL7ANCNFSM6AAAAAAXY2UE5U
.
You are receiving this because you authored the thread.Message ID:
@.***>


I18N-495

Add configurable application.properties setting to allow the user to add Analytics HTML string information to the main application

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.2 out of 5 committers have signed the CLA.:white_check_mark: maallen:white_check_mark: garionpin:x: DarKhaos:x: mattwilshire:x: byronantakYou have signed the CLA already but the status is still pending? Let us recheck it.


[Discussion] The learning design

I think about the learning design that is implemented here, and I just can't resolve to myself two questions. The core function for the learning is the environment step function. The chain of learning is [OBS_UNIT1 -> ACTION1 -> REWARD -> OBS_UNIT2 -> ACTION2 -> OBS_UNIT3 -> ACTION3 ... -> ALL TURN ACTIONS ARE ACTUALLY TAKEN] -> [THE SAME FOR THE NEXT TURN ...]. The questions are:

Less important. Only the first action gets reward. Doesn't it create significant problems, especially when the number of units per turn is big? Especially if the discount factor gamma is small, but also in general. Even this intermediate reward for most actions is delayed. I wonder how much harder the life is for the model because of this. One thing, - the ordering of the units to act can be important. I can imagine that the model can handle it. But is there an example of multi-unit problems that are designed like this?

More important. The algorithms like TD(0), Q-Learning, and more involved like PPO, all depend for the model update not only on the current state (or state-action pair) but also the next one. But the next step is a different unit, its observation is unit-dependent, its value function is completely different, and barely related. The process is basically not markovian, the states are heavily incomplete information, and each time different incomplete information. Isn't it a no-go? Or I miss-understand something major?

Please share your thought!

Some personal thoughts:

Personally I think this part is OK because of the large gamma. In many cases it's quite common to get the reward many steps afterwards of the action that caused the reward. The default gamma is 0.995, so if my understanding is correct an action will get a 0.995^100=0.61 factor of a reward given 100 steps later. This matches up pretty closely with the OpenAI Five setup where micro-rewards are sparse with large gammas: https://openai.com/blog/openai-five/
In the case of PPO it seems to predict the action value given current (which I think makes sense for our scenario, it takes current unit observations and guesses how likely the actions will effect the discounted reward), but into the advantage function is a critic model predicting just value given state. So I think the action-value function should be good. The second part it seems is the state value function, where like you said I think it's more of a problem. It's comparing the value of the current state versus the next step's state, and extends it into the future with the gamma discount factor. Part of the observation was included the values like num_units, num_cities, etc to purposely help with this state value estimate, but these only change each turn (not each action) like you said. So let's say we had an action that was good, I guess the advantage value from the action would be approximately something like `(value(state current)-value(state 1 step ahead))+gamma*value(state 2 step ahead)-value(state 3 step ahead)+...). Imagine it's action was building a city, and we get the reward for building a city ~4 steps later once the next turn starts, it'll still be included in the advantage calculation of that action despite it being a bunch of states earlier. So maybe the key here is just to make sure to include lots of good game-observations outside of the unit-observations so that the state value critic model can perform better? I had been hoping my CNN experiments where the whole map was driving the observation would work better, since it can fully-observe the state of the game at each decision - unfortunately I couldn't get it to outperform the simple models.

Hi @glmcdona , thanks for the feedback. Regarding 1, I agree that it should be OK. For the second point, you raise an interesting point about including more global game observations. Too risky though, I wouldn't want to try to make it work.
The CNN approach is what I also want to test. Have you just used CnnPolicy or something else? I want to take the network from that famous imitation learning CNN notebook and plug it instead of that one. If I checked correctly CnnPolicy is only 4 CNN layers. Additionally, I want the value part to have the same input per turn, and give additional input (unit location) only to the action part. I will let you know if it works for me!

Yeah, I think the built-in default CnnPolicy isn't a good fit. You can define your own layers:
https://stable-baselines3.readthedocs.io/en/master/guide/custom_policy.html
I just shared an example notebook with you on Kaggle that I've been using. I've tried a few architectures, and the latest one is inspired by that imitation learning notebook model layout. Note, although it did work, it didn't get to as high a reward solution as the simple non-CNN example in this repo. I'm personally working on implementing a solution more similar to the OpenAI Five observation setup now.

Ok, very interesting, I am reading your notebook now. Just a small remark, I believe theoretically it is called "private sharing", which is not allowed. Let's refrain from this in the future (as long as we are not in the same team!).

Oh wow, I didn't know we couldn't share code with each other if we aren't on the same team! Thanks for the heads up.

A few comments on that notebook

Good that you didn't use compressed_map_observation at the end ;). It is probably not good because of what we discussed.
I like how you build the observation layers.
The CNN model that you built is very strange. Three nn.Conv2d layers in a row without activations in between, no batch norm, two max pools, and none of them is at the end, no skip connections. It is far away from all designs that I know.
It is nice how you allow passing different types of observations, I will try to use it too. But do you use only self.obs['map']? For example all these global arguments, like night/day etc., can be good to concatenate to the output of CNN, instead of creating a separate layer.

Are you on the competition discord server @nosound2 ?
Regarding the architecture and whether or not the usage of skip/residual elements. The current "miner-state' has ~100 values (order of magnitude), any output of a CNN feature extractor is likely to be >10k values. Fancy architectures are great but the training time (and hyperparameters selection) is getting quickly out of hand (at least from my attempts).
I'm currently trying to inject as much human-knowledge in the observation to reduce what has to be learned from scratch to improve training speed.

The CNN model that you built is very strange. Three nn.Conv2d layers in a row without activations in between, no batch norm, two max pools, and none of them is at the end, no skip connections. It is far away from all designs that I know.

This is similar to a basic VGG16 model architecture, though looks like it should run a relu every 3x3 conv, eg:
https://neurohive.io/en/popular-networks/vgg16/

It is nice how you allow passing different types of observations, I will try to use it too. But do you use only self.obs['map']? For example all these global arguments, like night/day etc., can be good to concatenate to the output of CNN, instead of creating a separate layer. It seems like everything is ready for this too.

Yup, you are describing an earlier version of that notebook! I modified it to incorporate everything into the CNN layers to more closely match the imitation learning setup incase it helped. The original design had them added at the FC layer instead of adding them as layers to the CNN input.

Here is the example notebook shared now:
https://www.kaggle.com/glmcdona/python-environment-ppo-cnn-rl-example
Note that for kaggle submission, the main_lux-ai-2021.py needs to edited to include specifying the feature extractor in the model load operation, eg something like this:
from agent_policy import AgentPolicy, CustomCombinedExtractor
...
policy_kwargs = dict(
      features_extractor_class=CustomCombinedExtractor
)
model = PPO.load(f"model.zip", policy_kwargs=policy_kwargs)

Hi @glmcdona , thanks for the feedback. Regarding 1, I agree that it should be OK. For the second point, you raise an interesting point about including more global game observations. Too risky though, I wouldn't want to try to make it work.
The CNN approach is what I also want to test. Have you just used CnnPolicy or something else? I want to take the network from that famous imitation learning CNN notebook and plug it instead of that one. If I checked correctly CnnPolicy is only 4 CNN layers. Additionally, I want the value part to have the same input per turn, and give additional input (unit location) only to the action part. I will let you know if it works for me!

The MLp only has 4 layers 2 layers of 64 for both the actor and the critic.
The CnnPolicy only works good on images. The api gives us all of the information without any of the noise. Cnn approach would never be able to determine if there were multiple workers on city tile for example.

Geoff, btw do you have any idea how to get rid of the runtime error stacking error. At around 40-50 milllion steps, too many of the games stop early because the model hasn't quite learned to save fuel during the night.n And this causes there to be compile errors if too many games end early.

Geoff, btw do you have any idea how to get rid of the runtime error stacking error. At around 40-50 milllion steps, too many of the games stop early because the model hasn't quite learned to save fuel during the night.n And this causes there to be compile errors if too many games end early.

Not sure what would cause this. Do you have a copy of the error by any chance? Is it a memory leak, out of memory error?

Fun fact:

All have same reward function:

White: some reference
Blue gamma_0: higher episode length, lower reward
Orange gamma_1: lower episode length, higher reward

I still have to benchmark them.

ocess SpawnProcess-32:
Traceback (most recent call last):
File "C:\Users\18176\anaconda3\envs\pythree\lib\multiprocessing\process.py", line 297, in _bootstrap
self.run()
File "C:\Users\18176\anaconda3\envs\pythree\lib\multiprocessing\process.py", line 99, in run
self._target(*self._args, **self._kwargs)
File "C:\Users\18176\Desktop\luxlux20\examples\stable_baselines3\common\vec_env\subproc_vec_env.py", line 29, in _worker
observation, reward, done, info = env.step(data)
File "C:\Users\18176\Desktop\luxlux20\examples\luxai2021\env\lux_env.py", line 64, in step
obs = self.learning_agent.get_observation(self.game, unit, city_tile, team, is_new_turn)
File "C:\Users\18176\Desktop\luxlux20\examples\agent_policy.py", line 369, in get_observation
c = game.cities[game.map.get_cell_by_pos(closest_position).city_tile.city_id]
AttributeError: 'NoneType' object has no attribute 'city_id'
Traceback (most recent call last):
File "C:\Users\18176\anaconda3\envs\pythree\lib\multiprocessing\connection.py", line 312, in _recv_bytes
nread, err = ov.GetOverlappedResult(True)
BrokenPipeError: [WinError 109] The pipe has been ended
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "Other_train.py", line 191, in
train(local_args)
File "Other_train.py", line 163, in train
model.learn(total_timesteps=args.step_count, reset_num_timesteps=True)
File "C:\Users\18176\Desktop\luxlux20\examples\stable_baselines3\ppo\ppo.py", line 310, in learn
reset_num_timesteps=reset_num_timesteps,
File "C:\Users\18176\Desktop\luxlux20\examples\stable_baselines3\common\on_policy_algorithm.py", line 237, in learn
continue_training = self.collect_rollouts(self.env, callback, self.rollout_buffer, n_rollout_steps=self.n_steps)
File "C:\Users\18176\Desktop\luxlux20\examples\stable_baselines3\common\on_policy_algorithm.py", line 178, in collect_rollouts
new_obs, rewards, dones, infos = env.step(clipped_actions)
File "C:\Users\18176\Desktop\luxlux20\examples\stable_baselines3\common\vec_env\base_vec_env.py", line 162, in step
return self.step_wait()
File "C:\Users\18176\Desktop\luxlux20\examples\stable_baselines3\common\vec_env\subproc_vec_env.py", line 120, in step_wait
results = [remote.recv() for remote in self.remotes]
File "C:\Users\18176\Desktop\luxlux20\examples\stable_baselines3\common\vec_env\subproc_vec_env.py", line 120, in
results = [remote.recv() for remote in self.remotes]
File "C:\Users\18176\anaconda3\envs\pythree\lib\multiprocessing\connection.py", line 250, in recv
buf = self._recv_bytes()
File "C:\Users\18176\anaconda3\envs\pythree\lib\multiprocessing\connection.py", line 321, in _recv_bytes
raise EOFError

Hate to nag, but  recording playing command does not seem to work and the new updated files dont compile on the kaggle server for submissions

Hi @goforks12 , is it a different issue now? If so, can you please open a different issue per problem. Also, more details for the second issue will be helpful, I think.

ocess SpawnProcess-32: Traceback (most recent call last): File "C:\Users\18176\anaconda3\envs\pythree\lib\multiprocessing\process.py", line 297, in _bootstrap self.run() File "C:\Users\18176\anaconda3\envs\pythree\lib\multiprocessing\process.py", line 99, in run self._target(*self._args, **self._kwargs) File "C:\Users\18176\Desktop\luxlux20\examples\stable_baselines3\common\vec_env\subproc_vec_env.py", line 29, in _worker observation, reward, done, info = env.step(data) File "C:\Users\18176\Desktop\luxlux20\examples\luxai2021\env\lux_env.py", line 64, in step obs = self.learning_agent.get_observation(self.game, unit, city_tile, team, is_new_turn) File "C:\Users\18176\Desktop\luxlux20\examples\agent_policy.py", line 369, in get_observation c = game.cities[game.map.get_cell_by_pos(closest_position).city_tile.city_id] AttributeError: 'NoneType' object has no attribute 'city_id' Traceback (most recent call last): File "C:\Users\18176\anaconda3\envs\pythree\lib\multiprocessing\connection.py", line 312, in _recv_bytes nread, err = ov.GetOverlappedResult(True) BrokenPipeError: [WinError 109] The pipe has been ended
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "Other_train.py", line 191, in train(local_args) File "Other_train.py", line 163, in train model.learn(total_timesteps=args.step_count, reset_num_timesteps=True) File "C:\Users\18176\Desktop\luxlux20\examples\stable_baselines3\ppo\ppo.py", line 310, in learn reset_num_timesteps=reset_num_timesteps, File "C:\Users\18176\Desktop\luxlux20\examples\stable_baselines3\common\on_policy_algorithm.py", line 237, in learn continue_training = self.collect_rollouts(self.env, callback, self.rollout_buffer, n_rollout_steps=self.n_steps) File "C:\Users\18176\Desktop\luxlux20\examples\stable_baselines3\common\on_policy_algorithm.py", line 178, in collect_rollouts new_obs, rewards, dones, infos = env.step(clipped_actions) File "C:\Users\18176\Desktop\luxlux20\examples\stable_baselines3\common\vec_env\base_vec_env.py", line 162, in step return self.step_wait() File "C:\Users\18176\Desktop\luxlux20\examples\stable_baselines3\common\vec_env\subproc_vec_env.py", line 120, in step_wait results = [remote.recv() for remote in self.remotes] File "C:\Users\18176\Desktop\luxlux20\examples\stable_baselines3\common\vec_env\subproc_vec_env.py", line 120, in results = [remote.recv() for remote in self.remotes] File "C:\Users\18176\anaconda3\envs\pythree\lib\multiprocessing\connection.py", line 250, in recv buf = self._recv_bytes() File "C:\Users\18176\anaconda3\envs\pythree\lib\multiprocessing\connection.py", line 321, in _recv_bytes raise EOFError

It seems to be a problem in your custom code, in this line specifically:
c = game.cities[game.map.get_cell_by_pos(closest_position).city_tile.city_id]

ocess SpawnProcess-32: Traceback (most recent call last): File "C:\Users\18176\anaconda3\envs\pythree\lib\multiprocessing\process.py", line 297, in _bootstrap self.run() File "C:\Users\18176\anaconda3\envs\pythree\lib\multiprocessing\process.py", line 99, in run self._target(*self._args, **self._kwargs) File "C:\Users\18176\Desktop\luxlux20\examples\stable_baselines3\common\vec_env\subproc_vec_env.py", line 29, in _worker observation, reward, done, info = env.step(data) File "C:\Users\18176\Desktop\luxlux20\examples\luxai2021\env\lux_env.py", line 64, in step obs = self.learning_agent.get_observation(self.game, unit, city_tile, team, is_new_turn) File "C:\Users\18176\Desktop\luxlux20\examples\agent_policy.py", line 369, in get_observation c = game.cities[game.map.get_cell_by_pos(closest_position).city_tile.city_id] AttributeError: 'NoneType' object has no attribute 'city_id' Traceback (most recent call last): File "C:\Users\18176\anaconda3\envs\pythree\lib\multiprocessing\connection.py", line 312, in _recv_bytes nread, err = ov.GetOverlappedResult(True) BrokenPipeError: [WinError 109] The pipe has been ended
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "Other_train.py", line 191, in train(local_args) File "Other_train.py", line 163, in train model.learn(total_timesteps=args.step_count, reset_num_timesteps=True) File "C:\Users\18176\Desktop\luxlux20\examples\stable_baselines3\ppo\ppo.py", line 310, in learn reset_num_timesteps=reset_num_timesteps, File "C:\Users\18176\Desktop\luxlux20\examples\stable_baselines3\common\on_policy_algorithm.py", line 237, in learn continue_training = self.collect_rollouts(self.env, callback, self.rollout_buffer, n_rollout_steps=self.n_steps) File "C:\Users\18176\Desktop\luxlux20\examples\stable_baselines3\common\on_policy_algorithm.py", line 178, in collect_rollouts new_obs, rewards, dones, infos = env.step(clipped_actions) File "C:\Users\18176\Desktop\luxlux20\examples\stable_baselines3\common\vec_env\base_vec_env.py", line 162, in step return self.step_wait() File "C:\Users\18176\Desktop\luxlux20\examples\stable_baselines3\common\vec_env\subproc_vec_env.py", line 120, in step_wait results = [remote.recv() for remote in self.remotes] File "C:\Users\18176\Desktop\luxlux20\examples\stable_baselines3\common\vec_env\subproc_vec_env.py", line 120, in results = [remote.recv() for remote in self.remotes] File "C:\Users\18176\anaconda3\envs\pythree\lib\multiprocessing\connection.py", line 250, in recv buf = self._recv_bytes() File "C:\Users\18176\anaconda3\envs\pythree\lib\multiprocessing\connection.py", line 321, in _recv_bytes raise EOFError

It seems to be a problem in your custom code, in this line specifically: c = game.cities[game.map.get_cell_by_pos(closest_position).city_tile.city_id] Are there any changes in the agent that you run, in comparison with the git version?

I didn't mess with any of the game engine. I didnt change anything within the LuxAI computations. I was however using 16 cpu cores. And my MLP I was training had much larger layers.

Hi @goforks12 , is it a different issue now? If so, can you please open a different issue per problem. Also, more details for the second problem will be helpful, I think.

lux-ai-2021 --seed=100 ./kaggle_submissions/main_lux-ai-2021.py ./kaggle_submissions/main_lux-ai-2021.py --maxtime 100000
I try to do this command in bash with my Model.zip and my Agent_policy.py in the kaggle submission folder. Should lux-a-2021 be a python file? Or should it be the folder we cd into to run the evaluation?

Hi @goforks12 , is it a different issue now? If so, can you please open a different issue per problem. Also, more details for the second problem will be helpful, I think.

lux-ai-2021 --seed=100 ./kaggle_submissions/main_lux-ai-2021.py ./kaggle_submissions/main_lux-ai-2021.py --maxtime 100000
I try to do this command in bash with my Model.zip and my Agent_policy.py in the kaggle submission folder. Should lux-a-2021 be a python file? Or should it be the folder we cd into to run the evaluation?

lux-ai-2021 is a command added by the official Lux AI repo, check out the installation instructions here if the command isn't found in your environment:
https://github.com/Lux-AI-Challenge/Lux-Design-2021

ocess SpawnProcess-32: Traceback (most recent call last): File "C:\Users\18176\anaconda3\envs\pythree\lib\multiprocessing\process.py", line 297, in _bootstrap self.run() File "C:\Users\18176\anaconda3\envs\pythree\lib\multiprocessing\process.py", line 99, in run self._target(*self._args, **self._kwargs) File "C:\Users\18176\Desktop\luxlux20\examples\stable_baselines3\common\vec_env\subproc_vec_env.py", line 29, in _worker observation, reward, done, info = env.step(data) File "C:\Users\18176\Desktop\luxlux20\examples\luxai2021\env\lux_env.py", line 64, in step obs = self.learning_agent.get_observation(self.game, unit, city_tile, team, is_new_turn) File "C:\Users\18176\Desktop\luxlux20\examples\agent_policy.py", line 369, in get_observation c = game.cities[game.map.get_cell_by_pos(closest_position).city_tile.city_id] AttributeError: 'NoneType' object has no attribute 'city_id' Traceback (most recent call last): File "C:\Users\18176\anaconda3\envs\pythree\lib\multiprocessing\connection.py", line 312, in _recv_bytes nread, err = ov.GetOverlappedResult(True) BrokenPipeError: [WinError 109] The pipe has been ended
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "Other_train.py", line 191, in train(local_args) File "Other_train.py", line 163, in train model.learn(total_timesteps=args.step_count, reset_num_timesteps=True) File "C:\Users\18176\Desktop\luxlux20\examples\stable_baselines3\ppo\ppo.py", line 310, in learn reset_num_timesteps=reset_num_timesteps, File "C:\Users\18176\Desktop\luxlux20\examples\stable_baselines3\common\on_policy_algorithm.py", line 237, in learn continue_training = self.collect_rollouts(self.env, callback, self.rollout_buffer, n_rollout_steps=self.n_steps) File "C:\Users\18176\Desktop\luxlux20\examples\stable_baselines3\common\on_policy_algorithm.py", line 178, in collect_rollouts new_obs, rewards, dones, infos = env.step(clipped_actions) File "C:\Users\18176\Desktop\luxlux20\examples\stable_baselines3\common\vec_env\base_vec_env.py", line 162, in step return self.step_wait() File "C:\Users\18176\Desktop\luxlux20\examples\stable_baselines3\common\vec_env\subproc_vec_env.py", line 120, in step_wait results = [remote.recv() for remote in self.remotes] File "C:\Users\18176\Desktop\luxlux20\examples\stable_baselines3\common\vec_env\subproc_vec_env.py", line 120, in results = [remote.recv() for remote in self.remotes] File "C:\Users\18176\anaconda3\envs\pythree\lib\multiprocessing\connection.py", line 250, in recv buf = self._recv_bytes() File "C:\Users\18176\anaconda3\envs\pythree\lib\multiprocessing\connection.py", line 321, in _recv_bytes raise EOFError

It seems to be a problem in your custom code, in this line specifically: c = game.cities[game.map.get_cell_by_pos(closest_position).city_tile.city_id] Are there any changes in the agent that you run, in comparison with the git version?

I didn't mess with any of the game engine. I didnt change anything within the LuxAI computations. I was however using 16 cpu cores. And my MLP I was training had much larger layers.

If you didn't modify agent_policy.py to create your own agent yet, then I suspect there must be a rare game engine bug case where the Game.cities list is somehow not accurate, where it points to a City that actually doesn't belong to it's cell anymore. I'll have a quick look through to code to see if I can spot anything.

ocess SpawnProcess-32: Traceback (most recent call last): File "C:\Users\18176\anaconda3\envs\pythree\lib\multiprocessing\process.py", line 297, in _bootstrap self.run() File "C:\Users\18176\anaconda3\envs\pythree\lib\multiprocessing\process.py", line 99, in run self._target(*self._args, **self._kwargs) File "C:\Users\18176\Desktop\luxlux20\examples\stable_baselines3\common\vec_env\subproc_vec_env.py", line 29, in _worker observation, reward, done, info = env.step(data) File "C:\Users\18176\Desktop\luxlux20\examples\luxai2021\env\lux_env.py", line 64, in step obs = self.learning_agent.get_observation(self.game, unit, city_tile, team, is_new_turn) File "C:\Users\18176\Desktop\luxlux20\examples\agent_policy.py", line 369, in get_observation c = game.cities[game.map.get_cell_by_pos(closest_position).city_tile.city_id] AttributeError: 'NoneType' object has no attribute 'city_id' Traceback (most recent call last): File "C:\Users\18176\anaconda3\envs\pythree\lib\multiprocessing\connection.py", line 312, in _recv_bytes nread, err = ov.GetOverlappedResult(True) BrokenPipeError: [WinError 109] The pipe has been ended
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "Other_train.py", line 191, in train(local_args) File "Other_train.py", line 163, in train model.learn(total_timesteps=args.step_count, reset_num_timesteps=True) File "C:\Users\18176\Desktop\luxlux20\examples\stable_baselines3\ppo\ppo.py", line 310, in learn reset_num_timesteps=reset_num_timesteps, File "C:\Users\18176\Desktop\luxlux20\examples\stable_baselines3\common\on_policy_algorithm.py", line 237, in learn continue_training = self.collect_rollouts(self.env, callback, self.rollout_buffer, n_rollout_steps=self.n_steps) File "C:\Users\18176\Desktop\luxlux20\examples\stable_baselines3\common\on_policy_algorithm.py", line 178, in collect_rollouts new_obs, rewards, dones, infos = env.step(clipped_actions) File "C:\Users\18176\Desktop\luxlux20\examples\stable_baselines3\common\vec_env\base_vec_env.py", line 162, in step return self.step_wait() File "C:\Users\18176\Desktop\luxlux20\examples\stable_baselines3\common\vec_env\subproc_vec_env.py", line 120, in step_wait results = [remote.recv() for remote in self.remotes] File "C:\Users\18176\Desktop\luxlux20\examples\stable_baselines3\common\vec_env\subproc_vec_env.py", line 120, in results = [remote.recv() for remote in self.remotes] File "C:\Users\18176\anaconda3\envs\pythree\lib\multiprocessing\connection.py", line 250, in recv buf = self._recv_bytes() File "C:\Users\18176\anaconda3\envs\pythree\lib\multiprocessing\connection.py", line 321, in _recv_bytes raise EOFError

It seems to be a problem in your custom code, in this line specifically: c = game.cities[game.map.get_cell_by_pos(closest_position).city_tile.city_id] Are there any changes in the agent that you run, in comparison with the git version?

I didn't mess with any of the game engine. I didnt change anything within the LuxAI computations. I was however using 16 cpu cores. And my MLP I was training had much larger layers.

If you didn't modify agent_policy.py to create your own agent yet, then I suspect there must be a rare game engine bug case where the Game.cities list is somehow not accurate, where it points to a City that actually doesn't belong to it's cell anymore. I'll have a quick look through to code to see if I can spot anything. As a workaround, you can add a try/except to the get_observation() function in agent_policy.py to ignore and log errors.

i was doing an obscenely log training period. Will use shorter times now.

Here is an example training run from an 'okay' RL personal agent I've built. Notes:

This is the 'classic' reward function (the one here https://www.kaggle.com/glmcdona/reinforcement-learning-openai-ppo-with-python-game/notebook#Define-the-RL-agent-logic).
Opponent is a dummy agent that does nothing.
My agent here is a private version trying to get closer to the OpenAI Five approach.
This is CPU-only training after about 24 hours.

Learning curve for a few batch sizes:

Here are a couple replay files of the trained agent from the batch_size==10000 run, it's not great:
replays.zip
Unzip and you can view the replays here:
https://2021vis.lux-ai.org/


Error starting daemon: layer does not exist

please help

How are you trying to start the daemon?
What version are you using on what OS?

I am also having the same issue, with docker 1.12.5 and using btrfs as the storage driver on rhel 7.2
I am just trying out a move from 1.10.3 to 1.12.5 and have build a new environment, however scheduling multiple containers with mem_limit enforced caused the linux vm to reboot, and now docker doesn't start up.
I am using rancher with cattle as the scheduler.

@ibrokethecloud Have you tried to manually mount the layer (or the btrfs partition in general)?
Do you have any more details?

Same issue.
Docker version 1.12.6, build 78d1802
3.10.0-327.22.2.el7.x86_64
CentOS 7.2 64bit
Docker daemon crashed after adding a host in rancher, and can't start the service now.

https://github.com/docker/docker/issues/31506 seems a dupe of this issue.
Since #31506 has more detailed information than this issue, I'm closing this issue.

Same issue.
Docker version 1.12.6, build 78d1802
3.10.0-327.22.2.el7.x86_64
CentOS 7.2 64bit
Docker daemon crashed after adding a host in rancher, and can't start the service now.

have you fix this problem?


Cert:\ PSDrive is unavailable on PowerShell 5.1 launched by cmd when cmd is launched on PowerShell 7.3

Prerequisites

[X] Write a descriptive title.
[X] Make sure you are able to repro it on the latest released version
[X] Search the existing issues.
[X] Refer to the FAQ.
[X] Refer to Differences between Windows PowerShell 5.1 and PowerShell.

Steps to reproduce
Cert:\ PSDrive provided by Certificate provider is unavailable on PowerShell 5.1 launched by cmd.exe if and only if the cmd.exe was launched on PowerShell 7.3. This issue causes an error on running a batch script which launches PowerShell 5.1.
Although this issue happens on PowerShell 5.1, I file it for PowerShell 7.3 because it is reproduced only when cmd.exe is launched on PowerShell 7.3.

Environment
Issue

PS 5.1
Not repro

PS 5.1 on PS 7.3
Not repro

PS 5.1 on cmd on PS 5.1
Not repro

PS 5.1 on cmd on PS 7.2.7
Not repro

PS 5.1 on cmd on PS 7.3
Repro

Expected behavior
PS> cmd /c C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -NoProfile -Command Get-PSDrive

Name           Used (GB)     Free (GB) Provider      Root               CurrentLocation
----           ---------     --------- --------      ----               ---------------
Alias                                  Alias
C                 445.17         30.10 FileSystem    C:\                ..._<IP_ADDRESS>_x64__8wekyb3d8bbwe
Cert                                   Certificate   \
D                 906.38         25.13 FileSystem    D:\
E                   0.06          0.42 FileSystem    E:\
Env                                    Environment
Function                               Function
HKCU                                   Registry      HKEY_CURRENT_USER
HKLM                                   Registry      HKEY_LOCAL_MACHINE
Variable                               Variable
WSMan                                  WSMan

Actual behavior
PS> cmd /c C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -NoProfile -Command Get-PSDrive

Name           Used (GB)     Free (GB) Provider      Root               CurrentLocation
----           ---------     --------- --------      ----               ---------------
Alias                                  Alias
C                 445.17         30.10 FileSystem    C:\                ..._<IP_ADDRESS>_x64__8wekyb3d8bbwe
Cert                                   Certificate   \
D                 906.38         25.13 FileSystem    D:\
E                   0.06          0.42 FileSystem    E:\
Env                                    Environment
Function                               Function
HKCU                                   Registry      HKEY_CURRENT_USER
HKLM                                   Registry      HKEY_LOCAL_MACHINE
Variable                               Variable
WSMan                                  WSMan

Error details
No error is returned.

Environment data
PS> $PSVersionTable

Name                           Value
----                           -----
PSVersion                      7.3.0
PSEdition                      Core
GitCommitId                    7.3.0
OS                             Microsoft Windows 10.0.22623
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           <IP_ADDRESS>
WSManStackVersion              3.0

Visuals
CertPSDriveUnavailable.zip is a step recording capturing following steps.

Launch PowerShell 7.3 with no profile.
Run "C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -NoProfile -Command Get-PSDrive" and get Cert PSDrive as expected.
Run "cmd /c C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -NoProfile -Command Get-PSDrive" and get no Cert PSDrive.

It looks like this bug was introduced in 7.3.0-preview.7.  More specifically, I believe it was introduced by #16355.  Check this out:
PS7.3> cmd /c powershell -noprofile
Windows PowerShell
Copyright (C) Microsoft Corporation. All rights reserved.

PS5.1> Get-PSDrive cert
Get-PSDrive : Cannot find drive. A drive with the name 'cert' does not exist.
At line:1 char:1
+ Get-PSDrive cert
+ ~~~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (cert:String) [Get-PSDrive], DriveNotFoundException
    + FullyQualifiedErrorId : GetLocationNoMatchingDrive,Microsoft.PowerShell.Commands.GetPSDriveCommand

PS5.1> Import-Module Microsoft.PowerShell.Security
Import-Module : The following error occurred while loading the extended type data file: Error in TypeData "System.Security.AccessControl.ObjectSecurity":
The member AuditToString is already present.
Error in TypeData "System.Security.AccessControl.ObjectSecurity": The member AccessToString is already present.
Error in TypeData "System.Security.AccessControl.ObjectSecurity": The member Sddl is already present.
Error in TypeData "System.Security.AccessControl.ObjectSecurity": The member Access is already present.
Error in TypeData "System.Security.AccessControl.ObjectSecurity": The member Group is already present.
Error in TypeData "System.Security.AccessControl.ObjectSecurity": The member Owner is already present.
Error in TypeData "System.Security.AccessControl.ObjectSecurity": The member Path is already present.
At line:1 char:1
+ Import-Module Microsoft.PowerShell.Security
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (:) [Import-Module], RuntimeException
    + FullyQualifiedErrorId : FormatXmlUpdateException,Microsoft.PowerShell.Commands.ImportModuleCommand

PS5.1> Import-Module Microsoft.PowerShell.Security -RequiredVersion <IP_ADDRESS>
PS5.1> Get-PSDrive cert

Name           Used (GB)     Free (GB) Provider      Root                                                                                CurrentLocation
----           ---------     --------- --------      ----                                                                                ---------------
Cert                                   Certificate   \

Here's what I think is happening: PowerShell 7 modifies the value of PSModulePath for the special case of launching Windows PowerShell from PowerShell 7.  In this scenario, however, PowerShell 7 is launching cmd, not PowerShell.exe, so the special handling of PSModulePath doesn't happen, and Windows PowerShell launches with PowerShell 7 modules in its PSModulePath.  (What initially tipped me off was the surprise of seeing history-based prediction in Windows PowerShell.)
Since the PowerShell 7 version of Microsoft.PowerShell.Security is the latest version available to it in this scenario, that's what Windows PowerShell tries to load.  Until 7.3.0-preview.7, Windows PowerShell was able to load this PowerShell 7 module.  Now, when it tries to import the PowerShell 7 version, it throws errors for the types that were moved into the module in #16355.
In the meantime, as a workaround, if you're able to modify the scripts that started failing on you, have them explicitly import version <IP_ADDRESS> of Microsoft.PowerShell.Security before attempting to access cert:\

Thank you for the information, dwtaber. Importing Microsoft.PowerShell.Security <IP_ADDRESS> explicitly mitigated the error.

When we approved the change to move the types out of SMA, we didn't anticipate the impact on the cert: drive that multiple users have reported.  Committee will discuss this after the long weekend.

The root problem is that when doing pwsh -> cmd -> powershell, the $env:PSModulePath in the powershell process will be incorrectly using the module path from pwsh. This is because the cmd derives that environment value from pwsh, and then powershell derives that from cmd. Below is a screenshot of the value of that env variable from within powershell.

With the incorrect module path value, when doing Import-Module Microsoft.PowerShell.Security in powershell, it’s actually attempting to import the pwsh version of the Microsoft.PowerShell.Security module, not the Windows PowerShell version that comes in-box with powershell.exe.

Doing the pwsh -> cmd -> powershell is doomed to run into problems because the module paths are just wrong for powershell.exe. That’s why we special-cased the pwsh -> powershell.exe scenario to fix the PSModulePath environment variable before starting powershell.exe.
@masaru-iritani In your workflow, I suggest you unset the environment variable PSModulePath in cmd by set PSModulePath= before starting powershell.exe, so that the powershell.exe process will be guaranteed to use the correct module path.

Ok, what @daxian-dbw says makes sense to me based on how this works and "supposed" to work.  cmd being in the middle will not hit the code paths that explicitly fixes up PSModulePath but unsetting it will cause either pwsh or powershell to create it.

Copying my comment over from a duplicate bug:
I'm hitting this bug in a long-term product support codebase and probably can't check in the workaround. I also suspect we'll see more reports of this issue as more people move to 7.3, and the error message gives no easy indication of what the problem is.
If pwsh and powershell already know how to rebuild PSModulePath, could the pwsh -> powershell.exe special-case be applied to all executable calls?

I had azure pipeline that uses powershell inline script which runs on an self-hosted build agent where I have powershell standard and Ps7, Pipeline used to work like a charm until I upgraded 2 weeks back PS7 to 7.3
Script started to fail with the following error

I opened a ticket with VS community at
https://developercommunity.visualstudio.com/t/dotnet-coverage-not-showing-analysis-onl/10177746#T-ND10238787
They pointed me to this ticket
As you will notice in PS command line it calls the right <IP_ADDRESS> version and in PS7 as well it calls its correct version and script on both works fine
I issued  set "PSModulePath=" in PS command line even I added
import-module  -Name 'Microsoft.PowerShell.Security' -RequiredVersion <IP_ADDRESS>
in the inline script in the pipeline but nothing helped
So not sure what is needed to be doen in order to make the script in the pipeline work correctly


[FEATURE] Vim Movements

Is your feature request related to a problem? Please describe.
I did expect this application to support basic vim movements.
wiki-tui is addressed to people, heavily relying on the terminal, so it would make sense to me.
Other popular command line tools like man, less, ranger, partially even tmux and vim itself of course, use vim movements.
Describe the solution you'd like
I'd like to be able to use a few basic movements:

j/k for down and up
Ctrl-d/Ctrl-u half page down, half page up
Maybe even ? and / for keyword search (advanced)

Describe alternatives you've considered
Well, scrolling and klicking works fine, but feels a bit less fun.
Additional context
No additional context.
Checklist

[x] checked that this feature wasn't requested before in an issue
[x] read CONTRIBUTING.md

Foot note
If the feature is deemed acceptable or desirable to the community, I would be willing to try to implement it myself.

Hey @julianbuettner and thank you for your feature request!
Vim movements are already there (not all of them, but the basics, more info can be found in the docs. Note: they were added per request in this issue).
Currently, h/j/k/l works, but Ctrl-d/Ctrl-u and ?// aren't implemented. Keyword search would require a rewrite of the drawing mechanism (it's messy down there, but I'm working on it), but half page up and down would be a nice addition.
To implement page up and down, you would need to add the keys to the config (document the change in the docs) and then to the RootLayout::wrap_on_event function. Because we currently only support rewriting the keybindings of actions given by our UI library cursive, we cannot implement our own (yet).
If you look into the on_event function, you can see what events need to be sent in the RootLayout::wrap_on_event function.
Please also make sure to read the docs about implementing a feature
If you have any questions regarding the codebase, feel free to ask me!

Thanks to @Enoumy, all of your requested keybindings except for the search-related ones are implemented, meaning I'm going to close the issue for now. Feel free to open it up again if you have other suggestions for keybindings!


Games Page Love

Goal
Make the Games page less pathetic...

Completed for now maybe add back in the description at a later date? Closing.


caffe conversion

What is the command to convert caffe model to tensorflow? and can caffe model be converted to keras?

Hi @SANTO-P , here is something that might give you some inspiration and suggestion when using MMdnn. You can check it out at [Samples of conversion between different frameworks].(https://github.com/Microsoft/MMdnn#support-frameworks)

$ mmconvert -sf caffe -in resnet152-deploy.prototxt -iw resnet152.caffemodel -df cntk -om caffe_resnet152.dnn
As this command converts caffe to cntk....what parameters to be included in command for caffe to keras conversion?

Here is my suggestion.
First, convert the caffe model to IR structure by following caffe->IR.
$ mmtoir -f caffe -n resnet152-deploy.prototxt -w resnet152.caffemodel -o caffe_resnet_IR
.
.
.
IR network structure is saved as [caffe_resnet_IR.json].
IR network structure is saved as [caffe_resnet_IR.pb].
IR weights are saved as [caffe_resnet_IR.npy].

Then, convert the IR structure to the keras code by following IR->Keras.
$ mmtocode -f keras --IRModelPath caffe_resnet_IR.pb --dstModelPath keras_inception_v3.py

Parse file [caffe_resnet_IR.pb] with binary format successfully.
Target network code snippet is saved as [keras_inception_v3.py].

Hoping this could be helpful to you.


IE9/10 bug

On IE9/IE10, I'm getting the following error:
SCRIPT5007: Unable to get property 'width' of undefined or null reference
Looking at your codebase, it appears it's referring to the section of code below (specifically the line I've commented below). I did a console.log(this.props), and I got 'undefined', so it doesn't appear any props are making it through, despite the fact that I've assigned props and they work just fine on IE11 and non-IE browsers.
Please advise.
var _react2 = _interopRequireDefault(_react);

var _lodash = require('lodash.isequal');

var _lodash2 = _interopRequireDefault(_lodash);

var _utils = require('./utils');

var _responsiveUtils = require('./responsiveUtils');

var _ReactGridLayout = require('./ReactGridLayout');

var _ReactGridLayout2 = _interopRequireDefault(_ReactGridLayout);

function _interopRequireDefault(obj) { return obj && obj.__esModule ? obj : { default: obj }; }

function _objectWithoutProperties(obj, keys) { var target = {}; for (var i in obj) { if (keys.indexOf(i) >= 0) continue; if (!Object.prototype.hasOwnProperty.call(obj, i)) continue; target[i] = obj[i]; } return target; }

function _classCallCheck(instance, Constructor) { if (!(instance instanceof Constructor)) { throw new TypeError("Cannot call a class as a function"); } }

function _possibleConstructorReturn(self, call) { if (!self) { throw new ReferenceError("this hasn't been initialised - super() hasn't been called"); } return call && (typeof call === "object" || typeof call === "function") ? call : self; }

function _inherits(subClass, superClass) { if (typeof superClass !== "function" && superClass !== null) { throw new TypeError("Super expression must either be null or a function, not " + typeof superClass); } subClass.prototype = Object.create(superClass && superClass.prototype, { constructor: { value: subClass, enumerable: false, writable: true, configurable: true } }); if (superClass) Object.setPrototypeOf ? Object.setPrototypeOf(subClass, superClass) : subClass.__proto__ = superClass; }

var noop = function noop() {};

var ResponsiveReactGridLayout = function (_React$Component) {
  _inherits(ResponsiveReactGridLayout, _React$Component);

  function ResponsiveReactGridLayout() {
    var _Object$getPrototypeO;

    var _temp, _this, _ret;

    _classCallCheck(this, ResponsiveReactGridLayout);

    for (var _len = arguments.length, args = Array(_len), _key = 0; _key < _len; _key++) {
      args[_key] = arguments[_key];
    }

    return _ret = (_temp = (_this = _possibleConstructorReturn(this, (_Object$getPrototypeO = Object.getPrototypeOf(ResponsiveReactGridLayout)).call.apply(_Object$getPrototypeO, [this].concat(args))), _this), _this.state = _this.generateInitialState(), _temp), _possibleConstructorReturn(_this, _ret);
  }

  // This should only include propTypes needed in this code; RGL itself
  // will do validation of the rest props passed to it.

  _createClass(ResponsiveReactGridLayout, [{
    key: 'generateInitialState',
    value: function generateInitialState() {
      var _props = this.props;
      var width = _props.width;  // PROBLEM IS THIS LINE
      var breakpoints = _props.breakpoints;
      var layouts = _props.layouts;
      var verticalCompact = _props.verticalCompact;
      var cols = _props.cols;

      var breakpoint = (0, _responsiveUtils.getBreakpointFromWidth)(breakpoints, width);
      var colNo = (0, _responsiveUtils.getColsFromBreakpoint)(breakpoint, cols);
      // Get the initial layout. This can tricky; we try to generate one however possible if one doesn't exist
      // for this layout.
      var initialLayout = (0, _responsiveUtils.findOrGenerateResponsiveLayout)(layouts, breakpoints, breakpoint, breakpoint, colNo, verticalCompact);

      return {
        layout: initialLayout,
        breakpoint: breakpoint,
        cols: colNo
      };
    }
  }, {
    key: 'componentWillReceiveProps',
    value: function componentWillReceiveProps(nextProps) {

      if (nextProps.width != this.props.width) {
        var newBreakpoint = nextProps.breakpoint || (0, _responsiveUtils.getBreakpointFromWidth)(nextProps.breakpoints, nextProps.width);
        this.onWidthChange(nextProps.width, newBreakpoint);
      }

      // Allow parent to set breakpoint directly.
      if (nextProps.breakpoint !== this.props.breakpoint) {
        this.onWidthChange(nextProps.width, nextProps.breakpoint);
      }

      // Allow parent to set layouts directly.
      if (!(0, _lodash2.default)(nextProps.layouts, this.props.layouts)) {
        var _state = this.state;
        var _breakpoint = _state.breakpoint;
        var _cols = _state.cols;

        // Since we're setting an entirely new layout object, we must generate a new responsive layout
        // if one does not exist.

        var newLayout = (0, _responsiveUtils.findOrGenerateResponsiveLayout)(nextProps.layouts, nextProps.breakpoints, _breakpoint, _breakpoint, _cols, nextProps.verticalLayout);
        this.setState({ layout: newLayout });
      }
    }

    /**
     * When the width changes work through breakpoints and reset state with the new width & breakpoint.
     * Width changes are necessary to figure out the widget widths.
     */

  }, {
    key: 'onWidthChange',
    value: function onWidthChange(width, newBreakpoint) {
      var _props2 = this.props;
      var breakpoints = _props2.breakpoints;
      var verticalLayout = _props2.verticalLayout;
      var verticalCompact = _props2.verticalCompact;
      var cols = _props2.cols;

      var lastBreakpoint = this.state.breakpoint;

      // Breakpoint change
      if (lastBreakpoint !== newBreakpoint) {

        // Store the current layout
        var layouts = this.props.layouts;
        layouts[lastBreakpoint] = JSON.parse(JSON.stringify(this.state.layout));

        // Find or generate a new one.
        var newCols = (0, _responsiveUtils.getColsFromBreakpoint)(newBreakpoint, cols);
        var _layout = (0, _responsiveUtils.findOrGenerateResponsiveLayout)(layouts, breakpoints, newBreakpoint, lastBreakpoint, newCols, verticalLayout);

        // This adds missing items.
        _layout = (0, _utils.synchronizeLayoutWithChildren)(_layout, this.props.children, newCols, verticalCompact);

        // Store this new layout as well.
        layouts[newBreakpoint] = _layout;

        // callbacks
        this.props.onLayoutChange(_layout, layouts);
        this.props.onBreakpointChange(newBreakpoint, newCols);
        this.props.onWidthChange(width, this.props.margin, newCols);

        this.setState({ breakpoint: newBreakpoint, layout: _layout, cols: newCols });
      }
    }
  }, {
    key: 'render',
    value: function render() {
      var _props3 = this.props;
      var breakpoint = _props3.breakpoint;
      var breakpoints = _props3.breakpoints;
      var cols = _props3.cols;
      var layouts = _props3.layouts;
      var onBreakpointChange = _props3.onBreakpointChange;
      var onLayoutChange = _props3.onLayoutChange;
      var onWidthChange = _props3.onWidthChange;

      var other = _objectWithoutProperties(_props3, ['breakpoint', 'breakpoints', 'cols', 'layouts', 'onBreakpointChange', 'onLayoutChange', 'onWidthChange']);

      // wrap layouts so we do not need to pass layouts to child

      var onLayoutChangeWrapper = function onLayoutChangeWrapper(layout) {
        return onLayoutChange(layout, layouts);
      };

      return _react2.default.createElement(_ReactGridLayout2.default, _extends({}, other, {
        onLayoutChange: onLayoutChangeWrapper,
        layout: this.state.layout,
        cols: this.state.cols
      }));
    }
  }]);

  return ResponsiveReactGridLayout;
}(_react2.default.Component);

ResponsiveReactGridLayout.propTypes = {

  //
  // Basic props
  //

  // Optional, but if you are managing width yourself you may want to set the breakpoint
  // yourself as well.
  breakpoint: _react2.default.PropTypes.string,

  // {name: pxVal}, e.g. {lg: 1200, md: 996, sm: 768, xs: 480}
  breakpoints: _react2.default.PropTypes.object,

  // # of cols. This is a breakpoint -> cols map
  cols: _react2.default.PropTypes.object,

  // layouts is an object mapping breakpoints to layouts.
  // e.g. {lg: Layout, md: Layout, ...}
  layouts: function layouts(props) {
    _react2.default.PropTypes.object.isRequired.apply(this, arguments);
    Object.keys(props.layouts).forEach(function (key) {
      return (0, _utils.validateLayout)(props.layouts[key], 'layouts.' + key);
    });
  },

  // The width of this component.
  // Required in this propTypes stanza because generateInitialState() will fail without it.
  width: _react2.default.PropTypes.number.isRequired,

  //
  // Callbacks
  //

  // Calls back with breakpoint and new # cols
  onBreakpointChange: _react2.default.PropTypes.func,

  // Callback so you can save the layout.
  // Calls back with (currentLayout, allLayouts). allLayouts are keyed by breakpoint.
  onLayoutChange: _react2.default.PropTypes.func,

  // Calls back with (containerWidth, margin, cols)
  onWidthChange: _react2.default.PropTypes.func
};
ResponsiveReactGridLayout.defaultProps = {
  breakpoints: { lg: 1200, md: 996, sm: 768, xs: 480, xxs: 0 },
  cols: { lg: 12, md: 10, sm: 6, xs: 4, xxs: 2 },
  layouts: {},
  onBreakpointChange: noop,
  onLayoutChange: noop,
  onWidthChange: noop
};
exports.default = ResponsiveReactGridLayout;

Hmm. This could be a binding issue (this).
Could you fix the code above so it's complete?

this is part of the latest codebase for react-grid-layout... version 0.10.7

sorry... I just noticed my mistake

Just updated

Could you do me a favor (since you have easy access to a Windows machine with IE9/10):

Check out RGL
npm install
`npm install babel-preset-es2015-loose'
Replace es2015 in the .babelrc with es2015-loose
npm run build and copy the file over
Test with IE9/10

If it works, I'll move the official package over to loose mode.

Well first of all, I get the following when running npm run build...
ERROR in ./lib/ReactGridLayout.jsx
Module parse failed: /srv/www/testnew-my.quietyme.local/site/frontend/node_modules/react-grid-layout.checkedout/lib/ReactGridLayout.jsx Line 2: Unexpected token
You may need an appropriate loader to handle this file type.
| // @flow
| import React, {PropTypes} from 'react';
| import isEqual from 'lodash.isequal';
| import {autoBindHandlers, bottom, clone, compact, getLayoutItem, moveElement,
 @ ./index-dev.js 1:17-49

ERROR in ./lib/utils.js
Module parse failed: /srv/www/testnew-my.quietyme.local/site/frontend/node_modules/react-grid-layout.checkedout/lib/utils.js Line 2: Unexpected token
You may need an appropriate loader to handle this file type.
| // @flow
| export type LayoutItem = {w: number, h: number, x: number, y: number, i: string, placeholder?: boolean, moved?: boolean};
| export type Layout = Array<LayoutItem>;
| export type Position = {left: number, top: number, width: number, height: number};
 @ ./index-dev.js 2:23-45

ERROR in ./lib/ResponsiveReactGridLayout.jsx
Module parse failed: /srv/www/testnew-my.quietyme.local/site/frontend/node_modules/react-grid-layout.checkedout/lib/ResponsiveReactGridLayout.jsx Line 2: Unexpected token
You may need an appropriate loader to handle this file type.
| // @flow
| import React from 'react';
| import isEqual from 'lodash.isequal';
|
 @ ./index-dev.js 3:28-70

ERROR in ./lib/responsiveUtils.js
Module parse failed: /srv/www/testnew-my.quietyme.local/site/frontend/node_modules/react-grid-layout.checkedout/lib/responsiveUtils.js Line 3: Unexpected token
You may need an appropriate loader to handle this file type.
| // @flow
|
| import {compact, correctBounds} from './utils';
|
| import type {Layout} from './utils';
 @ ./index-dev.js 4:34-66

ERROR in ./lib/components/WidthProvider.jsx
Module parse failed: /srv/www/testnew-my.quietyme.local/site/frontend/node_modules/react-grid-layout.checkedout/lib/components/WidthProvider.jsx Line 3: Unexpected token
You may need an appropriate loader to handle this file type.
| // @noflow
| // Intentional; Flow can't handle the bind on L20
| import React from "react";
| import ReactDOM from 'react-dom';
|
 @ ./index-dev.js 5:31-72

Additionally, I get the following in the browser...
Uncaught TypeError: Cannot read property 'firstChild' of undefined
It's coming from the following line in the function "findComponentRoot"... presumably in the React core someplace. It does not appear to be coming directly from your codebase, and there isn't anything in the stack trace beyond what I have here.
firstChildren[0] = deepestAncestor.firstChild;

Before that, I also get the following:
Warning: React can't find the root component node for data-reactid value .0.0.1.$/=10.0.0. If you're seeing this message, it probably means that you've loaded two copies of React on the page. At this time, only a single copy of React can be loaded at a time.

That being said, I don't get the same errors as before and the grid displays more fully than it did before (albeit with different problems this time), so likely we're making progress. :-D It does now work the same way in IE10 as it does in Chrome and elsewhere, but now they're all broken to some degree.

0.10.8 is built with Loose Mode. Could you try it out?

narwold<EMAIL_ADDRESS>February 18, 2016 at 10:53 AM
That being said, I don't get the same errors as before and the grid
displays more fully than it did before (albeit with different problems
this time), so likely we're making progress. :-D It does now work the
same way in IE10 as it does in Chrome and elsewhere, but now they're
all broken to some degree.
—
Reply to this email directly or view it on GitHub
https://github.com/STRML/react-grid-layout/issues/168#issuecomment-185813493.
narwold<EMAIL_ADDRESS>February 17, 2016 at 4:49 PM
On IE9/IE10, I'm getting the following error:
SCRIPT5007: Unable to get property 'width' of undefined or null reference
Looking at your codebase, it appears it's referring to the section of
code below (specifically the line I've commented below). I did a
console.log(this.props), and I got 'undefined', so it doesn't appear
any props are making it through, despite the fact that I've assigned
props and they work just fine on IE11 and non-IE browsers.
Please advise.
_createClass(ResponsiveReactGridLayout, [{
key: 'generateInitialState',
value: function generateInitialState() {
var _props = this.props;
var width = _props.width; // this line is the problem
var breakpoints = _props.breakpoints;
var layouts = _props.layouts;
var verticalCompact = _props.verticalCompact;
var cols = _props.cols;
|var breakpoint = (0,
_responsiveUtils.getBreakpointFromWidth)(breakpoints, width);
var colNo = (0, _responsiveUtils.getColsFromBreakpoint)(breakpoint, cols);
// Get the initial layout. This can tricky; we try to generate one
however possible if one doesn't exist
// for this layout.
var initialLayout = (0,
_responsiveUtils.findOrGenerateResponsiveLayout)(layouts, breakpoints,
breakpoint, breakpoint, colNo, verticalCompact);
return {
layout: initialLayout,
breakpoint: breakpoint,
cols: colNo
};
}
|
}, {
—
Reply to this email directly or view it on GitHub
https://github.com/STRML/react-grid-layout/issues/168.

This is looking good. Thanks much!

I am getting "unable to get property 'width' of undefined or null reference using 0.10.8 in both IE 9 and 10, it is pointing to:
  function Resizable() {
    var _Object$getPrototypeO;

    var _temp, _this, _ret;

    _classCallCheck(this, Resizable);

    for (var _len = arguments.length, args = Array(_len), _key = 0; _key < _len; _key++) {
      args[_key] = arguments[_key];
    }
    return _ret = (_temp = (_this = _possibleConstructorReturn(this, (_Object$getPrototypeO = Object.getPrototypeOf(Resizable)).call.apply(_Object$getPrototypeO, [this].concat(args))), _this), _this.state = {
      resizing: false,
      **width: _this.props.width, height: _this.props.height,**
      slackW: 0, slackH: 0
    }, _temp), _possibleConstructorReturn(_this, _ret);
  }

Update: I went ahead and did the same configuration you mentioned onto the react-resizeable library and it works perfectly now. Although I still can't drag and drop with ie10/11 at least the widgets show up and can move things around in IE 11+

@finalfreq This should be fixed in the latest release and with Draggable v2 but I don't have easy IE9 access. Could you test?

@STRML tested and it is working like a charm 👍


Exception within runnerjob.py that prevents execution of jobs

Hello, there was an issue while running shadowclone.py  to resolve dns records. From the terminal output below, you can see that 4 functions activated however it looks like the service was unable to execute the run time. Upon inspection of poll output, there was an exception within runnerjob.py.
Exception
    Traceback (most recent call last):
      File "/function/lithops/worker/jobrunner.py", line 228, in run
        result = func(**data)
      File "shadowclone.py", line 94, in execute_command
    AttributeError: 'NoneType' object has no attribute 'replace'

Terminal Output
user@Administrators-MacBook-Pro-7 ~/arsenal/ShadowClone                                                                                [17:10:40]
(recon-venv) > $ sudo python shadowclone.py -i  ~/Desktop/compiled.txt  --split 3000000 -o ~/Desktop/test.txt -c "/go/bin/puredns resolve --resolvers ~/Desktop/resolvers.txt"                                         [±main ●●]
2023-02-11 17:11:33,138  [INFO] Splitting input file into chunks of 3000000 lines
2023-02-11 17:11:35,729  [INFO] Uploading chunks to storage
2023-02-11 17:14:08,275 [INFO] lithops.config -- Lithops v2.6.0
2023-02-11 17:14:08,301 [INFO] lithops.storage.backends.aws_s3.aws_s3 -- S3 client created - Region: us-west-1
2023-02-11 17:14:08,804 [INFO] lithops.serverless.backends.aws_lambda.aws_lambda -- AWS Lambda client created - Region: us-west-1
2023-02-11 17:14:08,805 [INFO] lithops.invokers -- ExecutorID f049ae-0 | JobID M000 - Selected Runtime: lithops_v2-6-0_gupt/sc-runtime - 512MB
2023-02-11 17:14:10,253 [INFO] lithops.invokers -- ExecutorID f049ae-0 | JobID M000 - Starting function invocation: execute_command() - Total: 4 activations
2023-02-11 17:14:10,292 [INFO] lithops.invokers -- ExecutorID f049ae-0 | JobID M000 - View execution logs at /private/tmp/lithops/logs/f049ae-0-M000.log
2023-02-11 17:14:10,295 [INFO] lithops.wait -- ExecutorID f049ae-0 - Getting results from 4 function activations

    0%|                                                                                | 0/4

2023-02-11 17:14:15,356 [INFO] lithops.executors -- ExecutorID f049ae-0 - Cleaning temporary data
2023-02-11 17:14:15,366  [ERROR] Could not execute the runtime.

Poll output
user@Administrators-MacBook-Pro-7 ~/arsenal/ShadowClone                                                                                [17:15:27]
(recon-venv) > $ lithops logs poll                                                                                [±main ●●]
Activation: 'python3' (6157671bc704)
[
    2023-02-11 15:59:51,378 [INFO] lithops.worker.handler -- Lithops v2.6.0 - Starting Localhost execution
    2023-02-11 15:59:51,379 [INFO] lithops.worker.handler -- Execution ID: c0571d-0-A000/00000
    2023-02-11 15:59:51,379 [INFO] lithops.storage.backends.localhost.localhost -- Localhost storage client created
    2023-02-11 15:59:51,385 [INFO] lithops.worker.jobrunner -- Going to execute 'hello()'
    ---------------------- FUNCTION LOG ----------------------
    ----------------------------------------------------------
    2023-02-11 15:59:51,386 [INFO] lithops.worker.jobrunner -- Success function execution
    2023-02-11 15:59:51,387 [INFO] lithops.worker.jobrunner -- Storing function result - Size: 41.0B
    2023-02-11 15:59:51,388 [INFO] lithops.worker.jobrunner -- Process finished
]

Activation: 'python38' (e7443555-f092-40ea-81a5-b4e863a77f7f)
[
    2023-02-12 00:47:42,450 [INFO] lithops.worker.handler -- Lithops v2.6.0 - Starting AWS Lambda execution
    2023-02-12 00:47:42,450 [INFO] lithops.worker.handler -- Execution ID: ec9eb2-0-A000/00000
    2023-02-12 00:47:42,472 [INFO] lithops.storage.backends.aws_s3.aws_s3 -- S3 client created - Region: us-west-1
    2023-02-12 00:47:42,554 [INFO] lithops.worker.jobrunner -- Going to execute 'hello()'
    ---------------------- FUNCTION LOG ----------------------
    ----------------------------------------------------------
    2023-02-12 00:47:42,554 [INFO] lithops.worker.jobrunner -- Success function execution
    2023-02-12 00:47:42,555 [INFO] lithops.worker.jobrunner -- Storing function result - Size: 41.0B
    2023-02-12 00:47:42,595 [INFO] lithops.worker.jobrunner -- Process finished
]

Activation: 'lithops_v2-6-0_gupt/sc-runtime' (9c11515c-055e-49d6-87b3-94138a4c106e)
[
    2023-02-12 01:01:44,053 [INFO] lithops.worker.handler -- Lithops v2.6.0 - Starting AWS Lambda execution
    2023-02-12 01:01:44,054 [INFO] lithops.worker.handler -- Execution ID: c94cbf-0-M000/00000
    2023-02-12 01:01:44,061 [INFO] lithops.storage.backends.aws_s3.aws_s3 -- S3 client created - Region: us-west-1
    2023-02-12 01:01:45,015 [INFO] lithops.worker.jobrunner -- Getting dataset from aws_s3://runtime-shadowclone/959adbc3-b543-4015-ae2b-90762961b742
    2023-02-12 01:01:45,121 [INFO] lithops.worker.jobrunner -- Going to execute 'execute_command()'
    ---------------------- FUNCTION LOG ----------------------
    ----------------------- EXCEPTION !-----------------------
    Traceback (most recent call last):
      File "/function/lithops/worker/jobrunner.py", line 228, in run
        result = func(**data)
      File "shadowclone.py", line 94, in execute_command
    AttributeError: 'NoneType' object has no attribute 'replace'
    ----------------------------------------------------------
    2023-02-12 01:01:47,095 [INFO] lithops.worker.jobrunner -- Process finished
]

Activation: 'lithops_v2-6-0_gupt/sc-runtime' (fbcb2251-f728-479f-b263-6050f2641d2f)
[
    2023-02-12 01:01:44,392 [INFO] lithops.worker.handler -- Lithops v2.6.0 - Starting AWS Lambda execution
    2023-02-12 01:01:44,392 [INFO] lithops.worker.handler -- Execution ID: c94cbf-0-M000/00003
    2023-02-12 01:01:44,412 [INFO] lithops.storage.backends.aws_s3.aws_s3 -- S3 client created - Region: us-west-1
    2023-02-12 01:01:45,370 [INFO] lithops.worker.jobrunner -- Getting dataset from aws_s3://runtime-shadowclone/337ac1fb-35b3-4fd3-8af3-4d3beb75a27d
    2023-02-12 01:01:45,493 [INFO] lithops.worker.jobrunner -- Going to execute 'execute_command()'
    ---------------------- FUNCTION LOG ----------------------
    ----------------------- EXCEPTION !-----------------------
    Traceback (most recent call last):
      File "/function/lithops/worker/jobrunner.py", line 228, in run
        result = func(**data)
      File "shadowclone.py", line 94, in execute_command
    AttributeError: 'NoneType' object has no attribute 'replace'
    ----------------------------------------------------------
    2023-02-12 01:01:47,507 [INFO] lithops.worker.jobrunner -- Process finished
]

@WarrDaddy , can you try by providing absolute path to the -i and -o flags instead of  short paths like ~/Desktop/compiled.txt.
Your command -c also looks wrong to me. you have to specify the placeholder {INPUT} somewhere for providing the chunk as input to the tool. There is no ~/Desktop/resolvers.txt inside the container too. take a look at the wiki for few examples

Hey there @fyoorer. Thanks for the response. I've corrected my command and the issue still persist.

user@Administrators-MacBook-Pro-7 ~/arsenal/ShadowClone                                                                                [23:38:42]
(recon-venv) > $ python shadowclone.py -i /Users/user/Desktop/compiled.txt --split 3000000 -o /Users/user/Desktop/shadowoutput.txt -c "/go/bin/puredns resolve {INPUT} --resolvers /function/resolvers.txt"                              [±main ●●]
2023-02-11 23:39:11,778  [INFO] Splitting input file into chunks of 3000000 lines
2023-02-11 23:39:14,301  [INFO] Uploading chunks to storage
2023-02-11 23:39:14,469 [INFO] lithops.config -- Lithops v2.6.0
2023-02-11 23:39:14,480 [INFO] lithops.storage.backends.aws_s3.aws_s3 -- S3 client created - Region: us-west-1
2023-02-11 23:39:14,951 [INFO] lithops.serverless.backends.aws_lambda.aws_lambda -- AWS Lambda client created - Region: us-west-1
2023-02-11 23:39:14,954 [INFO] lithops.invokers -- ExecutorID 170467-0 | JobID M000 - Selected Runtime: lithops_v2-6-0_gupt/sc-runtime - 512MB
2023-02-11 23:39:16,651 [INFO] lithops.invokers -- ExecutorID 170467-0 | JobID M000 - Starting function invocation: execute_command() - Total: 4 activations
2023-02-11 23:39:16,655 [INFO] lithops.invokers -- ExecutorID 170467-0 | JobID M000 - View execution logs at /private/var/folders/q2/v0knyw_n45598tj6xm9clf4r0000gp/T/lithops/logs/170467-0-M000.log
2023-02-11 23:39:16,656 [INFO] lithops.wait -- ExecutorID 170467-0 - Getting results from 4 function activations

  0%|                                                                                | 0/4

2023-02-11 23:39:21,678 [INFO] lithops.executors -- ExecutorID 170467-0 - Cleaning temporary data
2023-02-11 23:39:21,683  [ERROR] Could not execute the runtime.

Log poll output
(recon-venv) > $ lithops  logs poll                                                                                [±main ●●]
Activation: 'lithops_v2-6-0_gupt/sc-runtime' (fc364e0d-2132-4594-8da7-5281ad7f2a20)
[
    2023-02-12 07:38:28,561 [INFO] lithops.worker.handler -- Lithops v2.6.0 - Starting AWS Lambda execution
    2023-02-12 07:38:28,561 [INFO] lithops.worker.handler -- Execution ID: e90409-0-M000/00000
    2023-02-12 07:38:28,584 [INFO] lithops.storage.backends.aws_s3.aws_s3 -- S3 client created - Region: us-west-1
    2023-02-12 07:38:29,488 [INFO] lithops.worker.jobrunner -- Getting dataset from aws_s3://runtime-shadowclone/1a6d7cc8-ac31-44fb-b650-22699f5e927a
    2023-02-12 07:38:29,612 [INFO] lithops.worker.jobrunner -- Going to execute 'execute_command()'
    ---------------------- FUNCTION LOG ----------------------
    2023-02-12 07:38:41,270 [ERROR] lithops.worker.handler -- No completion message received from JobRunner process
    ----------------------- EXCEPTION !-----------------------
    Traceback (most recent call last):
      File "/function/lithops/worker/handler.py", line 202, in run_job
        raise MemoryError('HANDLER', msg)
    MemoryError: ('HANDLER', 'Function exceeded maximum memory and was killed')
    ----------------------------------------------------------
]

Activation: 'lithops_v2-6-0_gupt/sc-runtime' (4a47fb12-732a-4c28-80a5-0ad741ed78e1)
[
    2023-02-12 07:39:17,002 [INFO] lithops.worker.handler -- Lithops v2.6.0 - Starting AWS Lambda execution
    2023-02-12 07:39:17,002 [INFO] lithops.worker.handler -- Execution ID: 170467-0-M000/00003
    2023-02-12 07:39:17,008 [INFO] lithops.storage.backends.aws_s3.aws_s3 -- S3 client created - Region: us-west-1
    2023-02-12 07:39:17,849 [INFO] lithops.worker.jobrunner -- Getting dataset from aws_s3://runtime-shadowclone/652a70ec-2c67-4fe8-ab64-ea5c461cce67
    2023-02-12 07:39:17,961 [INFO] lithops.worker.jobrunner -- Going to execute 'execute_command()'
    ---------------------- FUNCTION LOG ----------------------
    ----------------------- EXCEPTION !-----------------------
    Traceback (most recent call last):
      File "/function/lithops/worker/jobrunner.py", line 228, in run
        result = func(**data)
      File "shadowclone.py", line 94, in execute_command
    AttributeError: 'NoneType' object has no attribute 'replace'
    ----------------------------------------------------------
    2023-02-12 07:39:18,986 [INFO] lithops.worker.jobrunner -- Process finished
]

@WarrDaddy should be fixed now, can you give it another try. Please let me know if you still encounter these errors


Some machines arent easily fitting through airlocks

for example thrusters cannot easily fit through, so you need either wrench and unwrench it either manually trying to alling it

Context Menu -> Rotate

Context Menu -> Rotate

Rotating won't help, you need to anchor and unanchor.

Context Menu -> Rotate

Rotating won't help, you need to anchor and unanchor.

I think that one of the rotation buttons resets the angle to something that aligns with anchoring

Context Menu -> Rotate

Rotating won't help, you need to anchor and unanchor.

A good example where anchoring won't help either is when salvage finds thrusters in a wreck. Pulled items are overall much harder to control in space as they maintain momentum, so it's really easy to have a thruster spin along walls rather unhelpfully.

Behold:
https://github.com/space-wizards/space-station-14/assets/62134478/2262a4af-df70-4e8f-88d7-9bf9217e3bc8

its also the same for altars among with other things


protobufc: fix cross compilation

Description of changes

Things done

Built on platform(s)

[x] x86_64-linux
[x] riscv64-linux (cross)
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
23.05 Release Notes (or backporting 22.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 208083 run on x86_64-linux 1

3 packages marked as broken and skipped:

postgresql14Packages.cstore_fdw
postgresql15Packages.cstore_fdw
postgresqlPackages.cstore_fdw

78 packages built:

acquire (python310Packages.acquire)
anbox
apkid
arch-install-scripts
bash-preexec
bashup-events32
bashup-events44
bats
buildah
collectd
cri-o
criu
crun
dgoss
fwupd
git-ftp
gnome-firmware
gnome.gnome-software
imhex
ix
kismet
krunvm
libgadu
libivykis
libnats-c
libsForQt5.discover (plasma5Packages.discover)
lxd
mapserver
mons
msmtp
nsjail
ocserv
packcc
pantheon.elementary-greeter
pantheon.switchboard-plug-about
pantheon.switchboard-with-plugs
pantheon.wingpanel-applications-menu
pantheon.wingpanel-with-indicators
pdf2odt
podman
postgresql11Packages.cstore_fdw
postgresql11Packages.postgis
postgresql12Packages.cstore_fdw
postgresql12Packages.postgis
postgresql13Packages.cstore_fdw
postgresql13Packages.postgis
postgresqlPackages.postgis (postgresql14Packages.postgis)
postgresql15Packages.postgis
protobufc
purple-googlechat
purple-hangouts
python310Packages.dissect
python310Packages.dissect-target
python310Packages.karton-autoit-ripper
python310Packages.karton-config-extractor
python310Packages.karton-yaramatcher
python310Packages.malduck
python310Packages.yara-python
python39Packages.acquire
python39Packages.dissect
python39Packages.dissect-target
python39Packages.karton-autoit-ripper
python39Packages.karton-config-extractor
python39Packages.karton-yaramatcher
python39Packages.malduck
python39Packages.yara-python
riemann_c_client
shunit2
spyre
stacs
syslogng
ttwatch
umurmur
unbound-full
volatility3
yadm
yara
zbackup


Questions about performance results

Hi!
Thanks for your great work in this repo.
I'm curious about the accuracy of your provided ResNet and ViT on the NCT-CRC-HE-100K dataset after training with your codebase.

Hi :D
Sure, here's the performance after 40 epochs:
ResNet: 1 for trainset, 0.93 for testset
ViT: 0.929 for trainset, 0.886 for testset
Hope it helps!

Thanks for your help!


Docs: dockerTools

https://github.com/NixOS/nixpkgs/blob/5622b6b6feb669edc227aaf000413d5b593d4051/doc/builders/images/dockertools.xml#L321

"popular"

"unpopular"

What the heck? By what "measurement"? :wink:

This issue has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/dockertools-imprecisions/8770/1

My prioritization algorithm is a simple graph-based popularity contest. The idea is to weight each node more heavily the deeper and more references they have.

out of: https://grahamc.com/blog/nix-and-layered-docker-images


readthedoc from pyproject.toml

Hi. Thank you for this great " Hypermodern Python" series.
With setup like this:
# pyproject.toml
...
[tool.poetry.dependencies]
# project deps
...
python = "<3.8,>=3.7"
# dev deps
black = { version = "^19.10b0", optional = true }
coverage = {version = "^5.1", optional = true }
...
sphinx-rtd-theme = { version = "^0.4.3", optional = true }
toml = { version = "^0.10.0", optional = true }
typeguard = { version = "^2.7.1", optional = true }

[tool.poetry.extras]
dev = ["black", "coverage", ... "toml", "typeguard"]
docs = ["m2r", "sphinx", "sphinx-autodoc-typehints", "sphinx-rtd-theme", "toml"]

# readthedocs.yml
version: 2

sphinx:
  configuration: docs/conf.py

python:
  version: 3.7
  install:
    - method: pip
      path: .
      extra_requirements:
        - docs

we can

read pyproject.toml from docs/conf.py for metadata
use pyproject.toml "as is" without additional docs/requirements.txt file

That's right, but the docs dependencies won't be pinned then. Read the Docs builds your project with pip, and pip uses Poetry to build a wheel, not for installing. The wheel does not include version pins.
The idea to read pyproject.toml from docs/conf.py for metadata is interesting. Do you have an example? If you install your package for building docs, you could also use importlib.metadata for that, though.
Many projects use extras for development dependencies, but I'm not a fan of that. I think extras are best used for optional user-facing features.

Do you have an example?

That's right, but the docs dependencies won't be pinned then.

That's right, but the docs dependencies won't be pinned then.

See docs build log for the repo above https://readthedocs.org/api/v2/build/10913417.txt Looks like all deps pinned

Reading the package metadata in docs/conf.py is a great idea!
As I said earlier, take a look at importlib.metadata. It allows you to read the package metadata directly from your installed package, the way every other packaging tool sees it, using only the standard library, without any need for parsing.
To use it, you also need to configure Read the Docs to use Python 3.8 via .readthedocs.yml.

Looks like all deps pinned

I was really hoping I would be wrong 😄But the deps are not pinned. Direct dependencies are constrained to major versions, as specified in pyproject.toml. Indirect dependencies ("subdependencies") are constrained by whatever depends on them, if at all. None of the dependencies are locked to a specific version.
Took me a while to find the right pip invocation in the log file, so I pasted it below:

pip output

/home/docs/checkouts/readthedocs.org/user_builds/multiconsumers-queue/envs/stable/bin/python -m pip install --upgrade --upgrade-strategy eager --no-cache-dir .[docs]
Processing /home/docs/checkouts/readthedocs.org/user_builds/multiconsumers-queue/checkouts/stable
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
    Preparing wheel metadata: started
    Preparing wheel metadata: finished with status 'done'
Collecting loguru<0.5.0,>=0.4.1
  Downloading loguru-0.4.1-py3-none-any.whl (54 kB)
Collecting attrs<20.0.0,>=19.3.0
  Downloading attrs-19.3.0-py2.py3-none-any.whl (39 kB)
Collecting arrow<0.16.0,>=0.15.5
  Downloading arrow-0.15.5-py2.py3-none-any.whl (46 kB)
Collecting toml<0.11.0,>=0.10.0; extra == "dev" or extra == "docs"
  Downloading toml-0.10.0-py2.py3-none-any.whl (25 kB)
Collecting sphinx-autodoc-typehints<2.0.0,>=1.10.3; extra == "docs"
  Downloading sphinx_autodoc_typehints-1.10.3-py3-none-any.whl (8.4 kB)
Collecting m2r<0.3.0,>=0.2.1; extra == "docs"
  Downloading m2r-0.2.1.tar.gz (16 kB)
Requirement already up-to-date: sphinx-rtd-theme<0.5.0,>=0.4.3; extra == "docs" in /home/docs/checkouts/readthedocs.org/user_builds/multiconsumers-queue/envs/stable/lib/python3.7/site-packages (from multiconsumers-queue==0.1.4) (0.4.3)
Collecting sphinx==2.3.1; extra == "docs"
  Downloading Sphinx-2.3.1-py3-none-any.whl (2.7 MB)
Collecting python-dateutil
  Downloading python_dateutil-2.8.1-py2.py3-none-any.whl (227 kB)
Collecting mistune
  Downloading mistune-0.8.4-py2.py3-none-any.whl (16 kB)
Collecting docutils
  Downloading docutils-0.16-py2.py3-none-any.whl (548 kB)
Collecting sphinxcontrib-serializinghtml
  Downloading sphinxcontrib_serializinghtml-1.1.4-py2.py3-none-any.whl (89 kB)
Collecting sphinxcontrib-applehelp
  Downloading sphinxcontrib_applehelp-1.0.2-py2.py3-none-any.whl (121 kB)
Collecting Pygments>=2.0
  Downloading Pygments-2.6.1-py3-none-any.whl (914 kB)
Collecting sphinxcontrib-qthelp
  Downloading sphinxcontrib_qthelp-1.0.3-py2.py3-none-any.whl (90 kB)
Requirement already up-to-date: alabaster<0.8,>=0.7 in /home/docs/checkouts/readthedocs.org/user_builds/multiconsumers-queue/envs/stable/lib/python3.7/site-packages (from sphinx==2.3.1; extra == "docs"->multiconsumers-queue==0.1.4) (0.7.12)
Collecting setuptools
  Downloading setuptools-46.1.3-py3-none-any.whl (582 kB)
Collecting sphinxcontrib-devhelp
  Downloading sphinxcontrib_devhelp-1.0.2-py2.py3-none-any.whl (84 kB)
Collecting sphinxcontrib-jsmath
  Downloading sphinxcontrib_jsmath-1.0.1-py2.py3-none-any.whl (5.1 kB)
Requirement already up-to-date: Jinja2>=2.3 in /home/docs/checkouts/readthedocs.org/user_builds/multiconsumers-queue/envs/stable/lib/python3.7/site-packages (from sphinx==2.3.1; extra == "docs"->multiconsumers-queue==0.1.4) (2.11.2)
Requirement already up-to-date: requests>=2.5.0 in /home/docs/checkouts/readthedocs.org/user_builds/multiconsumers-queue/envs/stable/lib/python3.7/site-packages (from sphinx==2.3.1; extra == "docs"->multiconsumers-queue==0.1.4) (2.23.0)
Collecting sphinxcontrib-htmlhelp
  Downloading sphinxcontrib_htmlhelp-1.0.3-py2.py3-none-any.whl (96 kB)
Requirement already up-to-date: babel!=2.0,>=1.3 in /home/docs/checkouts/readthedocs.org/user_builds/multiconsumers-queue/envs/stable/lib/python3.7/site-packages (from sphinx==2.3.1; extra == "docs"->multiconsumers-queue==0.1.4) (2.8.0)
Requirement already up-to-date: snowballstemmer>=1.1 in /home/docs/checkouts/readthedocs.org/user_builds/multiconsumers-queue/envs/stable/lib/python3.7/site-packages (from sphinx==2.3.1; extra == "docs"->multiconsumers-queue==0.1.4) (2.0.0)
Requirement already up-to-date: packaging in /home/docs/checkouts/readthedocs.org/user_builds/multiconsumers-queue/envs/stable/lib/python3.7/site-packages (from sphinx==2.3.1; extra == "docs"->multiconsumers-queue==0.1.4) (20.3)
Requirement already up-to-date: imagesize in /home/docs/checkouts/readthedocs.org/user_builds/multiconsumers-queue/envs/stable/lib/python3.7/site-packages (from sphinx==2.3.1; extra == "docs"->multiconsumers-queue==0.1.4) (1.2.0)
Requirement already up-to-date: six>=1.5 in /home/docs/checkouts/readthedocs.org/user_builds/multiconsumers-queue/envs/stable/lib/python3.7/site-packages (from python-dateutil->arrow<0.16.0,>=0.15.5->multiconsumers-queue==0.1.4) (1.14.0)
Requirement already up-to-date: MarkupSafe>=0.23 in /home/docs/checkouts/readthedocs.org/user_builds/multiconsumers-queue/envs/stable/lib/python3.7/site-packages (from Jinja2>=2.3->sphinx==2.3.1; extra == "docs"->multiconsumers-queue==0.1.4) (1.1.1)
Requirement already up-to-date: certifi>=2017.4.17 in /home/docs/checkouts/readthedocs.org/user_builds/multiconsumers-queue/envs/stable/lib/python3.7/site-packages (from requests>=2.5.0->sphinx==2.3.1; extra == "docs"->multiconsumers-queue==0.1.4) (2<IP_ADDRESS>)
Requirement already up-to-date: chardet<4,>=3.0.2 in /home/docs/checkouts/readthedocs.org/user_builds/multiconsumers-queue/envs/stable/lib/python3.7/site-packages (from requests>=2.5.0->sphinx==2.3.1; extra == "docs"->multiconsumers-queue==0.1.4) (3.0.4)
Requirement already up-to-date: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /home/docs/checkouts/readthedocs.org/user_builds/multiconsumers-queue/envs/stable/lib/python3.7/site-packages (from requests>=2.5.0->sphinx==2.3.1; extra == "docs"->multiconsumers-queue==0.1.4) (1.25.9)
Requirement already up-to-date: idna<3,>=2.5 in /home/docs/checkouts/readthedocs.org/user_builds/multiconsumers-queue/envs/stable/lib/python3.7/site-packages (from requests>=2.5.0->sphinx==2.3.1; extra == "docs"->multiconsumers-queue==0.1.4) (2.9)
Requirement already up-to-date: pytz>=2015.7 in /home/docs/checkouts/readthedocs.org/user_builds/multiconsumers-queue/envs/stable/lib/python3.7/site-packages (from babel!=2.0,>=1.3->sphinx==2.3.1; extra == "docs"->multiconsumers-queue==0.1.4) (2019.3)
Requirement already up-to-date: pyparsing>=2.0.2 in /home/docs/checkouts/readthedocs.org/user_builds/multiconsumers-queue/envs/stable/lib/python3.7/site-packages (from packaging->sphinx==2.3.1; extra == "docs"->multiconsumers-queue==0.1.4) (2.4.7)
Building wheels for collected packages: multiconsumers-queue, m2r
  Building wheel for multiconsumers-queue (PEP 517): started
  Building wheel for multiconsumers-queue (PEP 517): finished with status 'done'
  Created wheel for multiconsumers-queue: filename=multiconsumers_queue-0.1.4-py3-none-any.whl size=6392 sha256=6587f695831f380ca7037a45ea084155ed84fb0fc05dc205e60908b334ec9027
  Stored in directory: /tmp/pip-ephem-wheel-cache-o22648kv/wheels/0f/ff/96/13f72ca46f66c60bdc0fd096b28d7c93d5ba1b51adec72aab5
  Building wheel for m2r (setup.py): started
  Building wheel for m2r (setup.py): finished with status 'done'
  Created wheel for m2r: filename=m2r-0.2.1-py3-none-any.whl size=10466 sha256=6be74e69404760dbde46648fb6c0f385c87d33014abfa40a18d35a700c625897
  Stored in directory: /tmp/pip-ephem-wheel-cache-o22648kv/wheels/02/47/3a/e1c46c2cca442c8781612542397c9559a579f10e2dd87e7c9f
Successfully built multiconsumers-queue m2r
Installing collected packages: loguru, attrs, python-dateutil, arrow, toml, sphinxcontrib-serializinghtml, sphinxcontrib-applehelp, Pygments, sphinxcontrib-qthelp, setuptools, sphinxcontrib-devhelp, sphinxcontrib-jsmath, sphinxcontrib-htmlhelp, docutils, sphinx, sphinx-autodoc-typehints, mistune, m2r, multiconsumers-queue
  Attempting uninstall: Pygments
    Found existing installation: Pygments 2.3.1
    Uninstalling Pygments-2.3.1:
      Successfully uninstalled Pygments-2.3.1
  Attempting uninstall: setuptools
    Found existing installation: setuptools 41.0.1
    Uninstalling setuptools-41.0.1:
      Successfully uninstalled setuptools-41.0.1
  Attempting uninstall: docutils
    Found existing installation: docutils 0.14
    Uninstalling docutils-0.14:
      Successfully uninstalled docutils-0.14
  Attempting uninstall: sphinx
    Found existing installation: Sphinx 1.8.5
    Uninstalling Sphinx-1.8.5:
      Successfully uninstalled Sphinx-1.8.5
Successfully installed Pygments-2.6.1 arrow-0.15.5 attrs-19.3.0 docutils-0.16 loguru-0.4.1 m2r-0.2.1 mistune-0.8.4 multiconsumers-queue-0.1.4 python-dateutil-2.8.1 setuptools-46.1.3 sphinx-2.3.1 sphinx-autodoc-typehints-1.10.3 sphinxcontrib-applehelp-1.0.2 sphinxcontrib-devhelp-1.0.2 sphinxcontrib-htmlhelp-1.0.3 sphinxcontrib-jsmath-1.0.1 sphinxcontrib-qthelp-1.0.3 sphinxcontrib-serializinghtml-1.1.4 toml-0.10.0

Yep, I see. By idea we can install it from conf.py this way, with the help of install_with_constraints trick from your noxfile.py, but IMO it's a overkill for the most of doc build tasks :)

Hi @cjolowicz ,
Thanks for creating this resource! I've learned a lot from reading your blog posts and the hypermodern-python website.
I recently stumbled upon this comment describing another strategy for integrating poetry dependencies into a readthedocs build. I haven't tried it myself yet, but I think it might read the versions from poetry's lock file? Would this be a viable option?


[BUG] Remove generate subcommand

Describe the bug
We should deprecate the generate_chart command as this is not used extensively by users and currently does not work. Dashboards should be a central tool to build charts out of OSB metrics.
To Reproduce
When users run opensearch-benchmark generate --chart-type="time-series" from the documentation, the following error is revealed on the console:
opensearch-benchmark generate: error: the following arguments are required: artifact, --chart-spec-path

Hi @IanHoang I would like to work on this one.  I'll start digging the issue while anyone else with insights can also share in the comment. 😊


Contraction is broken

Somehow a nasty bug has snuck in, contraction is not working at all. Could someone please look urgently, thanks.

I have not been able to find any contraction example that works!
On Wed, 17 Jan 2024, 20:13 Calin Tataru, @.***> wrote:

Do you have an example?
—
Reply to this email directly, view it on GitHub
https://github.com/homotopy-io/homotopy-rs/issues/1312#issuecomment-1896615893,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACQ4OHXCKXDEP2HYC7F3O63YPAWF7AVCNFSM6AAAAABB63WQFWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOJWGYYTKOBZGM
.
You are receiving this because you authored the thread.Message ID:
@.***>

Also cannot reproduce

All working fine for me now as well, thanks for checking it.
On Thu, Jan 18, 2024 at 9:52 AM Nick Hu @.***> wrote:

Also cannot reproduce
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you authored the thread.Message ID: @.***>


[HELP] How could I validate the algorithm?

Hi @JK-Pair,
You will have to provide much more information about your problem.

Hi @JK-Pair,
You will have to provide much more information about your problem.

I'm sorry for the unclear question
I mean if I want to compare this algorithm with another algorithm, How could I do that?
Maybe you could suggest some blog about this to me?

This is still too general question and not related to sentinelhub-py.
Feel free to read some blog bost about the work we do with satellite data: https://medium.com/sentinel-hub


Add a contentEncoding property to compressed files.

This can be used later on by other parts of a pipeline to:

Tell that data was indeed compressed (without brittle extension checking),
Tell how the data was compressed.

It follows roughly the same semantics as the HTTP 'Content-Encoding' header and allows a consumer of the file to observe all the encoding processes that occurred on the file. See: http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html section 14.11.

See also: https://github.com/izaakschroeder/vinyl-s3/pull/5

Looks good. Thanks for the PR!
Would you mind adding a test to test/test.js?

Bah, it looks like some of the test are broken. Not sure how that happened. I'll investigate when I get home tonight.

Alright, I fixed the broken tests. Would you mind adding a test for this feature? I'd by happy to merge afterwards.

Thanks! On it.

All yours @jstuckey. :tada:

Awesome, thanks! :shipit:

Published to npm.

:+1: :gem:


Something went wrong with Vagrant.app

The vagrant cask has been successfully installed but at the very last stage it has written the following weird line:

Is that normal?

@denpatin I think that's just an output oddity from when your password was mistyped above.
If installer reports successful, you should be good to go.

@phinze Yeah, everything works well. But I got a bit confused with this last line. So ok, maybe it is because the first unsuccessful password try. Thanks for comment on this! But still I think (of course it isn't that important) the last line shouldn't appear in such cases.

Yeah we may just need to handle stderr piping somewhere. Will leave open to investigate.

Already being tracked.


feat: Unify Android/Desktop and remove AlgoSDK

Previously we had an android folder and a desktop folder wit duplicated code.
Now we have an androidModule, jvmModule and a sharedModule. sharedModule contains an Bip32Ed25519Base abstract class which implements everything, but leaves the choice of LazySodium instantiation free. androidModule and jvmModuel respectively contain their own kotlin file that implements Base but specifies LazySodium Java or LazySodium Android, specifying nothing else.
jvmModule also contains tests using the LazySodium Java instantiation. Because the androidModule and jvmModule fundamentally share the same codebase this is equivalent to partially testing the androidModule.

LGTM


ci: use PAT for triggering subsequent workflows

Description
This updates the release.yaml to trigger other workflows (e.g. gh-pages, discord), which is not possible when using the GITHUB_TOKEN.

Do you introduce a breaking change?

[ ] Yes
[x] No
[ ] I am unsure (no worries, we'll find out)

:tada: This PR is included in version 15.0.0-beta.1 :tada:
The release is available on:

npm package (@beta dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

:tada: This PR is included in version 14.2.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket: