rllib example after changing scenario

This is a:

[x] Question
[x] Feature request

Describe the issue
Reusing the example of rllib(tried with stabilizing highway.py) with different scenario and environment but with the same algorithm. Used 5 "human"(IDM Controller) vehicles and 5 "rl"(RL Controller) vehicles. Did the following changes from the example:

Changed the scenario and environment to bottle neck scenario and bottle neck environment.
Also included the additional_net_params required i.e., scaling and speed_limit.
Then changed the inflows from "inflow_highway" and "inflow_merge" to "1" and "3".
Added env Params             "lane_change_duration": 5,
"disable_tb": True,
"disable_ramp_metering": True,
But the algorithm to setup_exps() is used the same.

To Reproduce
Leave empty if not applicable.

File in which error occurred.
Exact command that was run that caused the error to occur.
Please attach the full error output log.

Expected behavior
I expected the behavior to be the similar one as of "Stabilizing_highway" but with "bottle neck" projection. It was not possible to do training because of these errors.
Screenshots Error output:
== Status ==
Using FIFO scheduling algorithm.
Resources requested: 3/3 CPUs, 0/1 GPUs
Memory usage on this node: 8.4/33.7 GB
Result logdir: /home/niyathipriya/ray_results/stabilizing_open_network_merges
RUNNING trials:

PPO_BottleneckEnv-v0_0:	RUNNING, 5 failures: /home/niyathipriya/ray_results/stabilizing_open_network_merges/PPO_BottleneckEnv-v0_0_2019-02-20_13-52-00amqzv9ot/error_2019-02-20_13-54-04.txt

Closing connection to TraCI and stopping simulation.
Note, this may print an error message when it closes.
I0220 13:54:04.762437 16016 store.cc:599] Disconnecting client on fd 16
2019-02-20 13:54:10,459	WARNING ppo.py:146 -- By default, observations will be normalized with MeanStdFilter
Loading configuration... done.
Success.
Loading configuration... done.
2019-02-20 13:54:11,533	INFO policy_evaluator.py:264 -- Creating policy evaluation worker 0 on CPU (please ignore any CUDA init errors)
2019-02-20 13:54:11.533913: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX
/home/niyathipriya/anaconda3/envs/flow/lib/python3.6/site-packages/tensorflow/python/ops/gradients_impl.py:100: UserWarning: Converting sparse IndexedSlices to a dense Tensor of unknown shape. This may consume a large amount of memory.
"Converting sparse IndexedSlices to a dense Tensor of unknown shape. "
2019-02-20 13:54:12,398	INFO multi_gpu_optimizer.py:74 -- LocalMultiGPUOptimizer devices ['/cpu:0']
Loading configuration... done.
Success.
Loading configuration... done.
Loading configuration... done.
Success.
Loading configuration... done.
2019-02-20 13:54:26,174	INFO policy_evaluator.py:264 -- Creating policy evaluation worker 2 on CPU (please ignore any CUDA init errors)
2019-02-20 13:54:26.175518: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX
2019-02-20 13:54:26,227	INFO policy_evaluator.py:264 -- Creating policy evaluation worker 1 on CPU (please ignore any CUDA init errors)
2019-02-20 13:54:26.228473: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX
/home/niyathipriya/anaconda3/envs/flow/lib/python3.6/site-packages/tensorflow/python/ops/gradients_impl.py:100: UserWarning: Converting sparse IndexedSlices to a dense Tensor of unknown shape. This may consume a large amount of memory.
"Converting sparse IndexedSlices to a dense Tensor of unknown shape. "
/home/niyathipriya/anaconda3/envs/flow/lib/python3.6/site-packages/tensorflow/python/ops/gradients_impl.py:100: UserWarning: Converting sparse IndexedSlices to a dense Tensor of unknown shape. This may consume a large amount of memory.
"Converting sparse IndexedSlices to a dense Tensor of unknown shape. "
Loading configuration... done.
Success.
Loading configuration... done.
Success.
Loading configuration... done.
Loading configuration... done.
2019-02-20 13:54:29,068	ERROR trial_runner.py:413 -- Error processing event.
Traceback (most recent call last):
File "/home/niyathipriya/anaconda3/envs/flow/lib/python3.6/site-packages/ray/tune/trial_runner.py", line 378, in _process_events
result = self.trial_executor.fetch_result(trial)
File "/home/niyathipriya/anaconda3/envs/flow/lib/python3.6/site-packages/ray/tune/ray_trial_executor.py", line 228, in fetch_result
result = ray.get(trial_future[0])
File "/home/niyathipriya/anaconda3/envs/flow/lib/python3.6/site-packages/ray/worker.py", line 2132, in get
raise value
ray.worker.RayTaskError: ray_worker (pid=16608, host=linuxpriya)
File "/home/niyathipriya/anaconda3/envs/flow/lib/python3.6/site-packages/ray/rllib/agents/agent.py", line 284, in train
result = Trainable.train(self)
File "/home/niyathipriya/anaconda3/envs/flow/lib/python3.6/site-packages/ray/tune/trainable.py", line 151, in train
result = self._train()
File "/home/niyathipriya/anaconda3/envs/flow/lib/python3.6/site-packages/ray/rllib/agents/ppo/ppo.py", line 103, in _train
fetches = self.optimizer.step()
File "/home/niyathipriya/anaconda3/envs/flow/lib/python3.6/site-packages/ray/rllib/optimizers/multi_gpu_optimizer.py", line 125, in step
self.num_envs_per_worker, self.train_batch_size)
File "/home/niyathipriya/anaconda3/envs/flow/lib/python3.6/site-packages/ray/rllib/optimizers/rollout.py", line 28, in collect_samples
next_sample = ray.get(fut_sample)
ray.worker.RayTaskError: ray_worker (pid=16601, host=linuxpriya)
File "/home/niyathipriya/anaconda3/envs/flow/lib/python3.6/site-packages/ray/rllib/evaluation/policy_evaluator.py", line 368, in sample
batches = [self.input_reader.next()]
File "/home/niyathipriya/anaconda3/envs/flow/lib/python3.6/site-packages/ray/rllib/offline/input_reader.py", line 31, in next
batches = [self.sampler.get_data()]
File "/home/niyathipriya/anaconda3/envs/flow/lib/python3.6/site-packages/ray/rllib/evaluation/sampler.py", line 65, in get_data
item = next(self.rollout_provider)
File "/home/niyathipriya/anaconda3/envs/flow/lib/python3.6/site-packages/ray/rllib/evaluation/sampler.py", line 267, in _env_runner
preprocessors, obs_filters, unroll_length, pack, callbacks)
File "/home/niyathipriya/anaconda3/envs/flow/lib/python3.6/site-packages/ray/rllib/evaluation/sampler.py", line 354, in _process_observations
episode.last_action_for(agent_id),
File "/home/niyathipriya/anaconda3/envs/flow/lib/python3.6/site-packages/ray/rllib/evaluation/episode.py", line 99, in last_action_for
flat = _flatten_action(policy.action_space.sample())
File "/home/niyathipriya/anaconda3/envs/flow/lib/python3.6/site-packages/gym/spaces/box.py", line 37, in sample
return gym.spaces.np_random.uniform(low=self.low, high=self.high + (0 if self.dtype.kind == 'f' else 1), size=self.low.shape).astype(self.dtype)
File "mtrand.pyx", line 1316, in mtrand.RandomState.uniform
OverflowError: Range exceeds valid bounds
Closing connection to TraCI and stopping simulation.
Note, this may print an error message when it closes.
2019-02-20 13:54:29,084	INFO trial_runner.py:450 -- Attempting to recover trial state from last checkpoint.
I0220 13:54:29.090139 16016 store.cc:599] Disconnecting client on fd 17
Closing connection to TraCI and stopping simulation.
Note, this may print an error message when it closes.
I0220 13:54:29.127265 16016 store.cc:599] Disconnecting client on fd 13

Desktop (please complete the following information):

OS: Ubuntu 18.04.1 LTS, Intel Xeon(R) processor with 64 bit
flow-0.2.0 version

Kindly let me know if I had done any wrong. I am unable to debug this as the error message says error inside the installed packages of ray, which is wrong as it works for the same examples if the scenario and environment is unchanged.
Note : Can you add some more details for basics of these examples described if I want to create my own scenario and train the vehicles. Like what are the basic factors need to be taken care before I start writing my functions for creating my own env and scenario and to train with another algotithm. The
documentation states few information but as a new learner, I expect it to be well described with the basics.
Any help would highly be appreciated so that I can learn the unknown as I am still a learner.
Thanks & Regards,
pnp91

hi @pnp91 I looked into this and it seems that BottleneckEnv is not designed to be used by RL experiments, but rather just for simulations; instead, you will want to use BottleNeckAccelEnv. This environment, however, seems to be missing some features. I've created a PR meant to fix these bugs (see the branch bottleneck_env_fix). From this branch, the following script (adapted from stabilizing_highway.py) should work:
"""Open merge example.

Trains a a small percentage of rl vehicles to dissipate shockwaves caused by
merges in an open network.
"""
import json

import ray
try:
    from ray.rllib.agents.agent import get_agent_class
except ImportError:
    from ray.rllib.agents.registry import get_agent_class
from ray.tune import run_experiments
from ray.tune.registry import register_env

from flow.utils.registry import make_create_env
from flow.utils.rllib import FlowParamsEncoder
from flow.core.params import SumoParams, EnvParams, InitialConfig, NetParams, \
    InFlows, SumoCarFollowingParams
from flow.scenarios.bottleneck import ADDITIONAL_NET_PARAMS
from flow.core.params import VehicleParams
from flow.controllers import IDMController, RLController

# experiment number
# - 0: 10% RL penetration,  5 max controllable vehicles
# - 1: 25% RL penetration, 13 max controllable vehicles
# - 2: 33% RL penetration, 17 max controllable vehicles
EXP_NUM = 0

# time horizon of a single rollout
HORIZON = 600
# number of rollouts per training iteration
N_ROLLOUTS = 20
# number of parallel workers
N_CPUS = 2

# inflow rate at the highway
FLOW_RATE = 2000
# percent of autonomous vehicles
RL_PENETRATION = [0.1, 0.25, 0.33][EXP_NUM]
# num_rl term (see ADDITIONAL_ENV_PARAMs)
NUM_RL = [5, 13, 17][EXP_NUM]

# We consider a highway network with an upstream merging lane producing
# shockwaves
additional_net_params = ADDITIONAL_NET_PARAMS.copy()
# additional_net_params["merge_lanes"] = 1
# additional_net_params["highway_lanes"] = 1
# additional_net_params["pre_merge_length"] = 500

# RL vehicles constitute 5% of the total number of vehicles
vehicles = VehicleParams()
vehicles.add(
    veh_id="human",
    acceleration_controller=(IDMController, {
        "noise": 0.2
    }),
    car_following_params=SumoCarFollowingParams(
        speed_mode="obey_safe_speed",
    ),
    num_vehicles=5)
vehicles.add(
    veh_id="rl",
    acceleration_controller=(RLController, {}),
    car_following_params=SumoCarFollowingParams(
        speed_mode="obey_safe_speed",
    ),
    num_vehicles=5)

# Vehicles are introduced from both sides of merge, with RL vehicles entering
# from the highway portion as well
inflow = InFlows()
inflow.add(
    veh_type="human",
    edge="1",
    vehs_per_hour=(1 - RL_PENETRATION) * FLOW_RATE,
    departLane="free",
    departSpeed=10)
# inflow.add(
#     veh_type="rl",
#     edge="1",
#     vehs_per_hour=RL_PENETRATION * FLOW_RATE,
#     departLane="free",
#     departSpeed=10)
inflow.add(
    veh_type="human",
    edge="3",
    vehs_per_hour=100,
    departLane="free",
    departSpeed=7.5)

flow_params = dict(
    # name of the experiment
    exp_tag="stabilizing_open_network_merges",

    # name of the flow environment the experiment is running on
    env_name="BottleNeckAccelEnv",

    # name of the scenario class the experiment is running on
    scenario="BottleneckScenario",

    # simulator that is used by the experiment
    simulator='traci',

    # sumo-related parameters (see flow.core.params.SumoParams)
    sim=SumoParams(
        sim_step=0.2,
        render=False,
    ),

    # environment related parameters (see flow.core.params.EnvParams)
    env=EnvParams(
        horizon=HORIZON,
        sims_per_step=5,
        warmup_steps=0,
        additional_params={
            "max_accel": 1.5,
            "max_decel": 1.5,
            "target_velocity": 20,
            "num_rl": NUM_RL,
            "lane_change_duration": 5,
            "disable_tb": True,
            "disable_ramp_metering": True,
            "add_rl_if_exit": False,
        },
    ),

    # network-related parameters (see flow.core.params.NetParams and the
    # scenario's documentation or ADDITIONAL_NET_PARAMS component)
    net=NetParams(
        inflows=inflow,
        no_internal_links=False,
        additional_params=additional_net_params,
    ),

    # vehicles to be placed in the network at the start of a rollout (see
    # flow.core.vehicles.Vehicles)
    veh=vehicles,

    # parameters specifying the positioning of vehicles upon initialization/
    # reset (see flow.core.params.InitialConfig)
    initial=InitialConfig(),
)

def setup_exps():

    alg_run = "PPO"

    agent_cls = get_agent_class(alg_run)
    config = agent_cls._default_config.copy()
    config["num_workers"] = N_CPUS
    config["train_batch_size"] = HORIZON * N_ROLLOUTS
    config["gamma"] = 0.999  # discount rate
    config["model"].update({"fcnet_hiddens": [32, 32, 32]})
    config["use_gae"] = True
    config["lambda"] = 0.97
    config["kl_target"] = 0.02
    config["num_sgd_iter"] = 10
    config['clip_actions'] = False  # FIXME(ev) temporary ray bug
    config["horizon"] = HORIZON

    # save the flow params for replay
    flow_json = json.dumps(
        flow_params, cls=FlowParamsEncoder, sort_keys=True, indent=4)
    config['env_config']['flow_params'] = flow_json
    config['env_config']['run'] = alg_run

    create_env, gym_name = make_create_env(params=flow_params, version=0)

    # Register as rllib env
    register_env(gym_name, create_env)
    return alg_run, gym_name, config

if __name__ == "__main__":
    alg_run, gym_name, config = setup_exps()
    ray.init(num_cpus=N_CPUS + 1, redirect_output=False)
    trials = run_experiments({
        flow_params["exp_tag"]: {
            "run": alg_run,
            "env": gym_name,
            "config": {
                **config
            },
            "checkpoint_freq": 20,
            "max_failures": 999,
            "stop": {
                "training_iteration": 200,
            },
        }
    })

Hello @AboudyKreidieh
Thanks a lot for the help. It worked for me. My experiment was successful. But I have a doubt. Why can we not add an inflow for vehicle type "rl"? Why is it removed? I tried running with it. But I was not successful. I want to is there a rule for the inflows to add

hi @pnp91 sorry for the late reply. The issue is just a matter of the way the environment was written. Some other environemnts, however, do allow you to have an unfixed number of automated vehicles, see for example WaveAttenuationMergePOEnv in flow/envs/merge.py
Also, I noticed i didn't reply to your original question about details for creating custom environments. For that, I suggest you look into our tutorials: https://github.com/flow-project/flow/tree/master/tutorials
Hope this helps! Feel free to ask more about the details of any tutorial/example

Hi @AboudyKreidieh I sincerely thank you so much for taking out your time and helping a beginner as me. I know as a novice, my doubts may sound unwanted and simple. But still, thank you for taking out your time to help me understand. I appreciate a lot.
Also, I had gone through these tutorials and was trying to develop my own scenario with new environment and train with a new algorithm. So I wanted to analyse all the minute details on how to build this. I have also read the documentation on how to create a new env. But I was unclear with some small details. Hence I reached out here. May be that is where I failed and your answer now helped me. Thanks for being patient with me. It was little unclear in docs where the additional_parameter "add_rl_if_exit" was not mentioned explicitly. This made me little skeptic on how to proceed while creating new.
And now I am trying to change the algorithm from PPO to A3C in the same example. As mentioned in the docs, I have also changed the hyper-parameters in "default_config" as required. But still, I am not successful in this. Is there anything else apart from that which is required to be changed to make this successful?

@AboudyKreidieh Can I use a different algorithm instead of PPO as mentioned in all the examples of rllib? I wish to use A3C algorithm and had changed the snippet of code parameters as follows:
def setup_exps():
alg_run = "A3C"

agent_cls = get_agent_class(alg_run)
config = agent_cls._default_config.copy()
config["num_workers"] = N_CPUS
config["sample_batch_size"] = HORIZON * N_ROLLOUTS
config["gamma"] = 0.999  # discount rate
config["lambda"] = 1.0
config["grad_clip"] = 40.0
config["lr"] = 0.0001
config["lr_schedule"] = 0.0001
config["vf_loss_coeff"] = 0.5
config["entropy_coeff"] = -0.01
config["sample_async"] = True
config["min_iter_time_s"] = 5
config["use_lstm"] = True
config["horizon"] = HORIZON

# save the flow params for replay
flow_json = json.dumps(
    flow_params, cls=FlowParamsEncoder, sort_keys=True, indent=4)
config['env_config']['flow_params'] = flow_json
config['env_config']['run'] = alg_run

create_env, gym_name = make_create_env(params=flow_params, version=0)

# Register as rllib env
register_env(gym_name, create_env)
return alg_run, gym_name, config

But I encounter errors directing towards to tensor flow which is not used here. I have tried with other parameters as well. But nothing was helpful. Error still persists as follows:
Error:
Traceback (most recent call last):
File "A3C_Try.py", line 10, in
from ray.rllib.agents.agent import get_agent_class
File "/home/niyathipriya/anaconda3/envs/flow/lib/python3.6/site-packages/ray/rllib/init.py", line 9, in
from ray.tune.registry import register_trainable
File "/home/niyathipriya/anaconda3/envs/flow/lib/python3.6/site-packages/ray/tune/init.py", line 6, in
from ray.tune.tune import run_experiments
File "/home/niyathipriya/anaconda3/envs/flow/lib/python3.6/site-packages/ray/tune/tune.py", line 12, in
from ray.tune.suggest import BasicVariantGenerator
File "/home/niyathipriya/anaconda3/envs/flow/lib/python3.6/site-packages/ray/tune/suggest/init.py", line 2, in
from ray.tune.suggest.basic_variant import BasicVariantGenerator
File "/home/niyathipriya/anaconda3/envs/flow/lib/python3.6/site-packages/ray/tune/suggest/basic_variant.py", line 9, in
from ray.tune.config_parser import make_parser, create_trial_from_spec
File "/home/niyathipriya/anaconda3/envs/flow/lib/python3.6/site-packages/ray/tune/config_parser.py", line 14, in
from ray.tune.trial import Trial, json_to_resources
File "/home/niyathipriya/anaconda3/envs/flow/lib/python3.6/site-packages/ray/tune/trial.py", line 22, in
from ray.tune.logger import pretty_print, UnifiedLogger
File "/home/niyathipriya/anaconda3/envs/flow/lib/python3.6/site-packages/ray/tune/logger.py", line 21, in
import tensorflow as tf
File "/home/niyathipriya/anaconda3/envs/flow/lib/python3.6/site-packages/tensorflow/init.py", line 24, in
from tensorflow.python import pywrap_tensorflow  # pylint: disable=unused-import
File "/home/niyathipriya/anaconda3/envs/flow/lib/python3.6/site-packages/tensorflow/python/init.py", line 55, in
'tensorflow_estimator.python.estimator'))
File "/home/niyathipriya/anaconda3/envs/flow/lib/python3.6/site-packages/tensorflow/python/tools/component_api_helper.py", line 56, in package_hook
child_pkg = importlib.import_module(child_package_str)
File "/home/niyathipriya/anaconda3/envs/flow/lib/python3.6/importlib/init.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "/home/niyathipriya/anaconda3/envs/flow/lib/python3.6/site-packages/tensorflow_estimator/init.py", line 8, in
from tensorflow_estimator._api.v1 import estimator
File "/home/niyathipriya/anaconda3/envs/flow/lib/python3.6/site-packages/tensorflow_estimator/_api/v1/estimator/init.py", line 8, in
from tensorflow_estimator._api.v1.estimator import experimental
File "/home/niyathipriya/anaconda3/envs/flow/lib/python3.6/site-packages/tensorflow_estimator/_api/v1/estimator/experimental/init.py", line 8, in
from tensorflow_estimator.python.estimator.canned.dnn import dnn_logit_fn_builder
File "/home/niyathipriya/anaconda3/envs/flow/lib/python3.6/site-packages/tensorflow_estimator/python/estimator/init.py", line 25, in
import tensorflow_estimator.python.estimator.estimator_lib
File "/home/niyathipriya/anaconda3/envs/flow/lib/python3.6/site-packages/tensorflow_estimator/python/estimator/estimator_lib.py", line 22, in
from tensorflow_estimator.python.estimator.canned.baseline import BaselineClassifier
File "/home/niyathipriya/anaconda3/envs/flow/lib/python3.6/site-packages/tensorflow_estimator/python/estimator/canned/baseline.py", line 64, in
from tensorflow_estimator.python.estimator import estimator
File "/home/niyathipriya/anaconda3/envs/flow/lib/python3.6/site-packages/tensorflow_estimator/python/estimator/estimator.py", line 66, in
from tensorflow_estimator.python.estimator import model_fn as model_fn_lib
File "/home/niyathipriya/anaconda3/envs/flow/lib/python3.6/site-packages/tensorflow_estimator/python/estimator/model_fn.py", line 36, in
from tensorflow_estimator.python.estimator.export import export_lib
File "/home/niyathipriya/anaconda3/envs/flow/lib/python3.6/site-packages/tensorflow_estimator/python/estimator/export/export_lib.py", line 25, in
from tensorflow.python.saved_model.model_utils import build_all_signature_defs
File "/home/niyathipriya/anaconda3/envs/flow/lib/python3.6/site-packages/tensorflow/python/saved_model/model_utils/init.py", line 22, in
from tensorflow.python.saved_model.model_utils.export_utils import build_all_signature_defs
File "/home/niyathipriya/anaconda3/envs/flow/lib/python3.6/site-packages/tensorflow/python/saved_model/model_utils/export_utils.py", line 51, in
ModeKeys.TRAIN: signature_constants.DEFAULT_TRAIN_SIGNATURE_DEF_KEY,
AttributeError: module 'tensorflow.python.saved_model.signature_constants' has no attribute 'DEFAULT_TRAIN_SIGNATURE_DEF_KEY'

hi @pnp91, first of happy to help whenever you need it! some details may not be very well documented, and issues like these help us identify some of the missing components in our codebase. Also, in terms of the A3C thing, can you share your new script (at least the A3C component) so that I can try it locally?

Hello @AboudyKreidieh thank you for your reply. Regarding A3C, I have now succeeded in implementing it. I will share the code as well. As per my investigation, I have realized that the problem was with the hyper parameter "train_batch_size". As per 'A3C-specific configs' stated in documentation , it requires "sample_batch_size" parameter. Unfortunately, using this does not support in the algorithm for A3C. The parameter "train_batch_size" should be used in place of "sample_batch_size". Please correct me if my observations are incorrect. Attached below, is the code which I have modified to make it run:
`
def setup_exps():
alg_run = "A3C"

agent_cls = get_agent_class(alg_run)
config = agent_cls._default_config.copy()
config["num_workers"] = N_CPUS
config["train_batch_size"] = HORIZON * N_ROLLOUTS
config["gamma"] = 0.999  # discount rate
config["model"].update({"fcnet_hiddens": [64, 64]})
config["lambda"] = 0.97
config["horizon"] = HORIZON

# save the flow params for replay
flow_json = json.dumps(
    flow_params, cls=FlowParamsEncoder, sort_keys=True, indent=4)
config['env_config']['flow_params'] = flow_json
config['env_config']['run'] = alg_run

create_env, gym_name = make_create_env(params=flow_params, version=0)

# Register as rllib env
register_env(gym_name, create_env)
return alg_run, gym_name, config

if name == "main":
alg_run, gym_name, config = setup_exps()
ray.init(num_cpus=N_CPUS + 1, redirect_output=False)
trials = run_experiments({
flow_params["exp_tag"]: {
"run": alg_run,
"env": gym_name,
"config": {
**config
},
"checkpoint_freq": 20,
"max_failures": 999,
"stop": {
"training_iteration": 200,
},
}
})`

hi @pnp91 you are correct! the parameters tend to change from algorithm to algorithm in rllib. For those I would generally recommend looking into their documentation: https://ray.readthedocs.io/en/latest/rllib-algorithms.html

assuming this has been resolved, so closing. feel free to open a new issue if anything else emerges!


Range End Not Working

I’ve been using the keyhunt tool, and I’ve noticed that the -r (range) option doesn’t seem to work as expected. Specifically, when I specify a range like -r 1:FFFFFFFF, it doesn’t behave as intended. Instead, I get unexpected results.
Could you please investigate this issue? It would be great if the tool could accurately handle the specified range.
Thank you for your attention to this matter!
Best regards

I guess that you are using it with multiple threads don't you?
If that is the case the default subrange per thread is 32 bits just like your specified range. If you are going to work with small ranges ( yes that 32 bit range  -r 1:FFF...  is small) please specify the N value to some lower value with  -n 0x1000000 or to -n 0x10000 ( 24 or 16 bit respectively) personally I prefer 24 bits.
With this I gues


Running trubar with trailing slashes causes missing first letters

I ran trubar translate orange-translations/orange3/si.yaml -s orange3/ -d orangeSI/ to integrate translations into source code. What happened is all non-hidden files were missing the first letter of the file name.

The hidden miss the first letter as well, I believe. Hidden files are hidden because they start with ., which was removed, I suppose. :)


Coursier doesn't seem to use maven metadata files to discover artifacts.

The issue is explained here: https://github.com/lihaoyi/mill/issues/1099

Actually, Mill seems to pull dependencies by directly building an URL to the supposed pom. However, this method doesn't work on some repositories like for Spigot dependencies.
Mill should first pull the first maven-metadata.xml (example) at <repoUrl>/<group separated by />/<artifactId>/maven-metadata.xml to retrieve the version's metadata file (example) and get the jar name from it.

After discussing with a Mill contributor, we found that the issue is coursier-related since Mill uses it for dependency management.

I can confirm the same issue from Ammonite when we want to fetch an artifact having only a "maven-metadata.xml" file (no POM file).


Unable to resolve action

Failed to resolve action download info. Error: Unable to resolve action<EMAIL_ADDRESS>unable to find version `v19.0.1 3`

This is the attempt with quotes - doesn't work without either at the moment. Is there a reason this action isn't on just standard SemVer?

Having the same issue here too.
temporary using
amondnet/vercel-action@@master
to resolve the issue.

Hi, @followbl
Until v2, I used semantic version (v2.0.3). From v19, I set it like that to match vercel clil's version code ( v19.0.1 ).

@amondnet - yea, sorry it's a pain (I know it's more than likely not a problem with this repo - a problem probably in how github actions yaml is parsing the line).
@shunkakinoki thanks for the workaround - confirmed it does fix the issue for now - thanks again!


Eloquent: Nested Lazy Eager Loading will erase relations from Nested Eager Loading

Laravel Version:  slim/slim 3.10.0
Eloquent: illuminate/database v5.6.20
PHP Version:  7.2.4
Database Driver & Version: MariaDB 10.3.6

Description:
When first using Nested Eager Loading, like
(1) $categories = Category::with(['products.adjunctProducts'])->get();
and put a Nested Lazy Eager Loading on top of it, like
(2) $categories->load(['products.subscriptionModel']);
then
after (1) you will get a Collection with 'products.adjunctProducts' relation, as expected.
But after (2) you will get a Collection with 'products.subscriptionModel' relation (again as expected),
but you will not find the relation 'products.adjunctProducts' anymore.
Remarks:

This does not happen with non-nested Eager Loads (e.g. $categories = Category::with('products')->get();
One could ask 'Why not just put the nested load directly in the first with()'.
The described case here is simplified as much as possible. Our Eager Loaded collection normally can be changed by different modules using an event. So if a modules needs further relations it would just use load()
Our temporary solution is to not use get() (until after the event) but work with the Builder-Object and concatenate several with()s. Not sure if that's the best solution!?

Steps To Reproduce:
Take a model with different relations on a first relation and reproduce with the description above.

That's the expected behavior. Use this:
$categories->products->load(['subscriptionModel']);

Sorry, I missed that. Use this:
$categories->pluck('products')->collapse()->load(['subscriptionModel']);

The purpose of load() is to get the relationship results – even if a model already has results. The idea behind overwriting existing results is that you might want to get the latest data.
I can see why you would expect a different behavior. For a single model and non-nested relationships you can use the loadMissing() method.
Your situation would require loadMissing() on the collection and support for nested relationships. The latter is not trivial to implement, since you would have to check for existing results in every single model (#23027).
You're welcome to propose or implement improvements.

Thank you very much for elaborating on that!
The suggested line of code unfortunately ends up with an "Uncaught BadMethodCallException: Method Illuminate\Support\Collection::load does not exist". But no worries, maybe I missed something specific to our implementation. I just gave it a quick try.
Anyway, I think that was the push in the right direction which I was hoping for!

Again, my bad:
use Illuminate\Database\Eloquent\Collection;

new Collection($categories->pluck('products')->collapse())->load(['subscriptionModel']);

A shorter way to achieve this would definitely be nice...

I updated my post. One would think that three attempts should be enough to write a working one-liner...

One has to admit, its certainly not the most obvious one-liner... Thanks! 🥇

I proposed Collection::loadMissing(): #24166


Allow setting a FQDN as RFC2136 authoritative nameserver

Issue description
When setting up a FQDN as nameserver for RFC2136, I noticed that cert-manager was not able to issue certiticates as the Certificate resources had the following status:
Status:
  Conditions:
    Last Transition Time:  2019-12-07T17:11:17Z
    Message:               Referenced issuer does not have a Ready status condition
    Reason:                Pending
    Status:                False
    Type:                  Ready

The referenced ClusterIssuer had the following status:
Status:
  Conditions:
    Last Transition Time:  2019-12-07T16:09:11Z
    Message:               Resource validation failed: spec.acme.solvers[0].dns01.rfc2136.nameserver: Invalid value: "": Nameserver invalid. Check the documentation for details.
    Reason:                ConfigError
    Status:                False
    Type:                  Ready
Events:                    <none>

Proposed solutions
The best solution would be for cert-manager to support FQDN in the spec.acme.solvers[].dns01.rfc2136.nameserver field and use DNS resolution to contact the server.
If fixing the issue is not possible and/or wanted, an alternative solution would be to deny the ClusterIssuer creation (through the webhook or the CRDs validation) in respect of the principle of least astonishment.
Additional context
In order to reproduce the issue, one can create the following cluster issuer:
---
apiVersion: cert-manager.io/v1alpha2
kind: ClusterIssuer
metadata:
  name: letsencrypt
spec:
  acme:
    email<EMAIL_ADDRESS>    server: https://acme-v02.api.letsencrypt.org/directory
    privateKeySecretRef:
      name: letsencrypt-staging-key
    solvers:
      - dns01:
          rfc2136:
            nameserver: ns1.example.com
            tsigAlgorithm: HMACSHA512
            tsigKeyName: cert-manager
            tsigSecretSecretRef:
              name: dns01-tsig-key
              key: key

Environment details

Kubernetes version (e.g. v1.10.2): 1.16
Cloud-provider/provisioner (e.g. GKE, kops AWS, etc): None
cert-manager version (e.g. v0.4.0): 0.12.0
Install method (e.g. helm or static manifests): Home made helm chart

/kind feature

/remove-lifecycle stale


What is sa_0_4?

I was aware of @crowsonkb's ava aesthetic models, but these ones are new to me, what is the dataset/where the sa_0_4 come from?

It's a linear model trained on clip vit-l/14 embeddings
Using 4000 samples annotated with https://github.com/JD-P/simulacrabot
I think the results look pretty great


silence detection using cutoff on the acoustic probabilities

I'm using the recognizer for real time transcription and I've noticed that even when the user don't speak the recognizer outputs some gibberish characters...I assume I can fix it using some cutoff probability of the acoustic model right?

@alanbekker You can try. But I'm a bit worried about the performance. The white space or blank id should bear the maximum probability when the user speaks noting, otherwise other characters will appear as what you saw. Only when figuring out a way to increase the  probability of blank/space and decrease the probability of other characters, you can solve this problem.


Bump Gloo Envoy to 1.4.15 for CVE-2020-25017 and CVE-2020-25018

included in https://github.com/rancher/rio/pull/1060.


[SPARK-32482][SS][TESTS] Eliminate deprecated poll(long) API calls to avoid infinite wait in tests

What changes were proposed in this pull request?
Structured Streaming Kafka connector tests are now using a deprecated poll(long) API which could cause infinite wait. In this PR I've eliminated these calls and replaced them with AdminClient.
Why are the changes needed?
Deprecated poll(long) API calls.
Does this PR introduce any user-facing change?
No.
How was this patch tested?
Existing unit tests.

Test build #126770 has started for PR 29289 at commit 8748691.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/31392/
Test PASSed.

Similar solution is planned on the driver side but that part is much more complicated and requires some time to answer all my questions and play around.

Test build #126773 has started for PR 29289 at commit 910fa1b.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/31395/
Test PASSed.

Test build #126770 has finished for PR 29289 at commit 8748691.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/126770/
Test PASSed.

cc @HeartSaVioR @zsxwing @xuanyuanking

Test build #126773 has finished for PR 29289 at commit 910fa1b.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/126773/
Test PASSed.

I'll leave the PR a day to see any further input, and merge tomorrow. Please take a look, or leave comment if anyone needs some time to review this.

retest this, please

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/31473/
Test PASSed.

Looks Jenkins is out of the order. GitHub Actions build passed so should be good to merge and go ahead.

Thanks @HyukjinKwon for reminding! I'll go ahead merging this.

Thanks! Merged into master.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/126850/
Test FAILed.

Thanks everybody to help me!

I'm proceeding further with SPARK-32032 but from tomorrow I'm going to have 3 weeks vacation so the PR will be delayed a bit...


Update CODE-OF-CONDUCT.md

Missed this in the last PR.

Fixed broken badge link

I guess I am not maintainer on this repo, so somebody else needs to approve


CMake Build Environment Support

Support for building the project via CMake scripts instead of Visual Studio projects.

Not feasible with the CMake environment


Follow typography rules for line length

Just one random link on typography: https://material.io/design/typography/understanding-typography.html#readability
My main concern with water.css is the characters per line. Different studies suggest optimal line length is between 40 and 90 characters. While it's curious that science has found such a big range for an optimum, I do want to point out that on the homepage of water.css I get at least one line with >110 characters. I'm not really a frontend and design guy, so if the font size and line length were chosen deliberately, I withdraw my complaints. I just want to say that I find lines too long and font too small in water.css
Thanks for this project anyway, have a great day!

I'm not a designer either but I think you raise a valid point. I'd also want to see if there were any accessibility concerns with our font readability.
@kognise could chime in on the typography design decisions.
Thanks for logging the issue!


missing dependencies

on install you need tabulate and jupyter to get the notebook running

Tabulate and ipykernel are now part of the requirements.


GUI - Import anonymizer as package

Web GUI makes use of anonymizer as module inside the src folder.
Anonymizer module inside src is called anonymizer_service.
Right now the service is executed using os.system library.
GUI should be updated so that any call to the service makes use of find_entities() instead.

Added in commit 055ebb88283fdaeb9f90361956034bac1e6adcbb .
Closing issue.


Allow multiple S3 destinations in a single account

coverage: 74.97% (-0.2%) from 75.148% when pulling 135d63119179d5eae093938f00e5ae671cdd50e2 on jtschladen:allow-duplicate-s3-destinations into 6d9b45e56893ab1134dbeeb77c411b994a7742f6 on Netflix:master.


Bug: "Exception Value: SpooledTemporaryFile instance has no attribute len"

I failed to run dbbackup, this exception keeps getting raised. It seems you are using SpooledTemporaryFile "file like objects" and pass them directly to dropbox client "put_file" method as "body" parameter, and it only supports file_like objects with "getvalue" method (like StringIO).
i'm attaching my "requirements.txt", the exception traceback, and my solution, which is to use cStringIO for the chunks.
if it makes any difference, this happens on my development setup, using sqlite. perhaps dropbox released a new package into pip that broke compatibility?
Thank you for this very useful module, Shlomi.
NOTE: Downgrading to dropbox==1.6.0 fixed the issue for me. It might be worth pinning requirements in setup.py, Michael.

The version of dropbox stored in PyPi is now outdated. It doesn't support Python3. A new version has been made by Dropbox here: https://github.com/dropbox/dropbox-sdk-python
It is supposed to be fully compatible with older versions and Python.
I've made a PR at django-storage-redux here: https://github.com/jschneier/django-storages/pull/76
If it is merged, I think we'll drop dropbox code from dbbabckup and use Redux's one.

I think this issue may be closed with this PR #117


compilation fails for "examples/grpc-server"

LocalAI version:

45370c212bbc379f65f2c77560958acc24877fba
Environment, CPU architecture, OS, and Version:

Linux fedora 6.5.6-300.fc39.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Oct  6 19:57:21 UTC 2023 x86_64 GNU/Linux
Describe the bug

After failures with CUDA and docker in
https://github.com/go-skynet/LocalAI/issues/1178
I try to compile and run LocalAI directly on the host: make BUILD_TYPE=cublas build
To Reproduce

make BUILD_TYPE=cublas build
Expected behavior

Successful build, binaries running with CUDA support
Logs

make -C go-llama BUILD_TYPE=cublas libbinding.a
make[1]: Verzeichnis „/home/sgw/LocalAI/go-llama“ wird betreten
I llama.cpp build info:
I UNAME_S:  Linux
I UNAME_P:  unknown
I UNAME_M:  x86_64
I CFLAGS:   -I./llama.cpp -I. -O3 -DNDEBUG -std=c11 -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wdouble-promotion -Wshadow -Wstrict-prototypes -Wpointer-arith -Wno-unused-function -pthread -march=native -mtune=native
I CXXFLAGS: -I./llama.cpp -I. -I./llama.cpp/common -I./common -O3 -DNDEBUG -std=c++11 -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -pthread
I CGO_LDFLAGS:
I LDFLAGS:
I BUILD_TYPE:  cublas
I CMAKE_ARGS:  -DLLAMA_AVX512=OFF -DLLAMA_CUBLAS=ON
I EXTRA_TARGETS:  llama.cpp/ggml-cuda.o
I CC:       cc (GCC) 13.2.1 20230918 (Red Hat 13.2.1-3)
I CXX:      g++ (GCC) 13.2.1 20230918 (Red Hat 13.2.1-3)

make[1]: „libbinding.a“ ist bereits aktuell.
make[1]: Verzeichnis „/home/sgw/LocalAI/go-llama“ wird verlassen
LLAMA_VERSION=24ba3d829e31a6eda3fa1723f692608c2fa3adda make -C backend/cpp/llama grpc-server
make[1]: Verzeichnis „/home/sgw/LocalAI/backend/cpp/llama“ wird betreten
cd llama.cpp && mkdir -p build && cd build && cmake .. -DLLAMA_AVX512=OFF -DLLAMA_CUBLAS=ON && cmake --build . --config Release
-- cuBLAS found
-- Using CUDA architectures: 52;61;70
-- CMAKE_SYSTEM_PROCESSOR: x86_64
-- x86 detected
CMake Error at examples/grpc-server/CMakeLists.txt:7 (find_package):
  Could not find a package configuration file provided by "absl" with any of
  the following names:

    abslConfig.cmake
    absl-config.cmake

  Add the installation prefix of "absl" to CMAKE_PREFIX_PATH or set
  "absl_DIR" to a directory containing one of the above files.  If "absl"
  provides a separate development package or SDK, be sure it has been
  installed.

-- Configuring incomplete, errors occurred!
make[1]: *** [Makefile:43: grpc-server] Fehler 1
make[1]: Verzeichnis „/home/sgw/LocalAI/backend/cpp/llama“ wird verlassen

Additional context

I also tried CMAKE_ARGS="-DLLAMA_AVX512=OFF"  make BUILD_TYPE=cublas build because my CPU doesn't support AVX512.
Maybe interesting:
$ gcc -v
Es werden eingebaute Spezifikationen verwendet.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-redhat-linux/13/lto-wrapper
OFFLOAD_TARGET_NAMES=nvptx-none
OFFLOAD_TARGET_DEFAULT=1
Ziel: x86_64-redhat-linux
Konfiguriert mit: ../configure --enable-bootstrap --enable-languages=c,c++,fortran,objc,obj-c++,ada,go,d,m2,lto --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-shared --enable-threads=posix --enable-checking=release --enable-multilib --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-gnu-unique-object --enable-linker-build-id --with-gcc-major-version-only --enable-libstdcxx-backtrace --with-libstdcxx-zoneinfo=/usr/share/zoneinfo --with-linker-hash-style=gnu --enable-plugin --enable-initfini-array --with-isl=/builddir/build/BUILD/gcc-13.2.1-20230918/obj-x86_64-redhat-linux/isl-install --enable-offload-targets=nvptx-none --without-cuda-driver --enable-offload-defaulted --enable-gnu-indirect-function --enable-cet --with-tune=generic --with-arch_32=i686 --build=x86_64-redhat-linux --with-build-config=bootstrap-lto --enable-link-serialization=1
Thread-Modell: posix
Unterstützte LTO-Kompressionsalgorithmen: zlib zstd
gcc-Version 13.2.1 20230918 (Red Hat 13.2.1-3) (GCC)

But the error message seems more like something is missing than it's the wrong gcc.
EDIT: search for absl, found and installed sudo dnf install python3-absl-py.noarch ... doesn't help.

Try to install libabsl-dev.
On ubuntu I do apt install libabsl-dev and it help

I also got this bug following the same https://localai.io/howtos/easy-setup-docker-gpu/ ... unforutunately didnt find the apt install libabsl-dev and am in the process of a new pull/build, will give it a try once it builds. Could be missing package on the Quay build?

No luck with the apt install libabsl-dev command, think the GPU installation instructions are borked for the moment.

Pretty sure the bug spawns from instructions:     image: quay.io/go-skynet/local-ai:master-cublas-cuda12
something was migrated between packages?
Im attempting to follow the How To https://localai.io/howtos/easy-setup-docker-gpu/
with using image instead: quay.io/go-skynet/local-ai:latest (its using cublas/cuda during its build anyway now)
will let know how it goes

No, that was not the issue... Man this rabbithole has got me on death spiral... depressed even. My ineptitude screaming at every docker recompose, compile. Docker has always been weak in my IT brain, I know I shouldn't have had to download these packages this many times over again... installing with cublas... Ive got be missing an important memo here... Before I checked cuda boxes the docker seemed to be installing with cuda libraries on its own... Im going to try a no frills install and check if cuda acceleration works, then work within the working base package to try to get cuda libraries upgraded?

@nbollman It's not just you - I'm also having the same problem attempting to compile the image with documented defaults.

same here. fyi: i got the same error when try installing/building locally. So it it is not docker/docker-compose specific..

@mounta11n how did you resolve absl with local build?

@nbollman I was able to make some progress by:

Set REBUILD=false in .env
Use local-ai:latest (I'm assuming "latest" is vanilla localai, I still want to try out master-cublas-cuda12-ffmpeg)

Issue persists: macOS Sonoma 14, M2

Workaround:
Instead of trying to compile and run LocalAI directly on the host, I utilized a prebuilt docker image: quay.io/go-skynet/local-ai:v1.30.0-cublas-cuda12.
Set BUILD_TYPE=cublas and REBUILD=true environment variables.
Explicitly defined -DLLAMA_CUBLAS=on within CMAKE_ARGS.
Used FORCE_CMAKE=1 to ensure CMake processes run.

By utilizing the above Docker run command, I was able to get LocalAI running successfully with CUDA support.
I hope this helps! Let me know if you face any other challenges.

Hi there,
I had the same issue. I fixed by compiling the gRPC locally and installing to a local directory, called ${HOME}/installed_packages.
Then I passed a bunch of variables to the CMake to let it find the gRPC, absl, protobuf, etc.
Then I ran:
export INSTALLED_PACKAGES="${HOME}/installed_packages/linux_x86_64"

export INSTALLED_LIB_CMAKE="${INSTALLED_PACKAGES}/lib/cmake"
export ADDED_CMAKE_ARGS="-Dabsl_DIR=${INSTALLED_LIB_CMAKE}/absl"
export ADDED_CMAKE_ARGS="${ADDED_CMAKE_ARGS} -DProtobuf_DIR=${INSTALLED_LIB_CMAKE}/protobuf"
export ADDED_CMAKE_ARGS="${ADDED_CMAKE_ARGS} -Dutf8_range_DIR=${INSTALLED_LIB_CMAKE}/utf8_range"
export ADDED_CMAKE_ARGS="${ADDED_CMAKE_ARGS} -DgRPC_DIR=${INSTALLED_LIB_CMAKE}/grpc"
export ADDED_CMAKE_ARGS="${ADDED_CMAKE_ARGS} -DCMAKE_CXX_STANDARD_INCLUDE_DIRECTORIES=${INSTALLED_PACKAGES}/include"

export _PROTOBUF_PROTOC=${INSTALLED_PACKAGES}/bin/protoc
export _GRPC_CPP_PLUGIN_EXECUTABLE=${INSTALLED_PACKAGES}/bin/grpc_cpp_plugin
export PATH=${PATH}:${INSTALLED_PACKAGES}/bin

CMAKE_ARGS="${ADDED_CMAKE_ARGS}" make build

If it can be useful, I can create a separate script with all the commands and create a PR containing it.

I am also now hanging on the protobuf error... anyone solved it (not on mac)?

Hi there,
I submitted the PR #1232 to solve it.

The issue was fixed. With the code  in the master branch, it is possible to build gRPC locally by running the command:
BUILD_GRPC_FOR_BACKEND_LLAMA=ON make build

The issue was fixed. With the code in the master branch, it is possible to build gRPC locally by running the command: BUILD_GRPC_FOR_BACKEND_LLAMA=ON make build
I think it is possible to close the issue.

For clarification for other noobs like me: that means adding BUILD_GRPC_FOR_BACKEND_LLAMA=ON in the .env

The issue was fixed. With the code in the master branch, it is possible to build gRPC locally by running the command: BUILD_GRPC_FOR_BACKEND_LLAMA=ON make build
I think it is possible to close the issue.

For clarification for other noobs like me: that means adding BUILD_GRPC_FOR_BACKEND_LLAMA=ON in the .env and not in CMAKE_ARGS or elsewhere

CMAKE_ARGS="-DLLAMA_AVX512=ON" BUILD_GRPC_FOR_BACKEND_LLAMA=ON make BUILD_TYPE=clblas build
results here in some error on current master branch (commit 562ac62f59b55c831207a908f07144473c2c4599 (HEAD -> master, origin/master, origin/HEAD))
gmake[4]: *** [examples/grpc-server/CMakeFiles/hw_grpc_proto.dir/build.make:89: examples/grpc-server/CMakeFiles/hw_grpc_proto.dir/backend.grpc.pb.cc.o] Error 1
gmake[4]: Leaving directory '/home/m/git/LocalAI/backend/cpp/llama/llama.cpp/build'
gmake[3]: *** [CMakeFiles/Makefile2:2492: examples/grpc-server/CMakeFiles/hw_grpc_proto.dir/all] Error 2
gmake[3]: Leaving directory '/home/m/git/LocalAI/backend/cpp/llama/llama.cpp/build'
gmake[2]: *** [Makefile:146: all] Error 2
gmake[2]: Leaving directory '/home/m/git/LocalAI/backend/cpp/llama/llama.cpp/build'
make[1]: *** [Makefile:43: grpc-server] Error 2
make[1]: Leaving directory '/home/m/git/LocalAI/backend/cpp/llama'
make: *** [Makefile:428: backend/cpp/llama/grpc-server] Error 2

This error is reproducible in the official container for 1.40.0-ffmpeg at least.
I attach screenshot.

@EchedeyLR can you share your .env ?

Is exactly the default one at https://github.com/mudler/LocalAI/blob/v1.40.0/.env with GO_TAGS and REBUILD uncommented.

AFAIK it's not a fix but a workaround while the team has figured out why this started happening.
Personnaly here's what I needed to add at the end of my .env:
CMAKE_ARGS="-DLLAMA_AVX512=OFF -DLLAMA_AVX2=OFF -DLLAMA_FMA=OFF -DLLAMA_CUBLAS=on"
BUILD_GRPC_FOR_BACKEND_LLAMA=ON
FORCE_CMAKE=1

(I also set REBUILD=true and BUILD_TYPE=cublas)

Would this work in the container image?
I am also worried about these fixed CMAKE_ARGS. OpenBLAS did not build for me in go-llama since several months ago, just a version stopped working and I thought that was related to instruction set, as in other computer everything worked so far.
I thought this was self-detected and not hard coded, this just makes it difficult for people to run it in non-exactly modern computers (and no, I am not speaking about something from 2008 or so, I imply 2014 even).

CMAKE_ARGS="-DLLAMA_AVX512=OFF -DLLAMA_AVX2=OFF -DLLAMA_FMA=OFF" BUILD_GRPC_FOR_BACKEND_LLAMA=ON make BUILD_TYPE=clblas build is working in v1.30.0, but not in v1.40.0 nor on current master (274ace289823a8bacb7b4987b5c961b62d5eee99).

BUILD_GRPC_FOR_BACKEND_LLAMA=ON

If this is needed, why was not included in the example .env file in the release? I also don't see mention of it outside the issues in the clear documentation.

Is this needed in the docs? Please ping me if so

We need to confirm this is the issue yet. I am yet to try that solution. Other user reported it as nonworking

My problem was slightly different in that it was giving me errors for absl::check, absl::log, etc.
It turns out Conda installs its own libraries and binaries, which also meant it had its own incompatible version of absl.
I completely uninstalled Anaconda3, then I cloned grpc with
git clone --recurse-submodules -b v1.58.0 --depth 1 --shallow-submodules https://github.com/grpc/grpc
which I got from the Dockerfile
I built and installed abseil-cpp, protobuf and grpc from the folder above into /usr/local, after all libraries were installed I built LocalAI as usual.
Here are my build args for LocalAI:
CMAKE_ARGS="-DLLAMA_CUBLAS=ON -DLLAMA_F16C=OFF -DLLAMA_AVX512=OFF -DLLAMA_AVX2=OFF -DLLAMA_AVX=OFF -DLLAMA_FMA=OFF" make BUILD_TYPE=cublas GO_TAGS=stablediffusion,tts CUDACXX=/usr/local/cuda/bin/nvcc build
Building as I type this, seems to be ok so far
Ubuntu Server 22.04, Xeon X5650

Just a little report, coming from #1386 using BUILD_GRPC_FOR_BACKEND_LLAMA=ON for a local compile did the trick. cmake did not find my installed libproto* from debian.
Notable comment there from @B4ckslash

AFAIK this is because not every distribution (Fedora, Debian and derivatives among them) builds Protobuf with CMake, which leads to the .cmake files missing for those distros, so CMake cannot find them when building llama.cpp.

The issue was fixed. With the code in the master branch, it is possible to build gRPC locally by running the command: BUILD_GRPC_FOR_BACKEND_LLAMA=ON make build
I think it is possible to close the issue.

Thank You so much this worked in my Ubuntu WSL make build issue

My problem was slightly different in that it was giving me errors for absl::check, absl::log, etc.
It turns out Conda installs its own libraries and binaries, which also meant it had its own incompatible version of absl.
I completely uninstalled Anaconda3, then I cloned grpc with git clone --recurse-submodules -b v1.58.0 --depth 1 --shallow-submodules https://github.com/grpc/grpc which I got from the Dockerfile
I built and installed abseil-cpp, protobuf (inside grpc/third_party) and grpc from the folder above into /usr/local, after all libraries were installed I built LocalAI as usual.
Here are my build args for LocalAI: CMAKE_ARGS="-DLLAMA_CUBLAS=ON -DLLAMA_F16C=OFF -DLLAMA_AVX512=OFF -DLLAMA_AVX2=OFF -DLLAMA_AVX=OFF -DLLAMA_FMA=OFF" make BUILD_TYPE=cublas GO_TAGS=stablediffusion,tts CUDACXX=/usr/local/cuda/bin/nvcc build
Building as I type this, seems to be ok so far
Ubuntu Server 22.04, Xeon X5650

Could you provide a script how to do it? I am facing same build problems

i got everything fixeed "solution"
it was just i completed removed that ubuntu WSL and got a new Debian WSL which is very much better than it !!
i just download the release "local-ai-avx-Linux-x86_64" and ran it with a .sh script.
localai.sh

#!/bin/sh
export GALLERIES='[{"url": "github:go-skynet/model-gallery/huggingface.yaml","name":"huggingface"}]'
export CORS=true
export ADDRESS=localhost:4040
export DEBUG=true
export PARALLEL_REQUESTS=true
# export WATCHDOG_IDLE=true
export THREADS=18
./local-ai-avx-Linux-x86_64

i am using the model : thebloke__wizardlm-13b-v1-0-uncensored-superhot-8k-ggml__wizardlm-13b-v1.0-superhot-8k.ggmlv3.q4_k_m.bin , only tested with this model for now.
It works like charm 🪄 🌟


Version 1.2.2 causes error in Angular DI: No provider for CookieOptions

We didn't yet have the time to investigate further but going back to 1.2.1 definitely solves the issue.
If we find out more we post an update here.
npm ls | grep angular
├──<EMAIL_ADDRESS>├──<EMAIL_ADDRESS>├──<EMAIL_ADDRESS>├──<EMAIL_ADDRESS>├──<EMAIL_ADDRESS>├──<EMAIL_ADDRESS>├──<EMAIL_ADDRESS>├─┬<EMAIL_ADDRESS>

Hey @macolo
Thanks for your feedback. I am inspecting the issue. It would be really helpful if you could post your bootstrap function and custom cookie options class.

Thank you for your help!
Here you go: https://gist.github.com/macolo/b37f0d859c863172e17359e76c11ef94

Any update on the Issue ? How to use this On the Application ?

Try to use the latest version. And there is a huge getting started section on the readme file.

I'm getting this error now when I do AOT compilation with angular-cli.  Works find with a non-AOT  (JiT) build.

Same here, getting the error when building with --aot

Getting the same with angular 2.4.2, ng-cli 1.0.0-beta24 and angular-cookies 1.2.6.
Error: No provider for CookieOptions!

@salemdar, can we re-open this?
The only way I can get it to work is if I import like this:
import { NgModule } from "@angular/core";
import { CommonModule } from "@angular/common";
import { CookieService, BaseCookieOptions, CookieOptions } from "angular2-cookie/core";

export function cookieServiceFactory(options: CookieOptions) {
  return new CookieService(options);
}

@NgModule({
  imports: [
    CommonModule,
    ...
 ],
  providers: [
    { provide: CookieOptions, useClass: BaseCookieOptions },
    { provide: CookieService, useFactory: cookieServiceFactory, deps: [CookieOptions] },
    ...
  ]
})
export class MyModule {
}

Which basically copies your core file: https://github.com/salemdar/angular2-cookie/blob/master/src/core.ts
I believe the issue is that you are not exposing a module for us to import, and by only recommending we declare the CookieService in our modules, DI cant find an implementation for CookieOptions. I think if you were to instead have a module that 'provided' both of these for us, we could import that module and be done.

Correction... here is my module:
import { NgModule } from "@angular/core";
import { CommonModule } from "@angular/common";
import { CookieService, BaseCookieOptions, CookieOptions } from "angular2-cookie/core";

export function cookieServiceFactory() {
  return new CookieService();
}

@NgModule({
  imports: [
    CommonModule,
    ...
 ],
  providers: [
    { provide: CookieService, useFactory: cookieServiceFactory },
    ...
  ]
})
export class MyModule {
}

If I try to import the CookieOptions, I get an unknown token error after building for AOT. However, I still think the issue is caused by the fact that a CookieModule is not being exposed with the proper things already 'provided' for us to import.

Having the same issue. Downgrading to previous version until this is fixed!

Same error

Still getting the error with angular 2.4.10.

Use ngx-cookie instead

I have used ngx-cookie but i still got the following error while iam doing ng serve --eot
Error: No provider for Token COOKIE_OPTIONS!

Are you importing the module properly @iravitejakumar ?

Yes @rolandoldengarm  I have imported module properly. I have declared my provider in app.module as follows.
@NgModule({
  declarations: [
    AppComponent,
  ],
  imports: [
    BrowserModule,
    FormsModule,
    HttpModule,
    RouterModule.forRoot(routes, { useHash: true })
  ],
  providers: [CookieService,{ provide: CookieOptions, useValue: {} }],
  bootstrap: [AppComponent]
})

It solved my problem and its working fine now.

@iravitejakumar
all I have is the import import { CookieModule } from 'ngx-cookie';
and:
imports: [
    CookieModule.forRoot()
]

Then, wherever I use it, I have import { CookieService } from 'ngx-cookie';
And I also inject it through the constructor, of course constructor(private cookieService: CookieService) {}
That's all, no providers. It's working perfect for me.

@iravitejakumar you are not importing the CookieModule... As I said.

I was getting the same error in karma testing and @MrCroft 's answer pointed in the right direction, I was missing the .forRoot() in the app.component.spec.ts file, same as the app.module.ts
describe('AppComponent', () => {   beforeEach(async(() => {     TestBed.configureTestingModule({       imports: [         RouterTestingModule,         HttpClientModule,         CookieModule.forRoot(),       ],       declarations: [ ],       providers: [ ]     }).compileComponents();   }));


try out by adding the needs field

Fixes #136

that is the culprit - might need to solve both issues in one go for this to pass
https://github.com/DiamondLightSource/blueapi/issues/411

Please give this PR a sensible title


about timer module, some advise

Timer module was single gen_server process, when create massive process with hnc_pool gen_statem module. That will face a Bottleneck.
Using the gen_statem  Generic Time-Outs like "{keep_state, Data, [{{timeout,Ref},TimoutMs, TimeoutMsg}]}".
I think It was better option.

Thanks, we will look into it :)

https://github.com/erlang/otp/pull/4811 😉


create interface navbar and login page (not functional yet)

What I did:

Made navbar for user's navigation (src/components/Navigation/Navbar.js)
Login interface (just the interface, not yet connected to api)
I accidentally push some api for category controller, I forgot to push it before (sorry ✌)

Approved


Failed to install grpcio using pip in Mac M1

What version of gRPC and what language are you using?
Tried versions from grpcio==1.46.5 to latest grpcio==1.50.0
What operating system (Linux, Windows,...) and version?
MacBook Pro M1
Darwin pata-mac 21.5.0 Darwin Kernel Version 21.5.0: Tue Apr 26 21:08:37 PDT 2022; root:xnu-8020.121.3~4/RELEASE_ARM64_T6000 arm64
What runtime / compiler are you using (e.g. python version or version of gcc)
Using virtualenv created with pyenv Python 3.9.13.
with pip install --upgrade pip updated to version  pip-22.3
pata@pata-mac:~ % pyenv --version
pyenv 2.3.2

What did you do?
Tried to install grpcio using pip like:
pip install grpcio
Also tried solutions from here: https://github.com/grpc/grpc/issues/30064
GRPC_PYTHON_BUILD_SYSTEM_OPENSSL=true GRPC_PYTHON_BUILD_SYSTEM_ZLIB=true pip install grpcio - same error
What did you expect to see?
Successful installation
What did you see instead?

      During handling of the above exception, another exception occurred:

      Traceback (most recent call last):
        File "/private/var/folders/y8/y8t_4g1113j90q4x_8_lz6j00000gp/T/pip-install-khdankw6/grpcio_11aeae1f2be942b7b5acc995495aeb9e/src/python/grpcio/commands.py", line 280, in build_extensions
          build_ext.build_ext.build_extensions(self)
        File "/Users/pata/.pyenv/versions/3.9.13/lib/python3.9/distutils/command/build_ext.py", line 449, in build_extensions
          self._build_extensions_serial()
        File "/Users/pata/.pyenv/versions/3.9.13/lib/python3.9/distutils/command/build_ext.py", line 474, in _build_extensions_serial
          self.build_extension(ext)
        File "/Users/pata/.pyenv/versions/3.9.13/envs/cookie-creator/lib/python3.9/site-packages/setuptools/command/build_ext.py", line 202, in build_extension
          _build_ext.build_extension(self, ext)
        File "/Users/pata/.pyenv/versions/3.9.13/lib/python3.9/distutils/command/build_ext.py", line 529, in build_extension
          objects = self.compiler.compile(sources,
        File "/private/var/folders/y8/y8t_4g1113j90q4x_8_lz6j00000gp/T/pip-install-khdankw6/grpcio_11aeae1f2be942b7b5acc995495aeb9e/src/python/grpcio/_parallel_compile_patch.py", line 58, in _parallel_compile
          multiprocessing.pool.ThreadPool(BUILD_EXT_COMPILER_JOBS).map(
        File "/Users/pata/.pyenv/versions/3.9.13/lib/python3.9/multiprocessing/pool.py", line 364, in map
          return self._map_async(func, iterable, mapstar, chunksize).get()
        File "/Users/pata/.pyenv/versions/3.9.13/lib/python3.9/multiprocessing/pool.py", line 771, in get
          raise self._value
        File "/Users/pata/.pyenv/versions/3.9.13/lib/python3.9/multiprocessing/pool.py", line 125, in worker
          result = (True, func(*args, **kwds))
        File "/Users/pata/.pyenv/versions/3.9.13/lib/python3.9/multiprocessing/pool.py", line 48, in mapstar
          return list(map(*args))
        File "/private/var/folders/y8/y8t_4g1113j90q4x_8_lz6j00000gp/T/pip-install-khdankw6/grpcio_11aeae1f2be942b7b5acc995495aeb9e/src/python/grpcio/_parallel_compile_patch.py", line 54, in _compile_single_file
          self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
        File "/private/var/folders/y8/y8t_4g1113j90q4x_8_lz6j00000gp/T/pip-install-khdankw6/grpcio_11aeae1f2be942b7b5acc995495aeb9e/src/python/grpcio/commands.py", line 263, in new_compile
          return old_compile(obj, src, ext, cc_args, extra_postargs,
        File "/Users/pata/.pyenv/versions/3.9.13/lib/python3.9/distutils/unixccompiler.py", line 120, in _compile
          raise CompileError(msg)
      distutils.errors.CompileError: command '/usr/bin/clang' failed with exit code 1

      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure

× Encountered error while trying to install package.
╰─> grpcio

note: This is an issue with the package mentioned above, not pip.
hint: See above for output from the failure.

Facing the same problem issue.

running this worked for me:
CFLAGS="-I /opt/homebrew/opt/openssl/include" LDFLAGS="-L /opt/homebrew/opt/openssl/lib" GRPC_PYTHON_BUILD_SYSTEM_OPENSSL=1 GRPC_PYTHON_BUILD_SYSTEM_ZLIB=1 pip install grpcio
But only in new virtualenv created from scratch. Nothing worked for my existing virtualenv.
GRPC_PYTHON_BUILD_SYSTEM_OPENSSL=true GRPC_PYTHON_BUILD_SYSTEM_ZLIB=true pip install grpcio - didn't work.


Add missing “:” for R code

See Issue https://github.com/jupyter/repo2docker/issues/898

Thanks a lot!


Update README.md

Purpose

...

Does this introduce a breaking change?

[ ] Yes
[ ] No

Pull Request Type
What kind of change does this Pull Request introduce?

[ ] Bugfix
[ ] Feature
[ ] Code style update (formatting, local variables)
[ ] Refactoring (no functional changes, no api changes)
[ ] Documentation content changes
[ ] Other... Please describe:

How to Test

Get the code

git clone [repo-address]
cd [repo-name]
git checkout [branch-name]
npm install

Test the code

What to Check
Verify that the following are valid

...

Other Information

I created this PR  because the original PR was accidentally closed.


Improve RUM integration tests

What does this PR do?
Add missing tests to get a better coverage of the RUM apis  in the Integration Tests

Codecov Report
Attention: Patch coverage is 92.30769% with 2 lines in your changes missing coverage. Please review.

Project coverage is 70.31%. Comparing base (4b83197) to head (fb18021).

Files with missing lines
Patch %
Lines

...droid/rum/tracking/ActivityViewTrackingStrategy.kt
88.24%
1 Missing and 1 partial :warning:

Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #2317      +/-   ##
===========================================
+ Coverage    70.29%   70.31%   +0.01%
===========================================
  Files          736      737       +1
  Lines        27469    27458      -11
  Branches      4607     4605       -2
===========================================
- Hits         19309    19305       -4
  Misses        6868     6868
+ Partials      1292     1285       -7

Files with missing lines
Coverage Δ

.../rum/internal/monitor/AdvancedNetworkRumMonitor.kt
0.00% <ø> (ø)

.../rum/tracking/ActivityLifecycleTrackingStrategy.kt
88.46% <ø> (-1.54%)
:arrow_down:

...tlin/com/datadog/android/rum/tracking/BundleExt.kt
100.00% <100.00%> (ø)

...droid/rum/tracking/FragmentViewTrackingStrategy.kt
85.71% <100.00%> (-3.17%)
:arrow_down:

...oid/rum/tracking/NavigationViewTrackingStrategy.kt
86.36% <100.00%> (ø)

...roid/compose/internal/ComposeNavigationObserver.kt
90.91% <100.00%> (+1.62%)
:arrow_up:

...droid/rum/tracking/ActivityViewTrackingStrategy.kt
87.76% <88.24%> (-2.87%)
:arrow_down:

... and 33 files with indirect coverage changes


test(twap): table-driven test for AfterEpochEnd

Background
#2294 added pruning logic to the AfterEpochEnd hook.
We should add some test cases to ensure full coverage of the hook
Suggested Design

test that it is no-op if current epoch != prune epoch
test that pruning occurs with the correct "time to keep records" set

Ensure that records earlier than the expected time are deleted
Ensure that records later or at the expected time are kept

test that it does not panic on pruning error

Acceptance Criteria

full test coverage as per suggested design

This is blocked by #2294

Closed by #2443


Correct Travis build status icon link in README.md

Currently, clicking link takes you to the badge image

Please rebase your PR against the current master. Failing tests were due to spurious eslint issues. Thanks.


consider passing the model directly to gui view objects

currently, GUI views don't touch the model... they just notify the controller of a change, and then the controller requests the changed value from the view before passing it to the model:
class SomeView:
    currentIndexChanged = Signal()
    def __init__(self):
        self.dims_sliders = ...
        self.dims_sliders.currentIndexChanged.connect(self.currentIndexChanged.emit)
    def current_index(self): ...

class Controller:
    def __init__(self):
        self.model = ...
        self.view = ...
        self.view.currentIndexChanged.connect(self._on_view_current_index_changed)

    def _on_view_current_index_changed(self) -> None:
        self.model.current_index.update(self.view.current_index())

That works fine, but it requires a lot of signal machinery to be on the View objects, and adds some indirection.   We could alternatively just pass the model to the frontend widget:
class SomeView:
    def __init__(self, model):
        self.model = model
        self.dims_sliders = ...
        self.dims_sliders.currentIndexChanged.connect(self._on_view_current_index_changed)

    def _on_view_current_index_changed(self, value) -> None:
        self.model.current_index.update(value)

class Controller:
    def __init__(self):
        self.model = ...
        self.view = ...

having the view control the model is not necessarily an unusual pattern.  Most of Qt view widgets have direct access to and mutate their models (it's mostly a model-view only pattern).
not sure yet whether this would make it harder to keep the different views behaving similarly

on a quick stab at implementing it, I ran into one other issue that would likely need addressing:  the view has the ability to reset the range of the canvas.  Ideally this would actually correspond to a change on the model, but we don't yet model the camera in the display model.    That's a big ball of wax that was a real sticking point in microvis ... but we'll need to tackle it someday


Fix JSON indentation

@civitaslearning/platform A long-standing annoyance for me... we return incorrectly indented JSON because at the time I added this to go-restful, I didn't understand the correct usage of json.MarshalIndent.
As a result, we get back stuff that looks like this:
$ curl http://api.private.civitaslearning.com/import/v1/version
{
  "version": "<IP_ADDRESS>7",
  "service": "import",
  "build": {
   "commit": "99a3f8b49b3a0e50e623b25c6cbaee54dd3761c8 origin/master",
   "build_time": "2016-04-21T22:37:14Z"
  }
 }
$

note the final } with a space before it.

:shipit:
BTW what is the way to make Firehose spit out plain text and not go automatically to json? That comment in simplified.go annoys me...

@troy-civitas I think the trick is to use a lower-level response writer; it's WriteEntity() that is doing the conversion (I think go-restful knows that the request wants a JSON response, or maybe that the endpoint is configured to return JSON, so renders the entity as JSON).  But there's a way to bypass the go-restful layer I'm pretty sure.


RichText URL link will not activate on click in certain conditions

axmol version: All current
devices test on:
developing environments

NDK version: r23c
Xcode version: 14.2
Visual Studio:

VS version: 2019 (16.11), 2022 (17.6+)
MSVC version: 1929, 1934, 19.35, 19.36, 19.37
Windows SDK version: 10.0.22621.0

cmake version:
Steps to Reproduce:

auto visibleSize = _director->getVisibleSize();

auto* layout = ui::Layout::create();
layout->setContentSize(visibleSize);
layout->setAnchorPoint(Vec2::ANCHOR_MIDDLE);
layout->setPosition(Vec2(visibleSize.width / 2, visibleSize.height));
this->addChild(layout);

auto* listener = ax::EventListenerTouchOneByOne::create();
listener->setSwallowTouches(true);
listener->onTouchBegan = [](Touch*, Event*) -> bool { return true; };
_eventDispatcher->addEventListenerWithSceneGraphPriority(listener, layout);

auto* richText = ui::RichText::createWithXML("click me");
richText->ignoreContentAdaptWithSize(false);
richText->setContentSize(Size(500, 200));
richText->setPositionNormalized(Vec2(0.5f, 0.5f));

layout->addChild(richText);

If a RichText with a clickable URL is a in the scene, and there is another node in the scene that has a EventListenerTouchOneByOne listener, and the onTouchBegan returns true (along with setSwallowTouches(true)), then the URL handler will not be triggered.
The URL touch handler is implemented with this code (in UIRichText.cpp, class ListenerComponent):
_touchListener                 = ax::EventListenerTouchAllAtOnce::create();
_touchListener->onTouchesEnded = AX_CALLBACK_2(ListenerComponent::onTouchesEnded, this);
Director::getInstance()->getEventDispatcher()->addEventListenerWithSceneGraphPriority(_touchListener, _parent);

All EventListenerTouchAllAtOnce listeners will never be triggered if any EventListenerTouchOneByOne exist that returns true on onTouchBegan.
The reason is in function EventDispatcher::dispatchTouchEvent(EventTouch* event), where the dispatcher first calls all EventListenerTouchOneByOne listeners, and then if any return true for onTouchBegan, it removes the entries from a variable named mutableTouches (which holds all current touches).
Once it loops through all EventListenerTouchOneByOne listeners, it then gets to the following code which should handle all the EventListenerTouchAllAtOnce event listeners:
if (allAtOnceListeners && !mutableTouches.empty())
{
... handles all `EventListenerTouchAllAtOnce` here
}

The problem is that the if condition is false, since mutableTouches is empty, so it never enters that block, and no EventListenerTouchAllAtOnce listeners ever receive the touch inputs.
The main issue is that EventListenerTouchOneByOne and EventListenerTouchAllAtOnce are not sorted together (by priority and Z order), so even if a node using EventListenerTouchAllAtOnce is on top of every other node in the scene, it would never be called because of other nodes using EventListenerTouchOneByOne.  This is what doesn't make sense to me.
My question is, does anyone know why EventListenerTouchAllAtOnce was used for the URL touch handler?  If EventListenerTouchOneByOne is used, then this issue does not occur, but I'm not sure if there would be any side-effects.
All Widget sub-classes (like ui::Button etc) use EventListenerTouchOneByOne, and they all work as intended, even to call openURL.
One other thing, EventListenerTouchAllAtOnce is ONLY used for the RichText URL handler; I can't find any usage of it at all in the Axmol source code (not including the cpp_tests project of course).

The way the current touch system works with regards to EventListenerTouchAllAtOnce and EventListenerTouchOneByOne is unexpected, and in my opinion, not correct.  For example:
root_node (EventListenerTouchOneByOne)/
├─ node_1 (EventListenerTouchOneByOne)/
   ├─ node_2 (EventListenerTouchOneByOne)/
      ├─ node_3 (EventListenerTouchAllAtOnce)/

If you have a node tree as such, where node_3 is the one with the highest Z value, I would expect touch handling to be done in the order:
node_3, node_2, node_1, and finally root_node.
What actually happens at the moment is this order:
node_2, node_1, root_node, and finally node_3
From my understanding, touch handling should be done from the the highest Z order to the lowest.  Splitting it into two different loops, one for EventListenerTouchAllAtOnce and another for EventListenerTouchOneByOne, means that the Z ordering between these two types of event listeners is not catered for.
I don't see any simple way to fix the EventDispatcher for this (without performance issues and such).
Now, regarding the RichText URL issue, I'll submit a PR to use EventListenerTouchOneByOne instead of EventListenerTouchAllAtOnce for the touch handling, and if anyone finds issues with it, then we'll figure out what to do next.


Verbose mode

Hi, nice work first of all!
Even though the output while loading the codes and vocab files can be helpful, I'd prefer it to be toggled by a verbose commandline parameter. Otherwise you can't directly use the stout while using the python module.
Thats the output:
Loading vocabulary from bpe.30000.vocab ...
Read<PHONE_NUMBER> words (30708 unique) from vocabulary file.
Loading codes from bpe.30000.codes ...

Ah ok, I see its printed to stderr. Forget everything...


add function buttons

add buttons which just add the function as a string to the function text field
example:

alternative: link to what to use functions
https://imagej.nih.gov/ij/developer/macro/functions.html

deprecated


doc: add reference from ignore_volume_az to cross_az_attach in doc

What this PR does / why we need it:
Which issue this PR fixes(if applicable):
fixes #
Special notes for reviewers:

Release note:

NONE

/lgtm
/approve


[client] Claiming hyperstructure does not update points

At the moment, claiming a hyperstructure does not compute points for the new owner

This will be covered by #1088


minMetricSamplesToAlarm uses incorrect evaluation period

Version
6.2.2
Steps and/or minimal code example to reproduce
Reproducible with unit tests included in the package: https://github.com/cdklabs/cdk-monitoring-constructs/blob/84609a4d81ad5367ee14a442411c82fa4d368407/test/common/alarm/AlarmFactory.test.ts#L275
Expected behavior
NoSamples child alarm should use the same datapointsToAlarm and evaluationPeriods as the alarm that we are attaching minMetricSamplesToAlarm to.
Example:
  factory.addAlarm(metric, {
    ...props,
    alarmNameSuffix: "none",
    comparisonOperator: ComparisonOperator.LESS_THAN_THRESHOLD,
    minMetricSamplesToAlarm: 500,
    datapointsToAlarm: 5,
    evaluationPeriods: 5,
  });

should create a NoSamples alarm with datapointsToAlarm: 5 and  evaluationPeriods: 5.
Actual behavior
NoSamples child alarm uses datapointsToAlarm and evaluationPeriods different than the main alarm - both are set 1 instead.
Other details
The fact that the datapointsToAlarm and evaluationPeriods  for NoSamples child alarm are using different values (i.e. 1), causes incorrect Composite Alarm behaviour.
Imagine monitor where you require 5 breaching datapoints to go into alarm with the minMetricSamplesToAlarm configured to an arbitrary value. Consider the following:
NoSamplesAlarm:         [X, X, X, X, 0, 0, 0, 0, 0]
MainAlarm:              [X, X, X, X, X, 0, 0, 0, 0]
---
CompositeWithSamples:   [0, 0, 0, 0, X, 0, 0, 0, 0]

where 0 is a non-breaching data point, X is a breaching data point

Even though there was only a single datapoint that satisfied both conditions, the monitor went into alarm despite the fact that the MainAlarm had datapointsToAlarm configured to 5 datapoints. NoSamplesAlarm generated by cdk-monitoring-constructs is based on datapointsToAlarm: 1, so if that single datapoint coincides with MainAlarm reaching its own threshold, the overall CompositeWithSamples will go into ALARM anyway. This is doesn't seem like a desirable behaviour - for the CompositeWithSamples to go into ALARM, all breaching datapoints should have sufficient number of samples.
If this is not a bug, is there any rationale behind the NoSamples using a different datapointsToAlarm and evaluationPeriods  than the MainAlarm?

So excited to see this happen. This will be such an improvement for our monitoring


Home page style changed for responsive design , fixed css .

Fixed some css style of "What we do?" and "Who we are?" .
Removed some unnecessary styles which was causing bad design .
Made Carousel size little small for better look .

Here are some screenshots:

issue number is #11 .


Rethink room aliases

Rooms need to be uniquely identified by something other than their name. Currently, a number is used. Should we use something else, like the name + #number, as is the case of item ids?

I don't think replacing numbers by names would make it any easier. Makers will probably still need to go to the target room to check and note it's name, and a number is faster and easier to type.
What I'll do is replace the "alias" name of the field by room_number, which I feel is more descriptive.


Lazy 1.20.6 fabric port

Thank you for the PR! I've made a release on CurseForge/Modrinth using it, and unascribed will merge this when she's able to - she's been dealing with a major personal emergency that's left her without computer access for the time being.


Suggest to Create Role by Terraform, Got: ClientException: Role is not valid

I got the error below when run your code
Error: ClientException: Role is not valid
	status code: 400, request id: dd60557f-6d91-4c15-8ab9-6d71b20f4609
  on ecs.tf line 19, in resource "aws_ecs_task_definition" "app":
  19: resource "aws_ecs_task_definition" "app" {

I suspect it because you hardcoded the aws_ecs_task_definition Role, I suggest to creat thr Role by terraform.

Hey @geekeren, originally this was part of a 2-part tutorial where the role was made manually in the AWS GUI in step one. Someone has made a PR with the role built out in terraform and I will be adding it soon! Thanks!!

@bradford-hamilton Got it. Thank you.

Creation of role in terraform was merged in from the other person's PR 👍


First create table using persistent in datasource

Obsolete PR. A similar feature is being developed in branch feature/persistentCatalog


Switch to cursor-based pagination with geared_pagination

Closes: https://github.com/Shopify/maintenance_tasks/issues/362
As mentioned in the issue, Pagy uses offset-based pagination, which decreases in performance as the collection of elements being paginated grows. It doesn't scale very well, and queries that use OFFSET are outright banned in Core for performance reasons. A cursor based pagination strategy queries based on a specific point in the dataset (the cursor), and works well for
sequential datasets / datasets with ordered attributes (maintenance_tasks_runs table is guaranteed to be sortable by created_at ).
Basecamp has actually written a gem that supports cursor-based pagination, which I think is a really good candidate for us to use, replacing Pagy.
Implementation

As the gem's README notes, we use set_page_and_extract_portion_from with the ordered_by  option to perform cursor-based pagination
Using this form of pagination requires the original relation to be unordered, so I've dropped the ordering on #previous_runs, since the pagination will handle this for us now
This provides us with an instance var @page, which contains the recordset of paginated elements which we can render in the view (similar to what was happening with Pagy)
What is config/initializers/geared_pagination.rb all about?

The gem works by including GearedPagination::Controller in ActionController::Base when the gem is initialized
-  GearedPagination::Controller sets an after_action filter, set_paginated_headers
- This currently breaks our test suite, because set_paginated_headers checks an ivar that doesn't exist (@page) which warns and thus raises in our test suite. I also don't really like that this filter runs before every single controller action in the engine and the host app.
- So, I've added an initializer that removes this filter and puts the onus on us to call set_paginated_headers when we need to (for now, this is only in TasksController#show)
- Arguably this is a little brittle. We could write a wrapper to isolate the call to set_paginated_headers in case we want to reuse pagination elsewhere, but I thought maybe that was premature

To 🎩

Create a bunch of runs in the dummy (ie. rails db:seed a couple times)
Check out http://localhost:3000/maintenance_tasks/tasks/Maintenance::TestTask and verify no pagination
Check out http://localhost:3000/maintenance_tasks/tasks/Maintenance::UpdatePostsTask and test the pagination

Tradeoffs

We lose the ability to show individual "pages" and to go back to a previous page. However, I think it will be pretty rare that a ton of pages will be needed
We also gain the ability to provide a URL to a given page based on a cursor now which is kind of nice

Opted for https://github.com/Shopify/maintenance_tasks/pull/371 instead


 Directly after cloning the repository I get:
~/projects/example-dvc-prototype main ❯ dvc list . -R --show-json --dvc-only
ERROR: unexpected error

Having any troubles? Hit us up at https://dvc.org/support, we are always happy to help!

Usually pulling the full repository can help this. However, I then run into:
~/projects/example-dvc-prototype main ❯ dvc pull                                                                   ✘ 255 24.76507s  3.0.0 14:49:28
WARNING: Some of the cache files do not exist neither locally nor on remote. Missing cache files:
name: None, md5: e4476c3400218a28ccf00ae81bf600f5.dir
WARNING: Some of the cache files do not exist neither locally nor on remote. Missing cache files:
name: None, md5: d8af0894c6df57665037bf9b94bfb520.dir
WARNING: No file hash info found for '/Users/mattseddon/projects/example-dvc-prototype/models'. It won't be created.
WARNING: No file hash info found for '/Users/mattseddon/projects/example-dvc-prototype/live/scalars'. It won't be created.
A       data/
1 file added and 2 files failed
ERROR: failed to pull data from the cloud - Checkout failed for following targets:
/Users/mattseddon/projects/example-dvc-prototype/models
/Users/mattseddon/projects/example-dvc-prototype/live/scalars
Is your cache up to date?
<https://error.dvc.org/missing-files>

Any subsequent attempt to call list with -R gives the same error as before until an experiment has been run.
Then I ran into
~/p/example-dvc-prototype main !3 ❯ dvc list . -R --dvc-only --show-json                                           0.02676s  .env  3.0.0 15:09:21
ERROR: URL 's3://' is supported but requires these missing dependencies: ['s3fs', 'boto3']. To install dvc with those dependencies, run:

        pip install 'dvc[s3]'

See <https://dvc.org/doc/install> for more info.

Finally, after all of that, I recreated your issue:
https://user-images.githubusercontent.com/37993418/176597628-7c737f58-fddf-49c2-af2b-518eadece56e.mov
The error here is that you have both the folder and the record selected when you right-click. To mitigate you must remove the selection of the folder before trying to pull:
https://user-images.githubusercontent.com/37993418/176598028-62f63b3f-b0d5-4b96-bafc-9ea04aa58c95.mov

There must be more rules that we need to put in place to decide if a record is selected or not.

Thanks @mattseddon for the research! Q - why was it partial though? (pulled only part of the files in the directory?)

Looks like you pulled everything in data/validation/cats. The first 1000 files are in data/train/cats.

Yes, you are right. Not sure tbh what is the best way to fix this ... anything similar to the project explorer and Git?

Default behaviour with delete in the explorer tree:
If multiple items are selected but the item right-clicked on is not within the selected list then only perform the action on the item that has been right-clicked on:
https://user-images.githubusercontent.com/37993418/176798664-ab44c642-8a7c-41d4-b624-ced698adae0a.mov
If multiple items are selected AND the item right-clicked is within the selected list then perform the operation on all items selected:
https://user-images.githubusercontent.com/37993418/176798772-abb2149d-74ff-4bc9-ae81-1f248d2156ca.mov
I will see if we can recreate this logic and raise a PR.


Add Frisian to lib/translations.py

Hi @chubin,
I added Frisian to lib/translations.py
For the words it would be this:

Morning: Moarns
Noon: Middeis
Evening: Jûns
Weather report for: Waarberjocht foar:

You've forgotten "Night", but I suppose it should be "Nachts" in this case.
Thank you very much for the translation; I will activate it now

It looks perfect!
Thank you very much @anned20
Do you want to translate the help file (help.txt => help-fy.txt)
so that we could finalize the translation and move "fy" from PARTIAL_TRANSLATION to FULL_TRANSLATION?


Adding the hot jar tracking info to the search results page

What is it and who's it for?
Search header research
To complete this research the hotjar code needs adding to the search results pages. Currently the code is only on the archives pages that was being used for previous research.
Note: There is new flexibility in hotjar to have "continous heatmaps". If the code is set on any page, the heat maps can be created and pages filtered within hot jar rather than having to alter the code everytime. The code and info is available if you go to https://insights.hotjar.com/sites/3858/heatmap/view and click the tracking button in the top right corner.
I am not 100% sure if this differs to what we were using previously.

If you need anymore info the help page is here https://help.hotjar.com/hc/en-us/articles/360034216634-Hotjar-JavaScript-Functions-Reference

Currently the code is only on the archives pages that was being used for previous research.

Has that research finished now (and can we subsequently remove the code that we'd used to run it)?


set primary-color as default focus color

Added --paper-input-container-focus-color: var(--primary-color);
For issue #51

Ah, please next time take a quick look to the CONTRIBUTING guidelines. It would be great if from now on everyone also fills CHANGELOG.md and other steps that are necessary. I will do it for you :)


During outbound provisioning user and role not able to sync in another store

Hi Team
I setup two wso2 identity server 5.10. It wso2-is-1 have store1 and another wso2-is-2 have store2. I configured both user store for outbound provisioning. Its working fine. Now when I stop the store2 server and create some users and group in store1 now I up the store2 server but those users and group which i created while store 2 was down not got synced in store2. How we can do ? second thing when any store1's user update his password how it will synced in store 2 server?

This issue is being closed due to extended inactivity. Please feel free to reopen it if further attention is needed. Thank you for helping us keep the issue list relevant and focused!


Flickering / redraws.

I'm using rather big svg images, not icons with react-inlinesvg.
There seems to be a lot of flickering and redraws when doing new routes / redraving parts of UI.
Somehow React seems to think it needs to redraw the svg elements a lot.
I have not yet investigated this yet, but I'll experiment with pure component mixins.

Off, yeah. We need to implement shouldComponentUpdate.

Hey @matthewwithanm
I'm having this problem too but appears that you aren't updating this repo anymore.
Are you accepting pull requests? new contributors?
I've been working a lot with svg svgporn and I'd love to help.
[s]

Fixed f36492593d7009ade7d972f5dbd386991643ab54


.NET should have a good practices guide about managing/transfer (maybe storing too) sensitive data and secrets

In advance, sorry for my english.
Help us make content visible
Searching about SecureString, Password, PasswordBox. And Password string fields/properties around in .net APIs.
In SecureString doc we can see a warning telling us "what happen" with SecureString and discouraging about using it.
Describe the new article
We can read in some issues in runtime repo about SecureString obsolescence, Secret, ShroudedBuffer... as new alternatives, etc...
But nowhere, any doc, teach or advice about how to manage passwords securely in .Net.
Workflows, classes, best practices, alternatives, common fails...
This should be linked in System.Security namespace documentation. And can be an "extra resource" or be in "learning".
Outline:
-Concerns about security and secrets.
-Light explanation about windows low-level sensitive data managing/certificates, etc... Or a link to existing doc.
-What to use in .Net for this.
-Managing passwords or sensitive data in .Net. What to do, what NOT to do.
-Code examples.
-Certificates/other solutions instead passwords.
-Asp.Net relations/SQL Server cryptography links.
-Code examples.

@Rick-Anderson Do we have a doc like this in the ASP.NET Core docs?

https://docs.microsoft.com/en-us/aspnet/core/security/app-secrets is a .NET tool


ModelBuilder Invalid overflow

https://github.com/libgdx/libgdx/blob/31ac4d900bffe10c4b194a912a1289fa1e42ad3c/gdx/src/com/badlogic/gdx/graphics/g3d/utils/ModelBuilder.java#L53
In my case
lastIndex == 65535
mb.lastIndex() == -1 (as Short)
mb.lastIndex() < Short.MAX_VALUE / 2 == true

Confirmed. I PRed a fix proposal.


[OpenAI] Precise Typechecking

Behind the scene, TSHY rewrites tsconfig.json to build the source code of the library but it ignores tests and samples. This PR breaks the monolith tsconfig.json into three ones, one for building the source, one for the tests, and one for the samples. It utilizes Project References to create a TS project each. This setup makes sure type errors are adequately caught in each of those projects.
Additionally, the PR disables rolling up d.ts files and fixes a type error in tests.

API change check
API changes are not detected in this pull request.


Feature/simplify annotation css

Quick fixes on the annotation hover design. These are temporary fixes but should improve selection/deleting of text and provide a more relevant annotation hover for decisions.

Show less annotations on hover (if no content value/translation is set, nothing is shown > easier to override in GN)
Make the annotation sidebar more compact
Remove block styling on inline elements
Small fixes to the flagged annotations: more relevant translations and a fix on selecting the text (after element was blocking selection)

@Dietr The "vul in" is quite prominent now and wasn't before, not sure if you intended this?


remove systemd-based distributions issue from  docs

The issue with systemd is closed and supposed to be fixed in v1.14, therefore manually adjusting systemd should not be necessary anymore.
Fixes: #18706

The CI failed due to a temporary breakage in main, this has since been resolve, please rebase on main to fix the CI checks.

Wondering if the doc should stay but instead be updated with the version of systemd where this error occurs and a note that it should no longer be an issue?
Not sure how quick distros bump their systemd versions, but I imagine slower distros like Debian may wait a bit?

If I understood correctly, this PR is meant to fix the problem for all systemd versions entirely? Maybe @NikAleksandrov can help us out here

Oh I miss understood this. I thought systemd the cause of an issue, not us.

Docs only change, no need to run full test suite.

Oh I miss understood this. I thought systemd the cause of an issue, not us.

That's true, but the PR mentioned above should make it work anyway. At least that's my understanding. Unfortunately I could not find anything about it in the release notes.


Upgrading and Securing RabbitMQ

Motivation and Context
This change upgrades the RabbitMQ operator, it upgrades the RabbitMQ docker container versions and also delivers a gaggle of hardening and security features for the RabbitMQ cluster.
What has changed
Changes are applied to the microservices, the tests, the docker development project and the central RabbitMQ provisioning code in the Kubernetes repository.

 Clean up Rabbit in all envs after account purge & lock down (8)

 Clean up Rabbit in all envs after account purge & lock down (8)

changing the default creds from guest/guest doesn't make things any more secure... it's just an annoying change in developer working practice, which will slow us down when working on features and bug fixes, I'm afraid.

Nick could you recheck this - guest/guest is still in place for the devs.

changing the default creds from guest/guest doesn't make things any more secure... it's just an annoying change in developer working practice, which will slow us down when working on features and bug fixes, I'm afraid.

Nick could you recheck this - guest/guest is still in place for the devs.


schemas: outlaw keys that contain pii data

https://trello.com/c/YhsT0jR9
Include self-tests of schema changes.

It should start to pass once https://github.com/alphagov/ckanext-datagovuk/pull/344 is merged and running on staging. I'm in the process of testing that now.
You could also point it at production instead. For some reason I haven't been able to find the issue there.

Just noted in discussion that I've forgotten to handle the data_dict and validated_data_dict fields in this.

:point_up: ok done this now.


Feature/document recurrence

The failing test is the python 2.6 test - all other tests are passing

I think you just saved this project, @cleder 😄


Login flow UI

Task/Issue URL: https://app.asana.com/0/488551667048375/1204259168623408/f
Description
Steps to test this PR
Feature 1

[ ]
[ ]

UI changes

Before
After

!(Upload before screenshot)
(Upload after screenshot)

Current dependencies on/for this PR:

develop

PR #2957

PR #2960
PR #2967

PR #2988

PR #3006   👈

This comment was auto-generated by Graphite.


ubuntu server caddy service can't active https

1. What version of Caddy are you running (caddy -version)?
caddy 0.9.5
2. What are you trying to do?
I wan't to active https
3. What is your entire Caddyfile?
www.example.com
root /var/www/example.com/www
gzip
ext .md
markdown {
    ext .md
    template default.html
}

When I run ‘caddy -conf /etc/caddy/Caddyfile' , It will active https, but run 'sudo service caddy start', work nothing.

I solved.
export  CADDYPATH=/etc/caddy
sudo chown ubuntu:www-data /etc/caddy
caddy -conf /etc/caddy/Caddyfile
change /etc/init/caddy.conf
setuid www-data
setgid www-data
to
setuid ubuntu
setgid www-data


Updated PixiShader to use standard uniform naming conventions (1f, 2f, 2fv, etc), fixed examples, added multiple shader support, updated Gruntfile.

Updated PixiShader to use standard uniform naming conventions (1f, 2f, 2fv, etc).. Added support for multiple sampler2D calls, allowing multi-texture shaders. Added support for Linear, Repeat, Nearest, Audio and Keyboard textures. Updated all the examples to use the new uniform naming convention. Updated Gruntfile to use the renamed GrayFilter and fixed indexAll in Example 15 to use the GrayFilter as well.

Cool

Cool